Image search method and device

Abstract

The embodiment of the present application discloses a method and device for image search. A specific implementation of the method includes: based on the search image uploaded by the user, selecting at least one image group from a preset image group set, a cluster group set obtained by product quantization of the image group association, a cluster association inverse linked list included in the cluster groups in the cluster group set, and the inverse linked list includes the image identifiers of the images in the image group that are related to the cluster; based on the matching relationship between the search image and the center points of the clusters included in the cluster groups in the cluster group set associated with each image group in the at least one image group and the inverse linked list associated with the cluster, selecting at least one image from the image set formed by the images in the each image group as the image to be pushed; generating search results including the determined images to be pushed. This implementation can save image search time and improve image search efficiency by introducing inverse linked list technology.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a method and device for image search. Background Art

[0002] With the rapid development of Internet technology, not only text search but also image search can be performed. For example, after obtaining the search image uploaded by the user, the search system can calculate the similarity between the search image and each image in the database, and then select at least one image from the images as the search result and return it to the user based on the result of the similarity calculation. Summary of the invention

[0003] The embodiments of the present application provide a method and device for image search.

[0004] In a first aspect, an embodiment of the present application provides an image search method, the method comprising: based on a search image uploaded by a user, selecting at least one image group from a preset image group set, wherein the image group association is a cluster group set obtained after product quantization, and the cluster groups in the cluster group set include a cluster association inverse linked list, and the inverse linked list includes image identifiers of images in the image group that are related to the cluster; based on a matching relationship between the search image and the center points of the clusters included in the cluster groups in the cluster group set associated with each image group in the above-mentioned at least one image group and the inverse linked list associated with the cluster, selecting at least one image from the image set formed by the images in the each image group as the image to be pushed; and generating search results including the determined images to be pushed.

[0005] In some embodiments, the picture group is also associated with a first cluster, which includes picture feature vectors corresponding to each picture in the picture group; and based on the search pictures uploaded by the user, at least one picture group is selected from a preset picture group set, including: obtaining a first picture feature vector corresponding to the search picture; determining the distance between the center point of the first cluster associated with each picture group in the picture group set and the first picture feature vector; based on the determined distance, selecting at least one picture group from the picture group set.

[0006] In some embodiments, selecting at least one picture group from the picture group set based on the determined distance includes: selecting a picture group from the picture group set having the smallest distance between center points of a first preset number of associated first class clusters and the first picture feature vector.

[0007] In some embodiments, based on the matching relationship between the search image and the center point of the cluster included in the cluster group in the cluster group set associated with each picture group in the above-mentioned at least one picture group and the inverted linked list associated with the cluster, at least one picture is selected from the picture set formed by the pictures in the each picture group as the picture to be pushed, including: for each cluster included in each cluster group in the cluster group set associated with each picture group in the above-mentioned at least one picture group, determining the distance between the center point of the cluster and the first picture feature vector; traversing each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, performing a distance accumulation operation on the picture indicated by the picture identifier; based on the distances corresponding to each picture in the picture set obtained by the distance accumulation operation, selecting at least one picture from the picture set as the picture to be pushed.

[0008] In some embodiments, based on the distances obtained by the distance accumulation operation corresponding to each picture in the picture collection, at least one picture is selected from the picture collection as the picture to be pushed, including: selecting the pictures with the smallest distances obtained by the distance accumulation operation corresponding to a second preset number of pictures from the picture collection as the pictures to be pushed.

[0009] In some embodiments, generating search results including determined pictures to be pushed includes: determining the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector; sorting the determined pictures to be pushed in order of distance from small to large to obtain a sequence of pictures to be pushed; generating search results including the sequence of pictures to be pushed.

[0010] In some embodiments, the above method further includes: returning the search results to the terminal device used by the user.

[0011] In a second aspect, an embodiment of the present application provides an image search device, which includes: a first selection unit, configured to select at least one image group from a preset image group set based on a search image uploaded by a user, wherein the image group association is a cluster group set obtained after product quantization, and the cluster groups in the cluster group set include a cluster association inverse linked list, and the inverse linked list includes image identifiers of images in the image group that are related to the cluster; a second selection unit, configured to select at least one image as a to-be-pushed image from a image set formed by images in each image group based on a matching relationship between the search image and the center points of the clusters included in the cluster groups in the cluster group set associated with each image group in the above-mentioned at least one image group and an inverse linked list associated with the cluster; a generation unit, configured to generate search results including the determined images to be pushed.

[0012] In some embodiments, the picture group is also associated with a first cluster, which includes picture feature vectors corresponding to each picture in the picture group; and the first selection unit includes: an acquisition subunit, configured to acquire a first picture feature vector corresponding to the search picture; a determination subunit, configured to determine the distance between the center point of the first cluster associated with each picture group in the picture group set and the first picture feature vector; and a first selection subunit, configured to select at least one picture group from the picture group set based on the determined distance.

[0013] In some embodiments, the first selection subunit is further configured to: select, from the picture group set, a picture group having a first preset number of picture groups with the smallest distances between center points of the associated first clusters and the first picture feature vector.

[0014] In some embodiments, the second selection unit includes: a distance accumulation subunit, which is configured to determine the distance between the center point of the cluster and the first image feature vector for each cluster included in each cluster group in the cluster group set associated with each picture group in the above-mentioned at least one picture group; traverse each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, perform a distance accumulation operation on the picture indicated by the picture identifier; the second selection subunit is configured to select at least one picture from the picture set as the picture to be pushed based on the distance corresponding to each picture in the picture set obtained by the distance accumulation operation.

[0015] In some embodiments, the second selection subunit is further configured to: select from the picture set a second preset number of pictures corresponding to the pictures with the smallest distance obtained through the distance accumulation operation as the pictures to be pushed.

[0016] In some embodiments, the generation unit is further configured to: determine the distance between the image feature vector corresponding to each image to be pushed and the first image feature vector; sort the determined images to be pushed in order of distance from small to large to obtain a sequence of images to be pushed; and generate search results including the sequence of images to be pushed.

[0017] In some embodiments, the above-mentioned apparatus further includes: a sending unit configured to return the search results to a terminal device used by the user.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation method in the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0020] The image search method and device provided by the embodiment of the present application first selects at least one image group from a preset image group set based on the search image uploaded by the user, wherein the image group association is a cluster group set obtained after product quantization, and the cluster group in the cluster group set includes a cluster association inverse linked list, and the inverse linked list includes the image identifiers of the images in the image group that are related to the cluster, and then based on the matching relationship between the search image and the center point of the cluster included in the cluster group in the cluster group set associated with each image group in the at least one image group and the inverse linked list associated with the cluster, at least one image is selected from the image set formed by the images in the each image group as the image to be pushed, so as to generate search results including the determined image to be pushed. The solution provided by the above embodiment of the present application can quickly determine the matching relationship between each image in the image set and the search image by introducing the inverse linked list technology, so that the image similar to the search image (such as the determined image to be pushed) can be quickly determined, and therefore the image search time can be saved and the image search efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0022] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;

[0023] Figure 2 is a flow chart of an embodiment of a picture search method according to the present application;

[0024] Figure 3 is a schematic diagram of an application scenario of the image search method according to the present application;

[0025] Figure 4 is a flowchart of another embodiment of the image search method according to the present application;

[0026] Figure 5 is a structural schematic diagram of an embodiment of a picture search device according to the present application;

[0027] Figure 6 It is a structural diagram of a computer system of an electronic device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0028] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the picture search method or picture search device of the present application can be applied.

[0031] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0032] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be installed with various communication client applications, such as web browser applications, information search applications, etc.

[0033] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, to provide distributed services), or they can be implemented as a single software or software module. No specific limitation is made here.

[0034] Server 105 can be a server that provides various services, such as an image search server for performing image search. The image search server can receive search images uploaded by users through terminal devices 101, 102, and 103, and then perform corresponding search operations based on the search images, and return the search results to the terminal device.

[0035] It should be noted that the image search method provided in some embodiments of the present application is generally executed by the server 105 , and accordingly, the image search device is generally disposed in the server 105 .

[0036] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] Continue to refer Figure 2 , which shows a process 200 of an embodiment of the image search method according to the present application. The process 200 of the image search method includes the following steps:

[0039] Step 201: based on the search pictures uploaded by the user, at least one picture group is selected from a preset picture group set.

[0040] In this embodiment, the execution subject of the image search method may be a server (for example Figure 1 The server can receive the user's data in real time through the terminal device (e.g. Figure 1 The terminal devices 101, 102, and 103 shown in the figure are used to upload search pictures, and then at least one picture group is selected from a preset picture group set based on the search pictures. Each picture group in the picture group set can be associated with a cluster group set obtained after product quantization. Each cluster included in each cluster group in the cluster group set can be associated with a posting linked list. The posting linked list can include picture identifiers of pictures in the picture group that are related to the cluster.

[0041] It should be noted that the picture group set can be stored locally on the server or in a picture storage device connected to the server, which is not specifically limited here. In addition, the picture group set can be obtained by clustering the pictures in the picture library through a clustering algorithm (such as a K-means clustering algorithm, etc.). K can represent the number of categories, and means can represent the mean. K-means is an algorithm that clusters data points by means.

[0042] For each image group in the image group set, the cluster group set associated with each image group can be obtained by product quantization of each image in the image group using a product quantization (PQ) algorithm. The purpose of the product quantization algorithm is to speed up the image retrieval speed, so it is a retrieval algorithm developed on the basis of vector quantization (VQ).

[0043] Specifically, for each picture group in the picture group set, each picture in the picture group can correspond to a picture feature vector. The picture feature vector can be used to characterize the picture to which it corresponds. When product quantization is performed on each picture in the picture group, the vector space where the picture feature vector corresponding to each picture is located can be divided into multiple subspaces. Then, based on the multiple subspaces, the picture feature vector corresponding to each picture can be divided into multiple sub-feature vectors. Then, for each subspace in the multiple subspaces, the sub-feature vectors belonging to the subspace can be clustered by a clustering algorithm to obtain a cluster group corresponding to the subspace. Then, the cluster groups corresponding to the multiple subspaces can be generated into a cluster group set corresponding to the picture group.

[0044] For the inverted linked list associated with each cluster included in each cluster group in the cluster group set, the inverted linked list can be formed by the picture identifiers of the pictures to which each sub-feature vector included in the cluster belongs. In addition, the cluster can also correspond to a center point. The center point can be, for example, the average value of each sub-feature vector included in the cluster.

[0045] In this embodiment, the server may use various selection methods to select at least one picture group from the picture group set based on the search pictures uploaded by the user. As an example, each picture group in the picture group set may correspond to a picture category. The server may use various analysis methods to perform content analysis on the search pictures, and determine the picture category to which the search pictures belong based on the analysis results. Then, the server may select a picture group from the picture group set that corresponds to the same picture category as the search pictures.

[0046] Optionally, each picture group in the picture group set may also be associated with a first cluster. The first cluster may include picture feature vectors corresponding to each picture in the picture group. In addition, the first cluster may correspond to a center point, which may be, for example, an average value of each picture feature vector in the first cluster. In order to improve the effectiveness of the picture group selected from the picture group set, the server may adopt the following selection method: obtaining a first picture feature vector corresponding to the search picture; determining the distance between the center point of the first cluster associated with each picture group in the picture group set and the first picture feature vector; based on the determined distance, selecting at least one picture group from the picture group set. For example, a first preset number (e.g., 1, 2, or 3, etc.) of picture groups with the smallest distance between the center point of the first cluster associated with them and the first picture feature vector are selected from the picture group set. Here, the first preset number may be set according to actual needs and is not specifically limited here.

[0047] It should be noted that the server can use various image feature extraction algorithms (such as convolutional neural network, local binary pattern algorithm, etc.) to extract features from the search image, and use the extracted features to generate a first image feature vector corresponding to the search image. In addition, the server can use various distance measurement methods (such as Euclidean distance, Manhattan distance, Ming's distance, etc.) to determine the distance between the center point of the first cluster and the first image feature vector.

[0048] It should be noted that the greater the distance between two vectors, the lower the similarity between the two vectors. The smaller the distance between two vectors, the higher the similarity between the two vectors.

[0049] Step 202, based on the matching relationship between the search image and the center points of the clusters included in the cluster groups in the cluster group set associated with each picture group in at least one picture group and the inverted linked list associated with the cluster, select at least one picture from the picture set formed by the pictures in each picture group as the picture to be pushed.

[0050] In this embodiment, the server can select at least one picture as the picture to be pushed from the picture set formed by the pictures in each picture group based on the matching relationship between the search picture and the center point of the cluster included in the cluster group set associated with each picture group in the above-mentioned at least one picture group and the inverted linked list associated with the cluster.

[0051] As an example, the server can determine the spatial distance between each image in the above-mentioned image set and the search image based on the matching relationship between the center points of the clusters included in the cluster group set associated with each image group in the above-mentioned at least one image group and the inverted linked list associated with the cluster. The distance can be used to measure the similarity between the image and the search image. Then, the server can select at least one image from the above-mentioned image set as the image to be pushed based on the determined distance.

[0052] Specifically, for each cluster included in each cluster group in the cluster group set associated with each picture group in the at least one picture group, the server can use the distance measurement method described above to determine the distance between the center point of the cluster and the first picture feature vector. Then, for each cluster included in each cluster group in the cluster group set associated with each picture group in the at least one picture group, the server can traverse the inverted linked list associated with the cluster, and read the distance between the center point of the cluster and the first picture feature vector determined previously. When traversing each picture identifier in the inverted linked list, the server can perform a distance accumulation operation on the picture indicated by the picture identifier based on the distance. Then, the server can select at least one picture from the picture set as the picture to be pushed based on the distance obtained by the distance accumulation operation corresponding to each picture in the picture set. For example, select the picture with the smallest distance obtained by the distance accumulation operation corresponding to the second preset number (for example, 50, 100 or 200, etc.) of pictures from the picture set as the picture to be pushed. Here, the second preset number can be set according to actual needs and is not specifically limited here.

[0053] It should be noted that the distance obtained by the distance accumulation operation corresponding to each picture in the above picture set may be the distance between the picture and the search picture in space. In fact, the distance obtained by the distance accumulation operation corresponding to each picture in the above picture set is the sum of the distances corresponding to the clusters to which each sub-feature vector corresponding to the picture feature vector corresponding to the picture belongs (the distance between the center point of the cluster and the first picture feature vector).

[0054] It should be noted that by referring to the inverted linked list technology, it is possible to avoid repeatedly reading the distance between the center point of the same cluster and the first image feature vector when determining the distance used to measure the similarity between each image in the above image set and the search image. In this way, the time spent in the image search process can be effectively saved, and therefore, the user's waiting time can be effectively saved, thereby improving the user experience.

[0055] Step 203: Generate search results including the determined pictures to be pushed.

[0056] In this embodiment, the server may generate a search result including the determined pictures to be pushed. As an example, the server may first sort the determined pictures to be pushed to obtain a sequence of pictures to be pushed. For example, the server may sort the pictures to be pushed based on the order of distances corresponding to the pictures to be pushed obtained by the distance accumulation operation from small to large. Then, the server may generate a search result including the sequence of pictures to be pushed.

[0057] In some optional implementations of this embodiment, the server may also return the search results to the terminal device used by the user via a wired connection or a wireless connection.

[0058] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the image search method according to this embodiment. Figure 3 In the application scenario of , the user can send the search picture A to the picture search server through the terminal device, as shown by reference numeral 301. Among them, the picture search server can store a picture group set locally. Each picture group in the picture group set can be associated with a cluster group set obtained after product quantization. Each cluster included in each cluster group in the cluster group set can be associated with a reversed linked list. The reversed linked list may include picture identifiers of pictures in the picture group that are related to the cluster. After receiving the search picture A, the picture search server can select at least one picture group from the picture group set based on the search picture A, as shown by reference numeral 302. Then, as shown by reference numeral 303, the picture search server can determine the spatial distance between each picture in the picture set formed by the pictures in each picture group and the search picture A based on the matching relationship between the search picture A and the center points of the clusters included in the cluster groups in the cluster group set associated with each picture group in the above-mentioned at least one picture group and the reversed linked list associated with the cluster. Next, as shown in reference numeral 304, the image search server may select at least one image from the image collection as the image to be pushed based on the determined distance in ascending order. Then, as shown in reference numeral 305, the image search server may generate search results including the determined image to be pushed. Finally, as shown in reference numeral 306, the image search server may return the search results to the terminal device.

[0059] The method provided by the above-mentioned embodiment of the present application first selects at least one picture group from a preset picture group set based on the search pictures uploaded by the user, wherein the picture group association is a cluster group set obtained after product quantization, and the cluster groups in the cluster group set include a cluster association inverse linked list, and the inverse linked list includes the picture identifiers of the pictures in the picture group that are related to the cluster, and then based on the matching relationship between the search picture and the center point of the cluster included in the cluster group in the cluster group set associated with each picture group in the at least one picture group and the inverse linked list associated with the cluster, at least one picture is selected from the picture set formed by the pictures in the pictures in each picture group as the picture to be pushed, so as to generate search results including the determined pictures to be pushed. The scheme provided by the above-mentioned embodiment of the present application can quickly determine the matching relationship between each picture in the picture set and the search picture by introducing the inverse linked list technology, so that pictures similar to the search picture (such as the determined pictures to be pushed) can be quickly determined, and therefore, the picture search time can be saved and the picture search efficiency can be improved.

[0060] Further references Figure 4 , which shows a process 400 of another embodiment of the image search method. The process 400 of the image search method includes the following steps:

[0061] Step 401, obtaining a first picture feature vector corresponding to a search picture uploaded by a user.

[0062] In this embodiment, the execution subject of the image search method may be a server (for example Figure 1 The server can receive the user's data in real time through the terminal device (e.g. Figure 1 The server may obtain a first picture feature vector corresponding to the search picture by using various picture feature extraction algorithms (such as convolutional neural network, local binary pattern algorithm, etc.) to extract features from the search picture, and generate a first picture feature vector corresponding to the search picture using the extracted features.

[0063] It should be noted that the image feature vector can be used to represent the image to which it corresponds.

[0064] Step 402: Determine the distance between the center point of the first cluster associated with each picture group in the preset picture group set and the first picture feature vector.

[0065] In this embodiment, each picture group in the preset picture group set can be associated with a first cluster. The first cluster can include picture feature vectors corresponding to each picture in the picture group. In addition, the picture group can also be associated with a cluster group set obtained after product quantization. The clusters included in the cluster groups in the cluster group set can be associated with a posting list, and the posting list can include picture identifiers of pictures in the picture group that are related to the cluster. Here, for an explanation of the first cluster, the cluster group set, and the posting list, please refer to Figure 2 The relevant descriptions in the illustrated embodiment will not be repeated here.

[0066] In this embodiment, after obtaining the first picture feature vector, the server can use various distance measurement methods (such as Euclidean distance, Manhattan distance, Ming distance, etc.) to determine the distance between the center point of the first type of cluster associated with each picture group in the picture group set and the first picture feature vector.

[0067] Step 403: Select at least one picture group from the picture group set based on the determined distance.

[0068] In this embodiment, after executing step 402, the server may select at least one picture group from the picture group set based on the determined distance. For example, a first preset number (e.g., 1, 2, or 3, etc.) of picture groups with the smallest distance between the center point of the associated first cluster and the first picture feature vector are selected from the picture group set. Here, the first preset number can be set according to actual needs and is not specifically limited here.

[0069] Step 404, for each cluster included in each cluster group in the cluster group set associated with each picture group in at least one picture group, determine the distance between the center point of the cluster and the first picture feature vector, traverse each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, perform a distance accumulation operation on the picture indicated by the picture identifier.

[0070] In this embodiment, for each cluster included in each cluster group in the cluster group set associated with each picture group in the at least one picture group, the server can use the distance measurement method described above to determine the distance between the center point of the cluster and the first picture feature vector. Then, the server can traverse each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, perform a distance accumulation operation on the picture indicated by the picture identifier.

[0071] Step 405 , for a picture set formed by pictures in each picture group in at least one picture group, based on the distances corresponding to each picture in the picture set obtained through the distance accumulation operation, at least one picture is selected from the picture set as the picture to be pushed.

[0072] In this embodiment, for a picture set formed by pictures in each picture group in the at least one picture group, the server may select at least one picture from the picture set as the picture to be pushed based on the distances corresponding to each picture in the picture set obtained by the distance accumulation operation. For example, the server may select pictures with the smallest distances obtained by the distance accumulation operation corresponding to a second preset number (e.g., 50, 100, or 200, etc.) of pictures from the picture set as the pictures to be pushed. Here, the second preset number can be set according to actual needs and is not specifically limited here.

[0073] Step 406: Determine the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector.

[0074] In this embodiment, after determining the pictures to be pushed corresponding to the search pictures, the server may use the distance measurement method described above to determine the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector.

[0075] Step 407 , sorting the determined pictures to be pushed in ascending order of distance to obtain a sequence of pictures to be pushed.

[0076] In this embodiment, the server may sort the determined pictures to be pushed in ascending order of the distances determined in step 406 to obtain a sequence of pictures to be pushed. In this way, the pictures to be pushed that are most similar to the search pictures may be placed in front.

[0077] Step 408: Generate search results including the sequence of pictures to be pushed.

[0078] In this embodiment, the server may generate search results including a sequence of images to be pushed, which can facilitate users to quickly obtain the required images, save users' browsing time, and improve user experience.

[0079] In some optional implementations of this embodiment, the server may also return the search results to the terminal device used by the user via a wired connection or a wireless connection.

[0080] from Figure 4 It can be seen that Figure 2Compared with the corresponding embodiment, the process 400 of the image search method in this embodiment highlights the steps of expanding the selection method of at least one image group, the selection method of the image to be pushed, and the method of generating the search results. Therefore, the scheme described in this embodiment can achieve the diversity of information processing. In addition, by obtaining the first image feature vector corresponding to the search image uploaded by the user, and then determining the distance between the center point of the first class cluster associated with each image group in the preset image group set and the first image feature vector, and then selecting at least one image group from the image group set based on the determined distance, the effectiveness of the selected image group can be improved, and the selected image group can have a higher correlation with the search image. In addition, for each cluster included in each cluster group in the cluster group set associated with each picture group in at least one picture group, by determining the distance between the center point of the cluster and the first picture feature vector, and traversing each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, performing a distance accumulation operation on the picture indicated by the picture identifier, the spatial distance between each picture included in each picture group in the at least one picture group and the search picture can be quickly obtained, which can effectively save time in the picture search process. In addition, by determining the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector, and then sorting the determined pictures to be pushed in order of distance from small to large, obtaining a sequence of pictures to be pushed, and then generating search results including the sequence of pictures to be pushed, the pictures ranked in front in the search results can be the pictures most similar to the search pictures, which can facilitate users to quickly obtain the required pictures, save users' browsing time, and improve user experience.

[0081] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a picture search device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0082] like Figure 5As shown, the image search device 500 of this embodiment includes: a first selection unit 501 is configured to select at least one image group from a preset image group set based on a search image uploaded by a user, wherein the image group can be associated with a cluster group set obtained after product quantization, and the clusters included in the cluster groups in the cluster group set can be associated with an inverted linked list, which can include image identifiers of images in the image group that are related to the cluster; a second selection unit 502 is configured to select at least one image as a to-be-pushed image from a image set formed by images in each image group based on a matching relationship between the search image and center points of clusters included in the cluster groups in the cluster group set associated with each image group in the at least one image group and an inverted linked list associated with the cluster; a generation unit 503 is configured to generate search results including the determined images to be pushed.

[0083] In this embodiment, in the image search device 500, the specific processing of the first selection unit 501, the second selection unit 502 and the generation unit 503 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of step 201, step 202 and step 203 in the corresponding embodiment are not repeated here.

[0084] In some optional implementations of the present embodiment, the picture group may also be associated with a first cluster, which may include picture feature vectors corresponding to each picture in the picture group; and the first selection unit 501 may include: an acquisition subunit (not shown in the figure), configured to acquire a first picture feature vector corresponding to the search picture; a determination subunit (not shown in the figure), configured to determine the distance between the center point of the first cluster associated with each picture group in the picture group set and the first picture feature vector; and a first selection subunit (not shown in the figure), configured to select at least one picture group from the picture group set based on the determined distance.

[0085] In some optional implementations of this embodiment, the first selection subunit may be further configured to: select, from the picture group set, a picture group having a first preset number of associated first class clusters with the smallest distance between the center point and the first picture feature vector.

[0086] In some optional implementations of the present embodiment, the second selection unit 502 may include: a distance accumulation subunit (not shown in the figure), configured to determine the distance between the center point of the cluster and the first image feature vector for each cluster included in each cluster group in the cluster group set associated with each image group in the above-mentioned at least one image group; traverse each image identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first image feature vector, perform a distance accumulation operation on the image indicated by the image identifier; a second selection subunit (not shown in the figure), configured to select at least one image from the image collection as the image to be pushed based on the distances corresponding to each image in the image collection obtained through the distance accumulation operation.

[0087] In some optional implementations of this embodiment, the second selection subunit may be further configured to: select from the picture set a second preset number of pictures corresponding to the pictures with the smallest distance obtained through the distance accumulation operation as the pictures to be pushed.

[0088] In some optional implementations of the present embodiment, the generation unit 503 may be further configured to: determine the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector; sort the determined pictures to be pushed in order of distance from small to large to obtain a sequence of pictures to be pushed; and generate search results including the sequence of pictures to be pushed.

[0089] In some optional implementations of this embodiment, the above-mentioned device 500 may also include: a sending unit (not shown in the figure), configured to return the search result to the terminal device used by the user.

[0090] The device provided by the above-mentioned embodiment of the present application first selects at least one picture group from a preset picture group set based on the search pictures uploaded by the user, wherein the picture group association is a cluster group set obtained after product quantization, and the cluster group in the cluster group set includes a cluster association inverse linked list, and the inverse linked list includes the picture identifiers of the pictures in the picture group that are related to the cluster, and then based on the matching relationship between the search picture and the center point of the cluster included in the cluster group in the cluster group set associated with each picture group in the at least one picture group and the inverse linked list associated with the cluster, at least one picture is selected from the picture set formed by the pictures in the pictures in each picture group as the picture to be pushed, so as to generate search results including the determined pictures to be pushed. The scheme provided by the above-mentioned embodiment of the present application can quickly determine the matching relationship between each picture in the picture set and the search picture by introducing the inverse linked list technology, so that pictures similar to the search picture (such as the determined pictures to be pushed) can be quickly determined, and therefore picture search time can be saved and picture search efficiency can be improved.

[0091] Reference below Figure 6 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer system 600 having a server 105 as shown. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0092] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0093] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0094] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0095] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0096] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a first selection unit, a second selection unit and a generation unit. The names of these units do not constitute a limitation on the units themselves in some cases. For example, the first selection unit may also be described as "a unit for selecting at least one picture group from a preset picture group set based on a search picture uploaded by a user".

[0099] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device: based on the search pictures uploaded by the user, select at least one picture group from a preset picture group set, wherein the picture group can be associated with a cluster group set obtained after product quantization, and the clusters included in the cluster groups in the cluster group set can be associated with an inverted linked list, which can include picture identifiers of pictures in the picture group that are related to the cluster; based on the matching relationship between the search picture and the center point of the cluster included in the cluster group in the cluster group set associated with each picture group in the above at least one picture group and the inverted linked list associated with the cluster, select at least one picture from the picture set formed by the pictures in the pictures in each picture group as the picture to be pushed; generate search results including the determined pictures to be pushed.

[0100] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for searching an image, comprising: Based on the search pictures uploaded by the user, at least one picture group is selected from a preset picture group set, wherein the picture group association is a cluster group set obtained by product quantization of the pictures in the picture group, and the cluster groups in the cluster group set include a cluster association inverted linked list, wherein the inverted linked list is formed by picture identifiers of pictures to which each sub-feature vector included in the cluster belongs respectively; Based on the matching relationship between the center points of the clusters included in the cluster groups in the cluster group set associated with each picture group in the at least one picture group and the inverted linked list associated with the cluster, determine the distance between the center point of the cluster and the first picture feature vector of the search picture; traverse each picture identifier in the inverted linked list associated with the cluster, and based on the distance between the center point of the cluster and the first picture feature vector, perform a distance accumulation operation on the picture indicated by the picture identifier; Based on the distances corresponding to the pictures in the picture set obtained through the distance accumulation operation, selecting at least one picture from the picture set formed by the pictures in each picture group as the picture to be pushed; Generate search results including the determined pictures to be pushed; Among them, the method of quantizing the products of pictures in the picture group to obtain a set of cluster groups includes: dividing the vector space where the picture feature vector corresponding to the picture is located into multiple subspaces, dividing the picture feature vector corresponding to the picture into multiple sub-feature vectors based on the multiple subspaces, clustering the sub-feature vectors belonging to each subspace in the multiple subspaces through a clustering algorithm to obtain a cluster group corresponding to the subspace, and generating a set of cluster groups corresponding to the picture group according to the cluster groups respectively corresponding to the multiple subspaces.

2. The method according to claim 1, wherein: The picture group is also associated with a first cluster, wherein the first cluster includes picture feature vectors corresponding to each picture in the picture group; as well as The selecting at least one picture group from a preset picture group set based on the search pictures uploaded by the user includes: Obtaining a first picture feature vector corresponding to the search picture; Determine the distance between the center point of the first cluster associated with each picture group in the picture group set and the first picture feature vector; At least one group of pictures is selected from the set of groups of pictures based on the determined distance.

3. The method according to claim 2, wherein: The selecting at least one picture group from the picture group set based on the determined distance comprises: A picture group having a first preset number of associated first clusters with the smallest distance between the center point and the first picture feature vector is selected from the picture group set.

4. The method according to claim 1, wherein: The selecting at least one picture from the picture set as the picture to be pushed based on the distances corresponding to the pictures in the picture set obtained by the distance accumulation operation includes: Select from the picture set a second preset number of pictures with the smallest distance obtained through the distance accumulation operation as the pictures to be pushed.

5. The method according to any one of claims 1 to 4, wherein: The generating of search results including the determined images to be pushed includes: Determine the distance between the picture feature vector corresponding to each picture to be pushed and the first picture feature vector; Sort the determined pictures to be pushed in order of distance from small to large to obtain a sequence of pictures to be pushed; Generate a search result including the sequence of pictures to be pushed.

6. The method according to any one of claims 1 to 4, wherein: The method further comprises: The search result is returned to the terminal device used by the user.

7. A picture search device, comprising: A first selection unit is configured to select at least one picture group from a preset picture group set based on a search picture uploaded by a user, wherein the picture group association is a cluster group set obtained by product quantization of pictures in the picture group, and the cluster groups in the cluster group set include a cluster association inverted linked list, wherein the inverted linked list is formed by picture identifiers of pictures to which each sub-feature vector included in the cluster belongs respectively; The distance calculation unit is configured to determine the distance between the center point of the cluster and the first picture feature vector of the search image based on the matching relationship between the center point of the cluster included in the cluster group in the cluster group set associated with each picture group in the at least one picture group and the inverted linked list associated with the cluster; traverse each picture identifier in the inverted linked list associated with the cluster, and perform a distance accumulation operation on the picture indicated by the picture identifier based on the distance between the center point of the cluster and the first picture feature vector; A second selection unit is configured to select at least one picture from the picture set formed by the pictures in each picture group as the picture to be pushed based on the distances corresponding to the pictures in the picture set obtained by the distance accumulation operation; A generating unit, configured to generate search results including the determined pictures to be pushed; Among them, the quantization of the products of the pictures in the picture group in the first selection unit to obtain a set of cluster groups includes: dividing the vector space where the picture feature vector corresponding to the picture is located into multiple subspaces, dividing the picture feature vector corresponding to the picture into multiple sub-feature vectors based on the multiple subspaces, clustering the sub-feature vectors belonging to the subspace divided out of each subspace in the multiple subspaces by a clustering algorithm to obtain a cluster group corresponding to the subspace, and generating a set of cluster groups corresponding to the picture group according to the cluster groups respectively corresponding to the multiple subspaces.

8. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images