Acceleration Method for Image Retrieval, Retrieval Device, Electronic Device, and Storage Medium

Abstract

The present application discloses an acceleration method for image retrieval, a retrieval device, an electronic device, and a storage medium. The acceleration method includes: obtaining an original feature vector of an image to be retrieved; processing the original feature vector based on a preset first classifier to obtain a residual feature vector of the original feature vector; inputting the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table; and performing accelerated retrieval on a query vector of the image to be retrieved based on the similarity distance query table and a preset retrieval table. By the above method, the present application can perform accelerated retrieval on the query vector of the image to be retrieved, making the search result faster and more accurate.

Description

Technical Field

[0001] The present application relates to the technical field of image retrieval, and particularly to an acceleration method for image retrieval, a retrieval device, an electronic device, and a storage medium. Background Art

[0002] Generally, the application fields of image retrieval are very extensive. It can be used in medical imaging to control the treatment of patients, and also in recommendation systems and search engines to query similar images. Maximum inner product search has gradually become a paradigm for solving large-scale image retrieval tasks.

[0003] However, accurately calculating the inner product between the query image and the image data in the database, and the calculation of the inner product is not only related to the magnitude of the vector modulus, but also needs to consider the direction between the vectors, which usually consumes a great deal of cost. And it is easier to obtain a larger inner product in the vector direction. The larger the inner product, the higher the similarity, which instead leads to ignoring the influence of the sorting and direction of the feature vectors on the loss function, resulting in a larger error result in image retrieval and reducing the speed of image retrieval. Summary of the Invention

[0004] To solve the above technical problems, the technical solution adopted in the first aspect of the present application is to provide an acceleration method for image retrieval, and the acceleration method includes: obtaining the original feature vector of the image to be retrieved; processing the original feature vector based on a preset first classifier to obtain the residual feature vector of the original feature vector; inputting the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table; and performing accelerated retrieval on the query vector of the image to be retrieved based on the similarity distance query table and a preset retrieval table.

[0005] To solve the above technical problems, the technical solution adopted in the second aspect of the present application is to provide a retrieval device, including:

[0006] An obtaining module, configured to obtain the original feature vector of the image to be retrieved;

[0007] A processing module, configured to process the original feature vector based on a preset first classifier to obtain the residual feature vector of the original feature vector;

[0008] A calculation module, configured to input the residual feature vector into a preset second classifier by the input module to calculate and obtain a similarity distance query table;

[0009] An accelerated retrieval module, configured to perform accelerated retrieval on the query vector of the image to be retrieved based on the similarity distance query table and a preset retrieval table.

[0010] To solve the above technical problems, the technical solution adopted in the third aspect of the present application is to provide an electronic device, which includes: a processor and a memory. A computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method according to the first aspect of the present application.

[0011] To solve the above technical problems, the technical solution adopted in the fourth aspect of the present application is to provide a computer-readable storage medium, which stores a computer program that can implement the method according to the first aspect of the present application when executed by a processor.

[0012] The beneficial effect of the present application is that: by obtaining the original feature vector of the image to be retrieved; processing the residual feature vector; calculating the residual feature vector to obtain a similarity distance query table; based on the similarity distance query table and a preset retrieval table, the query vector of the image to be retrieved can be retrieved quickly, thereby shortening the query time and improving the cost performance of image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0014] Figure 1 It is a schematic flowchart of the first embodiment of the acceleration method for image retrieval in the present application;

[0015] Figure 2 is Figure 1 a schematic flowchart of an implementation process before step S11 in

[0016] Figure 3 is Figure 1 a schematic flowchart of a specific implementation process for building an index before step S11 in

[0017] Figure 4 is based on Figure 3 a schematic flowchart of the query process in the specific embodiment in

[0018] Figure 5 is Figure 2 a schematic flowchart of a specific implementation process of step S23 in

[0019] Figure 6 is Figure 2 another schematic flowchart of a specific implementation process of step S23 in

[0020] Figure 7 is Figure 1Schematic diagram of a specific implementation process of step S13 in

[0021] Figure 8 is Figure 7 Schematic diagram of a specific implementation process of step S52 in

[0022] Figure 9 is Figure 1 Schematic diagram of a specific implementation process of step S14 in

[0023] Figure 10 Schematic diagram of a specific implementation example process of accelerated calculation for image retrieval in this application

[0024] Figure 11 Schematic diagram of storage of compressed index table for image retrieval in this application

[0025] Figure 12 Schematic diagram of distance retrieval for image retrieval in this application

[0026] Figure 13 Schematic block diagram of the structure of an embodiment of the retrieval device in this application

[0027] Figure 14 Schematic block diagram of the structure of an embodiment of the electronic device in this application

[0028] Figure 15 Schematic block diagram of the circuit of an embodiment of the computer-readable storage medium in this application Specific implementation manner

[0029] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of this application.

[0030] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0031] It should also be understood that the terms used in this specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in this specification of this application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0032] It should be further understood that the term "and / or" as used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] As used in this specification and the appended claims, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0035] To illustrate the technical solutions of the present application, the following specific embodiments are used to illustrate that the present application provides an acceleration method for image retrieval. Please refer to Figure 1 , Figure 1 is a schematic flowchart of the first embodiment of the acceleration method for image retrieval in the present application. The acceleration method specifically includes the following steps:

[0036] S11: Obtain the original feature vector of the image to be retrieved;

[0037] To solve the problem of fast image retrieval, generally two methods are used to accelerate the query. On the one hand, the amount of calculation is reduced by reducing the search range. On the other hand, the dimension of the image feature vector is reduced.

[0038] In this embodiment, the original vector space of the image to be retrieved can be extracted. Specifically, the original vector space is decomposed into the Cartesian product of m low-dimensional vector spaces, and kmeans clustering is performed on the decomposed low-dimensional vector spaces respectively. The clustering centers are learned through kmeans clustering, and the Cartesian product of these clustering centers is the index corresponding to the original feature vector. The optimization objective of kmeans clustering is shown in Equation (1). q is the original feature vector to be queried, x i is the feature vector in the database, is its clustering center, < > represents the inner product operation, E represents the expectation, Q represents the distribution of q, and l represents the loss function. The smaller the function value of this loss function l, the closer the inner product of q and is to the inner product of q and x iThe inner product, that is, the better the clustering effect.

[0039]

[0040] When the query vector satisfies isotropy, that is, qq T = cI, Equation (1) can be rewritten as follows. As shown in Equation (2) below:

[0041]

[0042] It can be seen from Equation (2) that when isotropy is satisfied, the minimum squared error used in nearest neighbor search can also be equivalent to the maximum inner product search discussed currently. Thus, processing the images to be retrieved in this way will lead to large errors. The specific analysis mainly has the following two reasons:

[0043] 1) If the inner product of q and x i is larger, then the similarity between the two is higher. Then x i is ranked more in the front of the query results. Then the error result of x i should have a higher proportion in the total error. This scheme does not consider the influence of this inner product.

[0044] 2) The calculation of the inner product is not only related to the magnitude of the vectors, but also the direction between the vectors needs to be considered. It is easier to obtain a large inner product in the direction parallel to the q vector. Therefore, we need to impose a larger penalty coefficient on it. This scheme does not consider the influence of the direction on the loss function. Therefore, it is particularly important to accelerate the speed of image retrieval while ensuring the accuracy of image retrieval.

[0045] Among them, the image to be retrieved contains the original feature vectors, which are used to represent the image to be retrieved. By obtaining the original feature vectors of the image to be retrieved, specifically, the features in the image set can be extracted through a feature extractor to obtain the original feature vectors of the image to be retrieved and store them in the database, which is conducive to improving the realizability of the accelerated image retrieval scheme.

[0046] S12: Based on a preset first classifier, process the original feature vectors to obtain the residual feature vectors of the original feature vectors;

[0047] Specifically, select a part of the original image data as training data. First, use the original feature vectors to train the preset first classifier, and then use the preset first classifier to process the original feature vectors to obtain the residual feature vectors of the original feature vectors.

[0048] Among them, a classifier is to learn a classification function or construct a classification model based on the existing data. This function or model can map the data records in the database to a certain one of the given categories, so that it can be applied to data prediction.

[0049] S13: Input the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table;

[0050] Specifically, in order to accelerate the image retrieval speed, map it to multiple sub-feature vector spaces respectively to train the same number of preset second classifiers to obtain multiple sub-residual feature vectors, input the multiple sub-residual feature vectors into the preset second classifier, calculate and obtain a similarity distance query table to index all image features.

[0051] S14: Based on the similarity distance query table and a preset retrieval table, perform accelerated retrieval on the query vector of the image to be retrieved.

[0052] Specifically, the inner product between each feature vector corresponds to a detection vector. According to the index, through the comparison and search of the similarity distance query table and the preset retrieval table, the vector set most similar to the original vector in the database can be found through sorting, so as to obtain the retrieval result.

[0053] Therefore, in this application, by obtaining the original feature vector of the image to be retrieved; processing the residual feature vector; then calculating the residual feature vector to obtain a similarity distance query table; and then based on the similarity distance query table and a preset retrieval table, the query vector of the image to be retrieved can be accelerated for retrieval, thereby shortening the query time and improving the cost performance of image retrieval.

[0054] In addition, before obtaining the original feature vector of the image to be retrieved, please refer to Figure 2 , Figure 2 is Figure 1 a schematic diagram of an implementation process before step S11 in

[0055] S21: Extract the features of the images in the image database to obtain image features;

[0056] Specifically, obtain an image set from the image database to get multiple images to be retrieved. Through a feature extractor, the features of the images in the image database can be extracted from the image set to obtain image features, that is, the original feature vectors mentioned above. Among them, please refer to Figure 3 , Figure 3 is Figure 1 a schematic diagram of a specific implementation process for building an index before step S11 in ; select a part from the image database as training data, use the original feature vectors to train a classifier C1 as the first classifier, and then use the classifier C1 to obtain the residual feature vectors of the original feature vectors.

[0057] S22: Divide the image features to obtain multiple sub-feature vectors;

[0058] After obtaining the original feature vectors, further proceed to two branches. One branch, as Figure 1 shown in, processes the original feature vectors using classifier C1 to obtain the residual feature vectors of the original feature vectors. The other branch is to divide the image features to obtain multiple sub-feature vectors for calculating the orientation loss and updating the established index.

[0059] S23: Calculate the multiple sub-feature vectors to obtain the orientation loss function;

[0060] Specifically, the larger the inner product, the higher the similarity between two vectors. In other words, the larger the inner product of q and x i , the higher x i should be ranked at the top of the query results. Therefore, the construction of the loss function l should also

[0061] consider the inner product value of q and x i . The re-constructed loss function is shown in Equation (3):

[0062]

[0063] where w represents the weight function, whose independent variable is the inner product value of q and x i , and has the property of non-decreasing monotonicity. From the calculation of the inner product, it can be seen that the feature vectors parallel to q will obtain larger values.

[0064] S24: Based on the multiple image sub-feature vectors and the orientation loss function, establish the indexes of the multiple image sub-feature vectors to obtain the preset retrieval table.

[0065] As Figure 3 shown, based on the multiple image sub-feature vectors and the orientation loss function, establish the indexes of the multiple image sub-feature vectors to obtain the preset retrieval table (i.e., the similarity distance query table). Please refer to Figure 4 , Figure 4 which is Figure 3 the schematic diagram of the query process in the specific embodiment of. Based on step S23, further, to optimize the orientation loss function, the second classifier uses the residual feature vectors as inputs for clustering, thereby optimizing the orientation loss function, and then updating the index table. Based on the updated index table, perform accelerated retrieval to obtain more accurate query results.

[0066] Even further, calculate the image features to obtain the orientation loss function. Please refer to Figure 5 , Figure 5 which is Figure 2 a specific implementation process schematic diagram of step S23 in, specifically including the following steps:

[0067] S31: Perform unsupervised training on a preset proportion of image features to obtain a preset first classifier;

[0068] First, an image dataset can be obtained and the images can be labeled. Specifically, a pre-trained feature extractor can be used to process the image data to obtain a 100-dimensional feature vector dataset. Using this labeled feature vector dataset, the dataset is saved as a file to provide data support for subsequent training and testing processes. The feature vector data set loaded into the memory is uniformly managed. In addition to providing functions for finding and adding feature vectors, basic numerical statistics functions are also provided for the data set.

[0069] Then, all feature vectors in the training data are read from the file, and a training set is obtained by random sampling according to a preset sampling ratio. The remaining feature vector data is used as a test set. Unsupervised training is performed using all the original feature vectors to obtain a classifier C1 with N cluster centers to narrow the search range, where N is a positive integer greater than or equal to 1.

[0070] Specifically, the unsupervised training here is the above-mentioned kmeans clustering method, and the distance metric for clustering is the inner product of vectors. The larger the inner product between vectors, the higher the similarity between the two. Conversely, the lower the similarity.

[0071] S32: Input the image features into the first classifier and calculate the residual feature vector corresponding to the image features;

[0072] Through the cluster centers of classifier C1, the residual feature vector corresponding to each original feature vector is calculated, thereby calculating the residual feature vector corresponding to the image features. Specifically, for example, if the subtraction of the original feature vector from the cluster center is defined as the residual of the feature vector, then the optimization objective of this classifier C1 is to reduce the inner product between the original feature vector and the residual feature vector.

[0073] S33: Map the residual feature vector to obtain multiple sub-residual feature vectors;

[0074] Specifically, the residual feature vector is further mapped to M subspaces to obtain multiple sub-residual feature vectors, where M is a positive integer greater than or equal to N. And unsupervised training is performed on multiple sub-residual feature vectors respectively to obtain multiple second classifiers; specifically, each subspace is separately subjected to unsupervised training to obtain the cluster centers of k subspaces, thereby obtaining multiple second classifiers, that is, classifier C2.

[0075] S34: Use multiple sub-residual feature vectors as inputs for clustering to optimize the direction loss function.

[0076] Specifically, before inputting multiple sub-residual feature vectors into the classifier C2, the multiple sub-residual feature vectors are used as inputs for clustering to optimize the direction loss function; then, the optimized direction loss is used to update the sub-cluster centers corresponding to the residual feature vectors for the indexes obtained by inputting the multiple sub-residual feature vectors into the classifier C2, thereby updating the indexes.

[0077] Furthermore, calculate the direction loss function for the multiple sub-feature vectors. Please refer to Figure 6 , Figure 6 which Figure 2 is another schematic diagram of the specific implementation process of step S23 in

[0078] S41: Decompose the multiple sub-residual feature vectors into parallel residuals parallel to the direction of the query feature vector and perpendicular residuals.

[0079] Specifically, from the calculation of the inner product, it can be seen that the feature vectors parallel to q will obtain larger values. Therefore, the residual feature vector can be decomposed into a residual r / / parallel to the direction of q and a residual r ⊥ .

[0080] Among them, during the training phase, the query feature vector q is x i , so the calculation of r / / is shown in Equation (4):

[0081]

[0082] The calculation of the residual r ⊥ perpendicular to the direction of q is shown in Equation (5):

[0083]

[0084] S42: Allocate weights to the parallel residuals and perpendicular residuals in the direction of the query feature vector to obtain the parallel weight corresponding to the parallel residual and the perpendicular weight corresponding to the perpendicular residual.

[0085] Specifically, for the image to be retrieved, corresponding to the direction of the query feature vector q, allocate weights to the parallel residual r / / and the perpendicular residual r ⊥ to obtain the parallel weight w / / corresponding to the parallel residual r / / and the perpendicular weight w ⊥ corresponding to the perpendicular residual r ⊥ .

[0086] S43: Substitute the parallel residual, vertical residual, parallel weight, and vertical weight into a preset formula to obtain a direction loss function.

[0087] Specifically, as can be seen from step S42 above, the weight function can also be decomposed into a vertical effect w ⊥ and a parallel effect w / / , and the finally obtained direction loss function is shown in Equation (6):

[0088]

[0089] where η is the ratio of the parallel component w / / to the vertical component w ⊥ , and ∝ indicates being directly proportional. In the direction loss function represented by Equation 6, substitute the residual feature vector into x i , which is the optimization objective of classifier C2.

[0090] Even further, input the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table. Please refer to Figure 7 , Figure 7 is Figure 1 a schematic diagram of a specific implementation process of step S13 in

[0091] S51: Split multiple sub-residual feature vectors into respective subspaces of the second classifier. Among them, use the feature space composed of a preset dimension vector as the sub-residual feature space for splitting the multiple sub-residual features, and use a preset status value to represent the preset dimension vector in the sub-residual feature space;

[0092] Generally, set a preset dimension vector to distinguish the sub-residual feature space. Among them, each sub-residual feature space can also use a preset status value to characterize the preset dimension vector.

[0093] S52: Based on a multi-threaded process, perform inner product calculations on multiple sub-residual feature vectors with the cluster centers of the sub-residual feature space respectively to obtain a similarity distance query table.

[0094] Even further, based on a multi-threaded process, perform inner product calculations on multiple sub-residual feature vectors with the cluster centers of the sub-residual feature space respectively to obtain a similarity distance query table. Please refer to Figure 8 , Figure 8 is Figure 7 a schematic diagram of a specific implementation process of step S52 in

[0095] S61: Divide the execution threads of the query feature vector into multi-threaded processes through a preset dimension;

[0096] Specifically, for the data to be loaded, first, the number of data loaded at one time can be calculated according to the number of bits supported by the instruction and the basic data type of the input feature data. Specifically, in combination with the fact that the inner product calculation in the clustering process can be accelerated by x86 instructions, and for a preset dimension, the execution threads for querying feature vectors are divided into multiple threads. For example, previously, one thread processed feature vectors from 1 to 100, and now it is divided into 10 dimensions, with 10 threads processing simultaneously.

[0097] S62: Based on multiple threads, cluster the query feature vectors for each sub-residual feature space to obtain the cluster centers;

[0098] Specifically, perform loop multiply-accumulate calculations on the loaded data to obtain the sum of multiply-accumulates for the number of loaded data. Finally, add up these sums of accumulates for the number of loaded data to obtain the final inner product, which is also the cluster center.

[0099] S63: Based on the direction loss function, calculate the direction loss between the sub-feature vectors of the query feature vectors and the cluster centers to obtain a similarity distance query table.

[0100] After obtaining the distance centers of the residuals, further calculate the loss of each point to the cluster center. First, select the minimum value from the subspace, and use the above formula (5) to calculate the minimum value of the direction loss. According to the minimum value of the direction loss, find the subspace state value corresponding to the original feature vector, and an index is established for each original feature vector in the training set.

[0101] Furthermore, based on the similarity distance query table and a preset retrieval table, perform accelerated retrieval on the query vectors of the images to be retrieved. Please refer to Figure 9 , Figure 9 Figure 1 is Figure 1 a schematic flowchart of a specific implementation process of step S14 in

[0102] S71: Compare the similarity distance query table with the preset retrieval table to obtain a comparison result;

[0103] S72: Accelerate the retrieval of the query feature vectors of the images to be retrieved by sorting the comparison results.

[0104] Furthermore, please refer to Figures 10 to 12 , Figure 10 is a schematic flowchart of a specific implementation example of the accelerated calculation for image retrieval in this application; Figure 11 is a schematic diagram of the storage of the compressed index table for image retrieval in this application; Figure 12 is a schematic diagram of the distance retrieval for image retrieval in this application.

[0105] Specifically, the inner product calculation in the clustering process can be accelerated through x86 instructions in a multi-threaded manner. For example, Figure 11 As shown, first, the data needs to be loaded. According to the number of bits supported by the instruction and the basic data type of the input feature data, the number of data items kBlockSize loaded at one time can be calculated. Then, the loaded data needs to be looped for multiply-accumulate calculation to obtain the sum of kBlockSize multiply-accumulates. Finally, adding these kBlockSize sums is the final inner product. The number of loop calculations is the vector dimension size divided by kBlockSize.

[0106] Specifically, to accelerate the calculation, for example, taking two-dimensional vectors as the minimum unit, the residual feature vectors are segmented. The feature space composed of two-dimensional feature vectors is called the sub-residual feature space. Each sub-residual feature space uses 16 state values to represent the two-dimensional vector. Suppose the original feature vector dimension is 100 dimensions, then the number of sub-spaces is 50. Each sub-feature space is clustered, and each sub-space obtains 16 cluster centers.

[0107] Since the state values are all between 0 and 15. Therefore, an 8-bit unsigned integer can be used to form a compressed index to store two state values. Each cluster center points to the compressed index table of the feature vectors included in its cluster. The stored view of the compressed index value is as Figure 12 shown. Each block stores 16 compressed indexes. Each compressed index can represent 2 real index values. For example, 8 bits correspond to 0 - 256, and 0 - 15 cannot use up all the 8-bit space. The 8-bit space is divided into two parts, the first 4 bits and the last 4 bits, to represent 2 real index values respectively, which can improve the space utilization rate. Therefore, each block actually stores the indexes of 2 feature vectors.

[0108] Furthermore, the trained classifier C1 is used to pre-group the feature vectors to be queried. Each feature vector to be queried will have a corresponding cluster center. In other words, each cluster center corresponds to a subset of the feature vector set to be queried. Subtracting each cluster center from the corresponding subset gives the residual feature vector of the feature vector set to be queried.

[0109] The residual feature vectors are segmented into the sub-spaces of classifier C2 and the inner product calculation is performed with each cluster center in the sub-space to obtain a distance retrieval table. This distance retrieval table is as Figure 12 shown. Figure 12 The part of sub-spaces 1 to 50 in is a single sub-space. A single sub-space includes 16 inner product values. The combination of the feature vector to be queried and the database feature vector constitutes a single piece of data in the distance retrieval table. According to Figure 10 the compressed index table shown and Figure 11The distance retrieval table can quickly obtain the inner product between the vector to be queried and the feature vectors included in its clustering cluster, and sorting it is the final retrieval result.

[0110] Therefore, the present application not only considers the direction of the vector in the optimization objective, but also constructs a weight function to consider the inner product between the vector to be queried and the database feature vectors. This is beneficial to reducing the error of the clustering center, making the search results more accurate. And a compressed index of the vector is constructed to save storage space. A distance query table and x86 instructions are also given to shorten the query time.

[0111] To illustrate the technical solution of the present application, the present application also provides a retrieval device, which can be a computer, a server, or a mobile terminal, and is not specifically limited here. Please refer to Figure 13 , Figure 13 FIG. is a structural schematic block diagram of an embodiment of the retrieval device of the present application. The retrieval device 7 includes: an acquisition module 71, a processing module 72, a calculation module 73, and an accelerated retrieval module 74.

[0112] The acquisition module 71 is used to acquire the original feature vector of the image to be retrieved;

[0113] The processing module 72 is used to process the original feature vector based on a preset first classifier to obtain the residual feature vector of the original feature vector;

[0114] The calculation module 73 is used to input the residual feature vector into a preset second classifier by the input module to calculate and obtain a similarity distance query table;

[0115] The accelerated retrieval module 74 is used to perform accelerated retrieval on the vector to be queried of the image to be retrieved based on the similarity distance query table and a preset retrieval table.

[0116] Therefore, the present application processes the original feature vector of the image to be retrieved obtained by the acquisition module 71 through the processing module 72 to obtain the residual feature vector; then uses the calculation module 73 to calculate the residual feature vector to obtain a similarity distance query table; and then based on the similarity distance query table and a preset retrieval table, the accelerated retrieval module 74 can perform accelerated retrieval on the vector to be queried of the image to be retrieved, thereby shortening the query time and improving the cost performance of image retrieval.

[0117] To illustrate the technical solution of the present application, the present application also provides an electronic device, which can be a computer or a mobile phone, etc., and is not specifically limited. Please refer to Figure 14 , Figure 14It is a structural schematic block diagram of an embodiment of the electronic device of the present application. The electronic device 8 includes: a processor 81 and a memory 82. A computer program 821 is stored in the memory 82. The processor 81 is configured to execute the computer program 821 to implement the method according to the first aspect of the embodiment of the present application, which will not be elaborated herein.

[0118] In addition, the present application also provides a computer-readable storage medium. Please refer to Figure 15 , Figure 15 It is a circuit schematic block diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 9 stores a computer program 91. When the computer program 91 can be executed by a processor, it can implement the method according to the first aspect of the embodiment of the present application, which will not be elaborated herein.

[0119] If it is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a device with storage function. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage device and includes several instructions (program data) for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods according to various embodiments of the present invention. The aforementioned storage device includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, as well as electronic devices such as computers, mobile phones, laptop computers, tablet computers, cameras, etc. having the above storage media.

[0120] The elaboration on the execution process of the program data in the device with storage function can refer to the elaboration in the embodiment of the image retrieval acceleration method of the present application above, which will not be elaborated herein.

[0121] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. An acceleration method for image retrieval, characterized in that, The acceleration method includes: Obtain the original feature vector of the image to be retrieved; Based on a preset first classifier, process the original feature vector to obtain the residual feature vector of the original feature vector; Input the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table; Based on the similarity distance query table and a preset retrieval table, perform accelerated retrieval on the query feature vector of the image to be retrieved; Wherein, before obtaining the original feature vector of the image to be retrieved, the acceleration method further includes: Extract the features of the images in the image database to obtain image features; Divide the image features to obtain multiple sub-feature vectors; Calculate the multiple sub-feature vectors to obtain a direction loss function; Based on the multiple sub-feature vectors and the direction loss function, establish an index for the multiple sub-feature vectors to obtain a preset retrieval table.

2. The acceleration method according to claim 1, wherein When calculating the multiple sub-feature vectors to obtain a direction loss function, the acceleration method further includes: Perform unsupervised training on a preset proportion of the image features to obtain a preset first classifier; Input the image features into the first classifier to calculate and obtain the residual feature vector corresponding to the image features; Map the residual feature vector to obtain multiple sub-residual feature vectors; Use the multiple sub-residual feature vectors as inputs for clustering to optimize the direction loss function.

3. The acceleration method according to claim 2, wherein Calculating the multiple sub-feature vectors to obtain a direction loss function includes: Decompose the multiple sub-residual feature vectors into parallel residuals parallel to the direction of the query feature vector and perpendicular residuals perpendicular thereto; On the direction of the query feature vector, perform weight assignment on the parallel residuals and the perpendicular residuals to obtain the parallel weight corresponding to the parallel residuals and the perpendicular weight corresponding to the perpendicular residuals; Substitute the parallel residuals, the perpendicular residuals, the parallel weight, and the perpendicular weight into a preset formula to obtain the direction loss function.

4. The acceleration method according to claim 2, wherein Inputting the residual feature vector into a preset second classifier to calculate and obtain a similarity distance query table includes: Respectively slice the multiple sub-residual feature vectors into each subspace of the second classifier, wherein a feature space composed of preset dimension vectors is used as the sub-residual feature space for slicing the multiple sub-residual feature vectors, and the sub-residual feature space uses a preset state value to represent the preset dimension vector; Based on multiple threads, perform inner product calculations on the multiple sub-residual feature vectors and the clustering centers of the sub-residual feature space respectively to obtain the similarity distance query table.

5. The acceleration method according to claim 4, wherein Based on multiple threads, performing inner product calculations on the multiple sub-residual feature vectors and the clustering centers of the sub-residual feature space respectively to obtain the similarity distance query table includes: Divide the execution threads of the query feature vector to be queried into the multiple threads through preset dimensions; Cluster the query feature vector to be queried for each of the sub-residual feature spaces based on the multiple threads to obtain the cluster centers; Calculate the direction loss between the sub-feature vector of the query feature vector to be queried and the cluster centers based on the direction loss function to obtain the similarity distance query table.

6. The acceleration method according to claim 5, wherein The accelerating the retrieval of the query feature vector of the image to be retrieved based on the similarity distance query table and a preset retrieval table includes: Compare the similarity distance query table with the preset retrieval table to obtain a comparison result; Sort the comparison result to accelerate the retrieval of the query feature vector of the image to be retrieved.

7. A retrieval device, characterized in that, The retrieval device includes: An acquisition module, configured to acquire the original feature vector of the image to be retrieved, and before acquiring the original feature vector of the image to be retrieved, extract the features of the images in the image database to obtain image features, divide the image features to obtain multiple sub-feature vectors, calculate the direction loss function for the multiple sub-feature vectors, and establish indexes for the multiple sub-feature vectors based on the multiple sub-feature vectors and the direction loss function to obtain a preset retrieval table; A processing module, configured to process the original feature vector based on a preset first classifier to obtain the residual feature vector of the original feature vector; A calculation module, configured to input the residual feature vector into a preset second classifier and calculate to obtain a similarity distance query table; An acceleration retrieval module, configured to accelerate the retrieval of the query feature vector of the image to be retrieved based on the similarity distance query table and a preset retrieval table.

8. An electronic device, characterized in that, including: A processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the acceleration method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program can be executed by a processor, it implements the acceleration method according to any one of claims 1-6.

Images