Image search method, device and apparatus

Abstract

The present invention discloses an image search method, equipment and device. First, based on a plurality of preset dimensions, the feature vector of an image is divided to obtain a first sub-vector set of a target image and a second sub-vector set of a to-be-matched image. Then, according to the sub-vector distance between any first sub-vector in the first sub-vector set and any second sub-vector in the second sub-vector set, a cost matrix is ​​obtained. A preset algorithm is used to solve the cost matrix, and based on the position invariance principle, a plurality of target sub-vector sets including the first sub-vector and the second sub-vector are determined. Finally, according to the distance between the first sub-vector and the sub-vector in the target sub-vector set, a metric distance is determined, and a search result is determined based on the metric distance. Since the feature vector of an image is divided based on a plurality of preset dimensions and then matched, the influence of background and noise in the feature vector on the metric distance can be reduced, thereby improving the search result of the image.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image search method, device and apparatus. Background Art

[0002] The search technology based on feature vectors is widely used in the field of computer vision. On the Internet, where massive digital image data is flooding, fast and accurate image search technology is particularly important.

[0003] In the prior art, when searching for images, the influence of background and noise is not considered, so the search results are not accurate enough. Summary of the invention

[0004] The present invention provides an image search method, equipment and device, which are used to solve the problem of inaccurate image search in the prior art.

[0005] In a first aspect, the present invention provides an image search method, the method comprising:

[0006] Based on a plurality of preset dimensions, the feature vectors of the image are divided to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched;

[0007] Calculating a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtaining a cost matrix based on a plurality of subvector distances;

[0008] Using a preset algorithm to solve the cost matrix, and based on the position invariance principle, determining a plurality of target sub-vector sets including a first sub-vector and a second sub-vector;

[0009] A metric distance is determined according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and a search result is determined based on the metric distance.

[0010] In a possible implementation, the cost matrix is ​​solved by using a preset algorithm, and a plurality of target sub-vector sets including a first sub-vector and a second sub-vector are determined based on a position invariance principle, including:

[0011] Using the preset algorithm to solve the cost matrix, and obtain an optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector;

[0012] Based on the position invariance principle, the target sub-vector set is selected from the multiple candidate sub-vector sets.

[0013] In a possible implementation, determining the metric distance according to the first subvector and the subvector distance in the target subvector set includes:

[0014] Determining a weight of the target subvector set according to a first subvector in the target subvector set;

[0015] After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set.

[0016] In a possible implementation manner, determining the weight of the target subvector set according to the first subvector in the target subvector set includes:

[0017] The reciprocal of the dimension of the first sub-vector is negated, and the exponent of the negated result is used as the weight of the target sub-vector set.

[0018] In a possible implementation, determining the metric distance according to the normalized weight and the sub-vector distance corresponding to the target sub-vector set includes:

[0019] For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set;

[0020] Each product is added together to obtain the metric distance.

[0021] In a possible implementation, before solving the cost matrix by using a preset algorithm and determining a plurality of target sub-vector sets including the first sub-vector and the second sub-vector based on the position invariance principle, the method further includes:

[0022] The distance of the sub-vectors in the cost matrix that is greater than a preset threshold is set to infinity.

[0023] In a possible implementation, if the dimensions of the first sub-vector and the second sub-vector are different, the sub-vector distance between the first sub-vector and the second sub-vector is determined in the following manner:

[0024] Using a nearest neighbor interpolation method or a dimensionality reduction algorithm, processing the high-dimensional sub-vectors in the first sub-vector and the second sub-vector to obtain two sub-vectors with the same dimension;

[0025] The distance between two sub-vectors with the same dimension is taken as the sub-vector distance.

[0026] In a second aspect, the present invention provides an image search device, comprising a processor, a memory, and a program stored in the memory and capable of running on the processor, wherein when the processor executes the program, it can implement any method described in the first aspect.

[0027] In a third aspect, the present invention provides an image search device, comprising:

[0028] A dimension division module, used for dividing the feature vector of the image based on a plurality of preset dimensions to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched;

[0029] a calculation module, configured to calculate a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtain a cost matrix based on a plurality of subvector distances;

[0030] A first determination module is used to solve the cost matrix by using a preset algorithm, and determine a plurality of target sub-vector sets including a first sub-vector and a second sub-vector based on a position invariance principle;

[0031] The second determination module is used to determine a metric distance according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and determine a search result based on the metric distance.

[0032] In a possible implementation manner, the first determining module is specifically configured to:

[0033] Using the preset algorithm to solve the cost matrix, and obtain an optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector;

[0034] Based on the position invariance principle, the target sub-vector set is selected from the multiple candidate sub-vector sets.

[0035] In a possible implementation manner, the second determining module is specifically configured to:

[0036] Determining a weight of the target subvector set according to a first subvector in the target subvector set;

[0037] After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set.

[0038] In a possible implementation manner, the second determining module is specifically configured to:

[0039] The reciprocal of the dimension of the first sub-vector is negated, and the exponent of the negated result is used as the weight of the target sub-vector set.

[0040] In a possible implementation manner, the second determining module is specifically configured to:

[0041] For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set;

[0042] Each product is added together to obtain the metric distance.

[0043] In a possible implementation, the device further includes a setting module;

[0044] Before solving the cost matrix by using a preset algorithm and determining a plurality of target sub-vector sets including the first sub-vector and the second sub-vector based on the position invariance principle, a setting module is used to:

[0045] The distance of the sub-vectors in the cost matrix that is greater than a preset threshold is set to infinity.

[0046] In a possible implementation, if the dimensions of the first subvector and the second subvector are different, the calculation module determines the subvector distance between the first subvector and the second subvector in the following manner:

[0047] Using a nearest neighbor interpolation method or a dimensionality reduction algorithm, processing the high-dimensional sub-vectors in the first sub-vector and the second sub-vector to obtain two sub-vectors with the same dimension;

[0048] The distance between two sub-vectors with the same dimension is taken as the sub-vector distance.

[0049] In a third aspect, the present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer programs or instructions, which, when executed, enables any method executed in the first aspect to be implemented.

[0050] The beneficial effects of the present invention are as follows:

[0051] In an embodiment of the present invention, the feature vector of the image is first divided based on multiple preset dimensions to obtain a first subvector set of the target image and a second subvector set of the image to be matched, and then the subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set is calculated, and based on multiple subvector distances, a cost matrix is ​​obtained, and then a preset algorithm is used to solve the cost matrix, and based on the principle of position invariance, multiple target subvector sets including the first subvector and the second subvector are determined, and finally, the metric distance is determined based on the first subvector and the subvector distance in the target subvector set, and the search result is determined based on the metric distance. Since the feature vector of the image is divided based on multiple preset dimensions and then matched, that is, the feature vector is segmented at multiple granularities, the influence of background and noise in the feature vector on the metric distance can be reduced, and the search result of the image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flowchart of an image search method provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a complete process of image search provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a target image provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of an image to be matched provided by an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of the structure of an image search device provided by an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of the structure of an image search device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The search and sorting technology based on feature vectors has a wide range of applications in the field of computer vision. On the Internet, where massive amounts of digital image data are flooding, fast and accurate image search technology is particularly important. Not only have a number of search engines for image search emerged, but they also play an important role in other fields of precise query. For example, in the process of face recognition, after extracting feature vectors from face images through neural networks, search and sort in the database to find the best matching image; for example, in the process of pedestrian re-identification, cross-camera search is also performed, which extracts features from pedestrian images, and then searches for one or more best matching images in the database according to a given distance measurement method. In short, these search technologies all use computers to extract and analyze features of the area of ​​interest in the image, and then store the obtained feature vectors in the database. After doing the same processing on the input search image, the similarity between the query image and the features in the database is calculated using the similarity criterion, and sorted according to the similarity, and finally output the search results under a given threshold.

[0061] There are two essential links in the image search process: 1. Extract features from the image. Traditional manual feature extraction methods, such as SIFT, HOG, SURF, etc., can be used, or other deep learning methods such as convolutional neural networks can be used for feature extraction. The main purpose is to reduce the dimension of high-dimensional data; 2. Calculate the distance between feature vectors. The main purpose of reducing the dimension of the image is to find a suitable low-dimensional space to measure the distance. Conventional methods include Euclidean distance, cosine distance, Harman Kardon distance, etc. Different distance measurement methods have a great impact on the search target. In the conventional feature extraction process, noise and other information will inevitably be introduced, resulting in a large difference in the feature vectors generated by the same target in the noise area. Conventional distance measurement methods often do not fully consider the impact of background and noise. The present invention is a feature distance measurement algorithm of multi-granularity cascade matching (Multi-Granulaity Matching Cascade) MGMC, which uses cascade matching after multi-granularity segmentation of feature vectors, and finally performs distance measurement on the matched features and weights them to obtain the final measurement distance.

[0062] Aiming at the search and sorting tasks in the field of machine vision, this paper proposes a more complete distance measurement method. First, the features of the image are extracted to obtain the feature vectors of the image to be searched and the query image. Secondly, the image features extracted by the convolutional network are translation invariant, so all feature vectors can be segmented at multiple granularities, and the traditional distance measurement can be performed on the feature vectors at different granularities. Then, the greedy algorithm is used to calculate the cost matrix at all granularities. Finally, according to the cost matrix, the best matching method is found, and the weighted metric distance of the feature vector under this matching method is calculated, thereby eliminating the influence of background and noise in the features on the distance.

[0063] It should be noted that, in the embodiments of the present invention, images and pictures may be equivalent. For example, searching for images may also be referred to as searching for pictures.

[0064] The embodiments of the present invention are described in detail below.

[0065] like Figure 1 FIG. 1 is a flow chart of an image search method provided by an embodiment of the present invention, which specifically includes the following steps:

[0066] S101, dividing the feature vector of the image based on a plurality of preset dimensions to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched;

[0067] S102, calculating a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtaining a cost matrix based on multiple subvector distances;

[0068] S103, using a preset algorithm to solve the cost matrix, and based on the position invariance principle, determining a plurality of target sub-vector sets including a first sub-vector and a second sub-vector;

[0069] S104: Determine a metric distance according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and determine a search result based on the metric distance.

[0070] In an embodiment of the present invention, the feature vector of the image is first divided based on multiple preset dimensions to obtain a first subvector set of the target image and a second subvector set of the image to be matched, and then the subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set is calculated, and based on multiple subvector distances, a cost matrix is ​​obtained, and then a preset algorithm is used to solve the cost matrix, and based on the principle of position invariance, multiple target subvector sets including the first subvector and the second subvector are determined, and finally, the metric distance is determined based on the first subvector and the subvector distance in the target subvector set, and the search result is determined based on the metric distance. Since the feature vector of the image is divided based on multiple preset dimensions and then matched, that is, the feature vector is segmented at multiple granularities, the influence of background and noise in the feature vector on the metric distance can be reduced, and the search result of the image can be improved.

[0071] The target image is an image input by a user, the image to be matched is an image in a search database, and the image search is to search the search database for an image matching the target image.

[0072] In an embodiment of the present invention, features are first extracted from a target image and an image to be matched to obtain a first feature vector of the target image and a second feature vector of the image to be matched. Then, based on a plurality of preset dimensions, the first feature vector and the second feature vector are divided to obtain a first sub-vector set corresponding to the first feature vector and a second sub-vector set corresponding to the second feature vector.

[0073] For example, the feature vector extracted from the target image is feature vector A, and the feature vector extracted from the image to be matched is feature vector B. Feature vector A and feature vector B are both 2048-dimensional feature vectors. The preset multiple dimensions are 128 dimensions, 256 dimensions, and 512 dimensions. Feature vector A and feature vector B are divided into 16 128-dimensional sub-vectors, 8 256-dimensional sub-vectors, and 4 512-dimensional sub-vectors.

[0074] After obtaining the first subvector set and the second subvector set, the distance between any first subvector in the first subvector set and any second subvector in the second subvector set is calculated. If the dimensions of the first subvector and the second subvector are the same, the distance between the first subvector and the second subvector is directly calculated. If the dimensions of the first subvector and the second subvector are different, the nearest neighbor interpolation method or the dimension reduction algorithm is used to process the high-dimensional subvectors in the first subvector and the second subvector to obtain two subvectors with the same dimension, and then the distance between the two subvectors with the same dimension is calculated, and the distance is used as the subvector distance of the two subvectors.

[0075] The above method, i.e., calculating the cost matrix between two sets of sub-vectors, i.e., performing distance measurement on sub-vectors at different granularities to obtain the distance between each pair, defines the calculation rules: for sub-vectors at different dimensions, use the nearest neighbor interpolation method or dimensionality reduction algorithm to construct sub-vectors of the same dimension.

[0076] After obtaining the cost matrix, it is necessary to find the optimal solution, that is, the optimal match, for the cost matrix. A preset algorithm can be used to solve the cost matrix to obtain the optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector, and then based on the principle of position invariance, a target sub-vector set is selected from the multiple candidate sub-vector sets.

[0077] The preset algorithm here can be the Hungarian algorithm, which can also be called the greedy algorithm.

[0078] Before using the Hungarian algorithm to solve the cost matrix, in order to facilitate calculation, the distance of the sub-vectors in the cost matrix that is greater than a preset threshold can be set to infinity.

[0079] The Hungarian algorithm is used to solve the cost matrix. After obtaining the optimal matching result, the optimal matching result is screened. According to the principle of position invariance, the position relationship between the first sub-vector and the second sub-vector in a set of best matching sub-vectors is one-to-one corresponding to the original vector and no intersection can occur.

[0080] In one embodiment, the metric distance is determined based on the first subvector in the target subvector set and the subvector distance. First, the weight of the target subvector set is determined based on the first subvector in the target subvector set; then, after the determined multiple weights are normalized, the metric distance is determined based on the normalized weights and the subvector distance corresponding to the target subvector set.

[0081] The metric distance is the distance between the feature vector of the target image and the feature vector of the image to be matched.

[0082] In a specific implementation, the weight of the target subvector set is determined according to the first subvector in the target subvector set. The reciprocal of the dimension of the first subvector may be negated, and the exponent of the negated result may be used as the weight of the target subvector set.

[0083] The metric distance is determined according to the normalized weight and the sub-vector distance corresponding to the target sub-vector set. First, for each target sub-vector set, the normalized weight is multiplied by the sub-vector distance corresponding to the target sub-vector set; then each product is added to obtain the metric distance.

[0084] In the above embodiment, for the filtered sub-vector set, the weight of each group of matching results to the final output distance is calculated according to the dimension and distance of the best match.

[0085] According to the calculated sub-vector distance, a greedy algorithm is used for matching calculation, and the optimal match between the two sub-vectors is calculated cyclically. For example, if a 128-dimensional sub-vector of vector A is closest to a 256-dimensional sub-vector of vector B, then the two sub-vectors are optimally matched and weighted. The remaining vectors are matched accordingly. During the matching process, the order of the sub-vectors must be kept unchanged, and the weight of the unmatched part is 0.

[0086] The weighted sum of each set of matching results is the final distance measure between the two feature vectors.

[0087] The search result is determined according to the metric distance. If the metric distance is less than a preset value, it is determined that the image to be matched and the target image match. If the metric distance is greater than or equal to the preset value, it is determined that the image to be matched and the target image do not match.

[0088] like Figure 2 As shown, it is a complete process of image search provided by an embodiment of the present invention. First, feature extraction is performed on the target image to obtain feature vector A, and feature extraction is performed on the image to be matched to obtain feature vector B; then feature vector A and feature vector B are segmented at multiple granularities to obtain subvector set A and subvector set B; a cost matrix is ​​obtained according to the distance between any subvector in subvector set A and any vector in subvector set B; the distance in the cost matrix that is greater than a preset threshold is set to infinity; the cost matrix is ​​solved using the Hungarian algorithm to obtain a matching result, and multiple matching groups (matching group 1, matching group 2...matching group n) are screened out from the matching result, and a weighted sum is performed on the multiple matching groups to obtain a distance measurement result of feature vector A and feature vector B, and finally, whether the image to be matched matches the target image is determined according to the distance measurement result.

[0089] For ease of understanding, the present invention is described below with reference to specific embodiments.

[0090] like Figure 3 The following is a schematic diagram of the target image: Figure 4 As shown, it is a schematic diagram of the image to be matched.

[0091] like Figure 3 and Figure 4 As shown in the figure, the same cartoon character is shown in different actions under different camera viewing angles. Due to the difference in background and action, the feature vectors extracted by the neural network may be locally different. The multi-granularity cascade matching distance measurement algorithm provided by the embodiment of the present invention can cope with such a situation. The following is a detailed description.

[0092] 1. Perform feature extraction on the target image to obtain feature vector A, and perform feature extraction on the image to be matched to obtain feature vector B. Among them, feature vector A and feature vector B are both 2048 dimensions.

[0093] 2. Split the feature vector A to obtain 16 128-dimensional vectors {A 11 ,A 12 ,......,A 116}, 8 256-dimensional vectors {A 21 ,A 22 ,......,A 18}, 4 512-dimensional vectors {A 31 ,A 32 ,......,A 34}, two 1024-dimensional vectors {A 41 ,A 42}, so we can get a sub-vector set A including 31 sub-vectors, namely Sa={A,A 11 ,..A 116 ,A 21 ,..A 28 ,A 31 ,..A 34 ,A 41 ,A 42};

[0094] Split the feature vector B to obtain 16 128-dimensional vectors {B 11 ,B 12 ,......,B 116}, 8 256-dimensional vectors {B 21 ,B 22 ,......,B 18}, 4 512-dimensional vectors {B 31 ,B 32 ,......,B 34}, 2 1024-dimensional vectors {B 41 ,B 42}, so we can get a sub-vector set B including 31 sub-vectors, that is, Sb = {B, B 11 ,..B 116 ,B 21 ,..B 28 ,B 31 ,..B 34 ,B 41 ,B 42}.

[0095] 3. Calculate the distance D between Sa and Sb ab , and get the cost matrix D.

[0096] The distance D between Sa and Sb ab , that is, the distance between any subvector in Sa and any subvector in Sb, for example, A 11 With B 11 The distance between 11 With B 21 The distance between.

[0097] 4. According to a preset threshold, the distance in the cost matrix that exceeds the threshold is set to infinity.

[0098] 5. Use the Hungarian algorithm to find the optimal match for the cost matrix;

[0099] 6. Filter the best matching results based on the principle of position invariance;

[0100] 7. Calculate D ab The weight, W ab =e -1 / x , x is A ij The vector dimension of

[0101] 8. According to the calculated W ab Normalize the weights to get W' ab ;

[0102] 10. Calculate the weighted metric distance L = ΣW' ab *D ab ;

[0103] Output: The metric distance between feature vector A and feature vector B is L.

[0104] It should be noted that Figure 3 and Figure 4 The background in is not drawn.

[0105] According to the translation invariance of the image corresponding to the feature vector, the present invention can only focus on the distance between the objects of interest and exclude the interference of background and noise; through the cascade matching relationship between vectors of different granularity, the objects can be aligned.

[0106] Based on the same inventive concept, the embodiment of the present invention also provides an image search device, such as Figure 5 As shown, the device includes, for example, a memory 510, a processor 500, and a program stored in the memory 510 and executable on the processor 500. Optionally, the terminal may further include a transceiver connected to the processor 500 and configured to communicate with other devices. Figure 5 The transceiver is not shown. Optionally, the device may also include a display screen to output a video ringback tone. Figure 5When the processor 500 executes the program stored in the memory 510, each step performed by the above method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0107] Specifically, when the processor 500 executes the program, it is used to:

[0108] Based on a plurality of preset dimensions, the feature vectors of the image are divided to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched;

[0109] Calculating a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtaining a cost matrix based on a plurality of subvector distances;

[0110] Using a preset algorithm to solve the cost matrix, and based on the position invariance principle, determining a plurality of target sub-vector sets including a first sub-vector and a second sub-vector;

[0111] A metric distance is determined according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and a search result is determined based on the metric distance.

[0112] Optionally, the processor 500 is specifically configured to:

[0113] Using the preset algorithm to solve the cost matrix, and obtain an optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector;

[0114] Based on the position invariance principle, the target sub-vector set is selected from the multiple candidate sub-vector sets.

[0115] Optionally, the processor 500 is specifically configured to:

[0116] Determining a weight of the target subvector set according to a first subvector in the target subvector set;

[0117] After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set.

[0118] Optionally, the processor 500 is specifically configured to:

[0119] The reciprocal of the dimension of the first sub-vector is negated, and the exponent of the negated result is used as the weight of the target sub-vector set.

[0120] Optionally, the processor 500 is specifically configured to:

[0121] For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set;

[0122] Each product is added together to obtain the metric distance.

[0123] Optionally, before solving the cost matrix by using a preset algorithm and determining a plurality of target sub-vector sets including the first sub-vector and the second sub-vector based on the position invariance principle, the processor 500 is further configured to:

[0124] The distance of the sub-vectors in the cost matrix that is greater than a preset threshold is set to infinity.

[0125] Optionally, if the dimensions of the first sub-vector and the second sub-vector are different, the processor 500 determines the sub-vector distance between the first sub-vector and the second sub-vector in the following manner:

[0126] A nearest neighbor interpolation method or a dimensionality reduction algorithm is used to process high-dimensional sub-vectors in the first sub-vector and the second sub-vector to obtain two sub-vectors with the same dimension; and a distance between the two sub-vectors with the same dimension is used as the sub-vector distance.

[0127] Based on the same inventive concept, an embodiment of the present invention further provides an image search device, such as Figure 6 As shown, the device comprises:

[0128] A dimension division module 601 is used to divide the feature vector of the image based on a plurality of preset dimensions to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched;

[0129] A calculation module 602, configured to calculate a sub-vector distance between any first sub-vector in the first sub-vector set and any second sub-vector in the second sub-vector set, and obtain a cost matrix based on a plurality of sub-vector distances;

[0130] A first determination module 603 is used to solve the cost matrix by using a preset algorithm, and determine a plurality of target sub-vector sets including a first sub-vector and a second sub-vector based on a position invariance principle;

[0131] The second determination module 604 is configured to determine a metric distance according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and determine a search result based on the metric distance.

[0132] Optionally, the first determining module 603 is specifically configured to:

[0133] Using the preset algorithm to solve the cost matrix, and obtain an optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector;

[0134] Based on the position invariance principle, the target sub-vector set is selected from the multiple candidate sub-vector sets.

[0135] Optionally, the second determining module 604 is specifically configured to:

[0136] Determining a weight of the target subvector set according to a first subvector in the target subvector set;

[0137] After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set.

[0138] Optionally, the second determining module 604 is specifically configured to:

[0139] The reciprocal of the dimension of the first sub-vector is negated, and the exponent of the negated result is used as the weight of the target sub-vector set.

[0140] Optionally, the second determining module 604 is specifically configured to:

[0141] For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set;

[0142] Each product is added together to obtain the metric distance.

[0143] Optionally, the device further comprises a setting module;

[0144] Before solving the cost matrix by using a preset algorithm and determining a plurality of target sub-vector sets including the first sub-vector and the second sub-vector based on the position invariance principle, a setting module is used to:

[0145] The distance of the sub-vectors in the cost matrix that is greater than a preset threshold is set to infinity.

[0146] Optionally, if the dimensions of the first sub-vector and the second sub-vector are different, the calculation module 602 determines the sub-vector distance between the first sub-vector and the second sub-vector in the following manner:

[0147] Using a nearest neighbor interpolation method or a dimensionality reduction algorithm, processing the high-dimensional sub-vectors in the first sub-vector and the second sub-vector to obtain two sub-vectors with the same dimension;

[0148] The distance between two sub-vectors with the same dimension is taken as the sub-vector distance.

[0149] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium. Optionally, the computer-readable storage medium has a computer program, and the computer program includes program instructions. When the program instructions are executed by the computer, the computer executes the steps of the above-mentioned image search method. Since the computer program in this embodiment and the aforementioned image search method are based on the invention under the same concept, through the detailed description of the aforementioned method, those skilled in the art can clearly understand the implementation process of the computer program in this embodiment, so for the sake of brevity of the specification, it will not be repeated here.

[0150] Based on the same inventive concept, an embodiment of the present application provides a computer program product. Optionally, the computer program product stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the above-mentioned image search method. Since the computer program in this embodiment and the aforementioned image search method are based on the invention under the same concept, through the detailed description of the aforementioned method, those skilled in the art can clearly understand the implementation process of the computer program in this embodiment, so for the sake of brevity of the specification, it will not be repeated here.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image search method, characterized in that: The method includes: Based on a plurality of preset dimensions, the feature vectors of the image are divided to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched; Calculating a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtaining a cost matrix based on a plurality of subvector distances; Using a preset algorithm to solve the cost matrix, and based on the position invariance principle, determining a plurality of target sub-vector sets including a first sub-vector and a second sub-vector; Determine a metric distance according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and determine a search result based on the metric distance; The determining of the metric distance according to the first sub-vector and the sub-vector distance in the target sub-vector set includes: Determining a weight of the target subvector set according to a first subvector in the target subvector set; After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set; The step of determining the weight of the target sub-vector set according to the first sub-vector in the target sub-vector set includes: Negate the reciprocal of the dimension of the first sub-vector, and use the exponent of the negated result as the weight of the target sub-vector set; The determining the metric distance according to the sub-vector distance corresponding to the normalized weight and the target sub-vector set includes: For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set; Each product is added together to obtain the metric distance.

2. The method according to claim 1, characterized in that The method adopts a preset algorithm to solve the cost matrix, and determines a plurality of target sub-vector sets including a first sub-vector and a second sub-vector based on the position invariance principle, including: Using the preset algorithm to solve the cost matrix, and obtain an optimal matching result of the cost matrix, wherein the optimal matching result is a set of multiple candidate sub-vectors including a first sub-vector and a second sub-vector; Based on the position invariance principle, the target sub-vector set is selected from the multiple candidate sub-vector sets.

3. The method according to claim 1, characterized in that The method further includes: solving the cost matrix by using a preset algorithm, and determining a plurality of target sub-vector sets including the first sub-vector and the second sub-vector based on the position invariance principle. The distance of the sub-vectors in the cost matrix that is greater than a preset threshold is set to infinity.

4. The method according to any one of claims 1 to 3, characterized in that: If the dimensions of the first subvector and the second subvector are different, the subvector distance between the first subvector and the second subvector is determined in the following manner: Using a nearest neighbor interpolation method or a dimensionality reduction algorithm, processing the high-dimensional sub-vectors in the first sub-vector and the second sub-vector to obtain two sub-vectors with the same dimension; The distance between two sub-vectors with the same dimension is taken as the sub-vector distance.

5. An image search device, characterized in that: The invention comprises a processor, a memory, and a program stored in the memory and capable of running on the processor. When the processor executes the program, the method according to any one of claims 1 to 4 can be implemented.

6. An image search device, characterized in that: include: A dimension division module, used for dividing the feature vector of the image based on a plurality of preset dimensions to obtain a first sub-vector set of the target image and a second sub-vector set of the image to be matched; a calculation module, configured to calculate a subvector distance between any first subvector in the first subvector set and any second subvector in the second subvector set, and obtain a cost matrix based on a plurality of subvector distances; A first determination module is used to solve the cost matrix by using a preset algorithm, and determine a plurality of target sub-vector sets including a first sub-vector and a second sub-vector based on a position invariance principle; A second determination module, configured to determine a metric distance according to the distance between the first sub-vector and the sub-vectors in the target sub-vector set, and determine a search result based on the metric distance; Wherein, the second determining module is specifically used for: Determining a weight of the target subvector set according to a first subvector in the target subvector set; After normalizing the determined multiple weights, the metric distance is determined according to the normalized weights and the sub-vector distance corresponding to the target sub-vector set; The second determining module is specifically used for: Negate the reciprocal of the dimension of the first sub-vector, and use the exponent of the negated result as the weight of the target sub-vector set; The second determining module is specifically used for: For each target sub-vector set, multiply the normalized weight by the sub-vector distance corresponding to the target sub-vector set; Each product is added together to obtain the metric distance.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer programs or instructions, which, when executed, enable the method executed by any one of claims 1 to 4 to be implemented.

Images