Image retrieval method, electronic equipment and storage medium

Abstract

The invention discloses an image retrieval method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-retrieved image; acquiring a first hash code corresponding to the image feature of the to-be-retrieved image by using a preset hash model, wherein the preset hash model is obtained by training a similarity error determined by training a first similarity feature set and a second similarity feature set of the image feature; acquiring a plurality of second hash codes; the Hamming distance between the second hash code and the first hash code is smaller than a preset distance threshold value; and calculating the similarity between the image feature corresponding to each second hash code and the image feature corresponding to the first hash code, and determining an image retrieval result of the to-be-retrieved image according to a similarity calculation result. According to the image retrieval method and device, the first hash code of the to-be-retrieved image is obtained through the preset hash model, then the image retrieval result is determined according to the similarity determined by the image features corresponding to the first hash code, the hash code quality can be effectively improved, and then the image retrieval accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image retrieval, and in particular to an image retrieval method, electronic equipment and storage medium. Background Art

[0002] With the rapid development of the Internet and social media, the explosive growth of image data has made retrieval technology a research field that has attracted much attention.

[0003] In current image retrieval schemes, it is usually necessary to rely on image feature extraction and matching. Specifically, commonly used methods include retrieval based on artificial features such as color histogram, texture, and shape, which are usually applied to label information. Therefore, there will be errors between the output label information and the original label information, thereby reducing the quality of the hash code and affecting the accuracy of image retrieval. Summary of the invention

[0004] The technical solution to the technical problem mainly solved by the present application is: to provide an image retrieval method, an electronic device and a storage medium, to obtain a first hash code corresponding to the image features of the image to be retrieved through a preset hash model, and then obtain multiple second hash codes whose Hamming distances corresponding to the first hash code are less than a preset distance threshold, and then determine the image retrieval result based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the quality of the hash codes and thus improve the accuracy of image retrieval.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide an image retrieval method, including: obtaining an image to be retrieved; using a preset hash model to obtain a first hash code corresponding to an image feature of the image to be retrieved, wherein the preset hash model is obtained by training a similarity error determined by a first similarity feature set and a second similarity feature set corresponding to each training image feature; the similarity between features in the first similarity feature set and the training image features is greater than the similarity between features in the second similarity feature set and the training image features; obtaining multiple second hash codes corresponding to the first hash code; wherein the Hamming distance between each of the second hash codes and the first hash code is less than a preset distance threshold; calculating the similarity between the image feature corresponding to each of the second hash codes and the image feature corresponding to the first hash code, and determining the image retrieval result of the image to be retrieved based on the similarity calculation result.

[0006] In some embodiments, the use of a preset hash model to obtain a first hash code corresponding to the image features of the image to be retrieved includes: training and obtaining the preset hash model; obtaining the image features of the image to be retrieved, and inputting the image features into the preset hash model to obtain the first hash code corresponding to the image features.

[0007] In some embodiments, each of the to-be-retrieved images obtains one of the image features, and each of the image features has a corresponding first hash code.

[0008] In some embodiments, the preset hash model is trained as follows: obtaining training image features of each training image, and obtaining a first similarity feature set and a second similarity feature set corresponding to each of the training image features, wherein the first similarity feature set includes multiple high-similarity features, and the similarity between each of the high-similarity features and the training image features is greater than a first similarity threshold; the second similarity feature set includes multiple low-similarity features, and the similarity between each of the low-similarity features and the training image features is less than a second similarity threshold, and each of the training images obtains one training image feature; obtaining a first hash feature corresponding to each of the high-similarity features, and obtaining a first similarity between each of the first hash features and each of the high-similarity features; obtaining a second hash feature corresponding to each of the low-similarity features, and obtaining a second similarity between each of the second hash features and each of the low-similarity features; determining the similarity error using the first similarity and the second similarity, and training a basic model with the similarity error to obtain the preset hash model.

[0009] In some embodiments, similarity sampling is performed using Beta distribution to determine a preset calibration value; and the similarity error is determined using the preset calibration value, the first similarity, and the second similarity.

[0010] In some embodiments, for each of the first hash feature and the second hash feature, binomial distribution sampling is performed under the hash feature dimension to obtain a discrete matrix; for each of the first hash feature and the second hash feature, one-dimensional uniform distribution random sampling is performed to obtain a first projection matrix; the discrete matrix is ​​projected using the first projection matrix to obtain a second projection matrix; and the first feature matrix is ​​projected using the first projection matrix to obtain a third projection matrix, wherein the first feature matrix is ​​composed of multiple first hash features and / or second hash features; the distribution difference between the hash feature distribution and the high-dimensional binomial distribution is determined using the second projection matrix and the third projection matrix; the basic model is trained with the similarity error to obtain the preset hash model, including: determining the model error using a preset balance coefficient, the similarity error and the distribution difference, and then adjusting the model parameters using the model error to obtain the preset hash model.

[0011] In some embodiments, obtaining multiple second hash codes whose Hamming distance corresponding to the first hash code is less than a preset distance threshold includes: constructing a hash code database, and pre-storing preset hash codes output by using a preset hash model in the hash code database; calculating the Hamming distance between the first hash code and the preset hash code, and taking the preset hash code whose Hamming distance is less than the preset distance threshold as the second hash code, and then obtaining several second hash codes from the hash code database, wherein the several second hash codes are at least a part of the preset hash code.

[0012] In some embodiments, the calculating the similarity between the image features corresponding to each of the second hash codes and the image features corresponding to the first hash code, and determining the image retrieval result of the image to be retrieved based on the similarity calculation result, includes: obtaining an original feature set corresponding to multiple second hash codes, wherein the original feature set includes the image features corresponding to each of the second hash codes; performing similarity calculation using the image features corresponding to the first hash code and the image features corresponding to each of the second hash codes to obtain multiple corresponding similarity values; using the image corresponding to the second hash code whose similarity value is greater than a third similarity threshold as a target image, and using the target image as the image retrieval result of the image to be retrieved.

[0013] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, the electronic device comprising a memory and a processor coupled to the memory, the memory storing at least one computer program, and the at least one computer program, when loaded and executed by the processor, is used to implement the method as described above.

[0014] To solve the above technical problems, another technical solution adopted in the present application is: providing a computer-readable storage medium, the computer-readable storage medium having at least one program, and when the at least one program is loaded and executed by a processor, it is used to implement the above method.

[0015] Different from the current technology, the image retrieval method provided by the present application includes: obtaining an image to be retrieved; using a preset hash model to obtain a first hash code corresponding to an image feature of the image to be retrieved, wherein the preset hash model is obtained by training a similarity error determined by a first similarity feature set and a second similarity feature set corresponding to each training image feature; the similarity between the features in the first similarity feature set and the training image features is greater than the similarity between the features in the second similarity feature set and the training image features; obtaining multiple second hash codes corresponding to the first hash code; wherein the Hamming distance between each second hash code and the first hash code is less than a preset distance threshold; calculating the similarity between the image feature corresponding to each second hash code and the image feature corresponding to the first hash code, and determining the image retrieval result of the image to be retrieved based on the similarity calculation result. That is, in the present application, a first hash code corresponding to the image features of the image to be retrieved is obtained through a preset hash model. After obtaining the accurate first hash code, multiple second hash codes whose Hamming distances corresponding to the first hash code are less than a preset distance threshold are obtained. Then, the image retrieval result is determined based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the quality of the hash codes and thus improve the accuracy of image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 It is a flowchart of an embodiment of an image retrieval method in this application; Figure 2 It is a flowchart of an embodiment of hash model training in this application; Figure 3 is a structural diagram of an embodiment of an image retrieval system in the present application; Figure 4 It is a structural schematic diagram of an embodiment of an electronic device in the present application; Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium in the present application. DETAILED DESCRIPTION

[0017] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0018] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0019] Traditional image retrieval methods mainly rely on image feature extraction and matching. That is, commonly used methods include retrieval based on artificial features such as color histograms, textures, and shapes. Label information is usually used, which has a high acquisition cost. In addition, different instances appearing in subsequent training sets will produce similar hash codes, resulting in reduced retrieval performance. That is, there will be errors between the output label information and the original label information, thereby reducing the quality of the hash code and affecting the accuracy of image retrieval.

[0020] Therefore, an image retrieval method is provided, which obtains a first hash code corresponding to an image feature of an image to be retrieved through a preset hash model, and after obtaining an accurate first hash code, obtains multiple second hash codes whose Hamming distances corresponding to the first hash code are less than a preset distance threshold, and then determines the image retrieval result based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the quality of hash codes and thus improve the accuracy of image retrieval.

[0021] See also Figure 1 , Figure 1 is a flow chart of an embodiment of the image retrieval method in this application; it should be noted that if there are substantial results, the method of this application is not based on Figure 1 The process sequence shown is limited.

[0022] like Figure 1 As shown, the image retrieval method of the present application may include the following steps: S10: Obtain the image to be retrieved.

[0023] The image to be retrieved refers to image data that has not been retrieved.

[0024] Specifically, a retrieved image to be retrieved is obtained, and image features of the image to be retrieved are obtained through a pre-trained model.

[0025] S20. Obtain a first hash code corresponding to an image feature of the image to be retrieved using a preset hash model, wherein the preset hash model is obtained by training a similarity error determined by a first similarity feature set and a second similarity feature set corresponding to each training image feature; and the similarity between features in the first similarity feature set and features of the training image is greater than the similarity between features in the second similarity feature set and features of the training image.

[0026] Among them, the preset hash model refers to the network model obtained after training the convolutional neural network, which is mainly used to input image features and then output the hash code corresponding to the image features; the first hash code refers to the hash code corresponding to the image features of the image to be retrieved, which is output by the preset hash model; the training image features refer to the image features corresponding to the training images; the first similarity feature set refers to a range set of similarities corresponding to the training image features, which may include multiple features with close similarities; the second similarity feature set refers to another range set of similarities corresponding to the training image features, which may also include multiple features with close similarities.

[0027] Specifically, after the preset hash model is obtained by training the similarity error determined by the first similarity feature set and the second similarity feature set corresponding to the training image features, the image features of the image to be retrieved are input into the preset hash model, and then the preset hash model outputs the first hash code corresponding to the image features of the image to be retrieved.

[0028] It can be understood that each image to be retrieved corresponds to an image feature, each training image corresponds to a training image feature, and each image feature corresponds to a first hash code.

[0029] S30, obtaining a plurality of second hash codes corresponding to the first hash code; wherein a Hamming distance between each second hash code and the first hash code is less than a preset distance threshold.

[0030] Among them, the second hash code refers to a hash code in the hash code database whose Hamming distance with the first hash code is less than a preset distance threshold, and is different from the first hash code; the Hamming distance refers to an indicator to measure the degree of difference between two binary numbers, indicating the maximum number of bits of difference between two binary numbers, which can be used to measure the error rate in data transmission.

[0031] Specifically, after obtaining the first hash code corresponding to the image feature of the image to be retrieved, the first hash code is used to calculate the Hamming distance between hash codes in the hash code database, the Hamming distance between the first hash code and multiple hash codes pre-stored in the hash code database is determined, and the hash code whose Hamming distance is less than a preset distance threshold is used as the second hash code, thereby obtaining multiple second hash codes corresponding to the first hash code.

[0032] S40, calculating the similarity between the image feature corresponding to each second hash code and the image feature corresponding to the first hash code, and determining the image retrieval result of the image to be retrieved according to the similarity calculation result.

[0033] The similarity calculation result refers to the similarity comparison between the image features corresponding to the first hash code and the image features corresponding to the second hash code. Since there are multiple second hash codes, multiple corresponding similarity results can be calculated.

[0034] Specifically, after obtaining multiple second hash codes corresponding to the first hash code, the similarities between the image features corresponding to the hash codes are calculated, and then the similarities between the image features corresponding to each second hash code and the image features corresponding to the first hash code are obtained, that is, there is a corresponding similarity calculation result between each second hash code and the first hash code. Because there are multiple second hash codes, multiple similarity calculation results are obtained, and then multiple target images are determined based on the image features corresponding to several second hash codes with larger similarities in the similarity calculation results, and the obtained multiple target images are determined as image retrieval results of the images to be retrieved.

[0035] In the present application, a first hash code corresponding to the image features of the image to be retrieved is obtained through a preset hash model, which can effectively improve the quality of the first hash code; then a plurality of second hash codes whose Hamming distances corresponding to the first hash code are obtained and are less than a preset distance threshold, and then the image retrieval result is determined based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the accuracy of image retrieval.

[0036] In some embodiments, S10 obtaining the image to be retrieved may include the following operations.

[0037] The image to be retrieved is acquired by using an image acquisition device.

[0038] In some embodiments, S20 uses a preset hash model to obtain a first hash code corresponding to the image feature of the image to be retrieved, which may include the following operations.

[0039] First, train and obtain the preset hash model.

[0040] Among them, the preset hash model is obtained by training the similarity error determined by the first similarity feature set and the second similarity feature set corresponding to each training image feature; the similarity between the features in the first similarity feature set and the training image features is greater than the similarity between the features in the second similarity feature set and the training image features; it can be understood that the first similarity feature set refers to a range set of similarities corresponding to the training image features, which may include multiple features with close similarities; the second similarity feature set refers to another range set of similarities corresponding to the training image features, which may also include multiple features with close similarities.

[0041] Specifically, multiple training images are obtained, and training image features corresponding to each training image are obtained, and a first similarity feature set and a second similarity feature set corresponding to each training image feature are determined, and then the similarity error determined by the first similarity feature set and the second similarity feature set can be obtained, and then the basic model is trained using the similarity error to obtain a preset hash model.

[0042] Next, the image features of the image to be retrieved are obtained, and the image features are input into a preset hash model to obtain a first hash code corresponding to the image features.

[0043] For the image to be detected, an image feature corresponding to the image to be detected is obtained, and this image feature can represent the image to be detected.

[0044] Specifically, for the image to be retrieved, the image features of the image to be retrieved are obtained through the pre-trained model, and then the obtained image features are input into the preset hash model, and then the preset hash model outputs the first hash code corresponding to the image features of the image to be retrieved.

[0045] In this embodiment, a preset hash model is obtained by training the similarity error determined by the first similarity feature set and the second similarity feature set corresponding to each training image feature, and then the preset hash model is used to output the first hash code corresponding to the image feature of the image to be retrieved, which can effectively improve the quality of the hash code.

[0046] In some embodiments, each image to be retrieved obtains an image feature, and each image feature has a corresponding first hash code.

[0047] Specifically, each search only searches for one image to be searched, and each image to be searched has a corresponding image feature that can characterize the image to be searched. Therefore, each image to be searched has a corresponding image feature.

[0048] In some embodiments, the training of the preset hash model may include the following operations.

[0049] First, a training image feature of each training image is obtained, and a first similarity feature set and a second similarity feature set corresponding to each training image feature are obtained.

[0050] Among them, the first similarity feature set includes multiple high-similarity features, and the similarity between each high-similarity feature and the training image feature is greater than the first similarity threshold; the second similarity feature set includes multiple low-similarity features, and the similarity between each low-similarity feature and the training image feature is less than the second similarity threshold, and each training image obtains a training image feature.

[0051] It can be understood that a high similarity feature refers to a training image feature other than itself that has a higher similarity with a current training image feature; a low similarity feature refers to a training image feature other than itself that has a lower similarity with a current training image feature.

[0052] Specifically, in the case of multiple training images, the pre-training model is used to obtain the training image features of each training image, and for each training image feature, multiple high-similarity features other than itself are obtained to form a first similarity feature set, and multiple low-similarity features other than itself are obtained to form a second similarity feature set.

[0053] For example, there are n training images. The n training images are passed through the pre-training model to obtain n training image features. , and then for each training image feature, record the topk high similarity features except itself.

[0054]

[0055] Among them, the following conditions are met:

[0056] And record the topk low similarity features.

[0057]

[0058] Among them, the following conditions are met:

[0059] in, Indicates that among the n training image features, the i-th training image feature and the jth training image feature The similarity between them. The value of k can be k=10 or k=n-1; when k=n-1, .

[0060] Indicates the similarity between the i-th training image feature and the j-th training image feature among the n training image features, j does not belong to One of analogy.

[0061] Represents the features among n features that are consistent with the current training image features For comparison, the training image features corresponding to the first k maximum similarities are located in the n training image features. , the corresponding similarity is ; corresponding Represents the number of n training image features that are consistent with the current training image features For comparison, the training image features corresponding to the first k minimum similarities are located in the n training image features. , the corresponding similarity is .

[0062] Next, a first hash feature corresponding to each high-similarity feature is obtained, and a first similarity between each first hash feature and each high-similarity feature is obtained.

[0063] The first hash feature refers to the hash feature obtained after the high similarity feature passes through the initial hash model; the first similarity refers to the similarity between the first hash feature and the corresponding high similarity.

[0064] Specifically, after obtaining multiple high-similarity features and multiple low-similarity features corresponding to each training image feature, the initial hash model is used to obtain the first hash feature corresponding to each high-similarity feature, and the similarity between each first hash feature and each high-similarity feature, i.e., the first similarity, is calculated.

[0065] Next, a second hash feature corresponding to each low-similarity feature is obtained, and a second similarity between each second hash feature and each low-similarity feature is obtained.

[0066] The second hash feature refers to the hash feature obtained after the low-similarity feature passes through the initial hash model; the second similarity refers to the similarity between the second hash feature and the corresponding low-similarity.

[0067] Specifically, after obtaining multiple high-similarity features and multiple low-similarity features corresponding to each training image feature, the initial hash model is used to obtain the second hash features corresponding to each low-similarity feature, and the similarity between each second hash feature and each low-similarity feature, i.e., the second similarity, is calculated in the same way as the first similarity.

[0068] For example, the current training image features are , the first similarity feature set is , the second similarity feature set is , then: Current training image features The hash characteristics are: .

[0069] The first hash feature corresponding to the first similarity feature set is: .

[0070] The second hash feature corresponding to the second similarity feature set is: .

[0071] Then, the first similarity is: .

[0072] The second similarity is: .

[0073] Among them, is the first hash feature corresponding to the k-th high similarity feature, is the second hash feature corresponding to the k-th low similarity feature; is the current training image feature The first similarity between and the k-th high similarity feature, is the second similarity between the current training image feature and the kth low similarity feature, and the other parameters are similar.

[0074] It can be understood that the similarity calculation here is to calculate the cosine similarity of two feature vectors given two feature vectors of the same dimension; that is, the inner product operation of the two feature vectors is performed to obtain a constant; then the lengths of the two feature vectors are obtained, and then the constant is divided by the product of the two lengths to obtain the cosine similarity; among which, the length of the feature vector is obtained by taking the square of each dimension of the feature, then summing it, and taking the square root of the sum.

[0075] Then, a similarity error is determined using the first similarity and the second similarity, and an initial hash model is trained using the similarity error to obtain a preset hash model.

[0076] Specifically, after obtaining the first similarity and the second similarity, the similarity error between the first similarity and the second similarity can be determined, and then the parameters of the initial hash model are adjusted using the similarity error, and then the initial hash model is trained to obtain the preset hash model.

[0077] In this embodiment, high-similarity features and low-similarity features of each training image feature are obtained, and then the similarity between the features is traced back to obtain the corresponding first similarity and second similarity. This can avoid using labels to record high-similarity features and low-similarity features to distinguish different types of hash codes, thereby providing an accurate basis for the subsequent use.

[0078] Furthermore, the following operations may also be included.

[0079] Similarity sampling is performed using Beta distribution to determine the preset calibration value.

[0080] Among them, Beta distribution refers to a density function that is a conjugate prior distribution of Bernoulli distribution and binomial distribution, and has important applications in machine learning and mathematical statistics. In probability theory, Beta distribution, also known as Β distribution, refers to a set of continuous probability distributions defined in the interval (0, 1).

[0081] Specifically, since the similarity region in the original feature space is a continuous value (infinite) in the entire space [-1, 1], and the similarity value (Hamming distance) in the hash space is a discrete value (finite), {0, 1, 2, ... K-1}, the essential difference between the two means that if the hash feature similarity (pre-hashing feature) is directly aligned to the original continuous similarity, the similarity will be destroyed and the positive and negative pairs cannot be accurately distinguished. Therefore, it is necessary to perform multiple similarity sampling based on the Beta distribution, for example, 2k similarity samplings are performed, then:

[0082] in, F For the random variable X to obey the cumulative distribution function of beta(α, β), then: , Among them, α and β are the parameters of Beta distribution, and the values ​​can be α=β=20.

[0083] Furthermore, in order to make full use of the original similarity range, a centralization process is performed again, and then: .

[0084] in, is the preset calibration value.

[0085] A similarity error is determined using the preset calibration value, the first similarity, and the second similarity.

[0086] Specifically, the similarity difference between the preset calibration value and the first similarity and the second similarity can be used as the similarity error, then: .

[0087] in, is the similarity error, which is used to measure the difference between the original feature similarity and the similarity after feature hashing; is the second similarity corresponding to the jth low similarity, is the first similarity corresponding to the jth high similarity, is the current preset calibration value, is the k-1th preset calibration value.

[0088] In this embodiment, similarity sampling is performed through Beta distribution to determine a preset calibration value, and then the preset calibration value, the first similarity and the second similarity are used to determine the similarity error, which can make large similarities larger and small similarities smaller, that is, further distinguish the positive pairs and negative pairs corresponding to the similarities, and thus obtain a similarity error with higher accuracy.

[0089] In some embodiments, the following operations may also be included.

[0090] For each of the first hash feature and the second hash feature, binomial distribution sampling is performed under the hash feature dimension to obtain a discrete matrix.

[0091] Among them, in addition to maintaining the similarity size relationship, high-quality hash codes must also have quantization characteristics and bit balance characteristics. The current technology uses a heuristic method to measure quantization and bit balance errors respectively, and it is impossible to truly obtain high-quality hash codes. Therefore, this application considers the quantization characteristics, that is, each bit under the hash feature should be as close to 0 / 1 as possible, or -1 / 1. It should be noted that this application does not specify the use of any representation of the hash code. For the convenience of description, this application uses -1 / 1 for description; the bit balance characteristic is that different bits of the hash code should be as independent as possible. In fact, this requires that the distribution of the hash feature should be close to the binomial distribution with a parameter of 0.5. Therefore, this application uses the Wasserstein distance to measure the error between the hash feature distribution and the binomial distribution. The smaller the error, the higher the quality of the hash code.

[0092] Wasserstein distance, also known as Earth Mover's Distance (EMD), is a mathematical method for measuring the difference between two probability distributions. It originates from the Optimal Transport Problem and is used to measure the minimum "cost" required to "convert" one probability distribution into another.

[0093] Specifically, binomial distribution sampling is performed under the hash feature dimension, that is, when the hash code length is K, the number of training samples is randomly sampled from the K-dimensional binomial distribution, and a K-dimensional binary discrete vector is used to represent the binomial distribution. For example, the K-dimensional binomial distribution with a parameter of 0.5 has: .

[0094] Among them, the size is ( n , K )’s discrete matrix Z1.

[0095] For each of the first hash feature and the second hash feature, one-dimensional uniformly distributed random sampling is performed to obtain a first projection matrix.

[0096] Among them, Wasserstein distance can be used to measure the distance between hash feature distribution and binomial distribution. Since the Wasserstein distance of high-dimensional distribution is theoretically uncalculatable, this proposal uses projected Wasserstein distance to measure the distance between hash features and high-dimensional binomial distribution in hash retrieval scenarios.

[0097] Specifically, for each first hash feature and second hash feature, one-dimensional uniform distribution random sampling is performed to obtain multiple values, thereby forming a corresponding first projection matrix; for example: randomly sampling from a one-dimensional uniform distribution U(-1, 1) S values, forming a size of ( K , S ) Z 2.

[0098] The discrete matrix is ​​projected using the first projection matrix to obtain a second projection matrix; and the first feature matrix is ​​projected using the first projection matrix to obtain a third projection matrix, wherein the first feature matrix is ​​composed of a plurality of first hash features and / or second hash features.

[0099] The first projection matrix, the second projection matrix and the third projection matrix are all obtained by projection, that is, new matrices obtained after linear transformation.

[0100] Specifically, the discrete matrix is ​​projected using the first projection matrix, that is, the discrete matrix Z1 is projected, that is, a linear transformation is performed on the discrete matrix Z1, and the second projection matrix is ​​obtained through the first projection matrix.

[0101] For example, let the discrete matrix be Z1 and the first projection matrix be Z2. Then the second projection matrix is P 1 is calculated as follows: .

[0102] Among them, the size of the second projection matrix P1 is (n , S ).

[0103] Set the first characteristic matrix to F , and the first characteristic matrix F The size of n , K ), the first characteristic matrix F Characterizes n K-dimensional hash codes ( n , K ) size; then the third projection matrix P 2 is calculated as follows: .

[0104] Among them, the third projection matrix P The size of 2 is ( n , S ).

[0105] The distribution difference between the hash feature distribution and the high-dimensional binomial distribution is determined using the second projection matrix and the third projection matrix.

[0106] Among them, the sum of the differences of S values ​​on each bit of the hash feature is calculated as the distance between the high-dimensional hash feature distribution and the high-dimensional binomial distribution, that is, the F norm of the matrix difference, which is used to measure the quality of the hash code. Then, the distribution difference is The calculation is as follows: .

[0107] The value of S can be 10000; the norm is used to measure the length or size of a vector or matrix.

[0108] The basic model is trained with similarity error to obtain the preset hash model, including: The preset balance coefficient, similarity error and distribution difference are used to determine the model error, and then the model error is used to adjust the model parameters to obtain the preset hash model.

[0109] Specifically, after obtaining the similarity error and the distribution difference, the model error is calculated using the preset balance coefficient, the similarity error and the distribution difference, and then the model error is used to correct the model parameters of the initial hash model to obtain the preset hash model after correction.

[0110] For example, setting the preset balance coefficient to C , preset balance coefficient C The value range can be 0.2-0.8. Assuming the preset balance coefficient is 0.5, we have: .

[0111] in,C is the preset balance coefficient, is the similarity error, is the distribution difference.

[0112] In some embodiments, the training of the preset hash model can be described using a flowchart.

[0113] See also Figure 2 , Figure 2 It is a flowchart of an embodiment of hash model training in this application.

[0114] like Figure 2 As shown, the first step is to obtain the training image features of multiple training images; the second step is to obtain the first similarity feature set and the second similarity feature set corresponding to each training image feature; the third step is to extract hash features using the initial hash model; the fourth step is to obtain the similarity error; the fifth step is to obtain the distribution difference; wherein the fourth step and the fifth step can be performed simultaneously; the sixth step is to determine the model error using the similarity error and the distribution difference, and then use the model error to adjust the initial hash model to obtain the preset hash model; the seventh step is to output the first hash code for the image to be retrieved using the preset hash model. Please refer to the above content for the specific operation steps.

[0115] In this embodiment, the difference between the original feature similarity and the similarity after feature hashing is measured by the similarity error, and the hash code quality is measured by the distribution difference, and finally the model error is determined, which can effectively improve the quality of the hash code, and then improve the quality of the hash code output by the preset hash model.

[0116] In some embodiments, S30 obtains a plurality of second hash codes corresponding to the first hash code; wherein the Hamming distance between each second hash code and the first hash code is less than a preset distance threshold, and may include the following operations.

[0117] A hash code database is constructed, and preset hash codes output by using a preset hash model are pre-stored in the hash code database.

[0118] The hash code database refers to a database storing preset hash codes.

[0119] Specifically, the image features of the multiple images may be preset image features, and then a corresponding preset hash code is obtained for each image feature through a preset hash model.

[0120] It is understandable that after obtaining the preset hash model, for each image feature, a corresponding preset hash code may be obtained through the preset hash model, and then the obtained preset hash code may be stored in a hash code database.

[0121] The Hamming distance between the first hash code and the preset hash code is calculated, and the preset hash code whose Hamming distance is less than the preset distance threshold is used as the second hash code, and then a plurality of second hash codes are obtained from the hash code database, wherein the plurality of second hash codes are at least a part of the preset hash code.

[0122] Among them, the Hamming distance refers to the similarity value of the hash space, which represents the similarity between hash codes.

[0123] Specifically, a first hash code corresponding to an image feature of an image to be retrieved and a Hamming distance between multiple preset hash codes are calculated to obtain a series of Hamming distances, and then the series of Hamming distances are sorted, and a preset hash code whose Hamming distance is less than a preset distance threshold is used as a second hash code, and then a number of second hash codes are obtained from a hash code database; the preset distance threshold can be set according to actual conditions.

[0124] It is understandable that the Hamming distance may be calculated according to an existing calculation method, which is not limited here.

[0125] In some embodiments, S40 calculates the similarity between the image feature corresponding to each second hash code and the image feature corresponding to the first hash code, and determines the image retrieval result of the image to be retrieved according to the similarity calculation result; which may include the following operations.

[0126] First, an original feature set corresponding to a plurality of second hash codes is obtained, wherein the original feature set includes an image feature corresponding to each second hash code.

[0127] Specifically, after obtaining the plurality of second hash codes, the original image feature corresponding to each second hash code is determined, and then the original feature set corresponding to the plurality of second hash codes is obtained.

[0128] Next, similarity calculation is performed using the image feature corresponding to the first hash code and the image feature corresponding to each second hash code to obtain a plurality of corresponding similarity values.

[0129] Specifically, for the image feature corresponding to the first hash code, similarity calculation is performed with each image feature corresponding to the second hash code, thereby obtaining a plurality of corresponding similarity values.

[0130] It is understandable that the aforementioned similarity calculation can be used, that is, given two feature vectors of the same dimension, the cosine similarity of the two feature vectors is calculated; that is, the two feature vectors are inner-producted to obtain a constant; then the lengths of the two feature vectors are obtained, and then the constant is divided by the product of the two lengths to obtain the cosine similarity; wherein the length of the feature vector is obtained by square each dimension of the feature, then summing the squares, and taking the square root of the sum.

[0131] Then, the image corresponding to the second hash code whose similarity value is greater than the third similarity threshold is used as the target image, and the target image is used as the image retrieval result of the image to be retrieved.

[0132] Specifically, after obtaining multiple similarity values, a third similarity threshold is used for comparison, and an image corresponding to a second hash code greater than the third similarity threshold is used as a target image. There may be multiple target images, and then multiple target images are used as image retrieval results of the image to be retrieved, that is, multiple retrieval images related to the image to be retrieved are output.

[0133] In this embodiment, a first hash code corresponding to the image features of the image to be retrieved is obtained through a preset hash model, which can effectively improve the quality of the first hash code; then, multiple second hash codes whose Hamming distances corresponding to the first hash code are less than a preset distance threshold are obtained, and then the image retrieval result is determined based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the accuracy of image retrieval.

[0134] In some embodiments, the present application also provides an image retrieval system.

[0135] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of the structure of an embodiment of an image retrieval system in the present application. The image retrieval system can execute the steps of the image retrieval method in the above method.

[0136] The image retrieval system 400 includes: a first acquisition module 410, a second acquisition module 420, a third acquisition module 430, and a determination module 440. The first acquisition module 410 is used to acquire the image to be retrieved; the second acquisition module 420 uses a preset hash model to acquire a first hash code corresponding to the image feature of the image to be retrieved, wherein the preset hash model is obtained by training the similarity error determined by the first similarity feature set and the second similarity feature set corresponding to each training image feature; the similarity between the features in the first similarity feature set and the training image features is greater than the similarity between the features in the second similarity feature set and the training image features; the third acquisition module 430 is used to acquire multiple second hash codes corresponding to the first hash code; wherein the Hamming distance between each second hash code and the first hash code is less than a preset distance threshold; the determination module 440 calculates the similarity between the image feature corresponding to each second hash code and the image feature corresponding to the first hash code, and determines the image retrieval result of the image to be retrieved based on the similarity calculation result.

[0137] In some embodiments, the present application also provides an electronic device.

[0138] See also Figure 4 , Figure 4 1 is a schematic diagram of the structure of an electronic device in an embodiment of the present application. The electronic device can execute the steps of the image retrieval method in the above method.

[0139] The electronic device 500 includes a memory 520, a processor 510 coupled to the memory, and at least one computer program stored in the memory 520 and executable on the processor 510; when the processor 510 loads and executes the at least one computer program, it is used to implement the image retrieval step in the above method. For related contents, please refer to the detailed description of the above method, which will not be repeated here.

[0140] In some embodiments, a computer-readable storage medium is also included.

[0141] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium in the present application.

[0142] The computer readable storage medium 600 stores at least one program 610, which is used to implement the image retrieval step in the above method when loaded and executed by the processor. For related contents, please refer to the detailed description of the above method, which will not be repeated here.

[0143] The above scheme obtains the first hash code corresponding to the image features of the image to be retrieved through a preset hash model, which can effectively improve the quality of the first hash code; then obtains multiple second hash codes whose Hamming distances corresponding to the first hash code are less than a preset distance threshold, and then determines the image retrieval result based on the similarity between the image features corresponding to the second hash codes and the image features corresponding to the first hash codes, which can effectively improve the accuracy of image retrieval.

[0144] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0146] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0148] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image retrieval method, characterized in that: include: Get the image to be retrieved; Obtaining a first hash code corresponding to the image feature of the image to be retrieved by using a preset hash model, wherein the preset hash model is obtained by training with a similarity error determined by a first similarity feature set and a second similarity feature set corresponding to each training image feature; the similarity between the features in the first similarity feature set and the training image features is greater than the similarity between the features in the second similarity feature set and the training image features; Acquire a plurality of second hash codes corresponding to the first hash code; wherein a Hamming distance between each of the second hash codes and the first hash code is less than a preset distance threshold; The similarity between each image feature corresponding to the second hash code and the image feature corresponding to the first hash code is calculated, and the image retrieval result of the image to be retrieved is determined according to the similarity calculation result.

2. The method according to claim 1, characterized in that The method of using a preset hash model to obtain a first hash code corresponding to the image feature of the image to be retrieved includes: Training and obtaining the preset hash model; The image features of the image to be retrieved are obtained, and the image features are input into the preset hash model to obtain the first hash code corresponding to the image features.

3. The method according to claim 2, characterized in that Each of the to-be-retrieved images obtains one of the image features, and each of the image features has a corresponding first hash code.

4. The method according to claim 2, characterized in that: The training of the preset hash model is as follows: Acquire a training image feature of each training image, and acquire a first similarity feature set and a second similarity feature set corresponding to each of the training image features, wherein the first similarity feature set includes a plurality of high-similarity features, and the similarity between each of the high-similarity features and the training image feature is greater than a first similarity threshold; the second similarity feature set includes a plurality of low-similarity features, and the similarity between each of the low-similarity features and the training image feature is less than a second similarity threshold, and each of the training images obtains one training image feature; Obtaining a first hash feature corresponding to each of the high-similarity features, and obtaining a first similarity between each of the first hash features and each of the high-similarity features; Obtaining a second hash feature corresponding to each of the low-similarity features, and obtaining a second similarity between each of the second hash features and each of the low-similarity features; The similarity error is determined by using the first similarity and the second similarity, and an initial hash model is trained by using the similarity error to obtain the preset hash model.

5. The method according to claim 4, characterized in that Also includes: Similarity sampling is performed using Beta distribution to determine a preset calibration value; The similarity error is determined using the preset calibration value, the first similarity, and the second similarity.

6. The method according to claim 4, characterized in that Also includes: For each of the first hash feature and the second hash feature, perform binomial distribution sampling under the hash feature dimension to obtain a discrete matrix; Performing one-dimensional uniformly distributed random sampling on each of the first hash feature and the second hash feature to obtain a first projection matrix; Projecting the discrete matrix using the first projection matrix to obtain a second projection matrix; and projecting the first feature matrix using the first projection matrix to obtain a third projection matrix, wherein the first feature matrix is ​​composed of a plurality of first hash features and / or second hash features; Determine a distribution difference between a hash feature distribution and a high-dimensional binomial distribution using the second projection matrix and the third projection matrix; The step of training the basic model with the similarity error to obtain the preset hash model includes: The preset balance coefficient, the similarity error and the distribution difference are used to determine the model error, and then the model error is used to adjust the model parameters to obtain the preset hash model.

7. The method according to claim 1, characterized in that The obtaining of a plurality of second hash codes whose Hamming distances to the first hash code are smaller than a preset distance threshold comprises: Constructing a hash code database, and pre-storing a plurality of preset hash codes outputted by using the preset hash model in the hash code database; The Hamming distance between the first hash code and the preset hash code is calculated, and the preset hash code whose Hamming distance is less than a preset distance threshold is used as the second hash code, and then a plurality of second hash codes are obtained from the hash code database, wherein the plurality of second hash codes are at least a part of the preset hash code.

8. The method according to claim 1, characterized in that The calculating the similarity between each image feature corresponding to the second hash code and the image feature corresponding to the first hash code, and determining the image retrieval result of the image to be retrieved according to the similarity calculation result, includes: Acquire a plurality of original feature sets corresponding to the second hash codes, wherein the original feature set includes image features corresponding to each of the second hash codes; Calculate similarity using the image feature corresponding to the first hash code and the image feature corresponding to each of the second hash codes to obtain a plurality of corresponding similarity values; An image corresponding to the second hash code whose similarity value is greater than a third similarity threshold is used as a target image, and the target image is used as an image retrieval result of the image to be retrieved.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium has at least one program, and when the at least one program is loaded and executed by the processor, it is used to implement the method according to any one of claims 1 to 8.

Images