An image hash retrieval method based on center mapping

Abstract

The present invention discloses an image hash retrieval method based on center mapping. The method converts images into hash codes for retrieval, reduces storage and computational requirements, and improves retrieval efficiency. A deep model is used to map images into hash codes, which can better extract image features and retain more semantic information with a relatively small number of bits. The quantized hash codes are used as learning targets, making the errors caused by the final quantization negligible and improving the quality of the hash codes. A low-dimensional learning target is constructed using feature center mapping, so that the similarity information of images in high-dimensional feature space can be relatively completely retained in low-dimensional Hamming space, resulting in more accurate retrieval results. The model is trained in an unsupervised manner, greatly reducing the data annotation cost and expanding the scope of application of the method. The above method has been successfully applied in searching for decoration images, and can more quickly and intuitively determine the degree of decoration of a decoration image.

Description

Technical Field

[0001] The present invention relates to the technical field of image hash retrieval, and in particular to an image hash retrieval method based on center mapping. Background Art

[0002] In the era of big data, finding information quickly is a crucial issue, especially in the field of building renovation. To better assess a building's renovation status, historical images of known renovation levels are often needed for comparison. These images can help construction workers, developers, and owners further determine upcoming renovation plans and conduct preliminary assessments of the results. Using a new renovation image to find similar images from historical images can greatly improve the efficiency of related work. However, due to the vast number of historical renovation images in databases, how to quickly and accurately identify images with similar renovation levels has become a pressing issue.

[0003] Similarity retrieval, also known as approximate nearest neighbor retrieval, is a method that balances retrieval speed and accuracy. The nearest neighbor retrieval algorithm searches for the most similar historical image in the database based on the similarity between the query image and existing historical images. The core of approximate nearest neighbor retrieval is that the retrieval results are likely to be the nearest neighbors of the query image, improving retrieval efficiency while maintaining acceptable accuracy.

[0004] Hash retrieval, a branch of the approximate nearest neighbor search method, represents images as low-dimensional hash codes (i.e., binary codes) and determines image similarity based on the Hamming distance between the hash codes. During retrieval, the Hamming distance between the hash codes of the query image and previous images is calculated, and the images are sorted in ascending order of Hamming distance. Similar images are found by searching for the images corresponding to the hash codes that appear at the top of the sort. Hash retrieval has garnered widespread attention in both academia and industry due to its advantages, such as the efficient storage of binary codes in computers and the rapid calculation of Hamming distances using XOR and bit counting operations.

[0005] Existing image retrieval methods include floating-point vector feature retrieval, locality sensitive hashing method, spectral hashing method, reconstruction-based hash learning method, convolutional neural network hashing method and contrastive learning-based hash learning method.

[0006] Floating-point vector feature retrieval: Existing models and methods can generate high-dimensional floating-point vector features from images. The similarity between two floating-point vector features (such as Euclidean distance and cosine similarity) can be used to determine the similarity between two images, thereby finding similar images. However, floating-point vector feature retrieval requires high storage and computational complexity, resulting in slow retrieval speeds when the data volume is large, making it difficult to meet retrieval efficiency requirements in practical applications.

[0007] Locality-sensitive hashing (LSH) uses a series of random mappings or projections to assign similar images to the same hash bucket with a high probability, while dissimilar images are less likely to end up in the same bucket. When searching, the query image is hashed to obtain the bucket number, and then all the data within the corresponding bucket is retrieved and linearly matched to retrieve similar images. LSH and related methods suffer from data independence. They cannot learn a hash function that adapts to the data distribution based on the current data. Therefore, these methods often require longer hash codes to achieve the required retrieval accuracy, resulting in losses in computational and storage efficiency.

[0008] Spectral hashing method. This method transforms the hash function mapping problem into a spectral graph partitioning problem, ensuring that the hash codes of similar data in the original data space have a smaller Hamming distance and that the generated hash codes are balanced and independent. Balance means that the number of 0s and 1s in the generated hash codes is approximately equal, and independence means that the bits of the hash codes are unrelated, with as little redundant information as possible. This allows the hash code to represent as much information as possible within a given number of bits. Methods like spectral hashing suffer from large quantization errors, meaning that the model output is still a floating-point vector feature that needs to be further quantized into a hash code. This error can negatively impact the accuracy of image representation and retrieval precision.

[0009] Reconstruction-based hash learning method. Based on the AutoEncoder network structure, the original image and hash code are used as the input and output of the Encoder module, respectively. The Decoder then reconstructs the image based on the hash code. The network is trained with the goal of restoring the original image as much as possible from the Decoder output. This allows for the generation of high-quality hash codes that retain image features and can be used in the hash retrieval process. The reconstruction-based method aims to restore the original image as much as possible, which makes it difficult for the model to distinguish between important semantic information and secondary background information or even noise information in the image. This leads to the model learning this secondary information and ultimately the hash code retains insufficient semantic information, which cannot meet the retrieval requirements in practical applications.

[0010] Convolutional neural network hashing methods. This method first learns hash codes from paired semantic labels, then models the learning of the hash mapping function as a binary classification problem for each bit. Finally, this multi-classification model is jointly optimized with the feature learning portion of the convolutional neural network used for image classification to obtain the final deep hashing model. Similar to convolutional neural network hashing methods, which often require a large amount of annotated information, this method requires high initial data preparation costs, making it difficult to use in cost-sensitive scenarios, limiting its practical applicability.

[0011] A hash learning method based on contrastive learning. By enhancing an image with different methods, two images with identical semantics but different visual features can be obtained. By feeding these two images into the same model, two hash codes can be obtained. The goal of model training is to make the hash codes corresponding to the two enhanced images as identical as possible, while being as different as possible from the hash codes of other images. This results in hash codes that retain more semantic information, and searches using the hash retrieval process can yield semantically similar images. Contrastive learning methods treat all enhanced images except those from the same source as the original image as dissimilar samples. This approach easily overlooks similar samples in the data, hindering similarity learning. The final retrieval process relies heavily on the relative similarity between the data, leading to inconsistencies between model training and retrieval. Summary of the Invention

[0012] Aiming at the problems of low retrieval efficiency, high initial data annotation cost and inconsistency between model training and retrieval in existing hash retrieval methods, the present invention proposes an image hash retrieval method based on center mapping.

[0013] In order to achieve the above object, the present invention provides the following technical solutions:

[0014] The present invention provides an image hash retrieval method based on center mapping, which uses a deep model to map images into hash codes to extract image features, maps high-dimensional floating-point vector feature cluster centers to low-dimensional Hamming space anchor points, and constructs a target training model based on the distance between image features in high-dimensional space and cluster centers and low-dimensional anchor points; in the prediction stage, the image obtains a hash code through the model, and the similarity between images is judged according to the Hamming distance between the hash codes.

[0015] Furthermore, the above-mentioned image hash retrieval method based on center mapping includes the following steps:

[0016] S1. Build the dataset: Divide it into the search library data, training data, and test data in a ratio of 7:2:1;

[0017] S2. Construct a feature set of training data: Use the CNN model pre-trained on the ImageNet dataset to extract the features of all training set images. These features constitute a feature set of the training set.

[0018] S3, K-means clustering, perform K-means clustering on the feature set in S2 to obtain K cluster centers of the training set in the high-dimensional feature space

[0019] S4. According to the number of cluster centers K in S3, randomly select K columns from the Hadamard matrix as anchor points in the low-dimensional Hamming space, expressed as a matrix form H = (h1, h2, ..., h K ) T ;

[0020] S5. Build the model: Use several fully connected layers to build the model f(·), where the input dimension of the first layer is equal to the output size of the pre-trained CNN in S2, and the output dimension of the last layer is equal to the number of bits of the hash code;

[0021] S6, feature center mapping: the K cluster centers obtained in S2 are mapped one-to-one with the K anchor points obtained in S3, realizing the mapping between the high-dimensional floating-point vector feature cluster centers and the low-dimensional Hamming space hash code centers;

[0022] S7, construct training target: For a training image, use the CNN pre-trained network in S2 to obtain the feature x i , i = 1, 2, ..., n, calculate the Euclidean distance between the feature and the K centers obtained in S3:

[0023] d i，k =dist(x i , c k )=||x i -c k ||2, k=1, 2, ..., K

[0024] d i =(d i，1 , d i，2 ,...,d i，K ) T

[0025] Take the opposite number and perform SoftMax normalization as the weight of each center for the sample:

[0026] w i =softmax(-d i )

[0027] According to the mapping relationship in S6, the anchor points H in each low-dimensional Hamming space are weighted averaged; after symbolization using the sign function, they are used as the training target of this sample in the low-dimensional Hamming space:

[0028]

[0029] S8. Training model: Output the model With the training target y i The cosine distance between them is used as the loss to train the network, which is specifically expressed as follows

[0030]

[0031]

[0032] S9, hash code reasoning: For any image, input it into the CNN model in S2 to obtain the high-dimensional floating-point vector feature x, then input it into the trained model in S5 and use the sign function to symbolize it to obtain the hash code Y corresponding to the image:

[0033] Y=(sign(f(x))+1) / 2

[0034] S10, image retrieval: the images in the searched library and the new image are stored according to the hash codes corresponding to the images obtained in S9, the Hamming distance between the hash code of the new image and the hash codes of all the images in the searched library is calculated, and the first N images with the smallest Hamming distance are sorted in ascending order as the similar images retrieved.

[0035] The present invention also provides an application of the above-mentioned center mapping-based image hash retrieval method in decoration picture search.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The present invention converts images into hash codes for retrieval, which reduces the demand for storage space and calculation and improves retrieval efficiency.

[0038] 2. The present invention uses a deep model to map images into hash codes, which can better extract image features and retain more semantic information with a relatively small number of bits.

[0039] 3. The present invention uses the quantized hash code as the learning target, so that the error caused by the final quantization can be ignored, thereby improving the quality of the hash code.

[0040] 4. The present invention uses feature center mapping to construct a low-dimensional learning target and proposes a method of mapping the high-dimensional floating-point vector feature clustering center to the low-dimensional Hamming space anchor point, so that the similarity information of the image in the high-dimensional feature space can be relatively completely retained in the low-dimensional Hamming space, retaining the similarity of the data in the high-dimensional feature space as much as possible, and being able to retrieve similar images more accurately.

[0041] 5. The present invention uses an unsupervised approach to train the model, which greatly reduces the data annotation cost and increases the scope of application of the method.

[0042] 6. The present invention applies hash retrieval technology to the field of decoration images, which can more quickly and intuitively determine the decoration level of a decoration picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0044] Figure 1 Flowchart of the image hash retrieval method based on center mapping provided by an embodiment of the present invention.

[0045] Figure 2 A schematic diagram of the model structure of the image hash retrieval method based on center mapping provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] First, we introduce a solution that is somewhat similar to the present invention. If each image has its corresponding category label information and the dataset has M categories of labels, then we can randomly select M columns from the Hadamard matrix and record them as {h1, h2, ..., h M}, if a sample data belongs to the mth category, then its training target is h m , the model trained in this way can map images into hash codes that are close to their label features. However, this method requires labeling the categories of a large amount of data, and the cost of initial data labeling is high. If a single cluster center is used as a pseudo-label instead of the distance to each cluster center, assuming that K cluster centers are obtained through K-means clustering, for each sample, only the cluster center closest to it is selected as the pseudo-label and a low-dimensional hash code is selected as the training target based on this. This can make the model output closer to certain hash codes. However, this method does not take into account the distribution of high-dimensional features. Selecting only one cluster center as a pseudo-label will result in a large amount of information loss, and the hash code will find it difficult to retain sufficient similarity information.

[0047] The purpose of the present invention is to quickly and accurately find pictures with similar decoration levels. Based on this purpose, the design concept of the present invention is as follows: first, a hash retrieval model is constructed, and then unsupervised training is performed using pictures of various decoration levels. Then, for any decoration picture, a hash code can be obtained through the model. The Hamming distance between the hash code and the hash codes of historical pictures in the database is calculated, and the pictures corresponding to the hash codes with the smallest Hamming distance are selected as historical pictures with similar decoration levels, thereby obtaining the retrieval results.

[0048] In order to better understand the present technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

[0049] This paper provides an image hash retrieval method based on center mapping. This method uses a deep learning model to map images into hash codes to extract image features. The centers of high-dimensional floating-point vector feature clusters are mapped to anchor points in a low-dimensional Hamming space. A target training model is constructed based on the distance between the image features in the high-dimensional space and the cluster centers and the low-dimensional anchor points. During the prediction phase, images are hashed using the model, and the Hamming distance between the hash codes is used to determine the similarity between images.

[0050] Specifically, the above-mentioned image hash retrieval method based on center mapping, such as Figure 1 As shown, the following steps are included:

[0051] S1. Build a dataset: Collect a large number of images of various decoration levels and divide them into search library data, training data, and test data in a ratio of 7:2:1;

[0052] S2. Construct a feature set for training data: Use a CNN (Convolution Neural Network) model pre-trained on the ImageNet dataset, such as VGG-16, AlexNet, ResNet50, etc., to extract features from all training set images. These features constitute a feature set for the training set. Using an unsupervised approach to train the model greatly reduces the data annotation cost and increases the scope of application of this method.

[0053] S3, K-means clustering, perform K-means clustering on the feature set in S2 to obtain K cluster centers of the training set in the high-dimensional feature space

[0054] S4. According to the number of cluster centers K in S3, randomly select K columns from the Hadamard matrix as anchor points in the low-dimensional Hamming space, expressed as a matrix form H = (h1, h2, ..., h K ) T ;

[0055] S5. Build the model: Use several fully connected layers to build the model f(·), where the input dimension of the first layer is equal to the output size of the pre-trained CNN in S2, and the output dimension of the last layer is equal to the number of bits of the hash code;

[0056] S6, Feature Center Mapping: Since there are no usable labels for samples during unsupervised training, introducing cluster centers is equivalent to using the similarity of high-dimensional space to generate a pseudo-label for each sample based on the distance from the sample feature to the center, that is, the weight vector w of each center. After mapping to the low-dimensional space, the low-dimensional hash anchor points are weighted averaged according to the pseudo-label to obtain the target vector y that retains the similarity of the high-dimensional space features as much as possible. The specific method is as follows: the K cluster centers obtained in S2 are mapped one-to-one with the K anchor points obtained in S3 to achieve the mapping of the high-dimensional floating-point vector feature cluster centers to the low-dimensional Hamming space hash code centers, which can be expressed as follows:

[0057] g(c i )=hi, i=1, 2,...,K;

[0058] S7, construct training target: For a training image, use the CNN pre-trained network in S2 to obtain the feature x i , i = 1, 2, ..., n, calculate the Euclidean distance between the feature and the K centers obtained in S3:

[0059] d i，k =dist(x i , c k )=||x i -c k ||2, k=1, 2, ..., K

[0060] d i =(d i，1 , d i，2 ,...,d i，K ) T

[0061] Take the opposite number and perform SoftMax normalization as the weight of each center for the sample:

[0062] w i =softmax(-d i )

[0063] According to the mapping relationship in S6, the anchor points H in each low-dimensional Hamming space are weighted averaged; after symbolization using the sign function, they are used as the training target of this sample in the low-dimensional Hamming space:

[0064]

[0065] S8. Training model: Output the model With the training target y i The cosine distance between them is used as the loss to train the network, which is specifically expressed as follows

[0066]

[0067]

[0068] This loss can make the model output as consistent as possible with the semantic hash code target. At the same time, using the quantized vector as the target also reduces the error caused by quantization, so that the model can map image features to hash codes that retain semantic information and similarity between images.

[0069] S9, hash code reasoning: For any image, input it into the CNN model in S2 to obtain the high-dimensional floating-point vector feature x, then input it into the trained model in S5 and symbolize it using the sign function to obtain the hash code y corresponding to the image:

[0070] y=(sign(f(x))+1) / 2

[0071] S10, image retrieval: the images in the searched library and the new image are stored according to the hash codes corresponding to the images obtained in S9, the Hamming distance between the hash code of the new image and the hash codes of all the images in the searched library is calculated, and the first N images with the smallest Hamming distance are sorted in ascending order as the similar images retrieved.

[0072] In summary, the image hash retrieval method based on center mapping of the present invention converts images into hash codes for retrieval, with relatively small storage and computing requirements, thereby improving retrieval efficiency; using a deep model to map images into hash codes can better extract image features, and more semantic information can be retained with a relatively small number of bits; using quantized hash codes as learning targets makes the error caused by the final quantization negligible, thereby improving the quality of the hash codes; using feature center mapping to construct low-dimensional learning targets, so that the similarity information of images in high-dimensional feature space can be relatively completely preserved in low-dimensional Hamming space, and the retrieval results are more accurate; using an unsupervised method to train the model greatly reduces the data annotation cost and increases the scope of application of this method. The above method has achieved good application in the search of decoration pictures, and can more quickly and intuitively judge the degree of decoration of a decoration picture.

[0073] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image hash retrieval method based on center mapping, characterized in that: A deep model is used to map images into hash codes to extract image features, and high-dimensional floating-point vector feature cluster centers are mapped to low-dimensional Hamming space anchor points. A target training model is constructed based on the distance between the image features in the high-dimensional space and the cluster centers and the low-dimensional anchor points. In the prediction phase, the image is hashed using the model, and the similarity between images is determined based on the Hamming distance between the hash codes. The image hash retrieval method based on center mapping includes the following steps: S1. Build the dataset: Divide it into the search library data, training data, and test data in a ratio of 7:2:1; S2. Construct a feature set of training data: Use the CNN model pre-trained on the ImageNet dataset to extract the features of all training set images. These features constitute a feature set of the training set. S3, K-means clustering, perform K-means clustering on the feature set in S2 to obtain K cluster centers of the training set in the high-dimensional feature space S4. According to the number of cluster centers K in S3, randomly select K columns from the Hadamard matrix as anchor points in the low-dimensional Hamming space, expressed as a matrix form S5. Build the model: Use several fully connected layers to build the model f(·), where the input dimension of the first layer is equal to the output size of the pre-trained CNN in S2, and the output dimension of the last layer is equal to the number of bits of the hash code; S6, feature center mapping: the K cluster centers obtained in S2 are mapped one-to-one with the K anchor points obtained in S3, realizing the mapping between the high-dimensional floating-point vector feature cluster centers and the low-dimensional Hamming space hash code centers; S7, construct training target: For a training image, use the CNN pre-trained network in S2 to obtain the feature x i , i=1,2,…,n, calculate the Euclidean distance between the feature and the K centers obtained in S3: d i,k =dist(x i ,c k )=‖x i -c k ‖2,k=1,2,…,K Take the opposite number and perform SoftMax normalization as the weight of each center for the sample: <h2 style=";text-align:left;direction:ltr">w<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =softmax(-d<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> ) According to the mapping relationship in S6, the anchor points H in each low-dimensional Hamming space are weighted averaged; after symbolization using the sign function, they are used as the training target of this sample in the low-dimensional Hamming space: S8. Training model: Output the model With the training target y i The cosine distance between them is used as the loss to train the network, which is specifically expressed as follows: S9, hash code reasoning: For any image, input it into the CNN model in S2 to obtain the high-dimensional floating-point vector feature x, then input it into the trained model in S5 and symbolize it using the sign function to obtain the hash code y corresponding to the image: y=(sign(f(x))+1) / 2 S10, image retrieval: the images in the searched library and the new image are stored according to the hash codes corresponding to the images obtained in S9, the Hamming distance between the hash code of the new image and the hash codes of all the images in the searched library is calculated, and the first N images with the smallest Hamming distance are sorted in ascending order as the similar images retrieved.

2. Application of the image hash retrieval method based on center mapping according to claim 1 in searching for decoration pictures.

Images