An image retrieval method based on a locality sensitive hashing algorithm

Abstract

This invention discloses an image retrieval method based on the Locality Sensitive Hash (LSH) algorithm, comprising the following steps: Step 1, acquiring multiple image data, preprocessing the multiple image data to obtain a densely represented image dataset A; using a d-dimensional vector x (x∈A) to represent one image data in the image dataset A, x=[x1,x2,...,x... i ,...x d Step 2: Based on image dataset A, initialize the SoftHash parameters and update them based on the image data; where the SoftHash parameters include: synaptic weight matrix W and neuron bias vector b; Step 3: Given an image s with dense representation, use the synaptic weight matrix W and neuron bias vector b obtained in Step 2 to generate a binary representation v of image s with hash length k; Step 4: Based on the obtained binary representation v of image s with hash length k, perform fast nearest neighbor retrieval using Hamming distance to find images with the same object instance as the query image.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to an image retrieval method based on the locality-sensitive hashing algorithm. Background Technology

[0002] Image retrieval is a visual search task that, given a query image, aims to retrieve all images containing instances of the same object as the query image from a potentially very large image database. Image retrieval technology has wide applications in many fields, such as image search engines, image copyright authentication, and medical image analysis. Recently, work on image similarity retrieval has focused on large-scale, high-dimensional problems. To reduce computational complexity, a common approach is to first reduce the dimensionality of the data and then efficiently apply nearest neighbor search or spatial partitioning methods to the simplified data. Locality-Sensitive Hash (LSH) has been widely used and has achieved considerable success in effectively reducing the dimensionality of large datasets.

[0003] Locality-Sensitive Hashing (LSH) has become a widely studied technique in computer science, commonly used to solve image retrieval tasks. It maps input features to binary code, helping to reduce retrieval computation time. The goal of LSH is to cluster similar images closely together and separate dissimilar images far apart in the hash space. Classical LSH methods are typically used to convert high-dimensional features into low-dimensional binary code for fast and efficient nearest neighbor search. Recently, inspired by the olfactory circuits of fruit flies, some researchers have proposed a bio-inspired LSH called FlyHash, which performs sparse feature representation by converting high-dimensional features into even higher-dimensional binary code. FlyHash is a data-agnostic hashing algorithm where the projection space is randomly constructed and cannot adapt to the data flow. A learnable synaptic weight version of FlyHash is BioHash, which updates the synaptic weights of the hash network based on the input data, demonstrating higher performance than classic FlyHash. However, BioHash employs a hard winner-take-all (WTA) mechanism for weight updates, allowing only one winning neuron to be updated in each learning step, which severely limits its ability to capture semantic information from the input data.

[0004] Winner-take-all (WTA) is an important competitive mechanism in recurrent neural networks. It's a computational principle applied to neural network models where neurons compete to activate each other. WTA networks are commonly used in computational models of the brain, particularly for distributed decision-making or action selection in the cortex. It can be used to develop feature selectivity through competition in simple recurrent networks. The classic form of WTA is hard WTA, where only the neuron with the highest activation is updated, while other neurons remain unchanged during learning. This type of WTA is so coarse that the input signal may not be fully utilized by the neurons, thus affecting the semantic representation of the trained network. Compared to hard WTA, soft WTA allows multiple neurons to update based on the input signal, which is more practical and efficient in extracting semantic features from the input data.

[0005] To achieve fast retrieval in massive image datasets, Li et al. proposed Optimal Sparse Lifting (SOLHash), which learns the weights of a mapping function based on training data, thereby further improving the performance of hash representations. SOLHash comprises two key parts: optimal sparse lifting is a sparse binary vector representation of the input image in a high-dimensional space, which roughly preserves pairwise similarity between images. The sparse lifting operator is a sparse binary matrix that best maps the input image to the optimal sparse lifting. To optimize the objective function, SOLHash employs a Frank-Wolf parameter update, where each learned update involves solving a constrained linear procedure involving all training data, which is biologically impractical. From a computer science perspective, the scalability of SOLHash is highly limited; not only does each update step call a constrained linear procedure, but the procedure also involves pairwise similar matrices, which become incomprehensibly large for moderately sized datasets.

[0006] To improve image retrieval performance, Ryali et al., inspired by FlyHash, proposed a novel locality-sensitive hashing algorithm, BioHash. Compared to previous work, this algorithm generates sparse high-dimensional hash codes in a data-driven manner and learns synapses in a neurobiologically sound way. BioHash is a biologically sound, data-driven locality-sensitive hashing algorithm that combines Hebbian rules with a hard WTA mechanism to learn synaptic weights from data. The learning process of BioHash can be formalized as minimizing the following energy function:

[0007]

[0008] in, and It is the Lebesgue norm. The Rank operation sorts the inner product from largest to smallest, and

[0009]

[0010] From a computer science perspective, BioHash is simple to implement, scalable to large datasets, and performs well in similarity search, outperforming deep hashing methods trained via reverse engineering. However, the energy function of the locality-sensitive hashing algorithm BioHash approximates the K-means clustering algorithm, where the distance from the input point to the centroid is measured using a normalized inner product instead of Euclidean distance. Therefore, each input image can only belong to one neuron, such as... Figure 1 As shown.

[0011] To achieve fast retrieval using binary image representations, SOLHash generates sparse binary output vectors for input image data by preserving approximate pairwise similarities between the input image data. This is achieved through constrained linear programming that incorporates all training data in each learning step. However, in biological neural networks, neuronal responses are tuned through physically local synaptic changes. BioHash utilizes the Hebbian local learning rule to learn the synaptic weights of the mapping function, and experiments show that BioHash outperforms common LSH methods (such as classical LSH, data-driven hashing, and deep hashing) in retrieval performance. However, BioHash representations exhibit limitations when the hash length reaches a certain level. This may be attributed to the hard WTA mechanism employed by BioHash, which allows only one neuron to be activated for any input image. This hard WTA mechanism significantly limits the neural network's ability to learn and express semantic information, reducing the accuracy of image retrieval. Summary of the Invention

[0012] In view of this, the present invention provides an image retrieval method based on the locality-sensitive hashing algorithm, comprising the following steps:

[0013] Step 1: Acquire multiple image data, preprocess the multiple image data to obtain a densely represented image dataset A; use a d-dimensional vector x (x∈A) to represent one image data in image dataset A, x=[x1,x2,...,x... i ,...x d ];

[0014] Step 2: Initialize the SoftHash parameters; update the SoftHash parameters based on image dataset A;

[0015] The SoftHash parameters include: the synaptic weight matrix W and the neuron bias vector b. Where m > d;

[0016] Step 3: Given an image s with dense representation, use the updated synaptic weight matrix W and neuron bias vector b to obtain a binary representation v of image s with hash length k.

[0017] Step 4: Based on the binary representation v of the obtained image s, perform a fast nearest neighbor search using Hamming distance to find images with the same object instance as the query image.

[0018] Furthermore, the preprocessing in step 1 includes:

[0019] Step 11: Use Gaussian convolution to perform blurring and denoising processing on multiple image data;

[0020] Step 12: Normalize the multiple image data after blurring and denoising.

[0021] Furthermore, step 2 involves updating the SoftHash parameters, including:

[0022] Step 21: Given an input d-dimensional vector x (x∈A), map it to an m-dimensional space using a hash function, and output y. Where m > d;

[0023] Synaptic weight The update formula is:

[0024]

[0025] Where τ is a hyperparameter, a constant; η represents the learning rate, with a value between 0 and 0.1; y j Used to represent the softmax output of the j-th neuron; x i It is the i-th element of the d-dimensional vector x; u j It is the postsynaptic variable of the j-th output neuron;

[0026] The bias of the j-th neuron is b j The update method is as follows:

[0027] b j =b j-1 +Δb j

[0028] Among them, b j-1 For the bias of the (j-1)th neuron,

[0029] Step 22, based on synaptic weights and neuron bias b j Calculate the postsynaptic variables of the j-th neuron:

[0030]

[0031] Obtain its Bayesian posterior:

[0032] y j =Softmax(u j +b j );

[0033] Step 23: Repeat steps 21 and 22 to update the synaptic weight matrix W and the neuron bias vector b until the predetermined number of iterations is reached.

[0034] Furthermore, the binary representation v in step 3 is specifically as follows:

[0035]

[0036] Where top k represents all outputs y j The first k elements sorted from largest to smallest.

[0037] This invention discloses an image retrieval method based on the Locality Sensitive Hashing (LSH) algorithm. SoftHash maps densely represented images to a binary space to obtain new representations, thereby achieving fast retrieval through Hamming distance. The proposed SoftHash algorithm combines Hebbi-like learning rules and a soft winner-takes-all mechanism, learning the synaptic weights and neuron biases of the mapping function to generate sparse, binary hash representations. SoftHash allows weakly correlated data to learn, effectively generating more representative high-dimensional hash codes, thus improving retrieval performance while achieving fast retrieval. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the clustering process of the BioHash hashing algorithm;

[0039] Figure 2 This is a schematic diagram of the clustering process of the SoftHash hash algorithm;

[0040] Figure 3 A schematic diagram of the local class Heb learning rule;

[0041] Figure 4 A schematic diagram of the PN-KC-APL network architecture of the Drosophila mushroom body;

[0042] Figure 5 This is a flowchart of the image retrieval method based on the locality-sensitive hashing algorithm of the present invention. Detailed Implementation

[0043] This invention discloses an image retrieval method based on the Locality Sensitive Hashing (LSH) algorithm. By using the SoftHash algorithm, densely represented images are mapped to a binary space to obtain new representations, thereby achieving fast retrieval through Hamming distance. The energy function equation of SoftHash can be approximated as a mixture probability model, where each output neuron is treated as a separate probability model. In this framework, the clustering process can be interpreted as the probability that each input image belongs to a different neuron, such as... Figure 2 As shown, each input image can be associated with one or more neurons, allowing for a more flexible and detailed representation of the data. This probabilistic modeling approach in SoftHash enables a richer understanding of the semantic information in the input image data and captures complex patterns and associations within the data, thereby improving expressive power and discriminative ability.

[0044] The SoftHash proposed in this invention is a data-dependent hashing algorithm that uses Hebbi-like rules (such as...). Figure 3 (As shown) The synaptic weights and neuron biases of the mapping function are updated. This rule combines Bayes' theorem with a soft-winner-all (soft-WTA) mechanism using softmax smoothing to better learn semantic information in the input. This invention first generates a binary representation vector of the image through SoftHash, as shown in the following process. Figure 4 The PN-KC-APL neural network is illustrated. The input image is mapped to the KC neurons via a mapping function, which generates their posterior probabilities. The KC-APL recurrent network retains a small subset of activated KC neurons, assigning them state 1, while the remaining inactive KC neurons are assigned state 0. Next, a nearest neighbor search is performed on these representation vectors using Hamming distance to retrieve images with instances of the same object.

[0045] To achieve fast image retrieval, the location of the input image can be specified by observing the nearest few reference points among a set of reference points selected from the hash space, generating a sparse yet useful local representation. In this way, the nearest neighbors of the input image can be quickly found using Hamming distance. This invention models all output neurons as probabilistic models to support the entire data distribution. The values ​​on the output layer represent the probability that each input image belongs to them, calculated based on the input image data, synaptic weights, and neuron biases. The plasticity rules of the synaptic weights and the iterative updates of the neuron biases in the probabilistic model match their density to the input image data density. In other words, more output neurons are needed to provide high resolution where data density is high, and fewer neurons are needed where data density is low. Thus, the SoftHash algorithm has sufficient capability to activate similar output neurons from similar inputs, tightly clustering them in a hash space with a new dimension expansion. Therefore, the SoftHash algorithm proposed in this invention can learn the locality of the input representation.

[0046] From a mathematical perspective, the image retrieval method based on Locality Sensitive Hashing (LSH) proposed in this invention aims to achieve fast and more accurate image retrieval while representing the semantic information of images using binary values ​​(0 and 1). This invention learns the hash mapping function through a Hebbian-like learning rule and the Winner-Take-All (WTA) principle. Specifically, the synaptic weights of the mapping function are updated based on local learning rules, meaning that changes in weights depend only on the activity of presynaptic and postsynaptic neurons, which is logically sound in neurobiology. Furthermore, unlike previous studies using hard WTA rules, this invention introduces soft WTA rules, where non-winning neurons are not completely suppressed during the learning process. This allows weakly correlated data to be learned to generate more representative hash codes, thereby improving the accuracy of image retrieval.

[0047] Figure 5 The flowchart of the image retrieval method based on the locality-sensitive hashing algorithm of the present invention is shown. Figure 5 As shown, the method includes the following steps:

[0048] Step S1: Acquire multiple image data, preprocess the multiple image data, use Gaussian convolution to blur and denoise the multiple images, and then normalize them to obtain a densely represented image dataset A. Represent a single image data in image dataset A with a d-dimensional vector x (x∈A), denoted as x=[x1,x2,...,x...]. i ,...x d ].

[0049] Step S2: Initialize the parameters of SoftHash; based on the image dataset A, update the parameters of SoftHash: the synaptic weight matrix W and the neuron bias vector b.

[0050] Step S2-1: Given an input d-dimensional vector x (x∈A), map it to an m-dimensional space (m>d) using a hash function, and output y, where y is an m-dimensional vector, represented as... Synaptic weights based on Heb-like rules and soft winner-take-all mechanism The update formula is ( Figure 3 ):

[0051]

[0052] Where τ is a hyperparameter, a constant; η represents the learning rate, with a value between 0 and 0.1; y j Used to represent the softmax output of the j-th neuron; x i It is the i-th element of the d-dimensional vector x; u j It is the postsynaptic variable of the j-th output neuron.

[0053] The bias of the j-th neuron is b j The update method is as follows:

[0054] b j =b j-1 +Δb j

[0055] Among them, b j-1 For the bias of the (j-1)th neuron,

[0056] Step S2-2: Based on synaptic weights and neuron bias b j Calculate the postsynaptic variables of the j-th neuron. This leads to the further acquisition of its Bayesian posterior y. j =Softmax(u j +b j The calculation results.

[0057] Step S2-3: Based on the densely represented image dataset A, repeat steps S2-1 and S2-2 to update the parameters until the specified number of iterations is reached.

[0058] Step S3: Given an image s with dense representation, use the synaptic weight matrix W and neuron bias vector b updated in step S2 to generate a binary representation v of image s with hash length k, as follows:

[0059]

[0060] Where top k represents all outputs y j The first k elements sorted from largest to smallest.

[0061] Step S4: Based on the binary representation of image s, perform fast nearest neighbor retrieval using Hamming distance to find images that have the same object instance as the query image.

[0062] The performance of the SoftHash-based image retrieval method is then verified using two datasets, Fashion-MNIST and CIFAR10.

[0063] Table 1. Comparison of mAP@1000 results of different hash algorithms on the Fashion-MNIST and CIFAR10 datasets.

[0064]

[0065] Table 1 shows the mAP@1000 results of different hash algorithms on two benchmarks, with hash lengths ranging from 2 to 128. It is clear from Table 1 that among all hash algorithms, the image retrieval method based on SoftHash-2 exhibits the best retrieval performance at large hash lengths, especially when k=128. As the hash length k increases, SoftHash-2 achieves significantly better results than SoftHash-1. Comparing the results of FlyHash and ConvHash, it can be seen that FlyHash is significantly superior to the classic algorithm ConvHash at different hash lengths, demonstrating the good performance of high-dimensional sparse binary representation. BioHash-1 and BioHash-2 show the best retrieval performance at small hash lengths, but provide small improvements from k=32 to k=64, and even smaller improvements from k=64 to k=128, consistent with the conclusions in their previous work. Furthermore, it can be observed that SoftHash-2 and BioHash-2, with synaptic weights initialized according to a standard normal distribution, both achieve better results.

[0066] Experimental results show that the image retrieval method based on the Locality Sensitive Hash (SoftHash) algorithm proposed in this invention significantly outperforms retrieval methods based on other hash algorithms. Compared to classic and other biologically inspired Locality Sensitive Hash algorithms, SoftHash offers superior semantic representation capabilities through its binary representation.

[0067] In summary, the image retrieval method based on the Locality Sensitive Hash (SoftHash) algorithm proposed in this invention has superior retrieval accuracy compared to existing image retrieval methods based on hash algorithms. The above embodiments are merely illustrative of the technical solutions of this invention and not intended to limit it. Although the invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation methods of this invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of this invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in this invention.

Claims

1. An image retrieval method based on the locality-sensitive hashing algorithm, characterized in that, The method includes the following steps: Step 1: Acquire multiple image data, preprocess the multiple image data to obtain a densely represented image dataset A; use a d-dimensional vector x (x∈A) to represent one image data in image dataset A, x=[x1,x2,...,x... i ,...x d ]; Step 2, initialize the SoftHash parameters; Update the SoftHash parameters based on image dataset A; The SoftHash parameters include: the synaptic weight matrix W and the neuron bias vector b. Where m > d; Step 3: Given an image s with dense representation, use the updated synaptic weight matrix W and neuron bias vector b to obtain a binary representation v of image s with hash length k. Step 4: Based on the binary representation v of the obtained image s, perform a fast nearest neighbor search using Hamming distance to find images with the same object instance as the image to be searched; Step 2 involves updating the SoftHash parameters, including: Step 21: Given an input d-dimensional vector x (x∈A), map it to an m-dimensional space using a hash function, and output y, y=[y1,y2,...y j ,...y m ]; Synaptic weight The update formula is: Where τ is a hyperparameter, a constant; η represents the learning rate, with a value between 0 and 0.1; y j Used to represent the softmax output of the j-th neuron; x i It is the i-th element of the d-dimensional vector x; u j It is the postsynaptic variable of the j-th output neuron; The bias of the j-th neuron is b j The update method is as follows: b j =b j-1 +Δb j Among them, b j-1 For the bias of the (j-1)th neuron, Step 22, based on synaptic weights and neuron bias b j Calculate the postsynaptic variables of the j-th neuron: Obtain its Bayesian posterior: and j =Softmax(u j +b j ); Step 23: Repeat steps 21 and 22 to update the synaptic weight matrix W and the neuron bias vector b until the predetermined number of iterations is reached.

2. The image retrieval method as described in claim 1, characterized in that, Step 1 preprocessing includes: Step 11: Use Gaussian convolution to perform blurring and denoising processing on multiple image data; Step 12: Normalize the multiple image data after blurring and denoising.

3. The image retrieval method as described in claim 1, characterized in that, The binary representation v in step 3 is specifically as follows: Where top k represents all outputs y j The first k elements sorted from largest to smallest.

Images