Image retrieval method, system, device and medium based on center similarity

Abstract

The present application relates to an image retrieval method, system, device, and medium based on center similarity, wherein the method comprises: obtaining an image dataset, and using a Hadamard matrix and Bernoulli distribution to generate a hash center for each category in the image dataset; correcting the hash center to obtain a target hash center; constructing a training dataset based on the image dataset, extracting and activating features of each input image in the training dataset through a deep hash network to obtain low-dimensional real-valued features; calculating the center similarity Bayesian loss using the low-dimensional real-valued features and the target hash center, and training the deep hash network based on the center similarity Bayesian loss using backpropagation to obtain a target retrieval model; obtaining an image to be retrieved, and retrieving the image to be retrieved based on the target retrieval model to obtain a target similar image. The present application ensures that there is sufficient spacing between hash centers, reduces the difficulty and time of model training, and improves image retrieval accuracy.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image retrieval method, system, device and medium based on center similarity. Background Art

[0002] In recent years, with the rapid development of big data and artificial intelligence technologies, hash learning has become a mainstream research direction in approximate nearest neighbor search, achieving significant progress both domestically and internationally. Traditional hash learning methods can be divided into two categories based on whether they require data for pre-training: data-independent hashing and data-dependent hashing. Data-independent hashing is a hash learning method that does not rely on training data and uses a randomly generated projection matrix to binary encode the data. Locality-Sensitive Hashing (LSH), a representative data-independent hashing algorithm, uses random linear projections to map nearby data into similar binary codes. Kernelized Locality-Sensitive Hashing (KLSH), another approach, adds a kernel function to LSH to better learn nonlinear features. However, these methods suffer from two issues: first, they typically require very long hash codes, which results in low retrieval efficiency and significantly increases the storage cost of the hash codes; second, they suffer from low retrieval accuracy.

[0003] Due to the powerful feature extraction capabilities of deep neural networks, many recent works have attempted to combine deep learning with hash learning, using neural networks to learn hash codes, achieving impressive retrieval results. However, existing methods mostly rely on pairwise and triplet similarity learning. Pairwise similarity-based hash learning focuses on learning similarity relationships between pairs of data, which is localized, and the learned hash model fails to capture global similarity information. To ensure that the training data contains rich similarity information, triplet-based hash learning methods require sampling a large number of triplets from the training set, which significantly increases training time. Furthermore, due to the unclear hash center, the hash code lacks inter-class discriminability, resulting in low image retrieval accuracy. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to propose an image retrieval method, system, device and medium based on center similarity to reduce the difficulty and time of model training, avoid the problem of insufficient inter-class discriminability of hash codes, and improve image retrieval accuracy.

[0005] In order to solve the above technical problems, the present invention provides an image retrieval method based on center similarity, including:

[0006] Obtain an image dataset, and generate a hash center for each category in the image dataset using a Hadamard matrix and a Bernoulli distribution;

[0007] Correcting the hash center to obtain a target hash center;

[0008] Constructing a training dataset based on the image dataset, extracting and activating features of each input image in the training dataset through a deep hashing network to obtain low-dimensional real-valued features;

[0009] Calculating the center similarity Bayesian loss by using the low-dimensional real-valued features and the target hash center, and training the deep hash network based on the center similarity Bayesian loss by backpropagation to obtain a target retrieval model;

[0010] Acquire an image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image.

[0011] In order to solve the above technical problems, the present application provides an image retrieval system based on center similarity, including:

[0012] A hash center generating unit, configured to obtain an image data set and generate a hash center for each category in the image data set using a Hadamard matrix and a Bernoulli distribution;

[0013] A hash center correction unit, configured to correct the hash center to obtain a target hash center;

[0014] A feature extraction unit is used to construct a training dataset based on the image dataset, extract and activate the features of each input image in the training dataset through a deep hash network, and obtain low-dimensional real-valued features;

[0015] A model training unit is used to calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss using a back-propagation method to obtain a target retrieval model;

[0016] The target similar image retrieval unit is used to obtain the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain the target similar image.

[0017] To solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that the one or more processors can implement any one of the above-mentioned image retrieval methods based on center similarity.

[0018] To solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements any one of the above-mentioned image retrieval methods based on center similarity.

[0019] Embodiments of the present invention provide a method, system, device, and medium for image retrieval based on center similarity. The method includes: obtaining an image dataset and using a Hadamard matrix and Bernoulli distribution to generate a hash center for each category in the image dataset; correcting the hash center to obtain a target hash center; constructing a training dataset based on the image dataset, extracting and activating features of each input image in the training dataset through a deep hash network to obtain low-dimensional real-valued features; calculating a center similarity Bayesian loss using the low-dimensional real-valued features and the target hash center, and training the deep hash network based on the center similarity Bayesian loss using backpropagation to obtain a target retrieval model; obtaining an image to be retrieved, and searching the image to be retrieved based on the target retrieval model to obtain a target similar image. The embodiment of the present invention uses a Hadamard matrix and Bernoulli distribution to generate a hash center for each category in the image dataset, corrects the hash centers to ensure sufficient spacing between hash centers, and uses a center similarity Bayesian loss for model training, which helps reduce the difficulty and time of model training and improves image retrieval accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flowchart of the implementation process of the image retrieval method based on center similarity provided in an embodiment of the present application;

[0022] Figure 2 Schematic diagram of the framework structure of the image retrieval system based on center similarity provided in an embodiment of the present application;

[0023] Figure 3 This is a flowchart for implementing the first sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0024] Figure 4 This is a flowchart for implementing the second sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0025] Figure 5This is a flowchart for implementing the third sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0026] Figure 6 This is a flowchart for implementing the fourth sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0027] Figure 7 This is a flowchart for implementing the fifth sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0028] Figure 8 Schematic diagram of an image retrieval system based on center similarity provided in an embodiment of the present application;

[0029] Figure 9 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0031] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0034] It should be noted that the image retrieval method based on center similarity provided in the embodiment of the present application is generally executed by a server. Accordingly, the image retrieval system based on center similarity is generally configured in the server.

[0035] See also Figure 1 and Figure 2 , Figure 1 A specific implementation of the image retrieval method based on center similarity is shown. Figure 2 This is a schematic diagram of the framework structure of the image retrieval system based on center similarity provided in an embodiment of the present application.

[0036] It should be noted that the method of the present invention is not limited to the method of Figure 1 The process sequence shown is limited to the following steps:

[0037] S1: Obtain an image dataset, and use a Hadamard matrix and Bernoulli distribution to generate a hash center for each category in the image dataset.

[0038] Specifically, an image dataset is obtained from a preset dataset, where the preset dataset can be the CIFAR-10 dataset, the NUS-WIDE dataset, the COCO dataset, etc. The CIFAR-10 dataset is a single-label image dataset that contains 10 categories, and each category contains 6,000 images. The NUS-WIDE dataset is a multi-label image dataset that contains a total of 21 categories, and each image may belong to multiple categories. The COCO dataset is a popular image segmentation, recognition, and captioning dataset that contains 82,783 training images and 40,504 verification images, with a total of 80 categories, and each image may belong to multiple categories.

[0039] In the embodiment of the present application, the Hadamard matrix and Bernoulli distribution are used to generate the hash center of each category in the image data set. Among them, the Hadamard matrix is ​​a square matrix composed of binary values ​​1 and -1, and has the property that the row vector and the column vector are mutually orthogonal. The mutual orthogonality of two vectors indicates that their Hamming distance is large enough. Therefore, the Hadamard matrix can be used to generate a set of hash centers with a large Hamming distance between each two. Bernoulli distribution is a discrete probability distribution used to describe random experiments in which a single experiment has only two possible results (success or failure).

[0040] See also Figure 3 , Figure 3 A specific implementation of step S1 is shown, which is described in detail as follows:

[0041] S11: Acquire the image dataset.

[0042] S12: Construct the Hadamard matrix, and assign a column vector of the Hadamard matrix as the hash center to each category in the image dataset.

[0043] S13: If the number of categories in the image data set is greater than the length of the hash code, the Hadamard matrix is ​​expanded, and the column vectors are allocated to the unallocated categories based on the expanded Hadamard matrix as the hash centers.

[0044] S14: If there are still categories to be assigned in the image dataset, random sampling is performed using Bernoulli distribution to assign the categories to be assigned as the hash centers.

[0045] Specifically, we obtain the image dataset and construct 2 k Hadamard matrix of order:

[0046]

[0047] In the embodiment of the present application, a column vector of the Hadamard matrix is ​​assigned to each category as the hash center. When the number of categories in the image data set is greater than the length of the hash code, there may not be enough column vectors to assign, so The Hadamard matrix is ​​expanded, and the column vectors of the expanded Hadamard matrix are assigned to the unassigned categories as hash centers. If the column vectors of the expanded Hadamard matrix are still insufficient for assignment, that is, if there are still categories to be assigned in the image dataset, a Bernoulli distribution is used to randomly sample 1 and -1 values ​​for each bit to generate a hash center. This hash center is a binary hash center.

[0048] S2: Correct the hash center to obtain a target hash center.

[0049] Specifically, when using the Hadamard matrix and Bernoulli distribution to generate the hash center of each category in the image data set, there may be a situation where two hash centers are too close to each other. Therefore, in the embodiment of the present application, the hash centers need to be corrected to ensure that there is sufficient spacing between the hash centers.

[0050] See also Figure 4 , Figure 4 A specific implementation of step S2 is shown, which is described in detail as follows:

[0051] S21: If the image dataset is a single-label dataset, calculate the Hamming distance between the hash centers of each category to obtain a first Hamming distance.

[0052] S22: Taking the first Hamming distance that is smaller than the preset threshold as the second Hamming distance, and correcting the hash center corresponding to the second Hamming distance to obtain the target hash center.

[0053] S23: If the image dataset is a multi-label dataset, the number of 1s and -1s at the multiple hash center positions to which each category belongs is calculated to obtain the number of integer values ​​and the number of negative values.

[0054] S24: Correcting the hash center based on the comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

[0055] Specifically, when correcting the hash center, the embodiments of the present application adopt different correction methods for different image data sets. If the image data set is a single-label data set, the Hamming distance between the hash centers of each category is calculated to obtain the first Hamming distance; the first Hamming distance that is less than the preset threshold is used as the second Hamming distance, and the hash center corresponding to the second Hamming distance is corrected to obtain the target hash center. If the image data set is a multi-label data set, the number of 1s and -1s at the multiple hash center positions belonging to each category is calculated to obtain the number of integer values ​​and the number of negative values. The integer value is the number of 1s at the hash center position, and the negative value is the number of -1s at the hash center position. If the number of integer values ​​is greater than the number of negative values, the hash center at that position is set to 1; if the number of integer values ​​is less than the number of negative values, the hash center at that position is set to -1; if the number of integer values ​​is equal to the number of negative values, the hash center at that position is randomly set to 1 and -1.

[0056] S3: Construct a training dataset based on the image dataset, extract and activate features of each input image in the training dataset through a deep hash network, and obtain low-dimensional real-valued features.

[0057] Specifically, the image data in the image dataset is preprocessed to construct a training dataset. This preprocessing includes normalizing the image size. During model training, an optimizer is selected and model parameters such as weight decay, momentum, and learning rate are set. A deep hashing network is then used to extract and activate features for each input image in the training dataset using a convolutional neural network and fully connected layers, generating low-dimensional real-valued features.

[0058] See also Figure 5 , Figure 5 A specific implementation of step S3 is shown, which is described in detail as follows:

[0059] S31: Construct the training dataset based on the image dataset, and input the training dataset into the deep hash network.

[0060] S32: extracting features from each input image in the training data set by random sampling to obtain initial features.

[0061] S33: Activate the initial features through a hyperbolic tangent activation function to generate the low-dimensional real-valued features.

[0062] Specifically, a training dataset is constructed based on an image dataset and input into a deep hashing network. Initial features are extracted from each input image in the training dataset through random sampling. Because hash codes have binary constraints and cannot be directly back-propagated using gradients, the present embodiment uses a hyperbolic tangent activation function in the hash layer to learn low-dimensional real-valued features to approximate the hash code. Specifically, the initial features are activated using the hyperbolic tangent activation function to generate low-dimensional real-valued features.

[0063] S4: Calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and use back propagation to train the deep hash network based on the center similarity Bayesian loss to obtain a target retrieval model.

[0064] Specifically, the center similarity Bayesian loss is calculated by the low-dimensional real-valued features and the target hash center, and the center similarity Bayesian loss is used to drive the neural network to learn the hash code by maintaining the similarity between the low-dimensional real-valued features and the hash center, thereby training the deep hash network model and obtaining the target retrieval model.

[0065] In one specific embodiment, AlexNet was selected as the training model's base framework, and its parameters were initialized using weights pre-trained on ImageNet. During the training phase, SGD was used as the optimizer with a weight decay of 0.0005 and a momentum of 0.9. The initial learning rate was set to 0.01, which was reduced to 0.001 during training using a polynomial scheduling strategy.

[0066] See also Figure 6 , Figure 6 A specific implementation of step S4 is shown, which is described in detail as follows:

[0067] S41: Calculate the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and generate an inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product.

[0068] S42: Calculate the center similarity Bayesian loss based on the target inner product and the preset weight matrix using a preset formula, and calculate the angle loss and quantization loss of the Euclidean distance based on the low-dimensional real-valued features and the target hash center.

[0069] The preset formula is:

[0070]

[0071] Among them, J B is the model loss, θ ij is the target inner product, ω ij is the preset weight matrix, γ is a hyperparameter, C is the number of categories in the image dataset, N is the number of samples in the training dataset, and y ij is the j-th element value in the category label vector in one-hot encoding form.

[0072] S43: Calculate and generate model loss based on the center similarity Bayesian loss, the angle loss and the quantization loss.

[0073] S44: Adjust the model parameters of the deep hash network through model loss, and perform model iterative training on the adjusted deep hash network using back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0074] Specifically, the embodiment of the present application uses the center similarity Bayesian hash learning method to drive the neural network to learn hashes to generate a target retrieval model. The learning goal of the algorithm in the embodiment of the present application is to minimize the Hamming distance between the hash code and the hash center of its subordinate category in the Hamming space, and maximize the Hamming distance with the hash center of other categories. Based on this goal, the following objective function can be obtained:

[0075]

[0076] Among them, h i =Tanh(F(Θ,x i )) is the low-dimensional real-valued feature output by the hash layer of the neural network, Θ is the parameter of the neural network, Tanh(·) is the activation function, x i is the i-th input data or sample; G(h i ,l j ) represents the distance metric from the hash center, where l j is the jth hash center. H D H (h i ,l j ) is h i and l j The Hamming distance of y is also the third Hamming distance. ij is the category label vector y in one-hot encoding form i The j-th element value in is usually 0 or 1. C is the number of categories in the image dataset, and N is the number of samples in the training dataset.

[0077] Generate the inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain the target inner product. <hi ,l j > is inversely related to the third Hamming distance. Convert the above objective function (2) to:

[0078]

[0079] The transformed objective function (4) is learned by maximum a posteriori estimation, maximizing the following posterior probability:

[0080]

[0081] Among them, H, L, Y are the corresponding data h i , l i ,y i The collection of h i =Tanh(F(x i ,Θ)) is the real-valued feature after the neural network mapping, Θ is the parameter of the neural network, P(y ij |h i ,l i ) is the following likelihood function (5):

[0082]

[0083] Among them, σ(θ ij ) is a probability function used to convert θ ij Mapping to between 0 and 1 can be any function with a value range between 0 and 1 and monotonically increasing. The existing working method uses the following Sigmoid function (6):

[0084]

[0085] Substituting the Sigmoid function (6) into the objective function (4) converted from the above formula, we can obtain the loss function (7) of the current working mode:

[0086]

[0087] Among them, the category label Is a one-hot vector. For single-label data, y ij =1 and y ij = 0 is, according to the definition of likelihood function (5), for each sample, it will be pulled closer to its subordinate hash center once, and pulled away from other hash centers. This will cause the model learning direction to tend to move away from other hash centers and ignore the movement towards its subordinate hash centers. To solve this problem, this application designs a weight matrix (8), which is defined as follows:

[0088]

[0089] For moving the hash center farther away, a smaller weight is applied, while for moving closer, a larger weight is applied. Therefore, the weight matrix ω is ij Add J in the loss function (7) B Previously, the loss function of the center similarity Bayesian learning framework can be obtained (9):

[0090]

[0091] Existing work usually uses the Sigmoid function (6) to design the probability function of Bayesian learning. Substituting it into the center similarity Bayesian learning framework (9) can obtain the following loss function (10):

[0092]

[0093] Since the Sigmoid function (6) has the problem of too small gradient in the saturation region, the gradient may disappear during the back propagation process of the deep neural network, resulting in training difficulties. To solve this problem, the embodiment of the present application designs a probability function (11) whose gradient is easier to train:

[0094]

[0095] stx>0; (11)

[0096] Among them, γ is a hyperparameter. In order to make θ ij To meet the domain requirements of the probability function proposed in the embodiment of the present application, its value needs to be truncated, as shown in formula (12):

[0097] Φ ij =max(θ ij ,0.01); (12)

[0098] Substitute the probability function σ(x) of (11) and the formula Φ of (12) ij Substituting into the loss function (10), we get the loss function (13):

[0099]

[0100] By simplifying the loss function (13) and removing the constant term, the preset formula of the embodiment of the present application can be obtained:

[0101]

[0102] The purpose of the center similarity Bayesian loss is to maintain the similarity between the hash code and the hash center. In order to make the hash code sufficiently close to the hash center to which it belongs, it is further approximated from the perspective of Euclidean distance. The angle loss of Euclidean distance is calculated by formula (15):

[0103]

[0104] Since the model adopts a relaxation strategy during the training process, it is inevitable that a certain amount of quantization loss will occur. In order to reduce the error caused by the quantization process of the sign function, a quantization loss term is added. The quantization loss term is calculated using formula (16):

[0105]

[0106] By integrating the center similarity Bayesian loss, angle loss and quantization loss, we can get the final model loss of the model. The model loss is calculated by formula (17):

[0107] J=J B +λJ E +βJ Q ; (17)

[0108] Wherein, λ and β are both hyperparameters of the model. In a specific embodiment, the hyperparameters of CSBH are set to γ=100, λ=1 and β=0.01.

[0109] Finally, the model parameters of the deep hash network are adjusted by the model loss, and the adjusted deep hash network is iteratively trained by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0110] It should be noted that the preset number of iterations and the preset threshold are set according to actual conditions and are not limited here.

[0111] S5: Acquire an image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image.

[0112] Specifically, the target retrieval model has been trained in the above steps. Therefore, in actual application, the image to be retrieved is obtained, and the target retrieval model is used to retrieve images similar to the image to be retrieved from the hash code database to obtain the target similar image.

[0113] See also Figure 7 , Figure 7 A specific implementation of step S5 is shown, which is described in detail as follows:

[0114] S51: Acquire the image to be retrieved.

[0115] S52: Using the symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model.

[0116] S53: Extracting features of the image to be retrieved based on the new target retrieval model to obtain target real-valued features.

[0117] S54: Binary-encode the target real-valued feature using the symbolic function to obtain a target binary hash code.

[0118] S55: Calculate the Hamming distance between the target binary hash code and the hash codes of all images in the database to obtain a fourth Hamming distance.

[0119] S56: Determine the target similar image based on the fourth Hamming distance.

[0120] Specifically, an image to be retrieved is obtained; a symbolic function Sgn(·) is used as the activation function of the target retrieval model to obtain a new target retrieval model. Feature extraction is then performed on the image to be retrieved based on the new target retrieval model to obtain target real-valued features. The target real-valued features are then binary-encoded using the symbolic function to obtain a target binary hash code. The Hamming distance between the target binary hash code and the hash codes of all images in the database is calculated to obtain a fourth Hamming distance. If the fourth Hamming distance is less than a preset threshold, the image in the database corresponding to the calculated fourth Hamming distance is used as the target similarity result. The preset threshold is set based on actual conditions and is not limited here.

[0121] In a specific embodiment, the center-similarity-based image retrieval method and system of the present application can achieve a single image retrieval time of less than one thousandth of a second. For example, at 16 bits, the present application embodiment achieves a single image retrieval time of only 0.00052 seconds on the CIFAR-10 dataset; at 128 bits, the present application embodiment achieves a single image retrieval time of only 0.00087 seconds on the CIFAR-10 dataset.

[0122] In an embodiment of the present application, an image data set is obtained, and a Hadamard matrix and a Bernoulli distribution are used to generate a hash center for each category in the image data set; the hash center is corrected to obtain a target hash center; a training data set is constructed based on the image data set, and features of each input image in the training data set are extracted and activated through a deep hash network to obtain low-dimensional real-valued features; the center similarity Bayesian loss is calculated using the low-dimensional real-valued features and the target hash center, and the deep hash network is trained based on the center similarity Bayesian loss using back propagation to obtain a target retrieval model; an image to be retrieved is obtained, and the image to be retrieved is retrieved based on the target retrieval model to obtain a target similar image. The embodiment of the present invention uses a Hadamard matrix and a Bernoulli distribution to generate a hash center for each category in the image data set, and corrects the hash center to ensure that there is sufficient spacing between the hash centers. At the same time, the center similarity Bayesian loss is used for model training, which is conducive to reducing the difficulty and time of model training and improving image retrieval accuracy.

[0123] The present application proposes a unified center-similarity Hadamard matrix framework that generates hash centers for each category using a Hadamard matrix and Bernoulli distribution. A correction mechanism ensures sufficient spacing between hash centers, and Bayesian learning is used to maintain the similarity between hash codes and hash centers. This embodiment addresses the vanishing gradient problem in the saturation region of the Sigmoid function by designing a more easily trainable probability function for the center-similarity Bayesian learning framework. This in turn derives a center-similarity Bayesian hashing algorithm, which helps improve model training efficiency.

[0124] Please refer to Figure 8 , as a response to the above Figure 1 The present application provides an embodiment of an image retrieval system based on center similarity, which is similar to Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various computer devices.

[0125] like Figure 8 As shown, the image retrieval system based on center similarity of this embodiment includes: a hash center generation unit 61, a hash center correction unit 62, a feature extraction unit 63, a model training unit 64 and a target similar image retrieval unit 65, wherein:

[0126] A hash center generating unit 61 is used to obtain an image data set and generate a hash center for each category in the image data set using a Hadamard matrix and a Bernoulli distribution;

[0127] A hash center correction unit 62 is used to correct the hash center to obtain a target hash center;

[0128] A feature extraction unit 63 is configured to construct a training dataset based on the image dataset, extract and activate features of each input image in the training dataset through a deep hashing network, and obtain low-dimensional real-valued features;

[0129] A model training unit 64 is configured to calculate a center similarity Bayesian loss using the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss using a back-propagation method to obtain a target retrieval model;

[0130] The target similar image retrieval unit 65 is used to obtain the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain the target similar image.

[0131] Furthermore, the hash center generation unit 61 includes:

[0132] An image data set acquisition unit, configured to acquire the image data set;

[0133] a first hash center allocation unit, configured to construct the Hadamard matrix and allocate a column vector of the Hadamard matrix as the hash center to each category in the image dataset;

[0134] a second hash center allocation unit, configured to, if the number of categories in the image data set is greater than the length of the hash code, expand the Hadamard matrix, and allocate the column vector to the unallocated category based on the expanded Hadamard matrix as the hash center;

[0135] The first hash center allocation unit is configured to use Bernoulli distribution to perform random sampling if there are still categories to be allocated in the image data set, so as to allocate the categories to be allocated as the hash centers.

[0136] Furthermore, the hash center correction unit 62 includes:

[0137] a first Hamming distance calculation unit, configured to calculate the Hamming distance between the hash centers of each category to obtain a first Hamming distance if the image dataset is a single-label dataset;

[0138] A first correction unit is configured to use a first Hamming distance smaller than a preset threshold as a second Hamming distance, and correct a hash center corresponding to the second Hamming distance to obtain the target hash center;

[0139] a quantity calculation unit, configured to calculate the number of 1s and -1s at the multiple hash center positions to which each category belongs, if the image dataset is a multi-label dataset, to obtain the number of integer values ​​and the number of negative values;

[0140] The second correction unit is configured to correct the hash center based on a comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

[0141] Furthermore, the feature extraction unit 63 includes:

[0142] A training data set construction unit, configured to construct the training data set based on the image data set, and input the training data set into the deep hashing network;

[0143] An initial feature generation unit, configured to extract features from each input image in the training data set by random sampling to obtain initial features;

[0144] The low-dimensional real-valued feature generating unit is used to activate the initial feature through a hyperbolic tangent activation function to generate the low-dimensional real-valued feature.

[0145] Furthermore, the model training unit 64 includes:

[0146] a target inner product calculation unit, configured to calculate the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and generate an inner product between the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product;

[0147] A center similarity Bayesian loss calculation unit, configured to calculate the center similarity Bayesian loss based on the target inner product and a preset weight matrix using a preset formula, and to calculate the angle loss and quantization loss of the Euclidean distance based on the low-dimensional real-valued feature and the target hash center;

[0148] A model loss calculation unit, configured to calculate and generate a model loss based on the center similarity Bayesian loss, the angle loss, and the quantization loss;

[0149] The target retrieval model generation unit is used to adjust the model parameters of the deep hash network through model loss, and perform model iterative training on the adjusted deep hash network using back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0150] Furthermore, the preset formula is:

[0151]

[0152] Among them, J B is the model loss, θ ij is the target inner product, ω ij is the preset weight matrix, γ is a hyperparameter, C is the number of categories in the image dataset, N is the number of samples in the training dataset, and y ij is the j-th element value in the category label vector in one-hot encoding form.

[0153] Furthermore, the target similar image retrieval unit 65 includes:

[0154] A to-be-retrieved image acquisition unit, configured to acquire the to-be-retrieved image;

[0155] an activation function replacement unit, configured to use a symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model;

[0156] A target substantial feature acquisition unit is configured to extract features from the image to be retrieved using the new target retrieval model to obtain target real-valued features;

[0157] A binary encoding unit, configured to perform binary encoding on the target real-valued feature using the sign function to obtain a target binary hash code;

[0158] a calculating unit, configured to calculate a Hamming distance between the target binary hash code and hash codes of all images in a database to obtain a fourth Hamming distance;

[0159] A target similar image determining unit is configured to determine the target similar image based on the fourth Hamming distance.

[0160] To solve the above technical problems, the present application also provides a computer device. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0161] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that Figure 9 Only a computer device 7 having three components, memory 71, processor 72, and network interface 73, is shown. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead. It should be understood by those skilled in the art that a computer device herein is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0162] Computer devices can be desktop computers, laptops, PDAs, cloud servers, etc. Computer devices can interact with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0163] The memory 71 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as the hard disk or internal memory of the computer device 7. In other embodiments, the memory 71 may also be an external storage device of the computer device 7, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 7. Of course, the memory 71 may also include both the internal storage unit of the computer device 7 and its external storage devices. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as the program code of the center similarity-based image retrieval method. In addition, the memory 71 may also be used to temporarily store various types of data that have been output or are about to be output.

[0164] In some embodiments, the processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to execute program code stored in the memory 71 or process data, such as executing the program code of the aforementioned center similarity-based image retrieval method to implement various embodiments of the center similarity-based image retrieval method.

[0165] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0166] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores a computer program. The computer program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned image retrieval method based on center similarity.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of each embodiment of the present application.

[0168] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. An image retrieval method based on center similarity, characterized in that: include: Obtain an image dataset, and generate a hash center for each category in the image dataset using a Hadamard matrix and a Bernoulli distribution; Correcting the hash center to obtain a target hash center; Constructing a training dataset based on the image dataset, extracting and activating features of each input image in the training dataset through a deep hashing network to obtain low-dimensional real-valued features; Calculating the center similarity Bayesian loss by using the low-dimensional real-valued features and the target hash center, and training the deep hash network based on the center similarity Bayesian loss by backpropagation to obtain a target retrieval model; Acquire an image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image; The acquiring of the image dataset and generating the hash center of each category in the image dataset using the Hadamard matrix and Bernoulli distribution includes: Acquire the image dataset; Constructing the Hadamard matrix, and assigning a column vector of the Hadamard matrix as the hash center to each category in the image dataset; If the number of categories in the image data set is greater than the length of the hash code, the Hadamard matrix is ​​expanded, and the column vector is assigned to the unassigned category based on the expanded Hadamard matrix as the hash center; If there are still categories to be assigned in the image dataset, random sampling is performed using Bernoulli distribution to assign the categories to be assigned as the hash centers; The step of correcting the hash center to obtain a target hash center includes: If the image dataset is a single-label dataset, calculating the Hamming distance between the hash centers of each category to obtain a first Hamming distance; Taking the first Hamming distance smaller than the preset threshold as the second Hamming distance, and correcting the hash center corresponding to the second Hamming distance to obtain the target hash center; If the image dataset is a multi-label dataset, the number of 1s and -1s at the multiple hash center positions to which each category belongs is calculated to obtain the number of integer values ​​and the number of negative values; The hash center is corrected based on a comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

2. The image retrieval method based on center similarity according to claim 1, characterized in that: The step of constructing a training dataset based on the image dataset, extracting and activating features of each input image in the training dataset through a deep hashing network, and obtaining low-dimensional real-valued features includes: Constructing the training dataset based on the image dataset, and inputting the training dataset into the deep hashing network; Extract features from each input image in the training data set by random sampling to obtain initial features; The initial features are activated by a hyperbolic tangent activation function to generate the low-dimensional real-valued features.

3. The image retrieval method based on center similarity according to claim 1, characterized in that: The method calculates the center similarity Bayesian loss by using the low-dimensional real-valued features and the target hash center, and trains the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model, including: Calculating the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and generating an inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product; A preset formula is used to calculate the center similarity Bayesian loss based on the target inner product and a preset weight matrix, and an angle loss and a quantization loss of the Euclidean distance are calculated based on the low-dimensional real-valued feature and the target hash center; Calculating a generative model loss based on the center similarity Bayesian loss, the angle loss, and the quantization loss; The model parameters of the deep hash network are adjusted by the model loss, and the adjusted deep hash network is iteratively trained by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

4. The image retrieval method based on center similarity according to claim 3, characterized in that: The preset formula is: ； ； in, is the model loss, is the target inner product, is the preset weight matrix, is a hyperparameter, is the number of categories in the image dataset, N is the number of samples in the training dataset, is the j-th element value in the category label vector in one-hot encoding form.

5. The image retrieval method based on center similarity according to any one of claims 1 to 4, characterized in that: The step of obtaining an image to be retrieved and retrieving the image to be retrieved based on the target retrieval model to obtain a target similar image includes: Acquire the image to be retrieved; Using the symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model; Performing feature extraction on the image to be retrieved based on the new target retrieval model to obtain target real-valued features; Performing binary encoding on the target real-valued feature by using the sign function to obtain a target binary hash code; Calculating the Hamming distance between the target binary hash code and hash codes of all images in the database to obtain a fourth Hamming distance; The target similar image is determined based on the fourth Hamming distance.

6. An image retrieval system based on center similarity, characterized in that: include: A hash center generating unit, configured to obtain an image data set and generate a hash center for each category in the image data set using a Hadamard matrix and a Bernoulli distribution; A hash center correction unit, configured to correct the hash center to obtain a target hash center; A feature extraction unit is used to construct a training dataset based on the image dataset, extract and activate the features of each input image in the training dataset through a deep hash network, and obtain low-dimensional real-valued features; A model training unit is used to calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss using a back-propagation method to obtain a target retrieval model; A target similar image retrieval unit is used to obtain an image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image; The hash center generation unit includes: An image data set acquisition unit, configured to acquire the image data set; a first hash center allocation unit, configured to construct the Hadamard matrix and allocate a column vector of the Hadamard matrix as the hash center to each category in the image dataset; a second hash center allocation unit, configured to, if the number of categories in the image data set is greater than the length of the hash code, expand the Hadamard matrix, and allocate the column vector to the unallocated category based on the expanded Hadamard matrix as the hash center; a hash center allocating unit, configured to, if there are still categories to be allocated in the image dataset, perform random sampling using Bernoulli distribution to allocate the categories to be allocated as the hash centers; The hash center correction unit includes: a first Hamming distance calculation unit, configured to calculate the Hamming distance between the hash centers of each category to obtain a first Hamming distance if the image dataset is a single-label dataset; A first correction unit is configured to use a first Hamming distance smaller than a preset threshold as a second Hamming distance, and correct a hash center corresponding to the second Hamming distance to obtain the target hash center; a quantity calculation unit, configured to calculate the number of 1s and -1s at the multiple hash center positions to which each category belongs, if the image dataset is a multi-label dataset, to obtain the number of integer values ​​and the number of negative values; The quantity calculation unit is configured to correct the hash center based on a comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the image retrieval method based on center similarity according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the image retrieval method based on center similarity according to any one of claims 1 to 5.

Images