Image retrieval method, system and equipment based on center similarity and medium

Abstract

The invention relates to an image retrieval method, system and device based on center similarity and a medium, and the method comprises the steps: obtaining an image data set, and generating a Hash center of each category in the image data set by adopting a Hadamard matrix and Bernoulli distribution; correcting the hash center to obtain a target hash center; constructing a training data set based on the image data set, and extracting and activating features of each input image in the training data set through a deep hash network to obtain low-dimensional real value features; center similarity Bayesian loss is calculated through the low-dimensional real value features and the target Hash center, a deep Hash network is trained based on the center similarity Bayesian loss in a back propagation mode, and a target retrieval model is obtained; and obtaining a to-be-retrieved image, and retrieving the to-be-retrieved image based on the target retrieval model to obtain a target similar image. According to the method, sufficient spacing between the hash centers is ensured, the difficulty and time of model training are reduced, and the image retrieval accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image retrieval method, system, device and medium based on center similarity. Background Art

[0002] In recent years, with the rapid development of big data and artificial intelligence technology, hash learning has become the mainstream research direction of approximate nearest neighbor search, and has made significant progress both at home and abroad. Traditional hash learning methods can be divided into two categories based on whether data is needed for pre-training: data-independent hashing and data-dependent hashing. Data-independent hashing is a hash learning method that does not rely on training data. This type of method uses a randomly generated projection matrix to encode the data in binary values. There is a locality-sensitive hashing (LSH), which is a representative data-independent hashing algorithm that uses random linear projections to map nearby data to similar binary codes. Another kernelized locality-sensitive hashing (KLSH) adds a kernel function to LSH, which can better learn nonlinear features. However, there are two problems with this type of method. One is that it usually requires a very long hash code. An overly long hash code will lead to low retrieval efficiency and greatly increase the storage cost of the hash code; the other is that the retrieval accuracy is not high.

[0003] Due to the powerful ability of deep neural networks in feature extraction, many recent works have attempted to combine deep learning and hash learning, using neural networks to learn hash codes, and have achieved good retrieval results. However, most existing methods are based on pairwise similarity and triplet similarity for learning. Pairwise similarity-based hash learning focuses on learning the similarity relationship between pairs of data, which is local, and the learned hash model cannot capture global similarity information. In order to ensure that the training data has rich similarity information, the hash learning method based on triplet similarity needs to sample a large number of triplets from the training set, which will greatly increase the training time. In addition, due to the unclear hash center, the inter-class discrimination of the hash code is insufficient, resulting in low image retrieval accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an image retrieval method, system, device and medium based on center similarity to reduce the difficulty and time of model training, avoid the problem of insufficient inter-class discriminability of hash codes, and improve image retrieval accuracy.

[0005] In order to solve the above technical problems, the embodiment of the present application provides an image retrieval method based on center similarity, comprising:

[0006] Acquire an image data set, and use a Hadamard matrix and a Bernoulli distribution to generate a hash center for each category in the image data set;

[0007] Correcting the hash center to obtain a target hash center;

[0008] Building a training data set based on the image data set, extracting and activating features of each input image in the training data set through a deep hash network to obtain low-dimensional real-valued features;

[0009] The center similarity Bayesian loss is calculated by using the low-dimensional real-valued features and the target hash center, and the deep hash network is trained based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model;

[0010] Acquire the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image.

[0011] In order to solve the above technical problems, the embodiment of the present application provides an image retrieval system based on center similarity, including:

[0012] A hash center generating unit, used to obtain an image data set and generate a hash center of each category in the image data set using a Hadamard matrix and a Bernoulli distribution;

[0013] A hash center correction unit, used to correct the hash center to obtain a target hash center;

[0014] A feature extraction unit, configured to construct a training data set based on the image data set, extract and activate features of each input image in the training data set through a deep hash network, and obtain low-dimensional real-valued features;

[0015] A model training unit, used to calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model;

[0016] The target similar image retrieval unit is used to obtain the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain the target similar image.

[0017] To solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that the one or more processors can implement any one of the above-mentioned image retrieval methods based on center similarity.

[0018] To solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned image retrieval methods based on center similarity is implemented.

[0019] The embodiment of the present invention provides an image retrieval method, system, device and medium based on center similarity. The method includes: obtaining an image data set, and using Hadamard matrix and Bernoulli distribution to generate the hash center of each category in the image data set; correcting the hash center to obtain the target hash center; constructing a training data set based on the image data set, extracting and activating the features of each input image in the training data set through a deep hash network to obtain low-dimensional real-valued features; calculating the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and training the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model; obtaining an image to be retrieved, and retrieving the image to be retrieved based on the target retrieval model to obtain a target similar image. The embodiment of the present invention uses Hadamard matrix and Bernoulli distribution to generate the hash center of each category in the image data set, and corrects the hash center to ensure that there is sufficient spacing between the hash centers. At the same time, the center similarity Bayesian loss is used for model training, which is conducive to reducing the difficulty and time of model training and improving the accuracy of image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a flowchart of the implementation process of the image retrieval method based on center similarity provided in an embodiment of the present application;

[0022] Figure 2 Schematic diagram of the framework structure of the image retrieval system based on center similarity provided in the embodiment of the present application;

[0023] Figure 3 is a flowchart for implementing the first sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0024] Figure 4 is a flowchart for implementing the second sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0025] Figure 5is a flowchart for implementing the third sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0026] Figure 6 is a flowchart for implementing the fourth sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0027] Figure 7 is a flowchart for implementing the fifth sub-process in the image retrieval method based on center similarity provided in an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of an image retrieval system based on center similarity provided in an embodiment of the present application;

[0029] Fig. 9 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0031] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0034] It should be noted that the image retrieval method based on center similarity provided in the embodiment of the present application is generally executed by a server, and accordingly, the image retrieval system based on center similarity is generally configured in the server.

[0035] See also Figure 1 and Figure 2 , Figure 1 A specific implementation of the image retrieval method based on center similarity is shown. Figure 2 It is a schematic diagram of the framework structure of the image retrieval system based on center similarity provided in an embodiment of the present application.

[0036] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited to the following steps:

[0037] S1: Obtain an image data set, and use a Hadamard matrix and Bernoulli distribution to generate a hash center for each category in the image data set.

[0038] Specifically, an image dataset is obtained from a preset dataset, wherein the preset dataset may be a CIFAR-10 dataset, a NUS-WIDE dataset, a COCO dataset, and the like. The CIFAR-10 dataset is a single-label image dataset, which contains 10 categories, and each category contains 6,000 images. The NUS-WIDE dataset is a multi-label image dataset, which contains a total of 21 categories, and each image may belong to multiple categories. The COCO dataset is a popular image segmentation, recognition, and captioning dataset, which contains 82,783 training images and 40,504 verification images, a total of 80 categories, and each image may belong to multiple categories.

[0039] In the embodiment of the present application, the Hadamard matrix and Bernoulli distribution are used to generate the hash center of each category in the image data set. Among them, the Hadamard matrix is ​​a square matrix composed of 1 and -1 binary values, and has the property of orthogonality between row vectors and column vectors. The mutual orthogonality of two vectors indicates that their Hamming distance is large enough, so the Hadamard matrix can be used to generate a set of hash centers with a large enough Hamming distance between each other. Bernoulli distribution is a discrete probability distribution used to describe random experiments with only two possible results (success or failure) in a single experiment.

[0040] See also Figure 3 , Figure 3 A specific implementation of step S1 is shown, which is described in detail as follows:

[0041] S11: Acquire the image dataset.

[0042] S12: construct the Hadamard matrix, and assign a column vector of the Hadamard matrix as the hash center to each category in the image data set.

[0043] S13: If the number of categories in the image data set is greater than the length of the hash code, the Hadamard matrix is ​​expanded, and the column vectors are allocated to the unallocated categories based on the expanded Hadamard matrix as the hash centers.

[0044] S14: If there are still categories to be assigned in the image data set, random sampling is performed using Bernoulli distribution to assign the categories to be assigned as the hash centers.

[0045] Specifically, we obtain an image dataset and construct 2 using the following formula (1): k Hadamard matrix of order:

[0046]

[0047] In the embodiment of the present application, a column vector of the Hadamard matrix is ​​assigned to each category as the hash center. When the number of categories in the image data set is greater than the length of the hash code, there may not be enough column vectors to assign, so The Hadamard matrix is ​​expanded, and then the expanded Hadamard matrix is ​​used to allocate the column vector to the unassigned categories as the hash center. If the column vector of the expanded Hadamard matrix is ​​still insufficient for allocation, that is, there are still categories to be allocated in the image data set, the Bernoulli distribution is used to randomly sample the values ​​of 1 and -1 on each bit to generate a hash center, which is a binary hash center.

[0048] S2: Correct the hash center to obtain a target hash center.

[0049] Specifically, when using the Hadamard matrix and Bernoulli distribution to generate the hash center of each category in the image data set, there may be a situation where two hash centers are too close to each other. Therefore, in the embodiment of the present application, the hash centers need to be corrected to ensure that there is sufficient spacing between the hash centers.

[0050] See also Figure 4 , Figure 4 A specific implementation of step S2 is shown, which is described in detail as follows:

[0051] S21: If the image dataset is a single-label dataset, the Hamming distance between the hash centers of each category is calculated to obtain a first Hamming distance.

[0052] S22: Taking the first Hamming distance that is smaller than a preset threshold as the second Hamming distance, and correcting the hash center corresponding to the second Hamming distance to obtain the target hash center.

[0053] S23: If the image dataset is a multi-label dataset, the number of 1s and -1s at the multiple hash center positions to which each category belongs is calculated to obtain the number of integer values ​​and the number of negative values.

[0054] S24: Correcting the hash center based on the comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

[0055] Specifically, when correcting the hash center, the embodiments of the present application adopt different correction methods for different image data sets. If the image data set is a single-label data set, the Hamming distance between the hash centers of each category is calculated to obtain the first Hamming distance; the first Hamming distance that is less than the preset threshold is used as the second Hamming distance, and the hash center corresponding to the second Hamming distance is corrected to obtain the target hash center. If the image data set is a multi-label data set, the number of 1s and -1s at the multiple hash center positions to which each category belongs is calculated to obtain the number of integer values ​​and the number of negative values. The integer value is the number of 1s at the hash center position, and the number of negative values ​​is the number of -1s at the hash center position. If the number of integer values ​​is greater than the number of negative values, the hash center at that position is set to 1; if the number of integer values ​​is less than the number of negative values, the hash center at that position is set to -1; if the number of integer values ​​is equal to the number of negative values, the hash center at that position is randomly set to 1 and -1.

[0056] S3: construct a training data set based on the image data set, extract and activate the features of each input image in the training data set through a deep hash network, and obtain low-dimensional real-valued features.

[0057] Specifically, the image data in the image dataset is preprocessed to construct a training dataset. Preprocessing includes unifying the image size. When training the model, it is necessary to select an optimizer and set model parameters such as weight decay, momentum, and learning rate. Then, the deep hash network is used to extract and activate the features of each input image in the training dataset based on the convolutional neural network and the fully connected layer to obtain low-dimensional real-valued features.

[0058] See also Figure 5 , Figure 5 A specific implementation of step S3 is shown, which is described in detail as follows:

[0059] S31: construct the training dataset based on the image dataset, and input the training dataset into the deep hash network.

[0060] S32: extracting features from each input image in the training data set by random sampling to obtain initial features.

[0061] S33: Activate the initial features through a hyperbolic tangent activation function to generate the low-dimensional real-valued features.

[0062] Specifically, a training data set is constructed based on an image data set, and the training data set is input into a deep hash network; features are extracted from each input image in the training data set by random sampling to obtain initial features. Since the hash code has a binary constraint and cannot be directly back-propagated based on gradients, the embodiment of the present application uses a hyperbolic tangent activation function in the hash layer to learn low-dimensional real-valued features to approximate hash codes, that is, the initial features are activated by the hyperbolic tangent activation function to generate low-dimensional real-valued features.

[0063] S4: Calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model.

[0064] Specifically, the center similarity Bayesian loss is calculated by the low-dimensional real-valued features and the target hash center, and the center similarity Bayesian loss is used to drive the neural network to learn the hash code by maintaining the similarity between the low-dimensional real-valued features and the hash center, thereby training the deep hash network model to obtain the target retrieval model.

[0065] In a specific embodiment, AlexNet is selected as the basic skeleton of the training model, and its parameters are initialized using weights pre-trained on ImageNet. During the training phase, SGD is used as the optimizer, with a weight decay of 0.0005 and a momentum of 0.9. The initial learning rate is set to 0.01, which is reduced to 0.001 during the training process through a polynomial scheduling strategy.

[0066] See also Figure 6 , Figure 6 A specific implementation of step S4 is shown, which is described in detail as follows:

[0067] S41: Calculate the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and generate an inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product.

[0068] S42: Calculate the center similarity Bayesian loss based on the target inner product and the preset weight matrix using a preset formula, and calculate the angle loss and quantization loss of the Euclidean distance based on the low-dimensional real-valued features and the target hash center.

[0069] The preset formula is:

[0070]

[0071] Among them, J B is the model loss, θ ij is the target inner product, ω ij is the preset weight matrix, γ is a hyperparameter, C is the number of categories in the image dataset, N is the number of samples in the training dataset, and y ij is the j-th element value in the category label vector in one-hot encoding form.

[0072] S43: Calculate and generate model loss based on the center similarity Bayesian loss, the angle loss and the quantization loss.

[0073] S44: Adjust the model parameters of the deep hash network through model loss, and perform model iterative training on the adjusted deep hash network by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0074] Specifically, the embodiment of the present application uses the center similarity Bayesian hash learning method to drive the neural network to learn hashes to generate a target retrieval model. The learning goal of the algorithm in the embodiment of the present application is to minimize the Hamming distance between the hash code and the hash center of its subordinate category in the Hamming space, and maximize the Hamming distance with the hash center of other categories. According to this goal, the following objective function can be obtained:

[0075]

[0076] Among them, h i =Tanh(F(Θ,x i )) is the low-dimensional real-valued feature output by the hash layer of the neural network, Θ is the parameter of the neural network, Tanh(·) is the activation function, x i is the i-th input data or sample; G(h i ,l j ) represents the distance measure from the hash center, where l j is the jth hash center. H D H (h i ,l j ) is h i and l j The Hamming distance of y is also the third Hamming distance. ij is the category label vector y in one-hot encoding form i The jth element value in is usually 0 or 1. C is the number of categories in the image dataset, and N is the number of samples in the training dataset.

[0077] Generate the inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain the target inner product. <hi ,l j > is inversely related to the third Hamming distance. Convert the above objective function (2) to:

[0078]

[0079] The transformed objective function (4) is learned by maximum a posteriori estimation, maximizing the following posterior probability:

[0080]

[0081] Among them, H, L, Y are the corresponding data h i , l i ,y i The collection of h i =Tanh(F(x i ,Θ)) is the real-valued feature after the neural network mapping, Θ is the parameter of the neural network, P(y ij |h i ,l i ) is the following likelihood function (5):

[0082]

[0083] Among them, σ(θ ij ) is a probability function used to convert θ ij Mapped to between 0 and 1, it can be any function with a value range between 0 and 1 and monotonically increasing. The existing working method uses the following Sigmoid function (6):

[0084]

[0085] Substituting the Sigmoid function (6) into the objective function (4) converted by the above formula, we can obtain the loss function (7) of the current working mode:

[0086]

[0087] Among them, the category label is a one-hot vector. For single-label data, y ij =1 and y ij =0 is, according to the definition of likelihood function (5), for each sample, it will be pulled closer to its subordinate hash center once, and pulled away from other hash centers. This will cause the model learning direction to tend to move away from other hash centers and ignore the pull-in to its subordinate hash center. In order to solve this problem, the present application designs a weight matrix (8), which is defined as follows:

[0088]

[0089] A smaller weight is applied to the hash center when it is farther away, and a larger weight is applied to the hash center when it is closer, so the weight matrix ω is ij Add J in the loss function (7) B Previously, the loss function of the center similarity Bayesian learning framework can be obtained (9):

[0090]

[0091] Existing work usually uses the Sigmoid function (6) to design the probability function of Bayesian learning. Substituting it into the center similarity Bayesian learning framework (9) can obtain the following loss function (10):

[0092]

[0093] Since the Sigmoid function (6) has the problem of too small gradient in the saturation region, the gradient may disappear during the back propagation process of the deep neural network, resulting in difficulty in training. In order to solve this problem, the embodiment of the present application designs a probability function (11) whose gradient is easier to train:

[0094]

[0095] stx>0; (11)

[0096] Among them, γ is a hyperparameter. In order to make θ ij To meet the domain requirement of the probability function proposed in the embodiment of the present application, its value needs to be truncated, as shown in formula (12):

[0097] Φ ij =max(θ ij ,0.01); (12)

[0098] Substituting σ(x) of probability function (11) and Φ of formula (12) ij Substituting into the loss function (10), we get the loss function (13):

[0099]

[0100] By simplifying the loss function (13) and removing the constant term, the preset formula of the embodiment of the present application can be obtained:

[0101]

[0102] The purpose of the center similarity Bayesian loss is to maintain the similarity between the hash code and the hash center. In order to make the hash code sufficiently close to its subordinate hash center, it is further approximated from the perspective of Euclidean distance. The angle loss of Euclidean distance is calculated by formula (15):

[0103]

[0104] Since the model adopts a relaxation strategy during the training process, it is inevitable that a certain amount of quantization loss will occur. In order to reduce the error caused by the quantization process of the sign function, a quantization loss term is added. The quantization loss term is calculated using formula (16):

[0105]

[0106] By integrating the center similarity Bayesian loss, angle loss and quantization loss, we can get the final model loss of the model. The model loss is calculated by formula (17):

[0107] J=J B +λJ E +βJ Q ; (17)

[0108] Wherein, λ and β are both hyperparameters of the model. In a specific embodiment, the hyperparameters of CSBH are set to γ=100, λ=1 and β=0.01.

[0109] Finally, the model parameters of the deep hash network are adjusted by the model loss, and the adjusted deep hash network is iteratively trained by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0110] It should be noted that the preset number of iterations and the preset threshold are set according to actual conditions and are not limited here.

[0111] S5: Acquire the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image.

[0112] Specifically, the above steps have trained the target retrieval model, so in the actual application process, the image to be retrieved is obtained, and the target retrieval model is used to retrieve images similar to the image to be retrieved from the hash code database to obtain the target similar image.

[0113] See also Figure 7 , Figure 7 A specific implementation of step S5 is shown, which is described in detail as follows:

[0114] S51: Acquire the image to be retrieved.

[0115] S52: Use the symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model.

[0116] S53: Extract features of the image to be retrieved based on the new target retrieval model to obtain target real-valued features.

[0117] S54: Binary-encode the target real-valued feature through the symbolic function to obtain a target binary hash code.

[0118] S55: Calculate the Hamming distance between the target binary hash code and the hash codes of all images in the database to obtain a fourth Hamming distance.

[0119] S56: Determine the target similar image based on the fourth Hamming distance.

[0120] Specifically, obtain the image to be retrieved; use the symbolic function Sgn(·) as the activation function of the target retrieval model to obtain a new target retrieval model. Then, based on the new target retrieval model, perform feature extraction on the image to be retrieved to obtain the target real-valued features. Then, perform binary encoding on the target real-valued features through the symbolic function to obtain the target binary hash code. Calculate the Hamming distance between the target binary hash code and the hash codes of all images in the database to obtain the fourth Hamming distance. If the fourth Hamming distance is less than the preset threshold, the image in the database corresponding to the fourth Hamming distance is calculated as the target similarity result. The preset threshold is set according to the actual situation and is not limited here.

[0121] In a specific embodiment, the image retrieval method and system based on center similarity of the embodiment of the present application can achieve a single image retrieval time of less than one thousandth of a second. For example, at 16 bits, the retrieval time required for a single image on the CIFAR-10 dataset by the embodiment of the present application is only 0.00052 seconds; at 128 bits, the retrieval time required for a single image on the CIFAR-10 dataset by the embodiment of the present application is only 0.00087 seconds.

[0122] In an embodiment of the present application, an image data set is obtained, and a Hadamard matrix and a Bernoulli distribution are used to generate a hash center for each category in the image data set; the hash center is corrected to obtain a target hash center; a training data set is constructed based on the image data set, and the features of each input image in the training data set are extracted and activated through a deep hash network to obtain low-dimensional real-valued features; the center similarity Bayesian loss is calculated through the low-dimensional real-valued features and the target hash center, and the deep hash network is trained based on the center similarity Bayesian loss using a back-propagation method to obtain a target retrieval model; an image to be retrieved is obtained, and the image to be retrieved is retrieved based on the target retrieval model to obtain a target similar image. The embodiment of the present invention uses a Hadamard matrix and a Bernoulli distribution to generate a hash center for each category in the image data set, and corrects the hash center to ensure that there is sufficient spacing between the hash centers. At the same time, the center similarity Bayesian loss is used for model training, which is conducive to reducing the difficulty and time of model training and improving image retrieval accuracy.

[0123] The embodiment of the present application proposes a unified center similarity Hadamard matrix framework, which generates hash centers for each category through Hadamard matrix and Bernoulli distribution, ensures sufficient spacing between hash centers through correction mechanism, and uses Bayesian learning to maintain the similarity between hash codes and hash centers. The embodiment of the present application designs a probability function that is easier to train for the center similarity Bayesian learning framework to address the problem of gradient vanishing in the saturation region of the Sigmoid function, and then derives the center similarity Bayesian hashing algorithm, which is conducive to improving the efficiency of model training.

[0124] Please refer to Figure 8 , as a response to the above Figure 1 The present application provides an embodiment of an image retrieval system based on center similarity, and the system embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various computer devices.

[0125] like Figure 8 As shown, the image retrieval system based on center similarity of this embodiment includes: a hash center generation unit 61, a hash center correction unit 62, a feature extraction unit 63, a model training unit 64 and a target similar image retrieval unit 65, wherein:

[0126] A hash center generating unit 61 is used to obtain an image data set and generate a hash center of each category in the image data set using a Hadamard matrix and a Bernoulli distribution;

[0127] A hash center correction unit 62, used to correct the hash center to obtain a target hash center;

[0128] A feature extraction unit 63 is used to construct a training data set based on the image data set, extract and activate the features of each input image in the training data set through a deep hash network, and obtain a low-dimensional real-valued feature;

[0129] A model training unit 64 is used to calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model;

[0130] The target similar image retrieval unit 65 is used to obtain the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain the target similar image.

[0131] Further, the hash center generating unit 61 includes:

[0132] An image data set acquisition unit, used to acquire the image data set;

[0133] A first hash center allocation unit, configured to construct the Hadamard matrix and allocate a column vector of the Hadamard matrix as the hash center for each category in the image data set;

[0134] a second hash center allocation unit, configured to expand the Hadamard matrix if the number of categories in the image data set is greater than the length of the hash code, and allocate the column vector to the unallocated category based on the expanded Hadamard matrix as the hash center;

[0135] The first hash center allocation unit is used to use Bernoulli distribution to perform random sampling if there are still categories to be allocated in the image data set, so as to allocate the categories to be allocated as the hash centers.

[0136] Further, the hash center correction unit 62 includes:

[0137] A first Hamming distance calculation unit, configured to calculate the Hamming distance between the hash centers of each category to obtain a first Hamming distance if the image data set is a single-label data set;

[0138] A first correction unit, configured to use a first Hamming distance less than a preset threshold as a second Hamming distance, and correct a hash center corresponding to the second Hamming distance to obtain the target hash center;

[0139] a quantity calculation unit, configured to calculate the number of 1s and -1s at the multiple hash center positions to which each category belongs, if the image data set is a multi-label data set, to obtain the number of integer values ​​and the number of negative values;

[0140] The second correction unit is used to correct the hash center based on the comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

[0141] Further, the feature extraction unit 63 includes:

[0142] A training data set construction unit, configured to construct the training data set based on the image data set, and input the training data set into the deep hash network;

[0143] An initial feature generation unit, used for extracting features from each input image in the training data set by random sampling to obtain initial features;

[0144] The low-dimensional real-valued feature generating unit is used to activate the initial feature through a hyperbolic tangent activation function to generate the low-dimensional real-valued feature.

[0145] Further, the model training unit 64 includes:

[0146] a target inner product calculation unit, configured to calculate the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and to generate an inner product between the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product;

[0147] A center similarity Bayesian loss calculation unit, used to calculate the center similarity Bayesian loss based on the target inner product and a preset weight matrix using a preset formula, and to calculate the angle loss and quantization loss of the Euclidean distance based on the low-dimensional real-valued feature and the target hash center;

[0148] A model loss calculation unit, used to calculate and generate a model loss based on the center similarity Bayesian loss, the angle loss and the quantization loss;

[0149] The target retrieval model generation unit is used to adjust the model parameters of the deep hash network through the model loss, and perform model iterative training on the adjusted deep hash network by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

[0150] Furthermore, the preset formula is:

[0151]

[0152] Among them, J B is the model loss, θ ij is the target inner product, ω ij is the preset weight matrix, γ is a hyperparameter, C is the number of categories in the image dataset, N is the number of samples in the training dataset, and y ij is the j-th element value in the category label vector in one-hot encoding form.

[0153] Furthermore, the target similar image retrieval unit 65 includes:

[0154] A to-be-retrieved image acquisition unit, used for acquiring the to-be-retrieved image;

[0155] an activation function replacement unit, used to use a symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model;

[0156] A target substantial feature acquisition unit is used to extract features of the image to be retrieved based on the new target retrieval model to obtain target real-valued features;

[0157] A binary encoding unit, used for performing binary encoding on the target real-valued feature through the sign function to obtain a target binary hash code;

[0158] a calculating unit, configured to calculate a Hamming distance between the target binary hash code and hash codes of all images in a database to obtain a fourth Hamming distance;

[0159] A target similar image determining unit is used to determine the target similar image based on the fourth Hamming distance.

[0160] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0161] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 which are interconnected and communicated with each other through a system bus. It should be noted that: Fig. 9 Only a computer device 7 having three components, memory 71, processor 72, and network interface 73, is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0162] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0163] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk equipped on the computer device 7, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as the program code of the image retrieval method based on center similarity, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0164] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run the program code stored in the memory 71 or process data, such as running the program code of the above-mentioned image retrieval method based on center similarity to implement various embodiments of the image retrieval method based on center similarity.

[0165] The network interface 73 may include a wireless network interface or a wired network interface, and the network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0166] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores a computer program. The computer program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned image retrieval method based on center similarity.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0168] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An image retrieval method based on center similarity, characterized in that: include: Acquire an image data set, and use a Hadamard matrix and a Bernoulli distribution to generate a hash center for each category in the image data set; Correcting the hash center to obtain a target hash center; Building a training data set based on the image data set, extracting and activating features of each input image in the training data set through a deep hash network to obtain low-dimensional real-valued features; The center similarity Bayesian loss is calculated by using the low-dimensional real-valued features and the target hash center, and the deep hash network is trained based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model; Acquire the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain a target similar image.

2. The image retrieval method based on center similarity according to claim 1, characterized in that: The step of acquiring an image data set and using a Hadamard matrix and a Bernoulli distribution to generate a hash center of each category in the image data set includes: Acquire the image dataset; Constructing the Hadamard matrix, and assigning a column vector of the Hadamard matrix as the hash center to each category in the image data set; If the number of categories in the image data set is greater than the length of the hash code, the Hadamard matrix is ​​expanded, and the column vector is allocated to the unallocated category based on the expanded Hadamard matrix as the hash center; If there are still categories to be assigned in the image data set, random sampling is performed using Bernoulli distribution to assign the categories to be assigned as the hash centers.

3. The image retrieval method based on center similarity according to claim 1, characterized in that: The step of correcting the hash center to obtain a target hash center includes: If the image dataset is a single-label dataset, the Hamming distance between the hash centers of each category is calculated to obtain a first Hamming distance; Taking the first Hamming distance that is smaller than the preset threshold as the second Hamming distance, and correcting the hash center corresponding to the second Hamming distance to obtain the target hash center; If the image dataset is a multi-label dataset, the number of 1s and -1s at the multiple hash center positions to which each category belongs is calculated to obtain the number of integer values ​​and the number of negative values; The hash center is corrected based on the comparison result of the number of integer values ​​and the number of negative values ​​to obtain the target hash center.

4. The image retrieval method based on center similarity according to claim 1, characterized in that: The step of constructing a training data set based on the image data set, extracting and activating features of each input image in the training data set through a deep hash network, and obtaining low-dimensional real-valued features includes: Construct the training data set based on the image data set, and input the training data set into the deep hashing network; Extracting features from each input image in the training data set by random sampling to obtain initial features; The initial feature is activated by a hyperbolic tangent activation function to generate the low-dimensional real-valued feature.

5. The image retrieval method based on center similarity according to claim 1, characterized in that: The method calculates the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and trains the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model, including: Calculate the Hamming distance between each of the low-dimensional real-valued features and the target hash center to obtain a third Hamming distance, and generate an inner product of the low-dimensional real-valued feature and the target hash center based on the third Hamming distance to obtain a target inner product; A preset formula is used to calculate the center similarity Bayesian loss based on the target inner product and a preset weight matrix, and an angle loss and a quantization loss of the Euclidean distance are calculated based on the low-dimensional real-valued feature and the target hash center; Calculating a generation model loss based on the center similarity Bayesian loss, the angle loss, and the quantization loss; The model parameters of the deep hash network are adjusted by model loss, and the adjusted deep hash network is iteratively trained by back propagation until a preset number of iterations is obtained or the new model loss is lower than a preset threshold, thereby obtaining the target retrieval model.

6. The image retrieval method based on center similarity according to claim 5, characterized in that: The preset formula is: Among them, J B is the model loss, θ ij is the target inner product, ω ij is the preset weight matrix, γ is a hyperparameter, C is the number of categories in the image dataset, N is the number of samples in the training dataset, and y ij is the j-th element value in the category label vector in one-hot encoding form.

7. The image retrieval method based on center similarity according to any one of claims 1 to 6, characterized in that: The step of obtaining the image to be retrieved and retrieving the image to be retrieved based on the target retrieval model to obtain a target similar image includes: Acquire the image to be retrieved; Using the symbolic function as the activation function of the target retrieval model to obtain a new target retrieval model; Based on the new target retrieval model, feature extraction is performed on the image to be retrieved to obtain target real-valued features; Binary encoding the target real-valued feature by using the symbolic function to obtain a target binary hash code; Calculating the Hamming distance between the target binary hash code and the hash codes of all images in the database to obtain a fourth Hamming distance; The target similar image is determined based on the fourth Hamming distance.

8. An image retrieval system based on center similarity, characterized in that: include: A hash center generating unit, used to obtain an image data set and generate a hash center of each category in the image data set using a Hadamard matrix and a Bernoulli distribution; A hash center correction unit, used to correct the hash center to obtain a target hash center; A feature extraction unit, configured to construct a training data set based on the image data set, extract and activate features of each input image in the training data set through a deep hash network, and obtain low-dimensional real-valued features; A model training unit, used to calculate the center similarity Bayesian loss through the low-dimensional real-valued features and the target hash center, and train the deep hash network based on the center similarity Bayesian loss by back propagation to obtain a target retrieval model; The target similar image retrieval unit is used to obtain the image to be retrieved, and retrieve the image to be retrieved based on the target retrieval model to obtain the target similar image.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the image retrieval method based on center similarity as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image retrieval method based on center similarity according to any one of claims 1 to 7 is implemented.

Images