Depth collaborative graph Hash image retrieval method based on attention enhancement

Abstract

The invention relates to the technical field of computer vision and image processing, and provides a depth synergy graph Hash image retrieval method based on attention enhancement, which comprises the following steps: acquiring a to-be-retrieved image; extracting image features of the to-be-retrieved picture; performing weighting processing on the image features by using an attention mechanism to obtain weighted image features; encoding the weighted image features into a hash code; hash codes of pre-stored images in an image database are obtained, Hamming distances between the Hash codes of the to-be-retrieved images and the Hash codes of the pre-stored images are calculated, the pre-stored images are sorted according to the sequence of the Hamming distances between the Hash codes of the to-be-retrieved images and the Hash codes of the pre-stored images from small to large, and the first multiple pre-stored images are output to serve as similar images of the to-be-retrieved images; the Hash codes obtained through the image retrieval method are high in discrimination capability and high in image retrieval precision.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and more specifically, to a deep collaborative graph hashing image retrieval method based on attention enhancement. Background Art

[0002] With the development of Internet technology, especially the popularity of social networks and mobile devices, the amount of image data has exploded. This sharp increase in data volume has brought challenges to storage systems, and also posed unprecedented difficulties in how to quickly access and find required images.

[0003] A new method named Deep Collaborative Graph Hashing (DCGH) is proposed in the prior art to improve the efficiency and accuracy of large-scale image retrieval. The DCGH method captures the complex relationships between images by constructing a deep collaborative graph and learns effective hash codes. The method considers multi-level semantic embedding, latent common space construction, and intrinsic structure mining to achieve discriminative hash code learning.

[0004] Although the dual-stream feature encoding mechanism of the DCGH model integrates visual and semantic features, the refinement of feature extraction still needs to be improved. Specifically, the model fails to fully capture and utilize the inter-channel differences and spatial layout information of image data when processing images, which is crucial for distinguishing different pictures, especially those with subtle differences or complex backgrounds. The traditional convolutional neural network (CNN), as the core of the visual image representation learning module, ignores the uneven contribution of different regions in the image to the recognition performance through its global consistency processing method. For example, the recognition weights of key areas such as detail features and edge information should be higher. In addition, the global consistency processing method may cause the extracted features to contain a large amount of redundant or noisy information, thereby affecting the discriminative power and retrieval accuracy of the hash code, which limits the model's feature differentiation ability in complex scenes.

[0005] In summary, the existing technology has the problems of poor discrimination of hash codes and low image retrieval accuracy. Summary of the invention

[0006] In order to overcome the defects of the above-mentioned prior art in that the hash code has poor discriminability and low image retrieval accuracy, the present invention provides a deep collaborative graph hashing image retrieval method based on attention enhancement, which has strong hash code discriminability and high image retrieval accuracy.

[0007] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0008] Get the image to be retrieved;

[0009] Extracting image features of the image to be retrieved;

[0010] Performing weighted processing on the image features by using an attention mechanism to obtain weighted image features;

[0011] Encoding the weighted image features into a hash code;

[0012] Obtain hash codes of pre-stored images in an image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, sort the pre-stored images in ascending order of the Hamming distance between the two, and output the first several pre-stored images as similar images of the image to be retrieved.

[0013] The present invention also proposes a deep collaborative graph hash image retrieval system based on attention enhancement for implementing the above-mentioned deep collaborative graph hash image retrieval method based on attention enhancement, and the system comprises:

[0014] Image acquisition module, used to obtain the image to be retrieved;

[0015] An image feature generation module, used to extract image features of the image to be retrieved;

[0016] An image feature weighting module, used to perform weighted processing on the image features using an attention mechanism to obtain weighted image features;

[0017] An encoding module, used for encoding the weighted image features into a hash code;

[0018] The retrieval module is used to obtain the hash code of the pre-stored image in the image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, and sort the pre-stored images in the order of the Hamming distance between the two from small to large, and output the first several pre-stored images as similar images of the image to be retrieved.

[0019] The present invention also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the steps of the above-mentioned attention-enhanced deep collaborative graph hashing image retrieval method.

[0020] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0021] The present invention uses the attention mechanism to adaptively focus on important areas and channels in the image to be retrieved when processing image features, while suppressing the influence of noise and irrelevant information, thereby extracting more discriminative features. This enhanced feature representation helps to generate higher quality hash codes, thereby improving the accuracy of image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A first flow chart of the deep collaborative graph hashing image retrieval method based on attention enhancement proposed in Example 1;

[0023] Figure 2 A second flow chart of the deep collaborative graph hashing image retrieval method based on attention enhancement proposed in Example 2;

[0024] Figure 3 This is a schematic diagram of the first network structure of the image hash code generation model proposed in Example 2;

[0025] Figure 4 A pseudo code diagram of an algorithm proposed in Example 2;

[0026] Figure 5 It is a schematic diagram of the experimental results proposed in Example 2;

[0027] Figure 6 Schematic diagram of the change of MAP of different models proposed in Example 2 with the number of training rounds;

[0028] Figure 7 This is a schematic diagram of the MAP comparison results of hash codes of different lengths proposed in Example 2 on the CIFAR dataset;

[0029] Figure 8 This is a schematic diagram of MAP comparison results of hash codes of different lengths proposed in Example 2 on the MIRFLICKR dataset;

[0030] Fig. 9 This is a schematic diagram of MAP comparison results of hash codes of different lengths proposed in Example 2 on the NUS-WIDE dataset;

[0031] Fig.10 This is a schematic diagram of the results of hash codes of different lengths based on various data sets proposed in Example 2;

[0032] Fig.11 This is a graph showing changes in MAP of different data sets proposed in Example 2 with the number of training rounds;

[0033] Fig.12 This is a schematic diagram of parameter sensitivity analysis proposed in Example 2;

[0034] Fig.13 This is a schematic diagram of the second network structure of the image hash code generation model proposed in Example 2. DETAILED DESCRIPTION

[0035] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0036] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0037] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0039] Example 1

[0040] This embodiment proposes a deep collaborative graph hashing image retrieval method based on attention enhancement. Figure 1 This is a first flow chart of the deep collaborative graph hashing image retrieval method based on attention enhancement in this embodiment;

[0041] A deep collaborative graph hashing image retrieval method based on attention enhancement proposed in this embodiment includes the following steps:

[0042] S1: Get the image to be retrieved;

[0043] S2: extracting image features of the image to be retrieved;

[0044] S3: Using the attention mechanism to perform weighted processing on the image features to obtain weighted image features;

[0045] S4: Encoding the weighted image features into a hash code;

[0046] S5: Obtain hash codes of pre-stored images in the image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, and sort the pre-stored images in order of the Hamming distance between the two from small to large, and output the first several pre-stored images as similar images of the image to be retrieved.

[0047] In the specific implementation process, the attention mechanism is used to adaptively focus on important areas and channels in the image to be retrieved when processing image features, while suppressing the influence of noise and irrelevant information, thereby extracting more discriminative features. This enhanced feature representation helps to generate higher quality hash codes, thereby improving the accuracy of image retrieval.

[0048] In an optional embodiment, when extracting the image features of the image to be retrieved, the image to be retrieved is inputted using the data input layer, and the input image to be retrieved is extracted using the visual feature extraction module;

[0049] The input end of the first convolutional layer of the visual feature extraction module is connected to the data input layer, and the output end of the first convolutional layer is connected to the input end of the second convolutional layer through the first ReLU activation function layer and the first maximum pooling layer in sequence;

[0050] The output end of the second convolutional layer is connected to the input end of the third convolutional layer through the second ReLU activation function layer and the second maximum pooling layer in sequence;

[0051] The output end of the third convolutional layer is connected to the input end of the fourth convolutional layer through the third ReLU activation function layer;

[0052] The output end of the fourth convolutional layer is connected to the input end of the fifth convolutional layer through the fourth ReLU activation function layer.

[0053] As an exemplary illustration, the specific parameters of the convolution layer of this optional embodiment include: first convolution layer: input: 3-channel RGB image, first convolution layer parameters: convolution kernel size: 11×11, step size: 4, padding: 2, number of output channels: 64, ReLU activation function, maximum pooling layer parameters: pooling kernel size: 3×3, step size: 2; second convolution layer parameters: convolution kernel size: 5×5, padding: 2, number of input channels: 64, number of output channels: 192, ReLU activation function, maximum pooling layer parameters: pooling kernel size: 3×3, step size: 2; Parameters of the third convolution layer: convolution kernel size: 3×3, padding: 1, number of input channels: 192, number of output channels: 384, ReLU activation function; parameters of the fourth convolution layer: convolution kernel size: 3×3, padding: 1, number of input channels: 384, number of output channels: 256, ReLU activation function; parameters of the fifth convolution layer: convolution kernel size: 3×3, padding: 1, number of input channels: 256, number of output channels: 256, ReLU activation function, CBAM attention module, maximum pooling layer parameters: pooling kernel size: 3×3, step size: 2.

[0054] In an optional embodiment, when the image features are weighted by using an attention mechanism, the image features are weighted by using a convolutional block attention module or a bottleneck attention module;

[0055] The convolutional block attention module or the bottleneck attention module is connected to any convolutional layer through the ReLU activation function layer of the visual feature extraction module;

[0056] The convolutional block attention module and the bottleneck attention module both include a channel attention submodule and a spatial attention submodule connected in sequence.

[0057] In an optional embodiment, when encoding the weighted image features into hash codes, a hash layer is used to perform encoding operations;

[0058] The visual feature extraction module, the convolutional block attention module and the hash layer constitute a picture hash code generation model, wherein the output end of the visual feature extraction module is sequentially connected to the input end of the adaptive average pooling layer through the fifth ReLU activation function layer, the convolutional block attention module, the third maximum pooling layer;

[0059] The output end of the adaptive average pooling layer is connected to the input end of the first fully connected layer after a flattening operation; the output end of the first fully connected layer is connected to the input end of the second fully connected layer through the sixth ReLU activation function layer and the first Dropout layer in sequence; the output end of the second fully connected layer is connected to the input end of the hash layer through the seventh ReLU activation function layer and the second Dropout layer in sequence; the output end of the hash layer outputs the hash code after passing through the Tanh activation function layer.

[0060] As an exemplary illustration, the specific parameters of this optional embodiment include: adaptive average pooling: the output size is fixed to 6×6; flattening operation: converting the feature map into a one-dimensional vector; first fully connected layer: input dimension: 256×6×6, output dimension: 4096, ReLU activation function, Dropout (p=0.5); second fully connected layer: input dimension: 4096, output dimension: 4096, ReLU activation function, Dropout (p=0.5); hash layer: input dimension: 4096, output dimension: k (k is the target hash code length), Tanh activation function (compress the output to the [-1,1] interval.

[0061] In an optional embodiment, the operations of extracting the image features of the image to be retrieved, weighting the image features using an attention mechanism, and encoding the weighted image features into a hash code are all performed by the image hash code generation model;

[0062] The image hash code generation model is a model pre-trained using a preset image sample data set, wherein the image sample data set includes a plurality of images, and each image has a classification label;

[0063] When pre-training the image hash code generation model, the objective function is used The deep learning model is trained, and during the training process, the objective function is iteratively solved. When the number of iterations reaches the preset value or the objective function When the function value reaches the minimum, the training is ended and the trained image hash code generation model is obtained;

[0064] The objective function The expressions include:

[0065]

[0066] In the formula, represents the visual image representation learning objective function, represents the semantic feature encoding objective function, represents the first latent embedding objective function, represents the second latent embedding objective function, α represents the semantic feature encoding weight, and β represents the latent embedding weight.

[0067] In an optional embodiment, the visual image represents the learning objective function The expressions include:

[0068]

[0069] H v =tanh(f v (X;θ v )), B v =sggn(f v (X;θ v ))

[0070] Where T represents the matrix transpose symbol, tanh(·) represents the hyperbolic sine function, sggn(·) represents the discrete sign function, and f v (·) represents the image hash code generation model, θ v represents f v (·) corresponds to the learnable parameters, X represents the image matrix composed of images in the image sample data set, S represents the pairwise similarity matrix, and the matrix S is a binary matrix, wherein when the i-th row and j-th column element s ij When the value is 1, it means the i-th image x of the image matrix X i With the jth image x j have at least one common classification label, when s ij When the value is -1, it means x i With x j There is no public classification label, k represents the preset hash code length, represents the square of the Frobenius norm, γ represents the weight coefficient, B v Represents the binary hash coding matrix corresponding to the image matrix X, H v Indicates B v The corresponding image visual feature matrix, L represents the classification label matrix corresponding to the image matrix X, According to B v The predicted class label matrix.

[0071] As an exemplary explanation, k is the length of the hash code output by the image hash code generation model, and k is a constant. The length of k is generally selected depending on the size of the image data set to be detected. When the data set is small, such as thousands or tens of thousands of images, the hash code length selected is generally short. When applied to actual scenarios, the data set may reach millions or hundreds of millions, and the selected hash code length is relatively long. In actual applications, the shorter the hash code length, the faster the retrieval speed, but the relative retrieval accuracy is relatively low. The image hash code generation model of the present application can generate a more discriminative hash code to achieve better performance with a short hash code, that is, the image hash code generation model of the present application can make the hash code length shorter, ensuring the retrieval rate while ensuring that the hash code has good retrieval accuracy.

[0072] In an optional embodiment, the semantic feature encoding objective function The expressions include:

[0073]

[0074] H l =tanh(f l (L;θ l )), B l =sgn(f l (L;θ l ))

[0075] In the formula, represents the matrix transpose symbol, tanh(·) represents the hyperbolic sine function, sgn(·) represents the discrete sign function; f l (·) represents the semantic feature encoder, which is used to convert the image classification labels in the image sample dataset into semantic feature hash codes; θ l represents f l (·) corresponds to the learnable parameters, L represents the classification label matrix composed of image classification labels in the image sample dataset, B l Represents the binary hash encoding matrix of the classification label matrix L, H l Indicates B l The corresponding label semantic feature matrix.

[0076] In an optional embodiment, the first potential embedding objective function and the second latent embedding objective function The expressions include:

[0077]

[0078] stU=g e (H l θ e ), Hrv =g d (U;θ d )

[0079]

[0080] H g =tanh(f g (F m θ g )), B g =sgn(f g (F m θ g ))

[0081] In the formula, g e (·) represents the semantic feature encoder, g d (·) represents the semantic feature decoder, and is represented by g e (·) for H l The result of semantic feature encoding is obtained by g d The decoding reconstruction of (·) is the reconstructed image visual feature matrix H rv , H v Indicates H rv The corresponding original image visual feature matrix; θ e Indicates g e (·) is a learnable parameter, θ d Indicates g d (·) is a learnable parameter, U represents H l With H rv The shared potential embedded relations between g (·) represents the latent embedding feature encoder, θ g represents f g (·) is a learnable parameter, F m represents the output features of the mth layer of the encoder of the multi-layer graph coding network, wherein the encoder of the multi-layer graph coding network takes as input a potential embedded undirected graph with U as the node matrix and a ij is an edge, where a ij = l i × j , l i and l j Represent the i-th and j-th elements of the classification label matrix L respectively.

[0082] Example 2

[0083] This embodiment is based on the attention-enhanced deep collaborative graph hashing image retrieval method proposed in Embodiment 1, and proposes the following specific implementation examples:

[0084] Figure 2This is a second flow chart of the attention-enhanced deep collaborative graph hashing image retrieval method proposed in this embodiment.

[0085] like Figure 2 As shown, the present invention seamlessly integrates the convolutional block attention module (CBAM) into the visual image representation learning network (convolutional neural network CNN) of the deep collaborative graph hashing (DCGH) model, that is, the convolutional block attention module (CBAM) is inserted after the key convolutional layer of the visual image representation learning network, keeping the other parts of the original DCGH model unchanged.

[0086] To facilitate explanation and understanding, we first introduce the definition and usage of some symbols.

[0087] For image retrieval, there is a set of n image samples, denoted as Where d is the dimension of each sample and n is the number of samples. Each sample is labeled as at least one of the given categories, denoted by L = {l 1 , l 2 , l 3 , l 4 , ..., l n}∈{0,1} c×n , where c is the total number of categories, l i is the i-th sample x i In addition, a pairwise similarity matrix S is defined to represent the semantic relevance between data points. If the s in the i-th row and j-th column of the matrix S is ij =1 means the i-th sample x i and the jth sample x j Semantically similar, if s ij = -1 means dissimilar. In simple terms, if two samples have at least one common label, then s ij =1, otherwise s ij =-1.

[0088] To encode a high-dimensional visual image into a k-dimensional binary code, that is, X→B∈{-1,1} k×n , B is a compact hash code, B = sgn(H) = sgn(f(X; θ)), H is the output of the encoding network, sgn(·) is the sign function, and f(·; θ) is the encoding function with learnable parameters θ.

[0089] Given an intermediate feature map As input, the convolutional block attention module sequentially derives a one-dimensional channel attention weight map Channel and 2D spatial attention weight maps The entire convolutional block attention module can be summarized as:

[0090]

[0091] in Represents element-wise multiplication. F 1 is the intermediate output, F 2 is the final output. Figure 3 This is a schematic diagram of the first network structure of the image hash code generation model proposed in this embodiment.

[0092] Figure 3 The computation process of each attention map is described.

[0093] First, the intermediate feature map F aggregates the spatial information of the feature map through average pooling and maximum pooling operations to generate average pooling features Max pooling feature Then the average pooling features and the maximum pooling features are passed through a shared network, which consists of a multi-layer perceptron (MLP) with one hidden layer. To reduce the computational overhead, the reduction ratio r is set to Two new feature maps are obtained, and then feature fusion is performed. The corresponding elements of the two new feature maps are added to obtain a fused feature map and passed through the Sigmoid activation function to generate the final channel attention weight. In short, the calculation of the channel attention module is as follows:

[0094]

[0095] Where σ is the Sigmoid function, Note that the MLP weights W 0 and W 1 For two input and is shared, RELU activation function in W 0 and W 1 between.

[0096] First, the intermediate feature map F 1 Average and maximum pooling operations are applied along the channel axis to aggregate the channel information of the feature map and generate two two-dimensional feature maps: and Then the two two-dimensional feature maps are connected and convolved through the standard convolution layer, and finally passed through the Sigmoid activation function to generate the final spatial attention weight. In short, the calculation of the spatial attention module is as follows:

[0097]

[0098] Where σ is the Sigmoid function, f7×7 Indicates that a convolution operation is performed using a convolution kernel of size 7×7.

[0099] Figure 3 Describes how the convolutional attention module is seamlessly integrated into Alexnet (CNN), adding the convolutional attention module after the activation function ReLU of the 5th convolutional layer of Alexnet.

[0100] The purpose of the visual image representation learning network is to generate a discriminative visual hash code representation for each input sample. In visual feature learning, the pairwise similarity between samples is considered. When the i-th sample x i and the jth sample x j When they are semantically similar, i.e., s ij =1, then generate hash code b i and b j , and the generated hash code b i and b j It should have the minimum Hamming distance. Based on the deep CNN architecture, a deep visual pairwise similarity-preserving hash network is constructed, with:

[0101]

[0102] stb i =sgn(f v (x i θ v )), b i , b j ∈{-1,1} k ,

[0103] where b i is the sample x i The binary code of f v (·;θ v ) is a learnable parameter θ v Visual encoding network. We use a matrix form to simply relax equation (5) and introduce a classification loss term to ensure that the learned binary code is separable at the category level. The loss function can be expressed as:

[0104]

[0105] sB v =sgn(f v (X;θ v )), B∈{-1,1} k×n ,

[0106] where ||·|| F is the Frobenius norm of the matrix, and is the result of predicting the classification based on the learned binary code B. In addition, in order to make the visual encoding network globally differentiable, the discrete sign function sgn(·) is replaced by the hyperbolic sine function tanh(·), that is, H v is the output of the visual encoder. Basically, the goal is to make H v As close as possible to the binary code B. A quantization loss term is introduced, which measures H v The degree of closeness between and B drives the visual encoding network to optimize in the direction of minimizing the quantization error. The above objective function can be restated as Formula 7:

[0107]

[0108] H v =tanh(f v (X;θ v )), B v ∈{-1, 1} k×n ,

[0109] Among them, γ is a weight coefficient, which is used to characterize the importance of balancing different items in learning and ensure that the contribution of each component is properly considered.

[0110] The purpose of the semantic feature encoding network is to extract flexible and varied semantic representations from the existing label set, while capturing and integrating the deep semantic information that may be contained in the hash encoding process.

[0111] A semantic feature encoder is constructed to learn variable-length binary codes by taking semantic labels as binary features while preserving pairwise similarities in the target hash codes. The objective function of the semantic feature encoder module (Formula 8) is:

[0112]

[0113] H l =tanh(f l (L;θ l )), B l ∈{-1, 1} k×n ,

[0114] Among them, γ is a hyperparameter used to measure the importance of different items. l =tanh(f l (L;θ l )) represents the output of the semantic encoding network, which fully exploits the semantic information revealed by the semantic labels, which is subsequently incorporated into the end-to-end framework to guide the learning of hash codes.

[0115] The latent embedding network aims to generate a shared latent representation that captures the intrinsic connections between visual and semantic features, thereby enabling seamless information transfer and fusion across spaces.

[0116] A graph regularized latent space module is constructed, which uses the encoding-decoding mechanism to cross the semantic gap and obtains a shared latent embedding U, which can not only be encoded based on the semantic features H, but also can be decoded inversely into the reconstructed visual features H rv . Convert link to H l →U→H rv , that is, Formula 9:

[0117]

[0118] Among them, g e (·;θ e ) and g d (·;θ d ) are encoder and decoder networks respectively. In addition, a hash coding method that can preserve the structural features of the graph is constructed by using graph convolutional network technology. This method aims to capture the deep semantic and structural associations between data points and encode graph similarity through a multi-layer graph coding network.

[0119] Use a typical graph convolutional network to learn a specific encoder f gcn (·,·), the encoder is used to extract effective feature information on a given graph structure G = (N, E). Here N and E refer to the set of nodes and the set of edges in the graph, respectively. Each node in GCN is represented by a set of potential feature vectors, that is, the node feature matrix U = [u 1 ,u 2 , ..., u n ]. The edges are connected through the adjacency matrix To represent, define a ij = l i × j In order to process the undirected graph G and ensure the smooth propagation of information, the normalized adjacency matrix is ​​used It is obtained by adding the original matrix A to the identity matrix, that is, In the multi-layer structure of GCN, each layer passes through the encoder function f gcn (·,·) receives the output feature matrix of the previous layer and the normalized adjacency matrix as input, and outputs the updated node feature matrix. Specifically, the operation of the mth layer can be formalized as And m>1. For the initial layer, there is Therefore, we use the information propagation mechanism of multi-layer graph convolutional network to perform feature encoding, which is defined as follows:

[0120]

[0121] Among them, σ(·, ·) is an activation operator similar to “ReLU”. yes The diagonal node degree matrix of . The learnable weight parameters W in each layer m , which acts like a convolution kernel in CNN, responsible for learning and fusing features from nodes and their neighbors. We then input these features into the following pairwise similarity preservation loss function (Formula 11):

[0122]

[0123] H g =tanh(f g (F m θ g )), B g ∈{-1, 1} k×n ,

[0124] Among them, f g (·;θ g ) is the learnable parameter θ g The feature encoder.

[0125] Representing visual images into learning networks Semantic Feature Encoding Network Potential Embedded Network Incorporate into a unified learning framework. So the final objective function is:

[0126]

[0127] The final objective function includes three encoding networks: visual image representation learning network Semantic Feature Encoding Network Potential Embedded Network Specifically, an iterative learning method is used to optimize these networks, that is, first optimize Then based on The results are further optimized Finally based on The results are further optimized In this process, for three types of encoding B * (*=v,l,g), update the parameters θ corresponding to each type * (*=v,l,e,d,g) At the same time, set the final B=sgn(H v +αH l +βH g ). During the training process, we will focus on optimizing parameters and refining encoding quality.

[0128] For the parameter θ * , the network is automatically optimized based on Pytorch and the stochastic gradient descent (SGD) and back propagation (BP) algorithms are used to update the network parameters.

[0129] The discrete cyclic coordinate descent (DCC) strategy is used to deal with discrete constraints in binary quadratic programming (BQP), taking three modules based on pairwise similarity preservation loss as examples. With the remaining parameters fixed, it is simplified to an optimization problem, and similar strategies are applicable to other modules.

[0130]

[0131]

[0132] sB g ∈{-1, 1} k×n ,

[0133] where tr(·) is the trace operator of the matrix, and const represents the same as encoding B g , Q=kH g S+γH g .also Yes H g A row of , and U′ is H after removing u g , the same is a row of B, and B′ is B after z is removed. is a row of Q, and Q′ is Q after removing q. We optimize B by column. The optimal solution of z can be updated as (Formula 14):

[0134]

[0135] Figure 4 A pseudo code diagram of an algorithm proposed in this embodiment is shown in FIG. Figure 4 Pseudocode showing the optimization process of the final objective function.

[0136] This embodiment is based on the attention-enhanced deep collaborative graph hashing image retrieval method proposed in Embodiment 1, and proposes the following experimental examples:

[0137] Under the same hyperparameter settings, the experimental results may be affected by factors such as the randomness of the training image data and the query image data, GPU configuration, software package version, memory space, etc., resulting in slight fluctuations in the results. This is normal.

[0138] In this experiment, we used three datasets: CIFAR-10, MIRFLICKR, and NUS-WIDE for experimental evaluation. The three datasets are introduced as follows:

[0139] The CIFAR-10 dataset consists of 60,000 tiny color images of size 32×32. The dataset has 10 categories, each with 6,000 images for data segmentation. The 10 categories include airplane, bird, cat, dog, deer, frog, horse, truck, car. Each image is clearly labeled as one of the above categories. We randomly select 1,000 samples as query samples, 100 from each category, and the remaining images are used for model training.

[0140] The MIRFLICKR dataset consists of 25,000 images. The dataset predefines 38 categories. These images are manually labeled with at least one category. Specifically, the dataset contains 24 basic categories and 14 strict labels for salient image concepts. We randomly select 1,700 samples as query samples, and the remaining images are used for model training.

[0141] The NUS-WIDE dataset collects about 270,000 images from the Internet and defines 81 different categories. Each image is labeled with one or more categories. We selected samples from the first 21 categories for experimental evaluation, involving a total of 195,834 images. Each category contains at least 5,000 samples. We randomly selected 5,000 instances as query samples, and the remaining images were used for model training.

[0142] For the construction of the visual coding network, we can choose many kinds of CNN, and the choice of CNN is not the focus of this invention. For simplicity, we use the pre-trained AlexNet network in the original paper as the visual coding network for experiments. Specifically, AlexNet has 8 layers, namely 5 convolutional layers and 3 fully connected layers. We added a convolutional block attention module after the key convolutional layer. Then we replaced the 8th layer with a k-dimensional hash layer with a tanh activation function, where k is the length of the generated hash code. For the semantic coding network, we use three simple fully connected layers with a ReLU activation function to extract label relationship features, followed by a k-dimensional hash layer. Its network structure is y∈({0, 1} c )→4096→512→k. For the latent embedding network, we adopt two fully connected layers for the feature encoder and feature decoder respectively, and use a two-layer graph convolutional network (GCN) for further hash encoding.

[0143] We use the mean average precision (MAP) to measure the overall retrieval accuracy. Specifically, for each query sample x q, we calculate the average precision (AP) based on its retrieval results, and the calculation formula is as follows:

[0144]

[0145] where r k Representation and query sample x q The number of truly relevant samples, P(k) represents the accuracy of the first k returned samples, and I(k) is a binary function used to determine whether the kth search result is x. q The true neighbor of x, that is, the kth result and x q If at least one semantic label is shared, then I(k) = 1, otherwise I(k) = 0. By averaging the AP values ​​of a set of K query samples, we get MAP, that is:

[0146]

[0147] For CIFAR-10 and MIRFLICKR datasets, we use the entire retrieval database to evaluate MAP, while for the NUS-WIDE dataset with larger data volume, we use the first 5000 samples to evaluate MAP.

[0148] We selected the CIFAR-10 dataset and used 1000 images as the test set, and the rest of the images were used for training. The generated hash code length was 12 bits, the training rounds were set to 60 rounds, the learning rate was set to 0.001, and the minimum batch size was 64. According to the original paper DCGH, the hyperparameters were set as follows: α = 0.5, β = 0.5, γ = 100 (in the MIRFLICKR dataset in the 2.8.5 experiment, the hyperparameter γ = 1 was used). According to the original paper CBAM, a convolution kernel f of size 7 × 7 was used in the spatial attention module of CBAM. 7×7 Perform a convolution operation and set the reduction ratio r=16.

[0149] CBAM is inserted after the activation function ReLU of the first convolutional layer, denoted as CBAMDCGH1.

[0150] CBAM is inserted after the activation function ReLU of the third convolutional layer, denoted as CBAMDCGH3.

[0151] CBAM is inserted after the activation function ReLU of the fifth convolutional layer, denoted as CBAMDCGH5.

[0152] CBAM is inserted after the activation function ReLU of the first and fifth convolutional layers, denoted as CBAMDCGH15.

[0153] CBAM is inserted after the activation function ReLU of the third and fifth convolutional layers, denoted as CBAMDCGH35.

[0154] CBAM is inserted after the activation function ReLU of the first, third, and fifth convolutional layers, denoted as CBAMDCGH135.

[0155] CBAM is inserted after the activation function ReLU of all convolutional layers, denoted as CBAMDCGH12345.

[0156] We set up a control group: no CBAM was inserted and the original model was used, denoted as DCGH.

[0157] Figure 5 This is a schematic diagram of the experimental results proposed in this embodiment.

[0158] The MAP and training time of each experimental group are as follows Figure 5 shown.

[0159] from Figure 5 We can get the experimental results: CBAM inserted after the activation function ReLU of the 5th convolutional layer of the visual encoding network of DCGH can obtain the best MAP. At the same time, we can find that the training time of CBAMDCGH5 is not much different from that of DCGH, which also shows that CBAM is a lightweight module and its overhead can be ignored.

[0160] Figure 6 This is a schematic diagram of the change of MAP of different models proposed in this embodiment with the training rounds. Figure 6 The graph shows the change of mean average precision (MAP) of CBAMDCGH (CBAM at different insertion positions) and DCGH on 12-bit hash code with the number of training rounds. The number of training rounds is set to 60.

[0161] from Figure 6 In the results, inserting CBAM in the 5th convolutional layer has the best effect. Between the 50th and 60th rounds of training, the mean average precision (MAP) of CBAMDCGH5 is higher than that of CBAM. We can see that inserting CBAM in all convolutional layers has the worst effect. The more CBAMs are inserted, the more visual information will be lost, which may contain important visual information. The purpose of introducing CBAM is to reduce the impact of image noise and irrelevant information, so that the model can focus more on important visual information. The experimental results of CBAMDCGH15, CBAMDCGH35, CBAMDCGH135, and CBAMDCGH12345 also illustrate this point.

[0162] We determined that the best CBAM insertion position is after the activation function ReLU of the 5th convolutional layer of the visual encoding network of DCGH. In the following comparison, we use the above best model as our final model, denoted as CBAMDCGH.

[0163] This embodiment is based on the attention-enhanced deep collaborative graph hashing image retrieval method proposed in Embodiment 1, and proposes the following performance comparison example:

[0164] In this experiment, the experimental evaluation uses the same experimental settings and hyperparameters as the original paper DCGH. The experimental data of BGDH, SSDH, and DPQ deep hashing models refer to the original paper DCGH.

[0165] We compare some state-of-the-art image hashing methods to verify the effectiveness of CBAMDCGH. Specifically, there are the following deep hashing models CBAMDGCH, DCGH, BGDH, SSDH, DPQ.

[0166] Figure 7 Schematic diagram of MAP comparison results of hash codes of different lengths proposed in this embodiment on the CIFAR dataset; the experimental results of CIFAR-10 are shown in Figure 7 shown.

[0167] Figure 8 Schematic diagram of MAP comparison results of hash codes of different lengths proposed in this embodiment on the MIRFLICKR dataset; the experimental results of MIRFLICKR are shown in Figure 8 shown.

[0168] Fig. 9 This is a schematic diagram of the MAP comparison results of hash codes of different lengths proposed in this embodiment on the NUS-WIDE dataset; the experimental results of NUS-WIDE are as follows Fig. 9 shown.

[0169] according to Figure 7 , 8 and 9, we can get Fig.10 , Fig.10 A schematic diagram of the results of hash codes of different lengths based on various data sets proposed in this embodiment; Fig.10 The MAP improvement of CBAMDCGH over DCGH is shown based on hash codes of different lengths for each dataset.

[0170] exist Figure 7 , 8In Figure 9, by comparing the BGDH, SSDH, DPQ, and DCGH deep hashing models, we can find that the CBAMDCGH model is superior to BGDH, SSDH, and DPQ on the CIFAR-10, MIRFLICKR, and NUS-WIDE datasets. Fig.10 In the figure, we can clearly see that CBAMDCGH performs well on the three data sets, especially on the 12-bit hash code, with an average improvement of 1.41%. On the 24-bit, 32-bit, and 48-bit hash codes, the average improvements are 0.39%, 0.43%, and 0.05%, respectively. This shows that the overall performance of CBAMDCGH is better than that of DCGH. Specifically, CBAM can make up for the defect that short hash codes cannot store too much original image information. It makes the model pay more attention to important visual information and generate higher quality hash codes, thereby improving the performance of the model.

[0171] Fig.11 The MAP of different data sets proposed in this embodiment varies with the number of training rounds, such as Fig.11 As shown, part (a) shows the change of the mean average precision (MAP) of CIFAR-10, CBAMDCGH (left) and DCGH (right) based on the CIFAR-10 dataset with the training rounds, and we set the training rounds to 60. Part (b) shows the change of the mean average precision (MAP) of MIRFLICKR, CBAMDCGH (left) and DCGH (right) based on the MIRFLICKR dataset with the training rounds, and we set the training rounds to 10. Part (c) shows the change of the mean average precision (MAP) of NUS-WIDE, CBAMDCGH (left) and DCGH (right) based on the NUS-WIDE dataset with the training rounds, and we set the training rounds to 85.

[0172] exist Fig.11 In (a) CIFAR-10, (b) MIRFLICKR, and (c) NUS-WIDE, we can see that the MAP of CBAMDCGH and DCGH has increased significantly with the increase of training rounds, and (a) CIFAR-10, (b) MIRFLICKR, and (c) NUS-WIDE have stabilized from the 50th to the 60th round, from the 7th to the 10th round, and from the 75th to the 80th round, respectively. In addition, we can observe that CBAMDCGH can speed up the convergence of short hash codes (12 bits), and at the final convergence, the MAP value is higher than that of DCGH. At the same time, in the hash codes of other lengths (24 bits, 32 bits, 48 ​​bits), the convergence curves of CBAMDCGH and DCGH are quite close. Fig.11 and Fig.10 The results together prove that CBAM can reduce the impact of image noise and irrelevant information, while making the visual encoding network pay more attention to the important visual information in the image, further enhancing the model performance. In simple terms, the shorter the hash code, the less original information it contains in the image, and CBAM makes the short hash code contain more important visual information, so CBAMDCGH performs better on short hash codes. At the same time, CBAMDCGH can maintain the advantages of DCGH in long hash codes.

[0173] Fig.12 The schematic diagram of parameter sensitivity analysis proposed in this embodiment is as follows: Fig.12 As shown in the figure, we perform parameter sensitivity analysis to evaluate the stability of CBAMDCGH. Stability here refers to the fluctuation range of the MAP value of the model under different hyperparameters, that is, the trend and magnitude of the change in model performance when the parameters change. Our CBAMDCGH includes 5 hyperparameters, namely α, β, γ, r, and f. r is the reduction ratio and f is the convolution kernel size. We evaluate different combinations of hyperparameters by changing the values ​​of these parameters within a reasonable discrete range.

[0174] Specifically, the adjustment range of α, β, and γ is 10 -2 to 100, r is adjusted from 4 to 64, and f is adjusted to 3×3 and 7×7.

[0175] Our parameter sensitivity analysis is done on the CIFAR-10 dataset and 12 bits hash code, and we evaluate the stability of CBAMDCGH by varying a single hyperparameter while fixing other hyperparameters.

[0176] We from Fig.12 (a) It can be seen that CBAMDCGH is stable for α and β. When 10≤γ≤100, MAP remains between 0.91 and 0.93, and the fluctuation range does not exceed 0.02, indicating that the model has high stability within this range and can also obtain high retrieval accuracy.

[0177] Fig.12 (b) shows the model MAP corresponding to different reduction ratios r. When the r value gradually increases from 4 to 16, the MAP steadily rises to the optimal performance point, indicating that the feature extraction ability of the model is gradually enhanced at this stage, thereby improving the overall retrieval performance. When the r value further increases to 32, the MAP drops significantly, indicating that the r value is too large, which will cause the model to lose some important visual information, thus affecting the performance of the model. The r value continues to increase to 64, and the MAP rises slightly, but still does not reach the optimal value, indicating that the r value is too large, which makes the attention mechanism too coarse. Although it improves the learning ability of some features, the overall performance is still limited by the problem of information loss.

[0178] Fig.12 (c) shows the MAP training process corresponding to different convolution kernel sizes. We can observe that the MAP of the model using the 7×7 convolution kernel increases significantly faster than that of the model using the 3×3 convolution kernel in the early and mid-term training. This shows that when the convolution and the kernel are larger, the model can better capture the global feature information, thereby improving the convergence speed of the model. After the 50th round of training, both tend to be stable, and the use of the 7×7 convolution kernel can further improve the final retrieval accuracy of the model.

[0179] In summary, our model CBAMDCGH is stable to the five hyperparameters within reasonable ranges.

[0180] In order to evaluate the effectiveness of the spatial attention module and channel attention module in CBAM, we conduct ablation experiments on CBAMDCGH. Specifically, we remove the channel attention module in CBAM and obtain a degraded model, that is, only the spatial attention module, denoted as SAMDCGH. Similarly, remove the spatial attention module in CBAM and get CAMDCGH. The ablation experiment is completed based on the CIFAR-10 dataset and 12 bits hash code. We will compare the MAP of DCGH, CBAMDCGH, SAMDCGH, and CAMDCGH respectively.

[0181] from Fig.12 In (d), we can see that compared with other degradation models, our model CBAMDCGH can achieve the best performance. SAMDCGH and CAMDCGH both have different degrees of performance decline, among which the MAP value of SAMDCGH compared with CBAMDCGH decreases by 5.6%, and the MAP value of CAMDCGH compared with CBAMDCGH decreases by 1.6%, which also shows that the spatial attention module and channel attention module in CBAM play an important role in CBAMDCGH.

[0182] The present invention integrates the convolutional block attention module (CBAM) into the visual image representation learning network (CNN) of deep collaborative graph hashing (DCGH), producing the following beneficial effects:

[0183] The refinement and discrimination ability of feature extraction are improved. The CBAM attention mechanism in the present invention can adaptively focus on important areas and channels in the image, while suppressing the influence of noise and irrelevant information, thereby extracting more discriminative features. This enhanced feature representation helps to generate higher quality hash codes, thereby improving the accuracy of image retrieval. Specifically, the short hash codes generated by the original model are limited by the length of the hash code and cannot retain too much important image information. Too much noise and irrelevant information in the image will seriously affect the quality of the hash code, resulting in reduced image retrieval performance. CBAM enables the model to pay more attention to the truly important visual information in the image, so that the short hash code contains more important information, thereby improving the performance of the model image retrieval.

[0184] The advantages of DCGH in 24-bits, 32-bits, and 48-bits hash codes are maintained. The present invention improves on DCGH, retaining its advantages in multi-level semantic embedding, potential common space construction, and intrinsic structure mining, while further enhancing the model performance through the attention mechanism.

[0185] This paper seamlessly integrates the convolutional block attention module into the visual image representation learning network of deep collaborative graph hashing for the first time, realizing attention-enhanced deep collaborative graph hashing. This innovative combination fully utilizes the advantages of CBAM in feature extraction while maintaining the characteristics of DCGH in semantic preservation and structure mining.

[0186] Through the CBAM enhanced feature extraction mechanism, the discriminability of features is further improved while maintaining the original advantages of DCGH. This improvement enables the model to better cope with various challenges in practical applications, such as image noise, a large amount of irrelevant information, and complex background.

[0187] The Convolutional Block Attention Module (CBAM) is first introduced in the Deep Collaborative Graph Hashing framework (DCGH), which provides a new performance improvement method for large-scale image retrieval tasks.

[0188] This paper introduces the Convolutional Block Attention Module (CBAM) in the visual image representation learning network of Deep Collaborative Graph Hashing (DCGH), but this is not the only feasible attention enhancement scheme. As a potential alternative, we can consider using the Bottleneck Attention Module (BAM) to replace CBAM.

[0189] This embodiment also proposes an alternative to the Bottleneck Attention Module (BAM).

[0190] BAM is a lightweight and effective attention module that also focuses on the attention of channel and spatial dimensions, but differs from CBAM in structural design.

[0191] Application location. Both CBAM and BAM are placed after the key convolutional layers of the visual image representation learning network (CNN) of DCGH.

[0192] Channel attention calculation. CBAM uses a combination of average pooling and max pooling and generates channel attention weights through a shared multi-layer perceptron (MLP). BAM only uses global average pooling and then generates channel attention weights through a multi-layer perceptron (MLP).

[0193] Spatial attention calculation. CBAM uses a simple convolution operation. BAM uses multiple dilated convolution layers to expand the receptive field, which helps capture a wider range of spatial context information.

[0194] Attention Fusion. CBAM applies channel and spatial attention in series. BAM computes channel and spatial attention in parallel and then adds them together to get the final attention map.

[0195] Given an input feature map The output of BAM is the attention weight map The entire convolutional block attention module can be summarized as:

[0196]

[0197] in, represents element-wise multiplication, F 1 is the final output, In BAM we have two independent branches, which calculate the channel attention weight map and The attention weight map M(F) is calculated as follows:

[0198] M(F)=σ(M c (F)+M s (F)),

[0199] Where σ is the Sigmoid function, M c (F) and M s (F) Before adding, they are adjusted to

[0200] Channel attention branch. We first input feature map Perform global average pooling to obtain the channel vector Then it goes through a multi-layer perceptron (MLP) with one hidden layer, and we set the hidden activation size to r is the reduction ratio. We add a batch normalization layer (BN) after MLP to adjust the dimension size after the channel attention branch. In simple terms, we can summarize the channel attention branch as:

[0201] M c (F) = BN(MLP(AvgPool(F)))

[0202] =BN(W 1 (W 0 AvgPool(F)+b 0 )+b 1 ),

[0203] in

[0204] Spatial attention branch. In the channel dimension, we use 1×1 convolution to convert the feature map Projection to reduced dimension The reduction ratio r of the channel attention branch and the spatial attention branch is the same. Then we use two 3×3 dilated convolutions and finally use 1×1 convolution to reduce the dimension again to During the dimension adjustment, a batch normalization layer is applied at the end of the spatial attention branch. In simple terms, we can summarize the spatial attention branch as:

[0205]

[0206] Where f represents the convolution operation, the superscript represents the convolution kernel size, and BN represents the batch normalization operation.

[0207] Fig.13 This is a schematic diagram of the second network structure of the image hash code generation model proposed in this embodiment. Fig.13 We show that the bottleneck attention module is seamlessly integrated into Alexnet (CNN), where we add the bottleneck attention module after the key convolutional layer of Alexnet. By exploring this alternative, the present invention has the potential to further optimize its performance and efficiency in large-scale image retrieval tasks.

[0208] Example 3

[0209] This embodiment proposes a deep collaborative graph hash image retrieval system based on attention enhancement, which is used to implement a deep collaborative graph hash image retrieval method based on attention enhancement proposed in Example 1.

[0210] The attention-enhanced deep collaborative graph hashing image retrieval system includes:

[0211] Image acquisition module, used to obtain the image to be retrieved;

[0212] An image feature generation module, used to extract image features of the image to be retrieved;

[0213] An image feature weighting module, used to perform weighted processing on the image features using an attention mechanism to obtain weighted image features;

[0214] An encoding module, used for encoding the weighted image features into a hash code;

[0215] The retrieval module is used to obtain the hash code of the pre-stored image in the image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, and sort the pre-stored images in the order of the Hamming distance between the two from small to large, and output the first several pre-stored images as similar images of the image to be retrieved.

[0216] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the attention-enhanced deep collaborative graph hashing image retrieval method described in Example 1.

[0217] It can be understood that the attention-enhanced deep collaborative graph hashing image retrieval system and computer device of this embodiment improve the method of Example 1. The optional options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0218] The same or similar reference numerals correspond to the same or similar components;

[0219] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0220] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A deep collaborative graph hashing image retrieval method based on attention enhancement, characterized in that: The following steps are involved: Get the image to be retrieved; Extracting image features of the image to be retrieved; Performing weighted processing on the image features by using an attention mechanism to obtain weighted image features; Encoding the weighted image features into a hash code; Obtain hash codes of pre-stored images in an image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, sort the pre-stored images in ascending order of the Hamming distance between the two, and output the first several pre-stored images as similar images of the image to be retrieved.

2. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 1 is characterized in that: When extracting the image features of the image to be retrieved, the image to be retrieved is inputted using the data input layer, and the input image to be retrieved is extracted using the visual feature extraction module; The input end of the first convolutional layer of the visual feature extraction module is connected to the data input layer, and the output end of the first convolutional layer is connected to the input end of the second convolutional layer through the first ReLU activation function layer and the first maximum pooling layer in sequence; The output end of the second convolutional layer is connected to the input end of the third convolutional layer through the second ReLU activation function layer and the second maximum pooling layer in sequence; The output end of the third convolutional layer is connected to the input end of the fourth convolutional layer through the third ReLU activation function layer; The output end of the fourth convolutional layer is connected to the input end of the fifth convolutional layer through the fourth ReLU activation function layer.

3. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 2 is characterized in that: When the image features are weighted by using the attention mechanism, the image features are weighted by using a convolutional block attention module or a bottleneck attention module; The convolutional block attention module or the bottleneck attention module is connected to any convolutional layer through the ReLU activation function layer of the visual feature extraction module; The convolutional block attention module and the bottleneck attention module both include a channel attention submodule and a spatial attention submodule connected in sequence.

4. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to any one of claims 1 to 3, characterized in that: When encoding the weighted image features into hash codes, a hash layer is used to perform encoding operations; The visual feature extraction module, the convolutional block attention module and the hash layer constitute a picture hash code generation model, wherein the output end of the visual feature extraction module is sequentially connected to the input end of the adaptive average pooling layer through the fifth ReLU activation function layer, the convolutional block attention module, the third maximum pooling layer; The output end of the adaptive average pooling layer is connected to the input end of the first fully connected layer after a flattening operation; the output end of the first fully connected layer is connected to the input end of the second fully connected layer through the sixth ReLU activation function layer and the first Dropout layer in sequence; the output end of the second fully connected layer is connected to the input end of the hash layer through the seventh ReLU activation function layer and the second Dropout layer in sequence; the output end of the hash layer outputs the hash code after passing through the Tanh activation function layer.

5. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 4 is characterized in that: The operations of extracting the image features of the image to be retrieved, weighting the image features using the attention mechanism, and encoding the weighted image features into hash codes are all completed by the image hash code generation model; The image hash code generation model is a model pre-trained using a preset image sample data set, wherein the image sample data set includes a plurality of images, and each image has a classification label; When pre-training the image hash code generation model, the objective function is used The deep learning model is trained, and during the training process, the objective function is iteratively solved. When the number of iterations reaches the preset value or the objective function When the function value reaches the minimum, the training ends and the trained image hash code generation model is obtained; The objective function The expressions include: In the formula, represents the objective function of visual image representation learning, represents the semantic feature encoding objective function, represents the first latent embedding objective function, represents the second latent embedding objective function, α represents the semantic feature encoding weight, and β represents the latent embedding weight.

6. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 5 is characterized in that: The visual image represents the learning objective function The expressions include: s.t.H v =tanh(f v (X;θ v )),B v =sgn(f v (X;θ v )) Where T represents the matrix transpose symbol, tanh(·) represents the hyperbolic sine function, sgn(·) represents the discrete sign function, and f v (·) represents the image hash code generation model, θ v represents f v (·) corresponds to the learnable parameters, X represents the image matrix composed of images in the image sample data set, S represents the pairwise similarity matrix, and the matrix S is a binary matrix, wherein when the i-th row and j-th column element s ij When the value is 1, it means the i-th image x of the image matrix X i With the jth image x j have at least one common classification label, when s ij When the value is -1, it means x i With x j There is no public classification label, k represents the preset hash code length, represents the square of the Frobenius norm, γ represents the weight coefficient, B v Represents the binary hash coding matrix corresponding to the image matrix X, H v Indicates B v The corresponding image visual feature matrix, L represents the classification label matrix corresponding to the image matrix X, According to B v The predicted class label matrix.

7. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 6 is characterized in that: The semantic feature encoding objective function The expressions include: s.t.H l =tanh(f l (L;θ l )),B l =sgn(f l (L;θ l )) Where T represents the matrix transpose symbol, tanh(·) represents the hyperbolic sine function, sgn(·) represents the discrete sign function; f l (·) represents the semantic feature encoder, which is used to convert the image classification labels in the image sample dataset into semantic feature hash codes; θ l represents f l (·) corresponds to the learnable parameters, L represents the classification label matrix composed of image classification labels in the image sample dataset, B l Represents the binary hash encoding matrix of the classification label matrix L, H l Indicates B l The corresponding label semantic feature matrix.

8. The method for deep collaborative graph hashing image retrieval based on attention enhancement according to claim 7 is characterized in that: The first latent embedding objective function and the second latent embedding objective function The expressions include: s.t.U=g e (H l ;θ e ),H rv =g d (U;θ d ) s.t.H g =tanh(f g (F m ;θ g )),B g =sgn(f g (F m ;θ g )) In the formula, g e (·) represents the semantic feature encoder, g d (·) represents the semantic feature decoder, and is represented by g e (·) for H l The result of semantic feature encoding is obtained by g d The decoding reconstruction of (·) is the reconstructed image visual feature matrix H rv , H v Indicates H rv The corresponding original image visual feature matrix; θ e Indicates g e (·) is a learnable parameter, θ d Indicates g d (·) is a learnable parameter, and U represents H l With H rv The shared potential embedded relations between g (·) represents the latent embedding feature encoder, θ g represents f g (·) is a learnable parameter, F m represents the output features of the mth layer of the encoder of the multi-layer graph coding network, wherein the encoder of the multi-layer graph coding network takes as input a potential embedded undirected graph with U as the node matrix and a ij is an edge, where a ij = l i × j , l i and l j Represent the i-th and j-th elements of the classification label matrix L respectively.

9. A deep collaborative graph hash image retrieval system based on attention enhancement, used to implement the deep collaborative graph hash image retrieval method based on attention enhancement according to any one of claims 1 to 8, characterized in that: include: Image acquisition module, used to obtain the image to be retrieved; An image feature generation module, used to extract image features of the image to be retrieved; An image feature weighting module, used to perform weighted processing on the image features using an attention mechanism to obtain weighted image features; An encoding module, used for encoding the weighted image features into a hash code; The retrieval module is used to obtain the hash code of the pre-stored image in the image database, calculate the Hamming distance between the hash code of the image to be retrieved and the hash code of the pre-stored image, and sort the pre-stored images in the order of the Hamming distance between the two from small to large, and output the first several pre-stored images as similar images of the image to be retrieved.

10. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor performs the steps of the attention-enhanced deep collaborative graph hashing image retrieval method as described in any one of claims 1 to 8.

Images