Deep hashing image retrieval method and apparatus

Abstract

This application relates to the field of image retrieval technology, and particularly to a deep hash image retrieval method and apparatus. The method includes: extracting the original low-level features and local features of a preprocessed target image to generate a fused feature vector; analyzing the correlation between global features of the target image based on the long-distance dependencies between the fused feature vectors to generate a global feature vector of the target image according to the correlation; unifying the global feature vector to convert it into a binary hash code; and using the binary hash code to retrieve at least one image whose similarity to the target image meets the target requirements, generating a retrieval result. This application can effectively integrate local and global features of an image to generate a binary hash code with richer semantic information. The Hamming distance metric further improves the computation speed, thereby effectively improving the efficiency and accuracy of image retrieval and having a wide range of applications.

Description

Technical Field

[0001] This application relates to the field of image retrieval technology, and in particular to a deep hash image retrieval method and apparatus. Background Technology

[0002] Image retrieval technology refers to the technology of automatically searching, classifying, and retrieving images based on image content or metadata using computer vision and artificial intelligence algorithms. Its goal is to find the most relevant or similar images from a large image database according to the user's needs.

[0003] Image retrieval technologies are generally categorized as follows: Content-Based Image Retrieval (CBIR), which primarily retrieves images based on their visual content, using features such as color, texture, shape, and spatial relationships. Users search for similar images by querying the image or certain key features. Text-Based Image Retrieval (TBIR), which typically relies on image metadata or tags (such as filename, description, tags, and category). Through natural language processing, users can input text related to the image content for retrieval. Deep learning-based image retrieval, with its rise, has become mainstream, particularly image retrieval technologies based on deep learning models such as Convolutional Neural Networks (CNNs). Deep learning models can automatically extract high-level features from original images and perform efficient image similarity measurement through trained network models, significantly improving retrieval accuracy. Deep learning-based image retrieval technology, primarily relying on convolutional neural network technology, has become the mainstream image retrieval method today.

[0004] However, in related technologies, deep learning-based image retrieval technology mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local features of images and lack an understanding of the relationships between global features. This makes the image retrieval methods based on convolutional hashing still have certain limitations in terms of retrieval accuracy and efficiency, which urgently need to be solved. Summary of the Invention

[0005] This application provides a deep hash image retrieval method and apparatus to address the problem that in related technologies, deep learning-based image retrieval technology mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local features of images and lack an understanding of the relationships between global features. This makes the image retrieval methods based on convolutional hashing still have certain limitations in terms of retrieval accuracy and efficiency.

[0006] The first aspect of this application provides a deep hash image retrieval method, comprising the following steps: extracting the original low-level features and local features of a preprocessed target image to generate a fused feature vector containing the original low-level features and local features of the target image; analyzing the correlation between global features of the target image based on the long-distance dependency between the fused feature vectors to generate a global feature vector of the target image according to the correlation; performing unified processing on the global feature vector to convert the global feature vector into a binary hash code, and using the binary hash code to retrieve at least one image whose similarity to the target image meets the target requirement, thereby generating a retrieval result.

[0007] Optionally, in one embodiment of this application, the step of extracting the original low-level features and local features of the preprocessed target image to generate a fused feature vector containing the original low-level features and local features of the target image includes: deleting the fully connected layer classification function of the first target model used to extract the local features, and inserting an adaptive weighting mechanism into the first target model after deleting the fully connected layer classification function to obtain a processed first target model; assigning corresponding weight coefficients to the original low-level features and local features respectively using the processed first target model, and fusing the original low-level features and local features according to the weight coefficients to generate the fused feature vector.

[0008] Optionally, in one embodiment of this application, the step of analyzing the correlation between global features of the target image based on the long-distance dependency between the fused feature vectors, so as to generate a global feature vector of the target image according to the correlation, includes: generating a feature sequence that meets the input requirements of a second target model based on the fused feature vectors, and using the second target model to perform position embedding on the feature sequence to obtain the long-distance dependency between the fused feature vectors; understanding the order relationship of the feature sequence based on the long-distance dependency, so as to learn the global features of the target image and generate the global feature vector.

[0009] Optionally, in one embodiment of this application, the step of unifying the global feature vector to convert the global feature vector into a binary hash code includes: mapping the global feature vector to a preset interval to obtain the converted feature value, and converting the feature value into an integer that meets a preset numerical requirement; generating the binary hash code based on the integer.

[0010] Optionally, in one embodiment of this application, the step of retrieving at least one image whose similarity to the target image meets the target requirement using the binary hash code includes: calculating the Hamming distance between the binary hash code of the target image and the binary hash code of each image in the retrieved image library; and determining at least one image whose similarity to the target image meets the target requirement based on the Hamming distance.

[0011] A second aspect of this application provides a deep hash image retrieval apparatus, comprising: an extraction module for extracting the original low-level features and local features of a preprocessed target image to generate a fused feature vector containing the original low-level features and local features of the target image; an analysis module for analyzing the correlation between global features of the target image based on the long-distance dependency between the fused feature vectors to generate a global feature vector of the target image according to the correlation; and a retrieval module for performing unified processing on the global feature vector to convert the global feature vector into a binary hash code, and using the binary hash code to retrieve at least one image whose similarity to the target image meets the target requirement, thereby generating a retrieval result.

[0012] Optionally, in one embodiment of this application, the extraction module includes: a processing unit, configured to delete the fully connected layer classification function of the first target model used to extract the local features, and insert an adaptive weighting mechanism into the first target model after deleting the fully connected layer classification function to obtain a processed first target model; and a fusion unit, configured to use the processed first target model to assign corresponding weight coefficients to the original low-level features and local features respectively, and fuse the original low-level features and local features according to the weight coefficients to generate the fused feature vector.

[0013] Optionally, in one embodiment of this application, the analysis module includes: an embedding unit, configured to generate a feature sequence that meets the input requirements of a second target model based on the fused feature vector, so as to use the second target model to perform position embedding on the feature sequence to obtain the long-distance dependency between the fused feature vectors; and a learning unit, configured to understand the sequential relationship of the feature sequence based on the long-distance dependency, so as to learn the global features of the target image and generate the global feature vector.

[0014] Optionally, in one embodiment of this application, the retrieval module includes: a mapping unit, configured to map the global feature vector to a preset interval to obtain a converted feature value, and convert the feature value into an integer that meets a preset numerical requirement; and a generation unit, configured to generate the binary hash code based on the integer.

[0015] Optionally, in one embodiment of this application, the retrieval module includes: a calculation unit, configured to calculate the Hamming distance between the binary hash code of the target image and the binary hash code of each image in the retrieval image library; and a determination unit, configured to determine at least one image whose similarity to the target image meets the target requirement based on the Hamming distance.

[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep hash image retrieval method as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the depth hash image retrieval method described above.

[0018] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the depth hash image retrieval method described above.

[0019] This application's embodiments can extract the original low-level features and local features of the target image to generate a fused feature vector, and generate a global feature vector of the target image based on the long-distance dependencies between the fused feature vectors. This generates a binary hash code to retrieve at least one image whose similarity to the target image meets the target requirements. Thus, it achieves the comprehensive utilization of the excellent feature extraction capabilities of convolutional neural networks through a joint model, and uses Transformer to capture global feature correlations. By integrating the original low-level features, local features, and global features of the image to generate a binary hash code, the semantic information of the obtained binary hash code is richer. The Hamming distance metric used significantly improves the computation speed, thereby effectively improving the efficiency and accuracy of image retrieval. It has a wide range of applications and strong practical application capabilities. This solves the problem in related technologies where deep learning-based image retrieval technology mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local image features and lack an understanding of the relationships between global features, which limits the accuracy and efficiency of general convolutional hash-based image retrieval methods.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a depth hashing image retrieval method provided according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a novel deep hash image retrieval method according to an embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating an image retrieval process according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the search results according to one embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of a depth hash image retrieval device provided according to an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application.

[0028] Figure label:

[0029] 10-Deep hash image retrieval device: 100-Extraction module, 200-Analysis module and 300-Retrieval module; 601-Memory, 602-Processor and 603-Communication interface. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The following describes a deep hash image retrieval method and apparatus according to embodiments of this application with reference to the accompanying drawings. In the related technologies mentioned in the background section, deep learning-based image retrieval technology mainly relies on convolutional neural network (CNN) technology. However, CNNs are strong only in extracting local image features and lack an understanding of the relationships between global features. This limits the accuracy and efficiency of general convolutional hash-based image retrieval methods. This application provides a deep hash image retrieval method. In this method, the original low-level features and local features of the target image are appropriately preserved. A global feature vector of the target image is generated based on the long-distance dependency between local feature vectors to generate a binary hash code for retrieving at least one image whose similarity to the target image meets the target requirements. Thus, this method utilizes the excellent feature extraction capabilities of CNNs through a joint model and captures global feature correlations using Transformer. By combining local and global features of the image to generate a binary hash code, the semantic information of the resulting binary hash code is richer. The Hamming distance metric used significantly improves the computation speed, thereby effectively improving the efficiency and accuracy of image retrieval. It has a wide range of applications and strong practical application capabilities. This solves the problem that image retrieval technology based on deep learning mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local features of images and lack an understanding of the relationships between global features. This makes the image retrieval methods based on convolutional hashing still have certain limitations in terms of retrieval accuracy and efficiency.

[0032] Specifically, Figure 1 This is a flowchart of a depth hash image retrieval method provided in an embodiment of this application.

[0033] like Figure 1 As shown, the depth hashing image retrieval method includes the following steps:

[0034] In step S101, the original low-level features and local features of the preprocessed target image are extracted to generate a fused feature vector containing the original low-level features and local features of the target image.

[0035] It is understandable that the target image here can be interpreted as the reference image used when retrieving images, i.e., the reference object used to retrieve similar images. Original low-level features here refer to the basic low-level features such as edges and textures that may be lost after the image passes through convolutional layers. Local features here refer to unique and stable local region features in the target image.

[0036] In some embodiments, the global shape of the same object varies significantly from different viewpoints, but its local features remain relatively stable. To improve the accuracy of retrieval, this application, when retrieving similar images of a target image, can first extract the original low-level features and local features of the preprocessed target image, and then fuse the two to generate a processable fused feature vector.

[0037] The preprocessing of the target image here includes, but is not limited to, pixel adjustment and data augmentation operations. For example, the model input is the original CIFAR-10 image, each image is 32x32 pixels in size, and is an RGB image, meaning it has three channels: red, green, and blue. During the data preprocessing stage, to meet data processing requirements, this application may, but is not limited to, adjust the image to a size of 224x224 pixels and apply some data augmentation operations, such as random cropping, random horizontal flipping, and color dithering. After these processes, the image is converted to tensor form and normalized to ensure that the pixel values ​​are between -1 and 1, allowing it to be further converted into the input format required by the model.

[0038] The embodiments of this application can preprocess the target image to ensure that the image meets the data processing requirements of the model, and then extract relatively stable local features of the preprocessed target image, which helps to improve the accuracy of image retrieval.

[0039] Optionally, in one embodiment of this application, extracting local features from the preprocessed target image to generate a fused feature vector containing the original low-level features and local features of the target image includes: deleting the fully connected layer classification function of the first target model used to extract the original low-level features and local features, and inserting an adaptive weighting mechanism into the first target model after deleting the fully connected layer classification function to obtain a processed first target model; assigning corresponding weight coefficients to the original low-level features and local features respectively using the processed first target model, and fusing the original low-level features and local features according to the weight coefficients to generate the fused feature vector.

[0040] It is understandable that the first target model here can be understood as the relevant model used to extract the original low-level features and local features of an image during the deep learning process, such as the ResNet-18 model based on a convolutional neural network (ResNet-18 is both a specific implementation of a residual convolutional neural network and a deep learning model).

[0041] In some embodiments, in order to utilize the excellent feature extraction capabilities of convolutional neural networks, while preserving the original low-level features of the image and reducing the significant time loss caused by connecting and classifying local features when extracting local features of the preprocessed target image, this application can remove the fully connected layer classification function of the first target model used for extracting local features, so that the model no longer undertakes the classification task, but directly outputs the local feature vector. That is, the first target model only completes the task of extracting the original low-level features and local features of the target image.

[0042] For example, this application can use the ResNet-18 model to extract the original low-level features and local features of the target image. Figure 2 This is a flowchart of a novel deep hash image retrieval method according to an embodiment of this application, as follows: Figure 2 As shown, the process of extracting the original low-level features and local features of the target image can be represented, but is not limited to, as follows:

[0043] (1) After image preprocessing, it is converted into the input form required by the ResNet-18 model and input into the ResNet-18 network. The ResNet-18 network already has the initialization parameters learned from the ImageNet dataset, and the pre-trained weights in these initialization parameters are used to initialize the convolutional layers and other parts of the ResNet-18 network;

[0044] (2) Suppose that the input received by the ResNet-18 network is an image of size (batch_size, 3,224,224), where batch_size is the number of images input each time, 3 represents the three color channels of RGB, and 224 is the width and height of the image;

[0045] (3) The first convolutional layer of the ResNet-18 network performs a convolution operation to obtain the original low-level features: after passing through a 7x7 convolutional layer with 64 output channels and a stride of 2. After the convolution operation of this convolutional layer, the feature map of the original low-level features can be transformed from a 3-channel feature map to a 64-channel feature map. That is, it can be understood that the purpose of the first convolutional layer is to learn the original low-level features of the image, such as edges and textures, through the convolutional kernel. The output size of the image is: (batch_size, 64, 112, 112);

[0046] (4) Batch Normalization and ReLU Activation: First, the data of each batch is standardized to complete batch normalization, so as to reduce the internal covariate bias and accelerate model training; then, the ReLU activation function is applied to the convolution result (such as the feature map of the original low-level features) to introduce non-linear features, so that the network can learn more complex patterns. The output size of this step remains unchanged: (batch_size, 64, 112, 112).

[0047] (5) Max pooling: The feature map corresponding to the original low-level features is subjected to 3x3 max pooling operation using the max pooling layer. The stride of the max pooling layer is 2, which can reduce the size of the feature map while retaining the important features in the original low-level features to obtain local features. The output size of the feature map of the local features can be: (batch_size, 64, 56, 56).

[0048] (6) Residual Connections: The core of ResNet is residual connections. In the embodiments of this application, the original low-level features of the input are added and fused with the local features after convolution in each residual block to form skip connections. This helps to alleviate the gradient vanishing problem in deep network training, enabling the network to better propagate gradients and improve model performance.

[0049] Furthermore, in order to better integrate the original low-level features and local features, embodiments of this application may, but are not limited to, introduce at least one adaptive weighting mechanism in the original residual connections of the model. Based on this adaptive weighting mechanism, embodiments of this application may adaptively learn a weight coefficient for the original low-level features and convolutional features extracted by the first target model, so that the model can dynamically adjust the fusion ratio of the two.

[0050] Furthermore, embodiments of this application can add shorter residual branches to the residual blocks, allowing the original low-level features to be more directly passed to subsequent network layers. The structure of each residual block includes, but is not limited to: convolutional layers: typically composed of two convolutional layers (e.g., 3x3 convolutions), each followed by a batch normalization layer and ReLU activation; residual connections: the input features are directly added to the output of the convolutional layers. Residual connections ensure that the network maintains information flow during deep learning. For example, the first residual module in ResNet receives an input of size (batch_size, 64, 56, 56), which undergoes a series of convolutions, batch normalization, and ReLU activations, ultimately outputting a feature map of size (batch_size, 64, 56, 56).

[0051] (7) Subsequent residual processing: ResNet will repeat similar residual modules, i.e. subsequent residual blocks, gradually reducing the spatial size of the feature map and increasing the number of channels (e.g., 64->128->256). The output of each residual block will continue to be processed by convolution, pooling and other methods.

[0052] (8) Output: Finally, the ResNet network outputs a feature map (batch_size, 512, 7, 7) that combines the original low-level features and local features. This feature map retains the rich features extracted from the original image, including edge, shape, and texture information. Since the last fully connected layer has been removed (self.resnet.fc=nn.Identity()), the output is a feature tensor of shape (batch_size, 512, 7, 7).

[0053] It should be noted that the ResNet-18 model in this application embodiment is only an exemplary illustration of the first target model. In actual execution, the specific first target model can be determined by those skilled in the art based on the actual situation. The original low-level feature extraction and local feature extraction functions can be performed on the basis of deleting the fully connected layer classification function, and an adaptive weighting mechanism can be introduced. This application does not impose any specific restrictions.

[0054] The embodiments of this application can extract the original low-level features and local features of the target image by deleting the classification function of the fully connected layer and inserting an adaptive weighting mechanism in the first target model. This can effectively utilize the good feature extraction capability of the convolutional neural network while reducing the time loss caused by connection classification.

[0055] Step S102: Based on the long-distance dependency between fused feature vectors, analyze the correlation between global features of the target image to generate a global feature vector of the target image according to the correlation.

[0056] As one possible implementation, embodiments of this application can, after extracting the original low-level features and local features of the target image, analyze the correlation between global features of the target image based on the long-distance dependency between the fused feature vectors that fuse the two, and generate corresponding global feature vectors.

[0057] In this context, global features can be understood as features that summarize the overall attributes or structure of a target image. In contrast to local features, they capture the global layout of the target image. The relationships between global features refer to the semantic, spatial, or logical dependencies between different global feature vectors in the target image. Global features typically describe the overall attributes of an image (such as scene category or overall color distribution) or high-level semantics (such as "this is a picture of a beach"), while the relationships between them involve how these overall attributes interact, complement, or constrain each other.

[0058] In this context, long-distance dependencies between fused feature vectors refer to semantic associations or dependencies between geographically distant local regions in the target image. For example, local features in the upper left corner and those in the lower right corner of an image may have a dependency relationship; this cross-spatial association is known as a long-distance dependency.

[0059] The embodiments of this application can distribute the correlation between global features of the target image based on the long-distance dependency between fused feature vectors, which can form a more comprehensive understanding of the target image. By using the global feature vector of the target image generated based on the correlation, the accuracy of target image retrieval can be improved.

[0060] Optionally, in one embodiment of this application, the correlation between global features of the target image is analyzed based on the long-distance dependency between fused feature vectors to generate a global feature vector of the target image according to the correlation, including: generating a feature sequence that meets the input requirements of a second target model based on the fused feature vectors, and using the second target model to perform position embedding on the feature sequence to obtain the long-distance dependency between local feature vectors; understanding the order relationship of the feature sequence based on the long-distance dependency to learn the global features of the target image and generate a global feature vector.

[0061] It is understood that the second target model here can be understood as a related model used to realize the correlation between global features of the target image based on the long-distance dependency between fused feature vectors, so as to generate global feature vectors of the target image according to the correlation. In the embodiments of this application, ViT (VisionTransformer) may be used, but is not limited to.

[0062] In actual execution, when calculating the long-distance dependency between fused feature vectors, this application can convert the fused feature vectors into a format suitable for ViT processing based on the input requirements of the second target model (such as the ViT model) so as to perform position embedding, and further process them through a self-attention mechanism to obtain the long-distance dependency between fused feature vectors. Then, based on understanding the sequential relationship of the feature sequence, the global features of the target image are learned and a global feature vector is generated.

[0063] For example, the input requirement of ViT is a sequence of tokens, where each token represents a fixed region of the image. In order to convert the output of ResNet (batch_size, 512, 7, 7) into an input format that meets the input requirements of ViT, this application may perform the following processing, but is not limited to: (1) split each 7x7 spatial region into a 64-dimensional vector, that is, each region is converted into a token; (2) through a flattening operation, convert each 7x7 feature map into a feature vector of length 512, resulting in 49 tokens (because 7x7=49), each token having a dimension of 512, and the sequence of 49 tokens is the feature sequence; (3) the final input to ViT has a Tensor size of (batch_size, 49, 512), that is, 49 tokens (feature sequence), each token being a 512-dimensional vector.

[0064] To preserve the position information of the input token, this application embodiment can use ViT to perform position embedding on each token in the feature sequence. These embeddings represent the position of each local feature corresponding to the fused feature vector of each token in the target image, ensuring that ViT can distinguish information from different positions.

[0065] Then, in this embodiment of the application, the [CLS] Token in ViT (which is used to represent the comprehensive information of the entire image) can be used to process this Token together with other image Tokens through the Transformer during the forward propagation of the model. A special Token is initialized through cls_token and concatenated with other Tokens.

[0066] Self-attention is the core of the Transformer. For each input token, the self-attention mechanism calculates the similarity between that token and all other tokens, and then weights the representation of each token based on these similarities. Specifically, for each token, this embodiment can calculate the relationship between its query, key, and value with all other tokens. Through these relationships, the self-attention mechanism enables each token to pay attention to other relevant tokens during feature learning. In this way, ViT can capture long-distance dependencies between different regions in an image (e.g., relationships between object parts), thereby understanding the sequential relationships between each token in the feature sequence to learn the global features of the target image and generate a global feature vector.

[0067] The Transformer Encoder is a stack of multiple self-attention layers, each including a self-attention operation and a feedforward neural network. The output of each layer further enhances the representation of the input. After multiple Transformer Encoder layers, each token in the final output (including the [CLS] token) has an updated representation. Ultimately, the representation of the [CLS] token contains information about the entire image and can be used for final classification and image retrieval.

[0068] Furthermore, in this embodiment, the output of ViT includes not only the global feature vector but also the classification result of the target image. Specifically, inputting the feature_vector from ViT into the second fully connected layer fc2 yields an output vector with a length equal to the number of categories (10 categories), which can be used for target image classification. The final output is a binary classification result obtained through torch.sigmoid activation in ViT. This binary classification result can be used to determine the category of the target image, which is also the basis for adjusting model parameters during training. The complete model result is as follows: Figure 2 As shown.

[0069] It should be noted that after obtaining the fused feature vector and the global feature vector, this application can combine the models used in the generation process of the two so that they can be applied in practice after training. For example, ResNet and ViT can be combined to form a joint model, which facilitates the execution of the previous and subsequent steps in the actual execution process.

[0070] Therefore, when the joint model training begins, the ResNet part already has the initialization parameters learned from the ImageNet dataset, and the pre-trained weights in these initialization parameters are used to initialize the convolutional layers and other parts of the ResNet, which can effectively improve the training efficiency and application efficiency of the joint model.

[0071] Furthermore, instead of directly segmenting patches from the original image, ViT treats the fused features extracted by ResNet as a fixed-size sequence input, which can effectively utilize the feature extraction capabilities of convolutional neural networks for images and helps improve the training efficiency of the ViT model itself and the joint model.

[0072] Therefore, in practical applications, the embodiments of this application can use a joint model that combines convolutional neural networks and self-attention mechanisms. This allows the joint model to not only extract the original low-level features and local features of the image effectively, but also to adaptively assign different appropriate weights to the original low-level features and local features, thereby obtaining the correlation between global features and sequentially increasing the semantic information of the image contained in the feature vector.

[0073] Step S103: The global feature vector is uniformly processed to convert it into a binary hash code, and the binary hash code is used to retrieve at least one image that meets the target image similarity requirement, thereby generating the retrieval result.

[0074] In some embodiments, after outputting the global feature vector, this application requires the global feature vector to be uniformly processed so as to convert the global feature vector into a binary hash code that can be used to retrieve the target image. The binary hash code is then used to retrieve at least one image that meets the target image similarity requirement, and the image is output as the retrieval result.

[0075] Here, the target requirement can be understood as the requirements set during image retrieval, such as the retrieved image must have a similarity of at least 75% to a key part of the target image, or the number of similar images output. The specific requirements can be determined by those skilled in the art based on actual application needs. This embodiment is merely illustrative and does not impose any specific limitations.

[0076] The process will now be explained in more detail.

[0077] Optionally, in one embodiment of this application, the global feature vector is uniformly processed to convert the global feature vector into a binary hash code, including: mapping the global feature vector to a preset interval to obtain the converted feature value, and converting the feature value into an integer that meets the preset numerical requirements; generating a binary hash code based on the integer.

[0078] In some embodiments, when performing unified processing on the global feature vector to convert it into a binary hash code, this application may, but is not limited to, map the global feature vector to a preset interval to obtain the converted feature value, and then convert the feature value into an integer that meets the preset numerical requirements, thereby generating a binary hash code. Here, the binary hash code is a binary feature representation used to measure the features of an image.

[0079] Here, the preset interval can be understood as the range of values ​​that can satisfy the requirements of binary hash encoding when the global feature vector is uniformly processed to obtain binary hash encoding. Specifically, this application embodiment aims to obtain binary hash encoding where the numbers only involve 0 and 1, so this application embodiment can map the global feature vector to the interval [0, 1].

[0080] In practical applications, embodiments of this application may, but are not limited to, introduce a Sigmoid activation function into the joint model, thereby using the Sigmoid activation function to map each dimension of the global feature vector to between 0 and 1, and obtain the transformed feature values.

[0081] Furthermore, considering that the converted feature values ​​are between 0 and 1 but have a large number of decimals, in order to convert them into binary hash codes, the embodiments of this application can further convert these feature values ​​in [0, 1] into integers that meet preset numerical requirements, thereby generating binary hash codes based on these integers.

[0082] Here, the preset numerical requirement can be understood as the requirement for numerical values ​​in binary hash encoding. In this embodiment, the global feature vector mapping range is [0, 1], and binary hash encoding only involves the numbers 0 and 1. Therefore, the preset numerical requirement in this embodiment is that the feature value can only be 0 and 1. At this time, a threshold of 0.5 can be set, but is not limited to, to convert values ​​exceeding 0.5 to 1 and other values ​​to 0, ultimately obtaining the binary hash encoding of the target image that only contains 0 and 1.

[0083] This application embodiment can generate and use binary hash codes containing only 0 and 1 to retrieve target images. The similarity between images is measured in binary form, and the distance can be quickly calculated through XOR operation. In actual retrieval, it is faster than other retrieval methods and can greatly improve the retrieval efficiency and accuracy of target images.

[0084] Optionally, in one embodiment of this application, retrieving at least one image whose similarity to the target image meets the target requirement using binary hash encoding includes: calculating the Hamming distance between the binary hash encoding of the target image and the binary hash encoding of each image in the retrieved image library; and determining at least one image whose similarity to the target image meets the target requirement based on the Hamming distance.

[0085] In some embodiments, after obtaining the binary hash code of the target image, this application can determine at least one image whose similarity to the target image meets the target requirement by calculating the Hamming distance between the binary hash code of the target image and the binary hash code of each image in the retrieved image library.

[0086] Figure 3 This is a flowchart illustrating an embodiment of the image retrieval process of this application. Figure 3 As shown, there are 10,000 images in the image database. After the joint model is trained and tested, this embodiment of the application can first use the joint model to generate binary hash codes for the 10,000 images in the image database and put them into the hash_code array.

[0087] Then, in this embodiment of the application, the distance between the binary hash code of each image in the retrieved image library and the binary hash code of the target image can be compared in turn, and the Hamming distance can be used to calculate the similarity between each image in the retrieved image library and the target image, and finally output at least one image that is closest to the target image in terms of similarity.

[0088] The method proposed in this application was implemented in an experimental environment using a 12th Gen Intel® Core™ i7-12700H processor, 2.30 GHz, and 16.0 GB of RAM. Figure 4 The embodiments of this application demonstrate the retrieval results of a single target image on four datasets, including average accuracy, mAP, and average retrieval time per image.

[0089] The deep hashing image retrieval method proposed in this application extracts the original low-level features and local features of the target image to generate a fused feature vector. Based on the long-distance dependencies between the fused feature vectors, a global feature vector of the target image is generated to produce a binary hash code that retrieves at least one image whose similarity to the target image meets the target requirements. This achieves the integration of the excellent feature extraction capabilities of convolutional neural networks through a joint model, and utilizes the Transformer to capture global feature correlations. By combining the original low-level features, local features, and global features of the image to generate a binary hash code, the semantic information of the resulting binary hash code is richer. The Hamming distance metric used significantly improves the computation speed, thereby effectively improving the efficiency and accuracy of image retrieval. It has a wide range of applications and strong practical application capabilities. This solves the problem in related technologies where deep learning-based image retrieval technology mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local image features and lack an understanding of the relationships between global features, which limits the accuracy and efficiency of general convolutional hashing-based image retrieval methods.

[0090] Next, the deep hash image retrieval apparatus according to an embodiment of this application is described with reference to the accompanying drawings.

[0091] Figure 5 This is a schematic diagram of the structure of a deep hash image retrieval device according to an embodiment of this application.

[0092] like Figure 5 As shown, the deep hash image retrieval device 10 includes: an extraction module 100, an analysis module 200, and a retrieval module 300.

[0093] The extraction module 100 is used to extract the original low-level features and local features of the preprocessed target image to generate a local feature vector containing the original low-level features and local features of the target image.

[0094] Analysis module 200 is used to analyze the correlation between global features of the target image based on the long-distance dependency between fused feature vectors, so as to generate a global feature vector of the target image according to the correlation.

[0095] The retrieval module 300 is used to perform unified processing on the global feature vector, convert the global feature vector into binary hash code, and use the binary hash code to retrieve at least one image that meets the target image similarity requirement, and generate retrieval results.

[0096] Optionally, in one embodiment of this application, the extraction module 100 includes a deletion unit and an extraction unit.

[0097] The deletion unit is used to delete the fully connected layer classification function of the first target model used to extract the original low-level features and local features, and to insert an adaptive weighting mechanism into the first target model after deleting the fully connected layer classification function to obtain the processed first target model.

[0098] The extraction unit is used to assign corresponding weight coefficients to the original low-level features and local features respectively using the processed first target model, and to fuse the original low-level features and local features according to the weight coefficients to generate a fused feature vector.

[0099] Optionally, in one embodiment of this application, the analysis module 200 includes an embedding unit and a learning unit.

[0100] The embedding unit is used to generate a feature sequence that meets the input requirements of the second target model based on the fused feature vector, so as to use the second target model to embed the feature sequence at a position to obtain the long-distance dependency between local feature vectors.

[0101] The learning unit is used to understand the sequential relationship of feature sequences based on long-distance dependencies, so as to learn the global features of the target image and generate a global feature vector.

[0102] Optionally, in one embodiment of this application, the retrieval module 300 includes a mapping unit and a generation unit.

[0103] The mapping unit is used to map the global feature vector to a preset interval to obtain the transformed feature value, and to convert the feature value into an integer that meets the preset numerical requirements.

[0104] The generation unit is used to generate binary hash codes based on integers.

[0105] Optionally, in one embodiment of this application, the retrieval module 300 includes a calculation unit and a determination unit.

[0106] The computation unit is used to calculate the Hamming distance between the binary hash code of the target image and the binary hash code of each image in the retrieved image library.

[0107] The determining unit is used to determine at least one image whose similarity to the target image meets the target requirement based on the Hamming distance.

[0108] It should be noted that the foregoing explanation of the deep hash image retrieval method embodiment also applies to the deep hash image retrieval device of this embodiment, and will not be repeated here.

[0109] The deep hash image retrieval device proposed in this application can extract the original low-level features and local features of the target image to generate a fused feature vector, and generate a global feature vector of the target image based on the long-distance dependency relationship between the fused feature vectors, so as to generate a binary hash code to retrieve at least one image whose similarity to the target image meets the target requirements. Thus, it achieves the comprehensive utilization of the excellent feature extraction capabilities of convolutional neural networks through a joint model, and uses Transformer to capture the global correlation of features, integrating the original low-level features, local features, and global features of the image to generate a binary hash code. This results in richer semantic information in the obtained binary hash code, and the Hamming distance metric used greatly improves the computation speed, thereby effectively improving the efficiency and accuracy of image retrieval. It has a wide range of applications and strong practical application capabilities. This solves the problem in related technologies where deep learning-based image retrieval technology mainly relies on convolutional neural network technology. However, convolutional neural networks are only strong in extracting local features of images and lack an understanding of the relationships between global features, which limits the accuracy and efficiency of general convolutional hash-based image retrieval methods.

[0110] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0111] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0112] When the processor 602 executes the program, it implements the depth hash image retrieval method provided in the above embodiments.

[0113] Furthermore, electronic devices also include:

[0114] Communication interface 603 is used for communication between memory 601 and processor 602.

[0115] The memory 601 is used to store computer programs that can run on the processor 602.

[0116] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0117] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0118] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0119] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0120] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the depth hash image retrieval method described above.

[0121] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the deep hash image retrieval method provided in this application.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0124] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0125] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0126] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0127] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0129] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A deep hashing image retrieval method, characterized by, The method comprises the following steps: extracting the original low-level features and local features of the preprocessed target image to generate a fusion feature vector containing the original low-level features and local features of the target image; analyzing the correlation between the global features of the target image based on the long-distance dependency relationship between the fusion feature vectors to generate a global feature vector of the target image according to the correlation; uniformly processing the global feature vector to convert the global feature vector into a binary hash code, and using the binary hash code to retrieve at least one image with a similarity to the target image reaching a target requirement to generate a retrieval result; wherein the step of extracting the original low-level features and local features of the preprocessed target image to generate a fusion feature vector containing the original low-level features and local features of the target image comprises: deleting the full connection layer classification function of a first target model used for extracting the original low-level features and local features, and inserting an adaptive weighting mechanism into the first target model after deleting the full connection layer classification function to obtain a processed first target model; using the processed first target model to assign corresponding weight coefficients to the original low-level features and local features respectively, and fusing the original low-level features and local features according to the weight coefficients to generate the fusion feature vector; wherein the step of analyzing the correlation between the global features of the target image based on the long-distance dependency relationship between the fusion feature vectors to generate a global feature vector of the target image according to the correlation comprises: generating a feature sequence meeting the input requirements of a second target model according to the fusion feature vector, using the second target model to perform position embedding on the feature sequence to obtain the long-distance dependency relationship between the fusion feature vectors, and understanding the sequential relationship of the feature sequence based on the long-distance dependency relationship to learn the global features of the target image and generate the global feature vector.

2. The method of claim 1, wherein, The step of uniformly processing the global feature vector to convert the global feature vector into a binary hash code comprises: mapping the global feature vector to a preset interval to obtain a converted feature value, and converting the feature value into an integer meeting a preset numerical requirement; generating the binary hash code according to the integer.

3. The method of claim 1, wherein, The step of using the binary hash code to retrieve at least one image with a similarity to the target image reaching a target requirement comprises: calculating the Hamming distance between the binary hash code of the target image and the binary hash code of each image in a retrieval image library; determining at least one image with a similarity to the target image reaching a target requirement according to the Hamming distance.

4. A deep hashing image retrieval apparatus characterized by comprising: The method comprises the following steps: an extraction module configured to extract the original low-level features and local features of the preprocessed target image to generate a fusion feature vector containing the original low-level features and local features of the target image; The analysis module is configured to analyze the correlation between the global features of the target image based on the long-distance dependency relationship between the fusion feature vectors, and generate a global feature vector of the target image according to the correlation. The retrieval module is configured to uniformly process the global feature vector to convert the global feature vector into a binary hash code, and retrieve at least one image with a similarity to the target image reaching a target requirement by using the binary hash code to generate a retrieval result. The extraction module includes a processing unit configured to delete a full connection layer classification function of a first target model used for extracting the local features, and insert an adaptive weighting mechanism into the first target model after the full connection layer classification function is deleted to obtain a processed first target model; and a fusion unit configured to assign corresponding weight coefficients to the original low-level features and the local features respectively by using the processed first target model, and fuse the original low-level features and the local features according to the weight coefficients to generate the fusion feature vector. The analysis module includes an embedding unit configured to generate a feature sequence meeting an input requirement of a second target model according to the fusion feature vector, and perform position embedding on the feature sequence by using the second target model to obtain the long-distance dependency relationship between the fusion feature vectors; and a learning unit configured to understand the sequential relationship of the feature sequence based on the long-distance dependency relationship to learn the global features of the target image and generate the global feature vector.

5. An electronic device, comprising: The computer program is executed by the processor to implement the deep hash image retrieval method according to any one of claims 1-3. The computer program is executed by the processor to implement the deep hash image retrieval method according to any one of claims 1-3.

6. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the deep hash image retrieval method according to any one of claims 1-3.

7. A computer program product comprising a computer program, characterized in that, ​

Images