Image retrieval method based on decomposition Transform
By splitting the Transformer module into multiple parallel sub-modules and designing an adaptive routing module, the problem of insufficient adaptability in image retrieval of existing methods is solved, achieving more efficient feature extraction and discriminative improvement, especially performing well in retrieval scenarios of unknown class samples.
Patent Information
- Application Number
- CN202511242528.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-02
- Publication Date
- 2025-11-21
AI Technical Summary
Existing Transformer-based deep hashing methods lack the ability to adapt to the diversity of image categories and sample differences when processing image retrieval, resulting in mixed feature extraction and insufficient discriminativeness, especially when facing samples of unknown categories, the generalization ability drops significantly.
By decomposing the Transformer module, the multi-head self-attention module and the multilayer perceptron module are split into multiple parallel sub-modules, and an adaptive routing module is constructed. The sub-modules are dynamically matched through two sets of learnable selectors, thereby improving the generalization ability and discriminativeness of hash encoding.
It significantly improves the retrieval performance of image retrieval models on diverse image datasets, especially demonstrating excellent generalization and discrimination capabilities in zero-sample hash retrieval scenarios with unknown class samples.
Smart Images
Figure CN120994864A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of information retrieval, in particular to an image retrieval method based on a decomposition Transformer. BACKGROUND
[0002] With the popularity of social media networks, the storage and efficient retrieval of massive multimedia data face great challenges. Traditional retrieval methods based on real-valued features (such as Euclidean distance, cosine similarity) are difficult to meet the application requirements of large-scale data due to high computational complexity and large storage overhead. Approximate nearest neighbor (ANN) retrieval technology has emerged as the times require, among which the hash method maps high-dimensional data to compact binary codes, significantly reducing storage costs, and uses Hamming distance to achieve efficient retrieval, becoming the mainstream solution.
[0003] Early hash technology relies on shallow hash methods with artificial features, which have limited performance. Deep hash combines deep learning and hash learning, greatly improving feature expression ability and retrieval accuracy. According to whether label information is used, it can be divided into supervised and unsupervised, and the former usually has better performance. In recent years, the Vision Transformer (ViT) structure has been introduced into deep hash models due to its strong representation learning ability, further improving feature extraction effect.
[0004] However, the existing deep hash method based on Transformer still has significant limitations: a unified network architecture is generally used to process all input images, lacking the ability to adapt to the diversity of image categories and sample differences. This single processing path leads to mixed feature information and insufficient discriminability, making it difficult to effectively distinguish different visual patterns, especially when dealing with unknown category samples, the generalization ability significantly decreases, which becomes a key bottleneck restricting the discriminability of hash coding. It is urgent to develop new network structures and mechanisms to improve the adaptability of the model to image diversity. SUMMARY
[0005] In order to solve at least one problem in the prior art, the present application provides an image retrieval method based on a decomposition Transformer.
[0006] The first aspect of the present application provides an image retrieval method based on a decomposition Transformer, comprising the following steps: Obtain image data samples, divide the image into fixed-size blocks and flatten, then obtain input image features through linear projection; The image retrieval model is constructed to include a decomposition Transformer module, an adaptive routing module, and a hash coding module; wherein the decomposition Transformer module includes a plurality of parallel multi-head self-attention sub-modules and a plurality of parallel multi-layer perceptron sub-modules for feature extraction; the adaptive routing module includes two groups of learnable selectors, one group of learnable selectors is used to map the input image features to a multi-head self-attention sub-module, and the other group of learnable selectors is used to map the output of the multi-head self-attention sub-module to a multi-layer perceptron sub-module; the hash coding module is used to convert the output features of the decomposition Transformer module into hash codes; The parameters of the image retrieval model are updated according to the target loss to obtain a trained image retrieval model; The image retrieval is performed by using the trained image retrieval model.
[0007] In some optional embodiments, the multi-head self-attention sub-module is split from a multi-head self-attention module, and self-attention heads are divided into groups, each group being a multi-head self-attention sub-module; wherein represents a composite number greater than 4, represents a positive integer and is a divisor of . Preferably, is equal to 12, is equal to 3.
[0008] In some optional embodiments, the operation of the th multi-head self-attention sub-module is as follows:
[0009] wherein, represents a self-attention operation; represents a connection operation; , represents the decomposition features of the multi-head self-attention sub-module; , represents the projection matrix of the th multi-head self-attention sub-module; represents the length of the input sequence; represents the dimension of the input image features; ; represents a positive integer.
[0010] In some optional embodiments, the multi-layer perceptron sub-module is split from a multi-layer perceptron module, and the first full connection layer in the multi-layer perceptron module is divided into parts, each part being a first full connection sub-layer of the multi-layer perceptron sub-module; and the second full connection layer in the multi-layer perceptron module is divided into Each of the parts is a second full connection sublayer of a multi-layer perception sub-module.
[0011] In some optional embodiments, the first full connection layer of the i th multi-layer perception sub-module is calculated as follows:
[0012] wherein, represents the output of the first full connection layer of the i th multi-layer perception sub-module; represents the decomposed feature of the multi-layer perception sub-module; represents the weight of the first full connection layer; represents the bias term of the first full connection layer. The second full connection layer of the i th multi-layer perception sub-module is calculated as follows:
[0013] wherein, represents the weight of the second full connection layer; represents the bias term of the second full connection layer.
[0014] In some optional embodiments, the learnable selector is composed of a full connection layer and An activation function, and the input is processed by the learnable selector to generate a corresponding weighted score, which is calculated as follows:
[0015] wherein, , represents the weighted score of the i th sub-module of the multi-head self-attention, represents the weight matrix, represents the bias term. The selection is performed by maximizing the highest score to 1 and minimizing the rest of the scores to 0, which is calculated as follows: wherein,
[0016] represents the index of the maximum weighted score. In some optional embodiments, the target loss includes a triplet loss and a consistency loss.
[0017] The second aspect of the present application provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the above-mentioned image retrieval method based on the decomposed Transformer according to the computer program.
[0018]
[0019] The third aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium is used to store a computer program; and the computer program, when executed by a processor, implements the image retrieval method based on the decomposition Transformer.
[0020] The technical scheme of the embodiment of the present application has the following beneficial effects: a decomposition Transformer module is proposed, the multi-head self-attention and the multi-layer perception module in the Transformer module (Transformer Block) are decomposed into multiple functionally independent sub-modules, different sub-modules are used to encode diversified features, and the generalization ability and the discrimination ability of the hash coding are enhanced; meanwhile, two groups of learnable selectors are designed to construct an adaptive routing module, one group of learnable selectors dynamically selects a multi-head self-attention sub-module for the input image features, and the other group of learnable selectors dynamically selects a multi-layer perception sub-module for the output of the multi-head self-attention sub-module. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 The figure is a network structure diagram of an image retrieval model in the embodiment of the present application.
[0022] Figure 2 The figure is a precision-recall curve diagram of image hash based on 32-bit hash coding of each model, wherein (a) is a precision-recall curve diagram of image hash based on 32-bit hash coding on the MIRFLICKR-25K dataset; (b) is a precision-recall curve diagram of image hash based on 32-bit hash coding on the NUS-WIDE dataset; and (c) is a precision-recall curve diagram of image hash based on 32-bit hash coding on the MS COCO dataset.
[0023] Figure 3 The figure is a precision-recall curve diagram of zero-shot hash based on 64-bit hash coding of each model, wherein (a) is a precision-recall curve diagram of zero-shot hash based on 64-bit hash coding trained on the NUS-WIDE dataset and tested on the MS COCO dataset; and (b) is a precision-recall curve diagram of zero-shot hash based on 64-bit hash coding trained on the MS COCO dataset and tested on the NUS-WIDE dataset. DETAILED DESCRIPTION
[0024] The technical scheme in the embodiments of the present application will be described clearly and completely in combination with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.
[0025] Existing Transformer-based deep hashing methods generally use a unified network architecture to extract features from images, which is difficult to adapt to the diverse visual differences between images, resulting in mixed feature expression and insufficient discrimination, which further affects the effectiveness of hash coding. Especially when facing unknown class samples, the generalization ability of existing methods decreases significantly.
[0026] To solve the above problems, the present application proposes an image retrieval method based on a decomposed Transformer to improve the generalization ability and discriminability of hash coding. Specifically, the method splits the multi-head self-attention module and the multi-layer perceptron block module into multiple parallel sub-modules, constructs a decomposed Transformer module, and realizes the expression of diverse features through independent sub-modules. In addition, an adaptive routing module is designed, which uses two groups of learnable selectors to dynamically match a multi-head self-attention sub-module and a multi-layer perceptron sub-module for input respectively. This method can more effectively adapt to the feature differences of different images and improve the representation ability of hash coding.
[0027] As shown in Figure 1 , in the model training stage, a dataset is used to train a neural network containing a decomposed Transformer module, an adaptive routing module, and a hash coding module to obtain an image retrieval model. The dataset used can be various image datasets, such as the MIRFLICKR-25K dataset, the NUS-WIDE dataset, and the MS COCO dataset.
[0028] The Transformer module can be represented as , which mainly contains a multi-head self-attention module (Multi-Head Self-Attention, MHSA) and a multi-layer perceptron module (Multi-Layer Perceptron, MLP), which can be represented as . A complete visual Transformer model (Vision Transformer, ViT) contains multiple Transformer modules , where represents the number of modules; in the present application, 12-layer Transformer modules are used, i.e. . For convenience, the layer normalization (Layer Normalization, LN) in the Transformer module is ignored.
[0029] In embodiments of the present application, the decomposition Transformer module splits the multi-head self-attention module and the multi-layer perceptron module in the Transformer module into multiple sub-modules with the same structure but independent functions. For the multi-head self-attention module, , represents the number of sub-modules. The representation of the multi-layer perceptron module is similar to that of the multi-head self-attention module, .
[0030] Next, how to split the multi-head self-attention module and the multi-layer perceptron module in the Transformer module is specifically introduced. The calculation process of a complete multi-head self-attention module is shown in the following formula:
[0031] wherein, represents the complete feature in the multi-head self-attention module; represents the self-attention operation; represents the calculation of the th self-attention head, represents the number of self-attention heads; represents the projection matrix; represents the connection operation.
[0032] Specifically, the multi-head self-attention module sends the input into multiple attention heads at the same time, and each attention head performs self-attention calculation. Assuming that there are linear mapping matrices , , and in the th attention head, the input feature dimension is ; if there are heads, . The self-attention calculation is to map the input into the query vector , the key vector and the value vector of the attention head through the linear mapping matrix, and then the output of the th attention head is obtained by using the following formula:
[0033] wherein, , , ; represents that the output is converted into a probability distribution with a total sum of 1. Finally, the multi-head self-attention module splices the outputs of the attention heads to obtain the complete feature of the multi-head self-attention module output, and the dimension is still .
[0034] In embodiments of the present application, the purpose of decomposing the Transformer module is to split the complete multi-head self-attention module into several independently calculated multi-head self-attention sub-modules. Specifically, the self-attention heads are divided into groups, each group being a multi-head self-attention sub-module. For example, if there are 12 attention heads, divided into 3 groups, then the output of the first multi-head self-attention sub-module is obtained only by splicing the outputs of attention heads with serial numbers 1 to 4, the output of the second multi-head self-attention sub-module is obtained only by splicing the outputs of attention heads with serial numbers 5 to 8, and the output of the third multi-head self-attention sub-module is obtained only by splicing the outputs of attention heads with serial numbers 9 to 12. The operation of the multi-head self-attention sub-module is as follows:
[0035] wherein, represents the decomposed feature of the multi-head self-attention sub-module; represents the number of multi-head self-attention; represents the projection matrix of the multi-head self-attention sub-module.
[0036] The complete feature of the multi-head self-attention module is as follows:
[0037] In the Transformer, the multi-layer perceptron module includes two fully connected layers. The first layer of fully connected layer calculates the hidden layer output, the process being as follows:
[0038] wherein, represents the complete feature of the multi-layer perceptron module; represents the weight of the first layer of fully connected layer; represents the bias term of the first layer of fully connected layer; represents the output dimension of the first layer of fully connected layer. The second layer of fully connected layer calculates the output of the multi-layer perceptron module, the process being as follows:
[0039] wherein, represents the weight of the second layer of fully connected layer; represents the bias term of the second layer of fully connected layer.
[0040] The operation steps of the above two fully connected layers can be combined as follows:
[0041] In the embodiment of the present application, the decomposition Transformer module decomposes the output dimension of the first full connection layer into subspaces, and each multi-layer perceptron submodule processes dimensions, and the hidden layer output of the th submodule is calculated as follows:
[0042] wherein, represents the decomposed feature of the multi-layer perceptron submodule; represents the weight of the first full connection layer; represents the bias term of the first full connection layer. For the decomposition of the second full connection layer, the hidden layer output needs to be mapped back to the original dimension, and the output of the th multi-layer perceptron submodule after decomposition of the second full connection layer is calculated as follows:
[0043] wherein, represents the weight of the second full connection layer; represents the bias term of the second full connection layer.
[0044] The output of the multi-layer perceptron module is as follows:
[0045] In the training process, it is hoped that the decomposed features can maintain the original semantic relationship, that is, the features corresponding to the positive samples should have similar representations, while the negative samples should produce obviously distinguished representations. Therefore, for the decomposed features, the parameters are updated according to the triplet loss to improve the discriminability of the hash code, as follows:
[0046] wherein, represents the triplet loss of the decomposed feature; represents a distance function for measuring the similarity between two decomposed features; represents the maximum value (the larger of the two numbers); represents the anchor point in the triplet constructed by the decomposed feature; represents the positive sample of the anchor point in the triplet constructed by the decomposed feature; represents the negative sample of the anchor point in the triplet constructed by the decomposed feature; is an edge parameter to control the minimum difference requirement between the distance of anchor to positive and the distance of anchor to negative, to ensure the distance of anchor to positive (same class) is small enough and the distance of anchor to negative (different class) is at least the distance of anchor to negative.
[0047] In addition, for the complete feature, the parameters are also updated according to the triplet loss to further enhance the discriminability of the common space, as shown in the following formula:
[0048] wherein, denotes the triplet loss of the complete feature; denotes the anchor in the triplet constructed by the complete feature, denotes the positive sample of the anchor in the triplet constructed by the complete feature, denotes the negative sample of the anchor in the triplet constructed by the complete feature.
[0049] The decomposition of the Transformer module splits the multi-head self-attention module and the multi-layer perception module in the Transformer module into multiple sub-modules, which requires selecting a sub-module for the input. Therefore, an adaptive routing module is constructed, and a learnable selector is introduced to dynamically select a sub-module for each output. Taking the multi-head self-attention module as an example, the multi-head self-attention selector set is defined as wherein the th multi-head self-attention selector maps the input to a multi-head self-attention sub-module. Each learnable selector is composed of a fully connected layer and an activation function. After the input passes through the selector, the corresponding weighted score is generated, as shown in the following formula:
[0050] wherein, , denotes the weighted score of the th sub-module of the multi-head self-attention, denotes the weight matrix, denotes the bias term. The score reflects the importance of each multi-head self-attention sub-module to the input. In order to select the sub-module, the highest score is maximized to 1 and the rest of the scores are minimized to 0, as shown in the following formula:
[0051] wherein, denotes the index of the largest weighted score. In order to maintain discriminativeness, a consistency loss is introduced to minimize the difference between the decomposed output and the full output, as shown in the following formula:
[0052] wherein, denotes the consistency loss, denotes the L2 norm.
[0053] After passing through the adaptive routing module and the decomposed Transformer module, the output is a continuous vector of bits ( is a positive integer). In order to convert the continuous vector into a hash code, a hash code module is designed to generate a hash code from the continuous vector using a bit-based conversion function, as shown in the following formula:
[0054] wherein, denotes the binary code, denotes the output of the decomposed Transformer module; if , then , otherwise .
[0055] The retrieval process of the image retrieval method based on the decomposed Transformer is as follows: 1. Preprocessing of the image to be retrieved Standardize the image to be retrieved and convert it into a tensor in the input format of the image retrieval model; Generate a sequence of image patches, which, after being combined with the position encoding, serve as the initial input features of the decomposed Transformer module.
[0056] 2. Feature extraction by the decomposed Transformer module in cooperation with the adaptive routing module (1) The input image features are transmitted to the multi-head self-attention selector, which specifies a multi-head self-attention sub-module for it; (2) The input image features enter the selected multi-head self-attention sub-module for processing, obtaining the decomposed features of the multi-head self-attention sub-module; (3) The decomposed features output by the multi-head self-attention sub-module are connected through a residual connection and normalized by layer normalization, obtaining the normalized decomposed features; (4) The normalized decomposed features are transmitted to the multi-layer perceptron selector, which specifies a multi-layer perceptron sub-module for it; (5) The normalized decomposed features are input into the selected multi-layer perceptron sub-module for processing, obtaining the decomposed features of the multi-layer perceptron sub-module; (6) The decomposed features output by the multi-layer perceptron sub-module are connected by a residual connection and normalized by a layer, and then output as the output of the current Transformer module; Steps (1)-(6) are repeated, and after 12 rounds of feature extraction by the decomposed Transformer module and the adaptive routing module, the final image continuous feature vector is obtained.
[0057] 3. Hash coding The continuous feature vector output by the decomposed Transformer module is input into the hash coding module to convert it into a binary hash code.
[0058] 4. Retrieval library matching The Hamming distance between the hash code of the image to be retrieved and the hash codes of all images in the retrieval library is calculated.
[0059] 5. Output retrieval result The Top-K retrieval results are returned in order of Hamming distance, and the retrieval process is completed.
[0060] To verify the effectiveness of the image retrieval model in the embodiment of the application, experiments were conducted on the MIRFLICKR-25K dataset, NUS-WIDE dataset, and MS COCO dataset. Moreover, a plurality of mainstream image retrieval models (baseline models) were selected for comparative experiments, including DSH, HashNet, GreedyHash, CSQ, IDHN, DPN, HyP2Loss, HHF, CenterHash, SPRCH, and CrossHash. In addition, two zero-shot hash retrieval models, T-MLZSH and CPF (LIU C, SHE W H, CHEN M J, et al. Consistent penalizing field loss for zero-shot image retrieval [J]. Expert Systems with Applications, 2024, 236: 121287. DOI: 10.1016 / j.eswa.2023.121287.) were compared.
[0061] The MIRFLICKR-25K dataset contains about 25,000 images, each of which is manually labeled with 38 categories, covering complex concepts and scenes. After removing unlabeled samples, a total of 24,581 samples were obtained. For each category, 30 samples were selected for the query set, and 5,000 samples were selected for the training set, and the remaining samples were used for image retrieval.
[0062] The NUS-WIDE dataset contains 269,648 image samples from the Flickr website, each of which is manually labeled with multiple semantic tags covering a total of 81 categories. In the experiment, 21 commonly used categories with higher frequency were selected, and the number of corresponding images was 195,834. 2,100 samples were randomly selected for each category as the query set, 500 samples were randomly selected for each category as the training set, and the remaining as the retrieval database.
[0063] The MS COCO dataset consists of 40,504 validation images and 82,783 training images, each of which is labeled with one or more semantic tags from 91 categories. In the experiment, 5,000 images were randomly selected as the query set, 10,000 images were selected for training, and the remaining unselected images constituted the retrieval database.
[0064] Experimental scheme in image hashing: For the MIRFLICKR-25K dataset, 5,000 images were randomly selected as the training set, 1,000 images were used as the query set, and the remaining images were used as the database set. In the MS COCO dataset, 10,000 images were used for training, 5,000 images were randomly selected as the query set, and the remaining samples were used as the database set. For the NUS-WIDE dataset, 10,500 images were randomly selected as the training set, 2,100 images were randomly selected as the query set, and the remaining images were used as the database set.
[0065] Experimental scheme in zero-shot hashing: In the NUS-WIDE dataset, the concepts overlapping with MS COCO were removed to obtain a subset; this subset contains 100,303 samples and involves 17 categories; the MS COCO dataset remains unchanged.
[0066] To analyze the visual retrieval performance, two typical evaluation indicators were used to evaluate each model, namely mean average precision (mAP) and precision-recall (PR) curve. Mean average precision is used to measure the overall accuracy of the retrieval system on all queries, reflecting the ability of the algorithm to return relevant results; while the precision-recall curve shows the accuracy and coverage of the retrieval results under different thresholds, helping to evaluate the trade-off performance of the algorithm between precision and recall. The combination of the two can comprehensively evaluate the performance of the visual retrieval algorithm.
[0067] Image hashing performance: The NUS-WIDE dataset adopts the commonly used mAP@5000 protocol to evaluate the model trained on the dataset (NUS-WIDE@5000), and the other datasets all adopt the mAP@ALL protocol (MIRFLICKR@ALL, MS COCO@ALL), and the results are shown in Tables 1, 2, 3 and Figure 2 In Tables 1-3, the suffix "-FTH" represents that the decomposition Transformer module and the adaptive routing module are used to replace the Transformer module in the baseline model. Compared with other existing image retrieval models, the image retrieval model in the embodiment of the present application shows a sustained and significant performance improvement on all datasets. Specifically, SPRCH-FTH realizes a maximum performance improvement of 2.3% on the MIRFLICKR-25K dataset. On the MSCOCO dataset, the performance of SPRCH-FTH is improved by 0.2% to 1.9% compared with the SRPRCH model. At the same time, Figure 2 It also intuitively shows the superiority of image hashing. This improvement is mainly due to the decomposition design of the visual Transformer architecture, which enables the model to capture richer visual features by adaptively selecting the most suitable sub-module for each input image, thereby enhancing the discrimination ability of the learned hash code.
[0068] Table 1 Image Hashing mAP Evaluation Results of Each Model (MIRFLICKR@ALL)
[0069] Table 2 Image Hashing mAP Evaluation Results of Each Model (NUS-WIDE@5000)
[0070] Table 3 Image Hashing mAP Evaluation Results of Each Model (MS COCO@ALL)
[0071] Image hashing usually relies on training samples of existing categories and mainly focuses on fast image retrieval of known category images. Image hashing mainly emphasizes the discrimination ability of the model. Zero-shot hashing enables the model to generate discriminative hash codes for new category images without seeing the target category image samples, thereby realizing image retrieval of new categories. Zero-shot hashing emphasizes the generalization ability of the model.
[0072] Zero-shot hashing performance: Tables 4, 5 and Figure 3The mAP@1000 results of each model in the zero-shot retrieval scene are shown. Among them, NUS-WIDE→MS COCO means that the model is trained on the NUS-WIDE dataset and tested on the MS COCO; MS COCO→NUS-WIDE represents the opposite configuration. The results show that the model using the decomposition Transformer module has a significant advantage in the generalization ability across datasets. For example, compared with the SPRCH method, the SRPCH-FTH framework improves the mAP by about 5.1%. Compared with the zero-shot method (such as T-MLZSH, CPF), the model using the decomposition Transformer module also achieves satisfactory results. In addition, the precision-recall (PR) curve as shown in Figure 3 verifies the excellent generalization ability of the model using the decomposition Transformer module.
[0073] Table 4 mAP@1000 results of each model in the zero-shot retrieval scene (NUS-WIDE→MS COCO)
[0074] Table 5 mAP@1000 results of each model in the zero-shot retrieval scene (MS COCO→NUS-WIDE)
[0075] The above is only a preferred embodiment of the present application, and is not any form and substantial limitation of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the method of the present application, some improvements and supplements can also be made, which should be considered as the protection scope of the present application. For those skilled in the art, without departing from the spirit and scope of the present application, some changes, modifications and equivalent changes made by using the disclosed technical content are equivalent embodiments of the present application; at the same time, any equivalent changes, modifications and evolution of the above-mentioned embodiments according to the essential technology of the present application are still within the scope of the technical solutions of the present application.
Claims
1. A method for image retrieval based on decomposition Transformer, characterized in that, The method comprises the following steps: Obtaining image data samples, dividing the image into fixed-size blocks and flattening, and then obtaining input image features through linear projection; An image retrieval model comprising a decomposition Transformer module, an adaptive routing module and a hash coding module is constructed; the decomposition Transformer module comprises a plurality of parallel multi-head self-attention sub-modules and a plurality of parallel multi-layer perceptron sub-modules, and is used for feature extraction; The adaptive routing module comprises two groups of learnable selectors, one of which is used to map the input image features to a multi-head self-attention sub-module, and the other is used to map the output of the multi-head self-attention sub-module to a multi-layer perceptron sub-module; the hash coding module is used to convert the output features of the decomposition Transformer module into hash codes; The parameters of the image retrieval model are updated according to the target loss to obtain a trained image retrieval model; Image retrieval is performed through the trained image retrieval model.
2. The method of claim 1, wherein, The multi-head self-attention sub-module is split from the multi-head self-attention module, and each self-attention head is divided into groups, each group being a multi-head self-attention sub-module; wherein represents an even number, represents an even number, represents an even number, represents an even number.
3. The method of claim 2, wherein, equal to 12, equal to 3.
4. The method of claim 2, wherein, No. The operation of the multi-head self-attention submodule is shown in the following formula: ； wherein, denotes a self-attention operation; denotes a concatenation operation; denotes a decomposition feature of the multi-head self-attention sub-module; denotes a projection matrix of the th multi-head self-attention sub-module; denotes a length of the input sequence; denotes a dimension of the input image feature; ; denotes a positive integer.
5. The method of claim 2, wherein, The multi-layer perception sub-module is split from the multi-layer perception module, the first full connection layer in the multi-layer perception module is divided into parts, each part is a first full connection sub-layer of the multi-layer perception sub-module; the second full connection layer in the multi-layer perception module is divided into parts, each part is a second full connection sub-layer of the multi-layer perception sub-module.
6. The method of claim 5, wherein, The calculation process of the first full connection layer of the i-th multi-layer perceptron sub-module is as follows: ； wherein, represents an output of the first fully connected layer of the i-th multi-layer perceptron sub-module; represents a decomposed feature of the multi-layer perceptron sub-module; represents a weight of the first fully connected layer; represents a bias term of the first fully connected layer; The calculation process of the second full connection layer of the i-th multi-layer perceptron sub-module is as follows: ； wherein, represents the weights of the second layer fully connected layer; represents the bias term of the second layer fully connected layer.
7. The method of claim 2, wherein, The learnable selector is composed of a fully connected layer and an activation function. After the input is processed by the learnable selector, a corresponding weighted score is generated, as shown in the following formula: ； wherein, , denotes the weighted score of the multi-head self-attention th sub-module, denotes the weight matrix, denotes the bias term; The selection is performed in the manner of maximizing the highest score to 1 and minimizing the rest of the scores to 0, and the specific formula is as follows: ； wherein, denotes the index of the highest weighted score.
8. The method of claim 1, wherein, The target loss comprises a triplet loss and a consistency loss.
9. An electronic device, comprising: The memory is used to store a computer program; and the processor is used to execute the image retrieval method based on the decomposition Transformer according to the computer program.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store a computer program; and the computer program is used to implement the image retrieval method based on the decomposition Transformer when executed by the processor. The computer readable storage medium is used to store a computer program; and the computer program is used to implement the image retrieval method based on the decomposition Transformer when executed by the processor.