Image retrieval method based on attention fusion of local super features and global features

Abstract

The present invention provides an image retrieval method based on attention fusion of local super features and global features. The method mainly includes two parts: fusion feature training and real-time image retrieval. The fusion feature training includes two stages: extraction of local super features and fusion of local super features and global features. The extraction of local super features is to extract useful local features through an iterative attention model, and to find the most useful local super features through feature matching training with positive samples; the fusion of local super features and global features is to orthogonally fuse the information of the found local super features with the global features to obtain a single multi-dimensional feature vector for retrieval. The method of the present invention can accurately extract useful features of an image, and at the same time, the amount of feature data is small, the information is highly concentrated, the retrieval speed is faster, and the storage space requirement is lower, so it can be better used for image retrieval tasks.

Description

Technical Field

[0001] The present invention belongs to the field of image retrieval technology, and in particular relates to an image retrieval method based on attention fusion of local super features and global features. Background Art

[0002] With the recent development of the internet and mobile devices, every mobile device in our hands has become a data collection terminal. Compared to using text, people prefer to use pictures or videos to record the details of their lives and share them online. As a result, more and more image data is available to people, and finding the images they need within this massive amount of data has become a challenge.

[0003] Image retrieval technology has evolved through two stages. The earliest image retrieval method was text-based image retrieval (TBIR). This involves manually annotating images using text descriptions or extracting keywords based on the image context. Keywords are entered during the search, and the system returns the query results by matching the entered keywords with the image index. This method is also known as "searching images by words." The advantages of this method are its simplicity, ease of understanding, and fast query speed. However, different people may interpret the same image differently, so text annotation is also subjective.

[0004] To address this semantic gap, researchers have explored content-based image retrieval (CBIR), also known as "image search." CBIR extracts image content features, including color, texture, and shape, to generate a compact global image signature and multiple local feature descriptors. During a query, the global feature descriptor of the image being searched is compared with the global features stored in a feature library, and feature similarity is calculated to obtain candidate images. These candidates are then reranked using local features, and the final result is delivered to the client. Image feature representation is crucial for this task. Over the past decade, researchers have shifted from hand-crafted features (such as SIFT features, or scale-invariant feature transforms) to automatically learned features based on convolutional neural networks (CNNs). However, these features are not specifically optimized for image retrieval, so they lack the ability to discover semantically meaningful features and exhibit limited accuracy in practice. Furthermore, the two-stage retrieval approach (preliminary screening using global features and reranking using local features) requires the storage of more feature data and more similarity calculations, significantly increasing storage and computing power requirements.

[0005] In 2020, the VIT (Vision Transformer) model successfully applied the Transformer to the field of computer vision. Using the attention mechanism to extract key features of images is a hot and challenging direction. By using the attention mechanism to extract feature descriptors with a global perspective, better performance can be achieved in image retrieval. Summary of the Invention

[0006] In light of the above, the present invention proposes an image retrieval method based on the fusion of local super features and global features using an attention mechanism. By inputting the local features of an image into a designed iterative attention model, local super features of the image are trained. The resulting local super features are then orthogonally fused with the global features to produce a single multidimensional fused feature. This method can accurately extract useful features of the image while minimizing the amount of feature data, highly concentrating information, resulting in faster retrieval and lower storage space requirements, making it more suitable for image retrieval tasks.

[0007] In order to achieve the above-mentioned purpose of the invention, the present invention provides an image retrieval method based on the fusion of local super features and global features of the attention mechanism, which mainly includes two parts: fusion feature training and real-time image retrieval. Fusion feature training mainly includes two stages: extraction of local super features and fusion of local super features and global features. The extraction of local super features is to extract useful features from a large number of local features originally extracted from the convolutional neural network through the iterative attention model L, and then match them with the corresponding features of similar pictures in the same group to find the most useful local super features for image retrieval, and finally calculate the loss with the features of non-similar pictures to train the final local super feature extractor; the fusion of local super features and global features is to orthogonally fuse the information of the found local super features with the global features to obtain a single multi-dimensional feature vector for retrieval. Real-time image retrieval refers to processing the picture through this model to obtain a final single multi-dimensional feature vector, comparing this vector with the feature vectors of other pictures to obtain the most similar group of pictures, which is the final retrieval result.

[0008] The technical solutions specifically adopted in the present invention are as follows:

[0009] An image retrieval method based on attention fusion of local super features and global features, the specific steps of which are as follows:

[0010] Step S1: Based on an image dataset related to image retrieval, a retrieval image Q, positive sample images P belonging to the same class as the retrieval image Q, and negative sample images N belonging to a different class from the retrieval image Q are selected according to the classification of the data in the image dataset to form a training sample group G;

[0011] Step S2: Using a pre-trained convolutional neural network, input the training sample group G in step S1 into the convolutional neural network, extract the penultimate convolutional feature map in the convolutional neural network as the preliminary local feature LF, and extract the last convolutional feature map in the convolutional neural network as the preliminary global feature GF;

[0012] Step S3: Construct an iterative attention model L, and train the iterative attention model according to steps S3.1 and S3.2 to obtain the local super feature LSF;

[0013] Step S3.1: Input the local features LF of each image in the obtained set of training samples G into the iterative attention model L to obtain the local super features LSF of each image;

[0014] Step S3.2: Match the local super feature LSF of the search image Q and the positive sample image P to find the matching super feature pair; calculate the loss L for the super feature pair and the local super feature LSF of the negative sample image N. super ;

[0015] Step S4: For the obtained local super feature LSF and preliminary global feature GF, the preliminary global feature GF is subjected to pooling layer and fully connected layer operations to obtain a single multi-dimensional global feature SGF, and the orthogonal component OC of each local super feature LSF on the global feature SGF is calculated;

[0016] Step S5: The local super feature LSF of each image in the training sample group G is subtracted from its own orthogonal component OC, and then spatially concatenated with the global feature SGF. After passing through a fully connected layer, the final fused feature OCF is obtained for image retrieval.

[0017] Step S6: Calculate the cross entropy loss function for the obtained fusion feature OCF feature to obtain the loss L global , and then calculate the final total loss L = L super +L global ; Train the model based on the total loss until the model converges;

[0018] Step S7: Input all images in the image retrieval database into the trained model in the same way as the training samples to obtain the fusion feature OCF of each image in the image retrieval database. k ;

[0019] Step S8: The image to be retrieved Q * Input the trained model in the same way as the training sample to obtain the image Q to be retrieved. * Fusion feature OCF * , and combine it with the fusion feature OCF of each image in the image retrieval databasek Calculate the cosine similarity and take the image corresponding to the first K fusion features with the highest cosine similarity as the image to be retrieved Q * The most similar search results are returned.

[0020] Preferably, the iterative attention model L constructed in step S3 is used to map the local feature LF to the local super feature LSF, and its mapping function Φ(u) to the feature space is represented as follows:

[0021] Φ(u):R L*D →R N*d

[0022] Where u represents the input local feature, N represents the number of output local super features LSF, L represents the number of input local features LF, D and d represent the feature dimensions of local features LF and local super features LSF respectively, and R represents the feature vector space; the specific iterative process in the mapping function Φ(u) is expressed as follows:

[0023] Φ(u)=z T

[0024] z t =φ(u;z t-1 )

[0025] φ(u;z)=MLP(ψ(u;z))+ψ(u;z)

[0026] ψ(u;z)=V(u)·α+z

[0027] Among them, φ(u;z) represents the operation performed in one iteration, the first part of its input is the local feature u, and the second part of its input z is the output z of the previous iteration t-1 , where the second part of the input z in the first iteration is z 0 ∈R N*d It is a set of learnable templates, i.e., a learnable parameter matrix; the local super feature LSF is gradually formed by iteratively refining the template based on the local feature LF output by the convolutional neural network. After the φ in the mapping function Φ(u) is iterated T times, the output z obtained at the Tth iteration is T As the local super feature of the final output; MLP represents a fully connected layer, whose input is the output of the attention fusion operation ψ(u; z); in the attention fusion operation ψ(u; z), for the input local feature u, it is first linearly mapped to V9u) and K(u), and z is normalized to become Q(z), and then the attention weight α=l1norm(softmax(Q(z)*K(u)) is obtained by softmax and L1 normalization.

[0028] Preferably, when matching the local super features LSF of the search image Q and the positive sample image P in step S3.2, the matching rule of the super feature pair is:

[0029] For any set of positive sample pairs x,x + , with s∈S and s + ∈S + Represents two pictures x and x respectively + The local super features S and S + A single feature in; a super feature pair (s,s + ) When pairing, the following three conditions must be met at the same time: 1) s, s + Must be the features of each other's nearest neighbors; 2) s, s + Need to pass Lowe's first to second nearest neighbor ratio test; 3) s, s + Have the same super feature serial number;

[0030] A set of positive sample pairs of images x,x + All matched super feature pairs (s,s + ) constitutes the set P * .

[0031] Preferably, the three conditions in the matching rule of the super feature pair are expressed as the following formula:

[0032]

[0033] Where τ is the threshold, i(s) represents the sequence number of feature s, Representation feature s + The features of the nearest neighbors in the super feature S, S\{s} represents the feature set after removing feature s from S.

[0034] Preferably, the threshold value τ=0.9.

[0035] As a preference, the loss L in step S3.2 super The calculation formula is as follows:

[0036]

[0037] Where μ′ is a distance hyperparameter, which indicates the minimum interval between the positive sample feature and the corresponding multiple negative sample features in the local super feature pair, that is, the distance between features should be greater than the given μ′; The superscript + in the brackets indicates that the part must be a value not less than zero; N(i(s)) represents the set of all features with sequence number i(s) in the negative sample image N.

[0038] Preferably, the calculation formula of the orthogonal component OC in step S4 is as follows:

[0039]

[0040]

[0041]

[0042] Among them, f i represents the i-th local super feature LSF, f g represents the global feature SGF, f l,c and f g,c Respectively represent f i and f g The c-th dimension, f i,proj Represents a single local super feature f i In the global feature f g The orthogonal components on , C represents the dimensional information of the feature.

[0043] The method of the present invention has the following advantages: it improves the storage and computing requirements of the existing two-stage search, requiring less storage space. It also uses an attention mechanism and positive sample feature matching to select discriminative local super features, reducing the interference of other unimportant information. Feature fusion ensures that the final fused features contain both global and local features, improving the efficiency and accuracy of the retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the overall steps of the method of the present invention;

[0045] Figure 2 The structure of the iterative attention model L of the method of the present invention;

[0046] Figure 3 This is the orthogonal fusion model structure of the method of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the described embodiments are only intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0048] In a preferred embodiment of the present invention, an image retrieval method based on the fusion of local super features and global features using an attention mechanism is provided. The method mainly comprises two parts: fusion feature training and real-time image retrieval. Fusion feature training also includes two stages: extraction of local super features and fusion of local super features with global features. Figure 1 This is an overall process framework diagram, which shows the relationship between the various parts of the present invention and their processes. The specific implementation steps of the present invention are described in detail below:

[0049] Part 1: Fusion Feature Training

[0050] Step S1: Select an image dataset related to image retrieval, and select a retrieval image Q, a positive sample image P belonging to the same class as the retrieval image Q, and a negative sample image N belonging to a different class from the retrieval image Q according to the classification of the data in the image dataset to form a training sample group G.

[0051] In this example, the representative image dataset is the retrieval-SfM-120k dataset. Based on the classification of the data in the image dataset, a retrieval image Q is selected, along with a similar image P (positive sample) and five images N from different classes (negative samples). This constitutes a training set G = {Q, P, N1, N2, N3, N4, N5}, with corresponding labels {-1, 1, 0, 0, 0, 0}.

[0052] Part 2: Online fusion feature training

[0053] Step S2: Use the pre-trained convolutional neural network, input the training sample group G in step S1 into the convolutional neural network, extract the penultimate convolutional feature map of the model (i.e., the feature map output by the penultimate convolutional layer) and output it as the preliminary local feature LF, and extract the last convolutional feature map in the convolutional neural network (i.e., the feature map output by the last convolutional layer) and output it as the preliminary global feature GF. Figure 1 As shown in , the convolutional neural network used in this embodiment is the Resnet50 model.

[0054] Stage 1: Extraction of local super features

[0055] Step S3: Construct an iterative attention model L. Follow steps S3.1 and S3.2 to train the iterative attention model and obtain the local super feature LSF. The specific implementation of S3.1 and S3.2 is as follows:

[0056] Step S3.1: Input the local features LF of each image in the obtained set of training sample groups G into the iterative attention model L to obtain the local super features LSF of each image.

[0057] In this embodiment, the shape of the local feature LF is [B, C, H, W], where B represents the total number of photos in the training sample set, i.e., 7 photos in this embodiment. C is the output dimension, which is 1024 in this embodiment; H and W are the height and width of the output feature, which are both 32 in this embodiment. The local feature LF is input into the iterative attention model L to obtain the local super feature LSF of the group of images.

[0058] The structure of the iterative attention model L is as follows Figure 2As shown, the iterative attention model L is used to map the local feature LF to the local super feature LSF, and its mapping function Φ(u) to the feature space is expressed as follows:

[0059] Φ(u):R L*D →R N*d

[0060] Where u represents the input local features, that is, all LFs corresponding to an image, N represents the number of output local super features (LSFs), L represents the number of input local features (LFs), D and d represent the feature dimensions of local features (LFs) and local super features (LSFs), respectively, and R represents the feature vector space. The specific iterative process in the mapping function Φ(u) is as follows:

[0061] Φ(u)=z T

[0062] z t =φ(u;z t-1 )

[0063] φ(u;z)=MLP(ψ(u;z))+ψ(u;z)

[0064] ψ(u;z)=V(u)·α+z

[0065] Among them, φ(u;z) represents the operation performed in one iteration, the first part of its input is the local feature u, and the second part of its input z is the output z of the previous iteration t-1 , where the second part of the input z in the first iteration is z 0 ∈R N*d It is a set of learnable templates, i.e., a learnable parameter matrix; the local super feature LSF is gradually formed by iteratively refining the template based on the local feature LF output by the convolutional neural network. After the φ in the mapping function Φ(u) is iterated T times, the output z obtained at the Tth iteration is T As the local super feature of the final output; MLP represents a fully connected layer, whose input is the output of the attention fusion operation ψ(u; z); in the attention fusion operation ψ(u; z), for the input local feature u, it is first linearly mapped to V(u) and K(u), and z is normalized to become Q(z). A layer of softmax is first applied to Q(z)*K(u), and then the result is L1 normalized (L1 normalization) to obtain the attention weight α=l1norm(softmax(Q(z)*K(u)).

[0066] In this embodiment, a total of T=6 iterations are applied in Φ, z 0 ∈R N*dUsing random initialization parameters, the MLP performs two linear mappings on the input local feature u: [7, 1024, 32, 32] → [7, 1024, 1024, 1], transforming it into V(u) and K(u). Since the iterative attention model L maintains the size of each input and output dimension, the final LSF output format is [B, C, H, W] → [7, 1024, 1024, 1].

[0067] Step S3.2: Match the local super feature LSF of the search image Q and the positive sample image P to find the matching super feature pair. Calculate the loss L for the super feature pair and the local super feature LSF of the negative sample image N. super .

[0068] In this embodiment, when matching the local super features LSF of the search image Q and the positive sample image P, the matching rules of the super feature pairs are as follows:

[0069] For any set of positive sample pairs x,x + , with s∈S and s + ∈S + Represents two pictures x and x respectively + The local super features S and S + A single feature in; a super feature pair (s,s + ) When pairing, the following three conditions must be met at the same time: 1) s, s + Must be the features of each other's nearest neighbors; 2) s, s + Need to pass Lowe's first to second nearest neighbor ratio test; 3) s, s + Have the same super feature serial number.

[0070] In this embodiment, the three conditions in the matching rule of the super feature pair are expressed as the following formula:

[0071]

[0072] Where τ is the threshold value, in this embodiment, the threshold value τ = 0.9. i(s) represents the sequence number of feature s, Representation feature s + The features of the nearest neighbors in the super feature S, S\{s} represents the feature set after removing feature s from S.

[0073] A set of positive sample pairs of images x,x + All matched super feature pairs (s,s + ) constitutes the set P * .

[0074] It should be noted that if there is only one retrieval image Q and one positive sample image P in the training sample group, then the positive sample for the image x,x + That is, they correspond to Q and P respectively. If in other embodiments, there are multiple search images Q and positive sample images P, then they need to be combined into positive sample pairs of images x, x + .

[0075] In this embodiment, the loss L super The calculation formula is as follows:

[0076]

[0077] Where N(i(s)) represents the set of all features with sequence number i(s) in the negative sample image N; μ′ is a distance hyperparameter, which represents the minimum distance between the positive sample feature and the corresponding multiple negative sample features in the local super feature pair, that is, the distance between features should be greater than the given μ′; The superscript + in the brackets indicates that the part in the brackets must be a value not less than zero. In this embodiment, μ′ is set to 1.2. The closer the two matching positive sample pairs are, the The closer it is to 0, the super The smaller the difference between the query image feature and the negative sample image feature, the greater the difference. The larger it is, the smaller it will become with a negative sign. When it exceeds the given distance μ′, it meets the requirements of the model of this embodiment. It is ensured that this part is a value not less than zero, that is, it will become 0 instead of a negative number when it exceeds the given distance μ′. Therefore, the greater the difference between the query image feature and the negative sample image feature, the greater the difference L super This ensures that the extracted super features are features that can match similar images and reject features of unfamiliar images.

[0078] Stage 2: Fusion of local super features and global features

[0079] Step S4: For the obtained local super feature LSF and preliminary global feature GF, the preliminary global feature GF is subjected to pooling layer and fully connected layer operations to obtain a single multi-dimensional global feature SGF, and the orthogonal component OC of each local super feature LSF on the global feature SGF is calculated.

[0080] In this embodiment, the format of the preliminary global feature GF is [B, C, H, W]→[7, 2028, 16, 16]. The GF is transformed into [7, 2028, 1, 1] after the pooling layer, and becomes [7, 1024, 1, 1] after the fully connected layer. Finally, a single multi-dimensional global feature SGF is obtained: [7, 1024]. Each local super feature LSF is calculated, and its format is [B, C, H, W]→[7, 1, 1024, 1]. The orthogonal component OC on the global feature SGF (whose dimension is expanded to [B, _, C,]→[7, 1, 1024]) is obtained.

[0081] The structure of the orthogonal component OC is as follows Figure 3 As shown, the calculation formula is as follows:

[0082]

[0083]

[0084]

[0085] where f i represents the i-th local super feature LSF, f g represents the global feature SGF, f l,c and f g,c Respectively represent f i and f g The c-th dimension, f i,proj Represents a single local super feature f i In the global feature f g The orthogonal components on ; C represents the dimension information of the feature, which is 1024 in this embodiment.

[0086] Step S5: Subtract the orthogonal component OC from the local super feature LSF of each image in the training sample group G, and then spatially concatenate it with the global feature SGF. After passing through a fully connected layer, the final fused feature OCF is obtained for image retrieval. In this embodiment, the final format of the fused feature OCF is [7,512].

[0087] Step S6: Calculate the cross entropy loss function for the obtained fusion feature OCF feature to obtain the loss L global . Then calculate the final total loss L = L super +L global . The model is trained based on the total loss L until the model converges.

[0088] It should be noted that the model trained based on the total loss L here is a model framework composed of S2 to S5, denoted as model M, as shown in Figure 1 As shown in .

[0089] Part II: Real-time Image Retrieval

[0090] Step S7: Input all images in the image retrieval database into the trained model M, and obtain the fusion feature OCF of each image in the image retrieval database in the same way as the training samples. k , store it and the corresponding image index.

[0091] Step S8: The image to be retrieved Q * Input into the trained model M, and obtain the image Q to be retrieved in the same way as the training sample * Fusion feature OCF * :[C]=

[512] , and combine it with the fusion feature OCF of each image in the image retrieval database k Calculate the cosine similarity, sort the results in descending order, and take the images corresponding to the first K fusion features with the highest cosine similarity as the images to be retrieved Q * The most similar search results are returned.

[0092] The above description of the embodiments is intended to facilitate understanding and application of the present invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

Claims

1. An image retrieval method based on attention fusion of local super features and global features, characterized in that: The specific steps are as follows: Step S1: Based on an image dataset related to image retrieval, a retrieval image Q, positive sample images P belonging to the same class as the retrieval image Q, and negative sample images N belonging to a different class from the retrieval image Q are selected according to the classification of the data in the image dataset to form a training sample group G; Step S2: Using a pre-trained convolutional neural network, input the training sample group G in step S1 into the convolutional neural network, extract the penultimate convolutional feature map in the convolutional neural network as the preliminary local feature LF, and extract the last convolutional feature map in the convolutional neural network as the preliminary global feature GF; Step S3: Construct an iterative attention model L, and train the iterative attention model according to steps S3.1 and S3.2 to obtain the local super feature LSF; Step S3.1: Input the local features LF of each image in the obtained set of training samples G into the iterative attention model L to obtain the local super features LSF of each image; Step S3.2: Match the local super feature LSF of the search image Q and the positive sample image P to find the matching super feature pair; calculate the loss L for the super feature pair and the local super feature LSF of the negative sample image N. super ; Step S4: For the obtained local super feature LSF and preliminary global feature GF, the preliminary global feature GF is subjected to pooling layer and fully connected layer operations to obtain a single multi-dimensional global feature SGF, and the orthogonal component OC of each local super feature LSF on the global feature SGF is calculated; Step S5: The local super feature LSF of each image in the training sample group G is subtracted from its own orthogonal component OC, and then spatially concatenated with the global feature SGF. After passing through a fully connected layer, the final fused feature OCF is obtained for image retrieval. Step S6: Calculate the cross entropy loss function for the obtained fusion feature OCF feature to obtain the loss L global , and then calculate the final total loss L = L super +L global ; Train the model based on the total loss until the model converges; Step S7: Input all images in the image retrieval database into the trained model in the same way as the training samples to obtain the fusion feature OCF of each image in the image retrieval database. k ; Step S8: The image to be retrieved Q * Input the trained model in the same way as the training sample to obtain the image Q to be retrieved. * Fusion feature OCF * , and combine it with the fusion feature OCF of each image in the image retrieval database k Calculate the cosine similarity and take the image corresponding to the first K fusion features with the highest cosine similarity as the image to be retrieved Q * The most similar search results are returned.

2. The image retrieval method based on attention fusion of local super features and global features according to claim 1 is characterized in that The iterative attention model L constructed in step S3 is used to map the local feature LF to the local super feature LSF, and its mapping function Φ(u) to the feature space is represented as follows: Φ(u): R L*D →R N*d Where u represents the input local feature, N represents the number of output local super features LSF, L represents the number of input local features LF, D and d represent the feature dimensions of local features LF and local super features LSF respectively, and R represents the feature vector space; the specific iterative process in the mapping function Φ(u) is expressed as follows: Φ(u)=z T With t =φ(u;z t-1 ) φ(u;z)=MLP(ψ(u;z))+ψ(u;z) ψ(u;z)=V(u)·α+z Among them, φ(u;z) represents the operation performed in one iteration, the first part of its input is the local feature u, and the second part of its input z is the output z of the previous iteration t-1 , where the second part of the input z in the first iteration is z 0 ∈R N*d It is a set of learnable templates, i.e., a learnable parameter matrix; the local super feature LSF is gradually formed by iteratively refining the template based on the local feature LF output by the convolutional neural network. After the φ in the mapping function Φ(u) is iterated T times, the output z obtained at the Tth iteration is T As the local super feature of the final output; MLP represents a fully connected layer, whose input is the output of the attention fusion operation ψ(u;z); in the attention fusion operation ψ(u;z), for the input local feature u, it is first linearly mapped to V(u) and K(u), and z is normalized to become Q(z), and then the attention weight α=l1norm(softmax(Q(z)*K(u)) is obtained by softmax and L1 normalization.

3. The image retrieval method based on attention fusion of local super features and global features according to claim 1 is characterized in that When matching the local super features LSF of the search image Q and the positive sample image P in step S3.2, the matching rule of the super feature pair is: For any set of positive sample pairs of images x, x + , with s∈S and s + ∈S + Represents two pictures x and x respectively + The local super features S and S + A single feature in; a super feature pair (s, s + ) When pairing, the following three conditions must be met at the same time: 1) s, s + Must be the features of each other's nearest neighbors; 2) s, s + Need to pass Lowe's first to second nearest neighbor ratio test; 3) s, s + Have the same super feature serial number; A set of positive sample pairs of images x, x + All matched super feature pairs (s, s + ) constitutes the set P * .

4. The image retrieval method based on attention fusion of local super features and global features according to claim 3 is characterized in that The three conditions in the matching rule of the super feature pair are expressed as the following formula: Where τ is the threshold, i(s) represents the sequence number of feature s, Representation feature s + The features of the nearest neighbors in the super feature S, S\{s} represents the feature set after removing feature s from S.

5. The image retrieval method based on attention fusion of local super features and global features according to claim 4 is characterized in that: Threshold τ = 0.

9.

6. The image retrieval method based on attention fusion of local super features and global features according to claim 4 is characterized in that: The loss L in step S3.2 super The calculation formula is as follows: Where μ′ is a distance hyperparameter, which indicates the minimum interval between the positive sample feature and the corresponding multiple negative sample features in the local super feature pair, that is, the distance between features should be greater than the given μ′; The superscript + in the brackets indicates that the part must be a value not less than zero; N(i(s)) represents the set of all features with sequence number i(s) in the negative sample image N.

7. The image retrieval method based on attention fusion of local super features and global features according to claim 1 is characterized in that: The calculation formula of the orthogonal component OC in step S4 is as follows: Among them, f i represents the i-th local super feature LSF, f g represents the global feature SGF, f l，c and f g，c Respectively represent f i and f g The c-th dimension, f i，proj Represents a single local super feature f i In the global feature f g The orthogonal components on , C represents the dimensional information of the feature.

Images