Asymmetric image retrieval method based on decoupled differential knowledge distillation

Abstract

The present invention discloses an asymmetric image retrieval method based on decoupled differential knowledge distillation, which maintains a consistent retrieval order by optimizing the pairwise similarity relationship differences between the query network and the gallery network. The method calculates the pairwise similarity difference matrix in the gallery domain and decouples it into feature knowledge, inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge. This strategic decoupling effectively aligns the retrieval ranking of the query network with the gallery network, which can improve the retrieval accuracy of the query network at a low computational cost. The present invention has the characteristics of deep learning technology, and by optimizing the pairwise similarity relationship differences between the query network and the gallery network, and having the ability to maintain retrieval ranking consistency at a low computational cost, it effectively improves the retrieval accuracy of the query network while maintaining computational efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of asymmetric image retrieval and knowledge distillation, and in particular to an asymmetric image retrieval method based on decoupled differential knowledge distillation. Background Art

[0002] In the field of digital image retrieval, due to the dramatic increase in image data and the diversity of practical application requirements, how to quickly and accurately retrieve the required content from a huge image library has become a technical problem that needs to be solved urgently. Traditional image retrieval methods usually rely on feature matching technology for query images and library images. With the development of deep learning technology, the accuracy of feature expression and matching has been significantly improved, but it also brings high demands on computing resources and storage resources.

[0003] In some specific scenarios, such as mobile devices or edge computing environments, complex deep neural network models cannot be run efficiently due to hardware performance limitations. In addition, transmitting high-dimensional image feature data or a large number of query images to a central server for processing will also cause significant network bandwidth and latency issues, and may even bring privacy-related concerns. Therefore, the industry has proposed the concept of asymmetric image retrieval, which is to deploy a lightweight image query network on resource-constrained devices and place a high-capacity image feature library on the server. This setting is intended to reduce the burden on front-end devices and reduce the amount of data transmission.

[0004] However, in asymmetric image retrieval systems, due to the differences in complexity and capabilities between the query network and the gallery network, how to ensure that the retrieval results of the query network maintain consistent ranking with the gallery network is a major research challenge. Current methods, such as techniques based on correspondence learning and knowledge distillation, often encounter challenges when trying to compress and transfer complex feature structures in large networks to lightweight networks, resulting in reduced retrieval performance and consistency of results. In addition, in the process of pure knowledge transfer, the ability to correctly rank easily confused categories and difficult samples may be lost, further affecting the accuracy and robustness of the retrieval system.

[0005] In view of the above problems, there is an urgent need to seek new technical solutions that can effectively solve the bottleneck problems of limited performance of lightweight query networks and difficulty in maintaining retrieval ranking consistency, while providing solutions for asymmetric image retrieval tasks on edge devices without sacrificing performance. Therefore, developing a new asymmetric image retrieval method, optimizing the knowledge distillation process, and improving the consistency of query networks with gallery network retrieval results have become key research directions in this field. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an asymmetric image retrieval method based on decoupled differential knowledge distillation, which solves the problem of how to maintain the consistency of the retrieval order between the query network and the gallery network in asymmetric image retrieval. By focusing on the difference between the pairwise similarities of images rather than the absolute similarity values, the method effectively maintains a consistent retrieval order between the query network and the gallery network when the representation capacity of the query network is limited. In addition, the method further improves the retrieval performance by decoupling the pairwise similarity differential knowledge and independently transferring the feature knowledge and the pairwise similarity differential knowledge.

[0007] To achieve the above object, the technical solution provided by the present invention is: an asymmetric image retrieval method based on decoupled differential knowledge distillation, comprising the following steps:

[0008] 1) Data preparation:

[0009] Given a batch containing n images, resize the image set X to obtain a low-resolution image set X L and a high-resolution image set X H ;

[0010] 2) Set up the query and gallery network:

[0011] Set up a lightweight query network θ q (·), used to convert the low-resolution query image into a standardized d-dimensional vector; set a large gallery network θ g (·), used to convert high-resolution gallery images into standardized d-dimensional vectors;

[0012] 3) Feature extraction and similarity matrix construction:

[0013] By querying the network θ q (·) Extract low-resolution image set X L The features of the query image are obtained Represents the query image feature set V Q The features of the i-th image in, that is, the i-th query image features, i = 1, 2, ..., n; through the gallery network θ g (·) Extract high-resolution image set X H The features of the image library are obtained Represents the feature set V of the gallery image G The features of the i-th image in , that is, the i-th gallery image features; then, calculate the gallery image feature set V G and the query image feature set V Q The cosine similarity matrix G between Q And the gallery image feature set V GThe cosine similarity matrix G between the features of the internal image library G ; G Q and G G Contains the similarity information between the query image and the gallery image, providing a basis for subsequent retrieval ranking and similarity difference analysis;

[0014] 4) Construct the similarity matrix of the first k searches:

[0015] Based on the cosine similarity matrix G G , calculate the index of the top k most similar gallery images corresponding to each query image, and obtain a top-k index matrix R. Use the top-k index matrix R to perform the cosine similarity matrix G Q and G G Sort and construct the top-k retrieval similarity matrix C between the query image and the gallery image. Q And the top-k retrieval similarity matrix C between the gallery images themselves G , C Q and C G It will be used for the calculation of the subsequent feature distillation loss function and pairwise similarity difference distillation loss function;

[0016] 5) Decoupled pairwise similarity difference knowledge distillation:

[0017] Based on the obtained top-k retrieval similarity matrix C Q and C G In order to ensure the dominance of feature knowledge, a decoupled pairwise similarity difference knowledge distillation method is proposed, which is further decoupled into three types of knowledge: feature knowledge, inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge. Then, three loss functions are designed: feature distillation loss function L f , inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd , distill the above three types of knowledge respectively, and train and optimize the query network;

[0018] 6) Optimization of comprehensive loss function:

[0019] Design a comprehensive loss function L student , which is a function of feature distillation loss function L f and the decoupled pairwise similarity differential distillation loss function L rpd The decoupled pairwise similarity difference distillation loss function L is defined by a combination of rpd Including inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function Lcrpd , comprehensively considers the loss of feature knowledge distillation and pairwise similarity difference knowledge distillation, aiming to enhance the performance of the query network through this multi-dimensional loss function optimization in the training stage;

[0020] 7) Complete the retrieval of the image to be tested:

[0021] Use the trained and optimized query network to accurately retrieve the image to be tested and complete the retrieval task.

[0022] Further, in step 1), the image set containing n images is After adjusting the image size in image set X, a low-resolution image set is obtained and a high-resolution image set Among them, x i represents the i-th image in the image set X, Represents a low-resolution image set X L The i-th image in represents a high-resolution image set X H The i-th image in .

[0023] Further, in step 3), query the image features is obtained by querying the network θ q (·) Extract low-resolution image set X L The characteristics of

[0024]

[0025] Gallery Image Features is through the gallery network θ g (·) Extract high-resolution image set X H The characteristics of

[0026]

[0027] Next, calculate the image feature set V G and the query image feature set V Q The cosine similarity matrix G between Q :

[0028]

[0029] In the formula, represents the real number field, that is, the set of all real numbers, and the cosine similarity matrix G Q The dimension is n×n;

[0030] Calculate the image feature set V G The cosine similarity matrix G between the features of the internal image library G :

[0031]

[0032] Further, in step 4), obtaining the top-k index matrix R of the gallery images most similar to the query image is achieved by the argsort function, which is based on the cosine similarity matrix G G The top-k indices are returned in descending order of values, expressed as follows:

[0033]

[0034] In the formula, use argsort(G G , dim=2) function, returns the top-k indices of the cosine similarity values ​​arranged in descending order of the second dimension. The final R is a matrix of dimension n×k, where each row contains the indices of the top-k images most similar to the corresponding gallery image.

[0035] Further, in step 4), the obtained top-k index matrix R is used to perform the cosine similarity matrix G Q and G G Sort and construct two top-k retrieval similarity matrices C Q and C G :

[0036]

[0037]

[0038] In the formula, the sort function is to sort the cosine similarity matrix G according to the obtained top-k index Q and G G Sort and construct two top-k retrieval similarity matrices C Q and C G , C Q and C G They are the top-k retrieval similarity matrices generated by the query network and the gallery network, respectively, with a dimension of n×k. Each row in the matrix contains the similarity values ​​of the top-k most similar images corresponding to each image.

[0039] Further, in step 5), the decoupled pairwise similarity difference knowledge distillation method includes:

[0040] a. Decouple the pairwise similarity difference knowledge into feature knowledge and pairwise similarity difference knowledge;

[0041] b. For feature knowledge, retrieve the similarity matrix C based on top-k Q and C G, design a feature distillation loss function L f To align the representation space between the query network and the gallery network;

[0042] c. Further decoupling the pairwise similarity difference knowledge into inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge;

[0043] e. Retrieve similarity matrix C based on top-k Q and C G , construct two pairwise similarity difference matrices M Q and M G , describes the difference in similarity between each pair of images in the similarity ranking, and is calculated as follows:

[0044]

[0045]

[0046] In the formula, represents the difference in similarity between the j+1th and l+1th most similar images of the ith image in the query network and the gallery network, respectively, with a dimension of n×(k-1)×(k-1). Respectively represent the similarity values ​​of the i-th image to the j+1-th most similar image in the query network and the gallery network, Respectively represent the similarity values ​​of the i-th image in the query network and the j+1-th most similar image in the gallery network, where 1≤j,l≤k-1 means taking the first k-1 similarity values ​​in top-k for comparison;

[0047] Pairwise similarity difference matrix M Q and M G It captures the relative position changes between the query image and the gallery images in the similarity ranking;

[0048] f. Aiming at the inconsistent pairwise similarity difference knowledge, we design an inconsistent pairwise similarity difference distillation loss function L iprd , which is used to capture the inconsistent similarity ranking between the query network and the gallery network; for consistent pairwise similarity difference knowledge, a consistent pairwise similarity difference distillation loss function L is designed crpd , used to capture the parts with consistent similarity ranking between the query network and the gallery network;

[0049] g. Combined with inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd , according to different weights, the pairwise similarity difference distillation loss function L is formed rpd .

[0050] Furthermore, the feature distillation loss function L f The feature distillation loss function L is used to align the representation space of the query network with the representation space of the gallery network. f By computing the top-k retrieval similarity matrix C between the query image and its most similar gallery images Q and the top-k retrieval similarity matrix C between the gallery images themselves G The first column is to calculate the mean square error between the gallery images that are most similar to the query image to achieve the feature distillation loss function L f The form is as follows:

[0051]

[0052] In the formula, n represents the number of images, Represents the similarity value between the i-th image and its most similar first image in the query network and the gallery network respectively. It represents the square difference between the similarity values ​​of the first image that is most similar to the i-th image in the query network and the gallery network. All square differences are summed up and then the square root is taken to finally get the overall feature distillation loss L f .

[0053] Furthermore, the inconsistent pairwise similarity difference distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd The calculation formula is as follows:

[0054]

[0055]

[0056] In the formula, H(·) represents the Heaviside step function. For each image i, it traverses the first k-1 similar image pairs j and l. m is a constant used to avoid the denominator being less than the preset value. They represent the difference in similarity values ​​between the j+1th and l+1th most similar images of the i-th image in the query network and the gallery network, respectively.

[0057] Furthermore, the pairwise similarity difference distillation loss function L rpd The calculation formula is as follows:

[0058] L rpd =βL irpd +γL crpd

[0059] In the formula, β and γ are used to balance L iprd and Lcrpd Hyperparameters of weights.

[0060] Further, in step 6), during the training phase, the comprehensive loss function L of the query network is student as follows:

[0061] L student =αL f +βL irpd +γL crpd

[0062] In the formula, α is used to control L f Hyperparameters of weights.

[0063] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0064] 1. By optimizing the difference relationship of pairwise similarities between the query network and the gallery network, rather than just matching similarity values, the consistency of retrieval order between networks with different capabilities is significantly improved, especially when facing complex and large-scale datasets.

[0065] 2. By decoupling the transfer of feature knowledge and pairwise similarity difference knowledge, the query network is allowed to learn and simulate the retrieval behavior of the gallery network more accurately, achieving highly accurate retrieval results even when its representation capacity is limited.

[0066] In summary, the present invention combines the characteristics of image retrieval and knowledge distillation technology, and achieves a good balance between computational efficiency and retrieval performance. It is particularly suitable for processing asymmetric image retrieval tasks in large-scale image libraries, and can effectively improve the accuracy of query network retrieval while maintaining computational efficiency. It has practical application value and is worthy of promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Flow chart of the method of the present invention.

[0068] Figure 2 This is a principle framework diagram of the decoupled differential distillation method of the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with specific embodiments.

[0070] This embodiment discloses an asymmetric image retrieval method based on decoupled differential knowledge distillation. The method proposes a decoupled differential knowledge distillation method suitable for asymmetric image retrieval. The method maintains a consistent retrieval order by optimizing the difference in pairwise similarity relationships between the query network and the gallery network. The method calculates the pairwise similarity difference matrix in the gallery domain and decouples it into feature knowledge, inconsistent pairwise similarity difference knowledge, and consistent pairwise similarity difference knowledge. This strategic decoupling effectively aligns the retrieval ranking of the query network with the gallery network, which can improve the retrieval accuracy of the query network at a lower computational cost. Figure 1 As shown, the method of the present invention performs data preparation, query and gallery network setting, feature extraction and similarity matrix construction, construction of the similarity matrix of the top k searches, decoupling pairwise similarity difference knowledge distillation, optimization of the comprehensive loss function and query network search on the Stanford online product dataset. The steps include:

[0071] 1) Data preparation:

[0072] Stanford Online Products (SOP) is a product recognition dataset that includes a wide collection of 120,053 product images in 22,634 categories. The training set includes 59,551 training images in 11,318 categories, and the test set includes 60,502 images in 11,316 categories.

[0073] Take the 59,551 images in the training set as image set X, adjust the size of image set X, and obtain a low-resolution query image set X L and a collection of high-resolution gallery imagesX H , the image sizes are 64×64 and 256×256 respectively, such as Figure 2 shown.

[0074] Among them, the image set containing n images is The low-resolution image set is and a high-resolution image set for n=59551.

[0075] 2) Set up the query network and gallery network:

[0076] Use ImageNet pre-trained ResNet101 and ResNet18 as the gallery network θ g (·) and query network θ q (·), which are used to convert low-resolution query images into standardized d-dimensional vectors and high-resolution gallery images into standardized d-dimensional vectors, respectively. Figure 2 shown.

[0077] 3) Feature extraction and similarity matrix construction:

[0078] Using the query network θ q (·) and the gallery network θ g (·) Extract the query image set X respectively L and gallery image collection X H The features of each image in are used to obtain the query image feature set. With gallery image feature set like Figure 2 shown.

[0079] Query image features is obtained by querying the network θ q (·) Extract low-resolution image set X L The characteristics of

[0080]

[0081] in, Represents the query image feature set V Q The features of the i-th image in .

[0082] Gallery Image Features is through the gallery network θ g (·) Extract high-resolution image set X H The characteristics are obtained and defined as follows:

[0083]

[0084] in, Represents the feature set V of the gallery image G The features of the i-th image in .

[0085] Next, construct the image feature set V G and the query image feature set V Q The cosine similarity matrix G between Q , and the gallery image feature set V G The cosine similarity matrix G between the features of the internal image library G .

[0086] Cosine similarity matrix G Q The definition is as follows:

[0087]

[0088] In the formula, represents the real number field, that is, the set of all real numbers, and the cosine similarity matrix G Q The dimension is n×n.

[0089] Cosine similarity matrix G G The definition is as follows:

[0090]

[0091] 4) Construct the similarity matrix of the first k searches:

[0092] Based on the cosine similarity matrix G G , calculate the index of the top k most similar gallery images corresponding to each query image, and obtain a top-k index matrix R. This process is implemented by the argsort function, which is based on the cosine similarity matrix G G The top-k indexes are returned in descending order of the values, using the following formula:

[0093]

[0094] In the formula, use argsort(G G , dim=2) function, returns the top-k indices of the cosine similarity values ​​arranged in descending order of the second dimension. The final R is a matrix of dimension n×k, where each row contains the indices of the top-k images most similar to the corresponding gallery image.

[0095] Next, use the obtained top-k index matrix R to perform the cosine similarity matrix G Q and G G Sort and construct the top-k retrieval similarity matrix C between the query image and the gallery image Q , and the top-k retrieval similarity matrix C between the gallery images themselves G ,like Figure 2 The two matrices are constructed as follows:

[0096]

[0097]

[0098] Among them, C Q represents the top-k retrieval similarity matrix between the query image and the gallery image, C G Represents the top-k retrieval similarity matrix between the gallery images themselves, which is used for the calculation of the subsequent feature distillation loss function and the pairwise similarity difference distillation loss function. The sort function calculates the cosine similarity matrix G according to the obtained top-k index matrix R Q and G G Sort and construct two top-k retrieval similarity matrices C Q and C G .

[0099] 5) Decoupled pairwise similarity difference knowledge distillation:

[0100] Firstly, the pairwise similarity difference knowledge is decoupled into feature knowledge and pairwise similarity difference knowledge, and the latter is further decoupled into inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge.

[0101] For feature knowledge, the similarity matrix C is retrieved based on top-k Q With C G , design feature distillation loss function L f , the purpose is to align the representation space between the query network and the gallery network. By calculating C Q and C G The mean square error between the first column of (the most similar gallery images) is as follows:

[0102]

[0103] Where n represents the number of images, Represent the similarity value between the i-th image and its most similar first image in the query network and the gallery network respectively. It represents the square difference between the similarity values ​​of the first image that is most similar to the i-th image in the query network and the gallery network. All square differences are summed up and then the square root is taken to finally get the overall feature distillation loss L f .

[0104] Next, based on the top-k retrieval similarity matrix C Q and C G , construct two pairwise similarity difference matrices M Q With M G , describes the difference in similarity between each pair of images in the similarity ranking, and the formula is as follows:

[0105]

[0106]

[0107] In the formula, represents the difference in similarity between the j+1th and l+1th most similar images of the ith image in the query network and the gallery network, respectively, with a dimension of n×(k-1)×(k-1). Respectively represent the similarity values ​​of the i-th image to the j+1-th most similar image in the query network and the gallery network, They represent the similarity values ​​of the i-th image in the query network and the j+1-th most similar image in the gallery network, respectively, where 1≤j,l≤k-1 means taking the first k-1 similarity values ​​in top-k for comparison.

[0108] Aiming at the inconsistent pairwise similarity difference knowledge, we design an inconsistent pairwise similarity difference distillation loss function L iprd , captures the inconsistent similarity ranking between the query network and the gallery network, and is calculated using the following formula:

[0109]

[0110] Where H(·) represents the Heaviside step function. For each image i, the first k-1 similar image pairs j and l are traversed. m is a constant with a default value of 0.1 to avoid the denominator being too small. They represent the difference in similarity values ​​between the j+1th and l+1th most similar images of the i-th image in the query network and the gallery network, respectively.

[0111] Based on the consistent pairwise similarity difference knowledge, a consistent pairwise similarity difference distillation loss function L is designed. crpd , captures the part of the query network and the gallery network that has the same similarity ranking, and is calculated by the following formula:

[0112]

[0113] Where H(·) represents the Heaviside step function. For each image i, the first k-1 similar image pairs j and l are traversed. m is a constant with a default value of 0.1 to avoid the denominator being too small. They represent the difference in similarity values ​​between the j+1th and l+1th most similar images of the i-th image in the query network and the gallery network, respectively.

[0114] Finally, we combine the inconsistent pairwise similarity differential distillation loss L iprd With the consistent pairwise similarity differential distillation loss L crpd , combined with different weights to form a pairwise similarity difference distillation loss function L rpd , which is used to further improve the generalization ability of the network. The weights are controlled by hyperparameters β and γ, which are set to 0.2 and 0.1 respectively.

[0115] 6) Optimization of comprehensive loss function:

[0116] During the training phase, query the network’s comprehensive loss function L student It can be expressed as:

[0117] L student =αL f +βL irpd +γL crpd

[0118] Among them, α, β, and γ are the control L f , L iprdand L iprd The hyperparameters of the weights, where α is set to 100, reflects the dominant role of feature knowledge distillation in the comprehensive loss function.

[0119] 7) Complete the retrieval of the image to be tested: Use the trained and optimized query network to accurately retrieve the test data and complete the retrieval task.

[0120] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An asymmetric image retrieval method based on decoupled differential knowledge distillation, characterized in that: The following steps are involved: 1) Data preparation: Given a batch containing n images, resize the image set X to obtain a low-resolution image set X L and a high-resolution image set X H ; 2) Set up the query and gallery network: Set up a lightweight query network θ q (·), used to convert the low-resolution query image into a standardized d-dimensional vector; set a large gallery network θ g (·), used to convert high-resolution gallery images into standardized d-dimensional vectors; 3) Feature extraction and similarity matrix construction: By querying the network θ q (·) Extract low-resolution image set X L The features of the query image are obtained Represents the query image feature set V Q The features of the i-th image in, that is, the i-th query image features, i = 1, 2, ..., n; through the gallery network θ g (·) Extract high-resolution image set X H The features of the image library are obtained Represents the feature set V of the gallery image G The features of the i-th image in , that is, the i-th gallery image features; then, calculate the gallery image feature set V G and the query image feature set V Q The cosine similarity matrix G between Q And the gallery image feature set V G The cosine similarity matrix G between the features of the internal image library G ; G Q and G G Contains the similarity information between the query image and the gallery image, providing a basis for subsequent retrieval ranking and similarity difference analysis; 4) Construct the similarity matrix of the first k searches: Based on the cosine similarity matrix G G , calculate the index of the top k most similar gallery images corresponding to each query image, and obtain a top-k index matrix R. Use the top-k index matrix R to perform the cosine similarity matrix G Q and G G Sort and construct the top-k retrieval similarity matrix C between the query image and the gallery image. Q And the top-k retrieval similarity matrix C between the gallery images themselves G , C Q and C G It will be used for the calculation of the subsequent feature distillation loss function and pairwise similarity difference distillation loss function; 5) Decoupled pairwise similarity difference knowledge distillation: Based on the obtained top-k retrieval similarity matrix C Q and C G In order to ensure the dominance of feature knowledge, a decoupled pairwise similarity difference knowledge distillation method is proposed, which is further decoupled into three types of knowledge : Feature knowledge, inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge, and then through the three designed loss functions: feature distillation loss function L f , inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd , distill the above three types of knowledge respectively, and train and optimize the query network; The decoupled pairwise similarity difference knowledge distillation method comprises: a. Decouple the pairwise similarity difference knowledge into feature knowledge and pairwise similarity difference knowledge; b. For feature knowledge, retrieve the similarity matrix C based on top-k Q and C G , design a feature distillation loss function L f To align the representation space between the query network and the gallery network; c. Further decoupling the pairwise similarity difference knowledge into inconsistent pairwise similarity difference knowledge and consistent pairwise similarity difference knowledge; e. Retrieve similarity matrix C based on top-k Q and C G , construct two pairwise similarity difference matrices M Q and M G , describes the difference in similarity between each pair of images in the similarity ranking, and is calculated as follows: In the formula, represents the difference in similarity between the j+1th and l+1th most similar images of the ith image in the query network and the gallery network, respectively, with a dimension of n×(k-1)×(k-1), Respectively represent the similarity values ​​of the i-th image to the j+1-th most similar image in the query network and the gallery network, Respectively represent the similarity values ​​of the i-th image to the l+1th most similar image in the query network and the gallery network, where 1≤j,l≤k-1 means taking the first k-1 similarity values ​​in top-k for comparison; Pairwise similarity difference matrix M Q and M G It captures the relative position changes between the query image and the gallery images in the similarity ranking; f. Aiming at the inconsistent pairwise similarity difference knowledge, we design an inconsistent pairwise similarity difference distillation loss function L iprd , which is used to capture the inconsistent similarity ranking between the query network and the gallery network; for consistent pairwise similarity difference knowledge, a consistent pairwise similarity difference distillation loss function L is designed crpd , used to capture the parts with consistent similarity ranking between the query network and the gallery network; g. Combined with inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd , according to different weights, the pairwise similarity difference distillation loss function L is formed rpd ; The inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd The calculation formula is as follows: In the formula, H(·) represents the Heaviside step function. For each image i, it traverses the first k-1 similar image pairs j and l. m is a constant used to avoid the denominator being less than the preset value. Represents the similarity value difference between the j+1th and l+1th most similar images of the i-th image in the query network and the gallery network respectively; 6) Optimization of comprehensive loss function: Design a comprehensive loss function L student , which is a function of feature distillation loss function L f and the decoupled pairwise similarity differential distillation loss function L rpd The decoupled pairwise similarity difference distillation loss function L is defined by a combination of rpd Including inconsistent pairwise similarity differential distillation loss function L iprd and the consistent pairwise similarity differential distillation loss function L crpd , comprehensively considers the loss of feature knowledge distillation and pairwise similarity difference knowledge distillation, aiming to enhance the performance of the query network through this multi-dimensional loss function optimization in the training stage; 7) Complete the retrieval of the image to be tested: Use the trained and optimized query network to accurately retrieve the image to be tested and complete the retrieval task.

2. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 1, characterized in that: In step 1), the image set containing n images is After adjusting the image size in image set X, a low-resolution image set is obtained and a high-resolution image set Among them, x i represents the i-th image in the image set X, Represents a low-resolution image set X L The i-th image in represents a high-resolution image set X H The i-th image in .

3. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 2 is characterized in that: In step 3), query image features is obtained by querying the network θ q (·) Extract low-resolution image set X L The characteristics of Gallery Image Features is through the gallery network θ g (·) Extract high-resolution image set X H The characteristics of Next, calculate the image feature set V G and the query image feature set V Q The cosine similarity matrix G between Q : In the formula, represents the real number field, that is, the set of all real numbers, and the cosine similarity matrix G Q The dimension is n×n; Calculate the image feature set V G The cosine similarity matrix G between the features of the internal image library G :

4. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 3 is characterized in that: In step 4), the top-k index matrix R of the gallery images most similar to the query image is obtained by the argsort function, which is based on the cosine similarity matrix G G The top-k indices are returned in descending order of values, expressed as follows: In the formula, use argsort(G G , dim=2) function, returns the top-k indices of the cosine similarity values ​​arranged in descending order of the second dimension. The final R is a matrix of dimension n×k, where each row contains the indices of the top-k images most similar to the corresponding gallery image.

5. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 4, characterized in that: In step 4), the obtained top-k index matrix R is used to perform the cosine similarity matrix G Q and G G Sort and construct two top-k retrieval similarity matrices C Q and C G : In the formula, the sort function is to sort the cosine similarity matrix G according to the obtained top-k index Q and G G Sort and construct two top-k retrieval similarity matrices C Q and C G , C Q and C G They are the top-k retrieval similarity matrices generated by the query network and the gallery network, respectively, with a dimension of n×k. Each row in the matrix contains the similarity values ​​of the top-k most similar images corresponding to each image.

6. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 5, characterized in that: The feature distillation loss function L f The feature distillation loss function L is used to align the representation space of the query network with the representation space of the gallery network. f By computing the top-k retrieval similarity matrix C between the query image and its most similar gallery images Q and the top-k retrieval similarity matrix C between the gallery images themselves G The first column is to calculate the mean square error between the gallery images that are most similar to the query image to achieve the feature distillation loss function L f The form is as follows: In the formula, n represents the number of images, Represents the similarity value between the i-th image and its most similar first image in the query network and the gallery network respectively. It represents the square difference between the similarity values ​​of the first image that is most similar to the i-th image in the query network and the gallery network. Sum all the square differences and take the square root to get the overall feature distillation loss L. f .

7. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 6, characterized in that: The pairwise similarity difference distillation loss function L rpd The calculation formula is as follows: L rpd =βL irpd +γL crpd In the formula, β and γ are used to balance L iprd and L crpd Hyperparameters of weights.

8. The asymmetric image retrieval method based on decoupled differential knowledge distillation according to claim 7, characterized in that: In step 6), during the training phase, the comprehensive loss function L of the query network is student as follows: L student =αL f +βL irpd +γL crpd In the formula, α is used to control L f Hyperparameters of weights.

Images