A low-quality remote sensing image deep hashing retrieval method, system, device and medium based on vector quantization

Abstract

A low-quality remote sensing image depth hash retrieval method, system, device and medium based on vector quantization, the method comprising: first, obtaining high-quality and low-quality remote sensing images as input, extracting the features of high-quality and low-quality remote sensing images through a deep convolutional neural network respectively, and inputting them into a vector quantization module to quantize them into independent discrete spaces to generate quantized features, then generating hash codes for image retrieval through a deep hash network, and finally obtaining feature representations and constraining the feature representations by applying a loss function, which includes Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information retention, feature distance constraint and collaborative learning of the encoder, codebook and decoder; the system, device and medium are used to implement the method; the present application improves the retrieval accuracy of low-quality remote sensing images, reduces the storage space occupation and computing overhead, improves the robustness and generalization ability of the model, and ensures high retrieval performance.

Description

Technical Field

[0001] This invention relates to the field of image retrieval and image processing technology, and in particular to a method, system, device and medium for deep hash retrieval of low-quality remote sensing images based on vector quantization technology, which aims to improve the performance of low-quality images in remote sensing image retrieval tasks. Background Technology

[0002] Deep learning has demonstrated significant advantages in computer vision tasks, but most current vision tasks often rely on high-quality images for training and testing. However, in practical applications, image acquisition is often affected by various factors, such as sensor noise, camera motion blur, and atmospheric interference, leading to a decline in image quality. These low-quality images pose a challenge to the performance and generalization ability of models, as deep learning models struggle to accurately extract key features from low-quality images.

[0003] Existing research primarily utilizes feature distillation to improve the performance of low-quality images. This involves using deep neural networks to extract features from low-quality images that are similar to those in high-quality images, thereby improving the feature representation of low-quality images and applying them to downstream tasks. Additionally, some studies employ image restoration methods, first restoring low-quality images to high-quality images before performing feature extraction and subsequent processing. These methods have also achieved some success.

[0004] In the field of remote sensing image retrieval, existing research has largely focused on retrieving high-quality remote sensing images, with less attention paid to low-quality images. However, in practical applications, remote sensing images are often affected by various factors, such as weather, environment, and equipment noise, which lead to a decline in image quality and consequently affect retrieval performance.

[0005] Chinese patent application CN 118312636 A discloses a method for retrieving marine remote sensing ship images based on self-attention hashing. This method utilizes a self-attention mechanism to generate hash codes, thereby improving the accuracy and efficiency of image retrieval. However, because this method only focuses on remote sensing image retrieval under ideal conditions and fails to consider the impact of complex environments (such as weather changes and sea state interference) on remote sensing images, it has the drawback of being unable to overcome the challenges of image retrieval under complex environments.

[0006] Liu et al. published "Deep hash learning for remote sensing image retrieval" [J] (C.Liu, J.Ma, X.Tang, F.Liu, X.Zhang, and L.Jiao. Deep hash learning for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and RemoteSensing, 2020, Vol. 59(4): 3420-3443). This paper proposes a remote sensing image retrieval method based on deep hash learning, which improves retrieval efficiency by using deep neural networks for feature extraction and generating compact hash codes. However, because this method fails to fully consider the quantization error of low-quality image features during hash code generation, the retrieval accuracy of low-quality images decreases, resulting in a drawback of not being able to maintain consistency between low-quality and high-quality images.

[0007] Song et al. published "Asymmetric hash code learning for remote sensing image retrieval" [J] (W.Song, Z.Gao, R.Dian, P.Ghamisi, Y.Zhang, and J.A.Benediktsson. Asymmetric hash code learning for remote sensing image retrieval [J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, Vol. 60: 1-14). This paper proposes an asymmetric hash code learning method to improve the accuracy of remote sensing image retrieval by generating different hash codes for different modal data. However, because this method is insensitive to modal differences when processing low-quality remote sensing images, the retrieval performance of low-quality images deteriorates, resulting in the drawback of not being able to fully utilize multimodal information for accurate retrieval. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, the present invention aims to provide a method, system, device, and medium for deep hash retrieval of low-quality remote sensing images based on vector quantization. By introducing vector quantization into the deep hash retrieval task of low-quality remote sensing images, the essential features of low-quality remote sensing images are learned. By constructing codebooks for high-quality and low-quality remote sensing images, and utilizing three loss functions—pairwise loss, cross-entropy loss, and reconstruction loss—the mapping of features from low-quality remote sensing images to features from high-quality remote sensing images is achieved, thereby improving the retrieval performance of low-quality remote sensing images.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for deep hashing retrieval of low-quality remote sensing images based on vector quantization includes the following steps:

[0011] Step 1: Image Feature Extraction: By acquiring high-quality and low-quality remote sensing images as input, features of the high-quality and low-quality remote sensing images are extracted respectively using a deep convolutional neural network;

[0012] Step 2: Vector Quantization: Input the features extracted in Step 1 into the vector quantization module, quantize them into an independent discrete space, and generate quantized features;

[0013] Step 3: Hash code generation: The quantized features generated in Step 2 are used to generate hash codes for image retrieval through a deep hashing network;

[0014] Step 4: Application of loss function: After feature extraction, feature quantization and hash encoding in steps 1 to 3, feature representation is obtained. The feature representation is constrained by applying loss function, which includes Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information preservation, feature distance constraints and collaborative learning of encoder, codebook and decoder.

[0015] The specific steps of step 1 are as follows:

[0016] High-quality remote sensing images are represented as X = {x1, x2, ..., x} n} and the corresponding low-quality remote sensing images are represented as Simultaneously, the data is input into the backbone network AlexNet, which extracts image features. After feature extraction, dimensionality reduction is performed through two fully connected layers to generate real-valued hash codes. The real-valued hash code of a high-quality remote sensing image is represented as h = {h1, h2, ..., h...}. n The real-valued hash code representation of a low-quality remote sensing image is: Where h and All belong to R KK is the number of bits in the real-valued hash code. The formula for generating the real-valued hash code is as follows:

[0017] h i =Φ(x i ;θ), i=1,2,…,N

[0018]

[0019] Where θ represents the weight parameters of the backbone network AlexNet, the two fully connected layers, and the hash layer, and Φ represents a mapping function characterized by θ, which includes convolution, pooling, and fully connected operations.

[0020] The specific steps of step 2 are as follows:

[0021] Furthermore, to obtain the quantized image features, the vector quantization steps are as follows:

[0022] Two learnable codebooks E∈R m×K Added at the end of the feature extraction network, it is used to quantify the features of low-quality and high-quality remote sensing images, respectively, where m is the number of codewords in the codebook, and e∈R. K For codewords, i.e., codebook E∈R m ×K For a given vector in the image, the real-valued hash codes of high-quality and low-quality remote sensing images are vector-quantized using the corresponding codebooks to obtain quantization features. For any real-valued hash code h of a high-quality remote sensing image, its standard quantization process is expressed as:

[0023] s=E(h)=e q

[0024]

[0025] Where E(h) represents discretizing h into a codeword in the codebook, e q Let q represent the q-th codeword in the codebook, where q is the index of the codeword in codebook E that has the smallest Euclidean distance to the real-valued hash code h. Then, the L codewords with the smallest Euclidean distance to the real-valued hash code h are selected and weighted and summed to supplement the information of the quantization features, ultimately improving its representational ability. Specifically, for any real-valued hash code h of a remote sensing image, the L codewords with the smallest Euclidean distance to it are {e1, e2, ..., e...}. L The distance between the real-valued hash code h of the high-quality remote sensing image and the image itself is {d1, d2, ..., d}. L The weights {β1,β2,…,β} are calculated based on the distances between these codewords and the real-valued hash codes. L The final formula for the quantization vector z is:

[0026]

[0027] After vector quantization, the quantization features of a high-quality remote sensing image are represented as Z = {z1, z2, ..., z}. n The quantization features of low-quality remote sensing images are represented as follows:

[0028] The specific steps of step 3 are as follows:

[0029] Furthermore, in order to obtain a hash code that can be used for retrieval, the hash code generation steps are as follows:

[0030] To reduce quantization errors during the conversion of real-valued hash codes to hash codes, after generating the real-valued hash codes, the activation function tanh(·) is used to limit the value of each dimension of the real-valued hash code to between -1 and 1. The specific formula is as follows:

[0031]

[0032] For the real-valued hash code z of a high-quality remote sensing image i Its normalized hash code v i Represented as:

[0033] v i =tanh(z) i ), i = 1, 2, ..., N

[0034] Similarly, for real-valued hash codes of low-quality remote sensing images Its normalized hash code Represented as:

[0035]

[0036] To aid codebook learning in deep hashing networks, quantized features are constrained by cross-entropy loss in the classification layer to inject semantic information. The specific formula is as follows:

[0037] p i =fc(v i ), i = 1, 2, ..., N

[0038]

[0039] Where fc(·) represents the classification layer, p i Let represent the classification vector of the i-th high-quality remote sensing image. This represents the classification vector of the i-th low-quality remote sensing image;

[0040] The process of generating hash codes involves using vector p i and Binarization is performed to generate the hash code b of the final high-quality remote sensing image. iHash codes of low-quality remote sensing images The specific formula is as follows:

[0041]

[0042] b i =sign(v i ), i = 1, 2, ..., N

[0043]

[0044] Among them, b i v represents the hash code of the i-th high-quality remote sensing image. i It is its corresponding normalized real-valued hash code; similarly, This represents the hash code of the i-th low-quality remote sensing image. It is the normalized real-valued hash code of the corresponding low-quality remote sensing image. The real-valued hash code is converted into binary form by the sign(·) function, thereby generating the final hash code for retrieval.

[0045] The specific steps of step 4 are as follows:

[0046] To ensure the preservation of semantic information in the remote sensing image feature representation, the constraint of feature distance, and the collaborative learning of the encoder, codebook, and decoder, the loss function is applied in the following steps:

[0047] The Pairwise Loss function is used to constrain the feature space learning of high-quality and low-quality remote sensing images, ensuring that features from images of the same class are closer together and features from images of different classes are farther apart. Specifically, Pairwise Loss is used in the following three cases:

[0048] Case 1: Embedding Space Constraints for High-Quality Remote Sensing Images: This constraint is used to constrain the feature vectors of high-quality remote sensing images, ensuring that the feature distances between high-quality remote sensing images of the same class are closer, while the feature distances between high-quality remote sensing images of different classes are farther. The loss function formula is as follows:

[0049]

[0050] Where n represents the number of remote sensing image samples, i and j represent the indices of the remote sensing image samples, m represents the distance threshold, and θ ij For the similarity indicator matrix, v is defined as being similar to high-quality remote sensing images. i and v j These are the feature vectors of low-quality remote sensing images;

[0051] Case 2: Embedding Spatial Constraints for Low-Quality Remote Sensing Images: This constraint is used to constrain the features of low-quality remote sensing images, making the feature distances between low-quality images of the same class closer and the feature distances between low-quality images of different classes farther. The loss function formula is as follows:

[0052]

[0053] Where n represents the number of remote sensing image samples, i and j represent the indices of the remote sensing image samples, m represents the distance threshold, and θ ij The similarity indicator matrix is ​​defined to be similar to that of high-quality remote sensing images. and These are the feature vectors of low-quality remote sensing images;

[0054] Case 3: Pairing constraint for high-quality and low-quality remote sensing image features: This constraint brings high-quality and low-quality remote sensing image features of the same category closer together, while keeping features of different categories further apart. The loss function formula is as follows:

[0055]

[0056] Where, φ ij The similarity indicator matrix represents high-quality and low-quality remote sensing images, when φ ij When φ = 1, it indicates that the i-th high-quality remote sensing image and the j-th low-quality remote sensing image belong to the same category; when φ = 1, it indicates that the i-th high-quality remote sensing image and the j-th low-quality remote sensing image belong to the same category; ij When the value is 0, it indicates that they belong to different categories;

[0057] The overall Pairwise Loss function can be expressed as:

[0058] L pairwise =L pairwise-high +L pairwise-low +L pairwise-hl

[0059] The cross-entropy loss function guides the learning of the feature extraction network and codebook, ensuring that the quantized features of both high-quality and low-quality remote sensing images retain their semantic information. Specifically, the cross-entropy loss function optimizes the classification performance of high-quality and low-quality remote sensing images, as shown in the following formula:

[0060]

[0061] Among them, y ic For the true label of the i-th high-quality remote sensing image with respect to category c, p ic The predicted probability of a high-quality remote sensing image with respect to category c; Let c be the true label of the i-th low-quality remote sensing image with respect to category c. Let C be the predicted probability of a low-quality remote sensing image with respect to category c; where C is the number of categories.

[0062] The reconstruction loss function guides the collaborative learning of the feature extraction network and the codebook. Since the argmin operation in vector quantization is not differentiable, a straight-through estimator (STE) method is used. This method bypasses the codebook and passes the gradients from the pairwise loss and cross-entropy loss to the preceding feature extraction network, thereby updating the parameters of the feature extraction network. The specific loss function formula is as follows:

[0063] L commitment =|h-sg(e q )| 2

[0064] Here, sg(·) refers to the stop gradient operation, which is used to prevent gradients from propagating in the codebook. Although the pass-through estimation method can achieve gradient propagation, the codebook cannot directly receive gradient information from Pairwise Loss and cross-entropy loss, which limits codebook learning.

[0065] To ensure the codebook space can participate properly in learning, a dictionary learning method is adopted. This method optimizes the codebook learning process by calculating the L2 error between the features output by the feature extraction network and the corresponding quantized codewords. The specific loss function formula is as follows:

[0066] L vq =|sg(h)-e q | 2

[0067] Where h is the output of the feature extraction network, e q For the corresponding codeword;

[0068] The overall optimization problem of the model is to optimize the learnable parameters in the network framework by jointly using all loss functions. The overall loss function formula is as follows:

[0069]

[0070] Here, Θ represents all the learnable parameters in the network framework of this chapter.

[0071] A low-quality remote sensing image depth hashing retrieval system based on vector quantization, comprising:

[0072] Image feature extraction module: It takes high-quality and low-quality remote sensing images as input and extracts features from the high-quality and low-quality remote sensing images respectively through a deep convolutional neural network;

[0073] Vector quantization module: Input the features into the vector quantization module, quantize them into an independent discrete space, and generate quantized features;

[0074] Hash encoding generation module: Generates hash codes for image retrieval from features using a deep hashing network;

[0075] Loss function application module: After feature extraction, feature quantization and hash encoding, feature representation is obtained. The feature representation is constrained by applying loss functions, including Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information preservation, feature distance constraints and collaborative learning of encoder, codebook and decoder.

[0076] A low-quality remote sensing image depth hashing retrieval device based on vector quantization, comprising:

[0077] Memory: Used to store the computer program that implements the vector quantization-based deep hash retrieval method for low-quality remote sensing images;

[0078] Processor: Used to implement the vector quantization-based deep hash retrieval method for low-quality remote sensing images when executing the computer program.

[0079] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a vector quantization-based depth hash retrieval method for low-quality remote sensing images.

[0080] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0081] 1. This invention uses vector quantization to map the features of low-quality remote sensing images into a discrete space, thereby improving the retrieval accuracy of low-quality remote sensing images and effectively reducing quantization errors.

[0082] 2. This invention encodes quantized features using a deep hash network to generate compact hash codes, which significantly reduces storage space usage and computational overhead during image retrieval.

[0083] 3. This invention employs a combination of pairwise loss, cross-entropy loss, and reconstruction loss, which effectively ensures feature consistency between low-quality and high-quality remote sensing images, thereby improving the robustness and generalization ability of the model.

[0084] 4. This invention employs a combination of vector quantization and deep hashing learning, which is particularly effective for low-quality remote sensing images. It can maintain high retrieval performance even in complex environments (such as weather and noise conditions), overcoming the shortcomings of existing technologies in complex environments.

[0085] In summary, this invention improves the retrieval accuracy of low-quality remote sensing images, reduces storage space and computational overhead, enhances the robustness and generalization ability of the model, and ensures high retrieval performance. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the network structure of the present invention.

[0087] Figure 2 This is a schematic diagram of the standard vector quantization process of the present invention.

[0088] Figure 3 This is a schematic diagram of the through-pass estimation process of the present invention.

[0089] Figure 4 This is an example of a low-quality remote sensing image from the present invention.

[0090] Figure 5 This is a retrieval effect diagram of a query example of the AID(fog) dataset under the 64-bit hash encoding of this invention.

[0091] Figure 6 This is a retrieval result diagram of an example query for the AID (gaussian_noise) dataset under 64-bit hash encoding according to the present invention.

[0092] Figure 7 This is a retrieval effect diagram of an example query for the AID (motion_blur) dataset under 64-bit hash encoding according to the present invention. Detailed Implementation

[0093] The present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0094] This invention proposes a deep hashing retrieval method for low-quality remote sensing images based on vector quantization. For example... Figure 1 As shown, this method optimizes the feature extraction and retrieval performance of both high-quality and low-quality remote sensing images by combining vector quantization, deep hashing networks, and various loss functions. This primarily addresses the performance bottleneck of low-quality remote sensing images in remote sensing image retrieval.

[0095] A method for deep hashing retrieval of low-quality remote sensing images based on vector quantization includes the following steps:

[0096] Step 1: Image Feature Extraction: By acquiring high-quality and low-quality remote sensing images as input, features of the high-quality and low-quality remote sensing images are extracted respectively using a deep convolutional neural network;

[0097] Step 2: Vector Quantization: Input the features extracted in Step 1 into the vector quantization module, quantize them into an independent discrete space, and generate quantized features;

[0098] Step 3: Hash code generation: The quantized features generated in Step 2 are used to generate hash codes for image retrieval through a deep hashing network;

[0099] Step 4: Application of Loss Function: After feature extraction, feature quantization, and hash encoding in steps 1 to 3, feature representations are obtained. The feature representations are then constrained by applying loss functions, including Pairwise Loss, reconstruction loss, and cross-entropy loss, to ensure semantic information preservation, feature distance constraints, and collaborative learning among the encoder, codebook, and decoder.

[0100] The specific steps of step 1 are as follows:

[0101] High-quality remote sensing images are represented as X = {x1, x2, ..., x} n} and the corresponding low-quality remote sensing images are represented as Simultaneously, the data is input into the backbone network AlexNet, which extracts image features. After feature extraction, dimensionality reduction is performed through two fully connected layers to generate real-valued hash codes. The real-valued hash code of a high-quality remote sensing image is represented as h = {h1, h2, ..., h...}. n The real-valued hash code representation of a low-quality remote sensing image is: Where h and All belong to R K K is the number of bits in the real-valued hash code. The formula for generating the real-valued hash code is as follows:

[0102] h i =Φ(x i ;θ), i=1,2,…,N

[0103]

[0104] Where θ represents the weight parameters of the backbone network AlexNet, the two fully connected layers, and the hash layer, and Φ represents a mapping function characterized by θ, which includes convolution, pooling, and fully connected operations.

[0105] like Figure 2 As shown, the specific steps of step 2 are as follows:

[0106] Furthermore, to obtain the quantized image features, the vector quantization steps are as follows:

[0107] Two learnable codebooks E∈R m×KAdded at the end of the feature extraction network, it is used to quantify the features of low-quality and high-quality remote sensing images, respectively, where m is the number of codewords in the codebook, and e∈R. K For codewords, i.e., codebook E∈R m ×K For a given vector in the image, the real-valued hash codes of high-quality and low-quality remote sensing images are vector-quantized using the corresponding codebooks to obtain quantization features. For any real-valued hash code h of a high-quality remote sensing image, its standard quantization process is expressed as:

[0108] s=E(h)=e q

[0109]

[0110] Where E(h) represents discretizing h into a codeword in the codebook, e q Let q represent the q-th codeword in the codebook, where q is the index of the codeword in codebook E that has the smallest Euclidean distance to the real-valued hash code h. Then, the L codewords with the smallest Euclidean distance to the real-valued hash code h are selected and weighted and summed to supplement the information of the quantization features, ultimately improving its representational ability. Specifically, for any real-valued hash code h of a remote sensing image, the L codewords with the smallest Euclidean distance to it are {e1, e2, ..., e...}. L The distance between the real-valued hash code h of the high-quality remote sensing image and the image itself is {d1, d2, ..., d}. L The weights {β1,β2,…,β} are calculated based on the distances between these codewords and the real-valued hash codes. L The final formula for the quantized vector z is:

[0111]

[0112] After vector quantization, the quantization features of a high-quality remote sensing image are represented as Z = {z1, z2, ..., z}. n The quantization features of low-quality remote sensing images are represented as follows:

[0113] The specific steps of step 3 are as follows:

[0114] Furthermore, in order to obtain a hash code that can be used for retrieval, the hash code generation steps are as follows:

[0115] To reduce quantization errors during the conversion of real-valued hash codes to hash codes, after generating the real-valued hash codes, the activation function tanh(·) is used to limit the value of each dimension of the real-valued hash code to between -1 and 1. The specific formula is as follows:

[0116]

[0117] For the real-valued hash code z of a high-quality remote sensing imagei Its normalized hash code v i Represented as:

[0118] v i =tanh(z) i ), i = 1, 2, ..., N

[0119] Similarly, for real-valued hash codes of low-quality remote sensing images Its normalized hash code Represented as:

[0120]

[0121] To aid codebook learning in deep hashing networks, quantized features are constrained by cross-entropy loss in the classification layer to inject semantic information. The specific formula is as follows:

[0122] p i =fc(v i ), i = 1, 2, ..., N

[0123]

[0124] Where fc(·) represents the classification layer, p i Let represent the classification vector of the i-th high-quality remote sensing image. This represents the classification vector of the i-th low-quality remote sensing image;

[0125] The process of generating hash codes involves using vector p i and Binarization is performed to generate the hash code b of the final high-quality remote sensing image. i Hash codes of low-quality remote sensing images The specific formula is as follows:

[0126]

[0127] b i =sign(v i ), i = 1, 2, ..., N

[0128]

[0129] Among them, b i v represents the hash code of the i-th high-quality remote sensing image. i It is its corresponding normalized real-valued hash code; similarly, This represents the hash code of the i-th low-quality remote sensing image. It is the normalized real-valued hash code of the corresponding low-quality remote sensing image. The real-valued hash code is converted into binary form by the sign(·) function, thereby generating the final hash code for retrieval.

[0130] The specific steps of step 4 are as follows:

[0131] To ensure the preservation of semantic information in image feature representations, the constraint of feature distance, and the collaborative learning of the encoder, codebook, and decoder, the loss function is applied in the following steps:

[0132] The Pairwise Loss function is used to constrain the feature space learning of high-quality and low-quality remote sensing images, ensuring that features from images of the same class are closer together and features from images of different classes are farther apart. Specifically, Pairwise Loss is used in the following three cases:

[0133] Case 1: Embedding Space Constraints for High-Quality Remote Sensing Images: This constraint is used to constrain the feature vectors of high-quality remote sensing images, ensuring that the feature distances between high-quality remote sensing images of the same class are closer, while the feature distances between high-quality remote sensing images of different classes are farther. The loss function formula is as follows:

[0134]

[0135] Where n represents the number of remote sensing image samples, i and j represent the indices of the remote sensing image samples, m represents the distance threshold, and θ ij For the similarity indicator matrix, v is defined as being similar to high-quality remote sensing images. i and v j These are the feature vectors of low-quality remote sensing images;

[0136] Case 2: Embedding Spatial Constraints for Low-Quality Remote Sensing Images: This constraint is used to constrain the features of low-quality remote sensing images, making the feature distances between low-quality images of the same class closer and the feature distances between low-quality images of different classes farther. The loss function formula is as follows:

[0137]

[0138] Where n represents the number of remote sensing image samples, i and j represent the indices of the remote sensing image samples, m represents the distance threshold, and θ ij The similarity indicator matrix is ​​defined to be similar to that of high-quality remote sensing images. and These are the feature vectors of low-quality remote sensing images;

[0139] Case 3: Pairing constraint for high-quality and low-quality remote sensing image features: This constraint brings high-quality and low-quality remote sensing image features of the same category closer together, while keeping features of different categories further apart. The loss function formula is as follows:

[0140]

[0141] Where, φ ij The similarity indicator matrix represents high-quality and low-quality remote sensing images, when φ ij When φ = 1, it indicates that the i-th high-quality remote sensing image and the j-th low-quality remote sensing image belong to the same category; when φ = 1, it indicates that the i-th high-quality remote sensing image and the j-th low-quality remote sensing image belong to the same category; ij When the value is 0, it indicates that they belong to different categories;

[0142] The overall Pairwise Loss function can be expressed as:

[0143] L pairwise =L pairwise-high +L pairwise-low +L pairwise-hl

[0144] The cross-entropy loss function guides the learning of the feature extraction network and codebook, ensuring that the quantized features of both high-quality and low-quality remote sensing images retain their semantic information. Specifically, the cross-entropy loss function optimizes the classification performance of high-quality and low-quality remote sensing images, as shown in the following formula:

[0145]

[0146] Among them, y ic For the true label of the i-th high-quality remote sensing image with respect to category c, p ic The predicted probability of a high-quality remote sensing image with respect to category c; Let c be the true label of the i-th low-quality remote sensing image with respect to category c. Let C be the predicted probability of a low-quality remote sensing image with respect to category c; where C is the number of categories.

[0147] The reconstruction loss function is used to guide the collaborative learning of the feature extraction network and the codebook, such as... Figure 3 As shown, since the arg min operation in vector quantization is not differentiable, the Straight-Through Estimator (STE) method is used. This method bypasses the codebook and passes the gradients from the Pairwise Loss and cross-entropy loss to the preceding feature extraction network, thereby updating the parameters of the feature extraction network. The specific loss function formula is as follows:

[0148] L commitment =|h-sg(e q )| 2

[0149] Here, sg(·) refers to the stop gradient operation, which is used to prevent gradients from propagating in the codebook. Although the pass-through estimation method can achieve gradient propagation, the codebook cannot directly receive gradient information from Pairwise Loss and cross-entropy loss, which limits codebook learning.

[0150] To ensure the codebook space can participate properly in learning, a dictionary learning method is adopted. This method optimizes the codebook learning process by calculating the L2 error between the features output by the feature extraction network and the corresponding quantized codewords. The specific loss function formula is as follows:

[0151] L vq =|sg(h)-e q | 2

[0152] Where h is the output of the feature extraction network, e q For the corresponding codeword;

[0153] The overall optimization problem of the model is to optimize the learnable parameters in the network framework by jointly using all loss functions. The overall loss function formula is as follows:

[0154]

[0155] Here, Θ represents all the learnable parameters in the network framework of this chapter.

[0156] A low-quality remote sensing image depth hashing retrieval system based on vector quantization, comprising:

[0157] Image feature extraction module: By acquiring high-quality and low-quality remote sensing images as input, and extracting features from the high-quality and low-quality remote sensing images respectively through a deep convolutional neural network, this module implements step 1 of a deep hash retrieval method for low-quality remote sensing images based on vector quantization.

[0158] Vector quantization module: Input the features into the vector quantization module, quantize them into an independent discrete space, and generate quantized features to implement step 2 of a low-quality remote sensing image depth hash retrieval method based on vector quantization;

[0159] Hash encoding generation module: Generates hash codes for image retrieval from features through a deep hashing network, which is used to implement step 3 of a deep hashing retrieval method for low-quality remote sensing images based on vector quantization;

[0160] Loss function application module: After feature extraction, feature quantization and hash encoding are generated, feature representation is obtained. The feature representation is constrained by applying loss functions, including Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information preservation, feature distance constraints and collaborative learning of encoder, codebook and decoder, and to implement step 4 of a low-quality remote sensing image deep hash retrieval method based on vector quantization.

[0161] A low-quality remote sensing image depth hashing retrieval device based on vector quantization, comprising:

[0162] Memory: Used to store the computer program that implements the vector quantization-based deep hash retrieval method for low-quality remote sensing images;

[0163] Processor: Used to implement the vector quantization-based deep hash retrieval method for low-quality remote sensing images when executing the computer program.

[0164] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the described method for deep hash retrieval of low-quality remote sensing images based on vector quantization.

[0165] To verify the effectiveness of the proposed method in low-quality remote sensing image retrieval tasks, the method was simulated on the AID dataset under three degradation conditions: fog, Gaussian noise, and motion blur. Figure 4 As shown. Subsequent comparative and ablation experiments were conducted on these three datasets. The AID dataset contains 10,000 images in total, comprising 30 classes of remote sensing images, with 220 to 420 images per class, and a resolution of 600×600.

[0166] This invention employs AlexNet as the backbone network for feature extraction. To adapt to hash retrieval tasks, the final classification layer of the AlexNet network is replaced with a hash layer. The output dimension of the hash layer is K, i.e., the number of bits in the binary hash code. The entire network framework is trained using the Adam optimizer, with the learning rate and weight decay parameters set to 1×10⁻⁶. -4 and 5×10 -4In the vector quantization operation, the number of codewords L was set to 10, and the number of codewords m in the codebook was set to 5000. This codebook was used to quantize the features of both high-quality and low-quality remote sensing images. All input remote sensing images were resized to 224×224 resolution before entering the network. Images from each class of the AID dataset were split into 70% training set and 30% test set. All code was written in Python 3.8 and implemented using the PyTorch framework. Experiments were conducted on an Intel Xeon Gold 5218R CPU and an NVIDIA GeForce RTX 3080 GPU.

[0167] To verify the effectiveness of this invention in low-quality remote sensing image retrieval tasks, a comparative experiment was conducted with existing methods. The methods compared included the following:

[0168] (1) Scratch: The model is trained from scratch using low-quality remote sensing images as a baseline method.

[0169] (2) Finetune: First, use high-quality remote sensing images for pre-training, and then use low-quality remote sensing images for fine-tuning.

[0170] (3)FAH: A remote sensing image retrieval method that uses high-quality and low-quality remote sensing images for training simultaneously;

[0171] (4) AHCL: Another remote sensing image retrieval method that uses high-quality and low-quality remote sensing images for joint training.

[0172] The effectiveness of these methods is measured by Mean Average Precision (MAP) across all queries, and Precision@50 is used to measure the accuracy of the retrieval method; the higher the value, the better the retrieval performance.

[0173] Table 1. Different hash retrieval methods on different datasets

[0174]

[0175] The experiment was conducted under three degradation conditions (fog, Gaussian noise, motion blur) on the AID dataset, and the results are shown in Table 1.

[0176] In the AID(fog) dataset, as shown in the figure, the method of this invention performs best across all hash code bit lengths. The Scratch method performs poorly on low-quality remote sensing images, with MAP values ​​of 78.59 and 79.86 for 32-bit and 64-bit hash codes, respectively. The Finetune method, through pre-training and fine-tuning, achieves improved performance, reaching 86.03 for 64-bit hash codes. In contrast, the FAH and AHCL methods, trained jointly on high-quality and low-quality remote sensing images, achieve MAP values ​​of 91.86 and 92.02 for 64-bit hash codes, respectively; similarly, as shown in the figure... Figure 5 As shown, compared with other methods, this method has the highest value on the Precision@50 metric.

[0177] On the AID (gaussian noise) dataset, the Scratch method is significantly affected, with a MAP of 77.10 for 32-bit hash codes. The Finetune method shows a more significant performance improvement, achieving a MAP of 87.90 for 64-bit hash codes. The FAH and AHCL methods perform well with 64-bit hash codes, reaching 90.88 and 91.99 respectively. The method of this invention achieves the best MAP value of 92.39 with 64-bit hash codes, representing a 0.40% improvement over AHCL. Similarly, as... Figure 6 As shown, compared with other methods, the method of the present invention has the highest value on the Precision@50 index.

[0178] On the AID (motion_blur) dataset, all methods showed improved MAP. This invention achieved state-of-the-art results of 92.47 and 92.70 under 128-bit and 256-bit hash code conditions, respectively, representing improvements of 1.10% and 1.16% compared to the FAH method. Similarly, as... Figure 7 As shown, compared with other methods, the present invention has the highest value in the Precision@50 index.

Claims

1. A method for deep hashing retrieval of low-quality remote sensing images based on vector quantization, characterized in that, Includes the following steps: Step 1: Image Feature Extraction: High-quality and low-quality remote sensing images are acquired as input, and features are extracted from the high-quality and low-quality images respectively using a deep convolutional neural network. The specific steps are as follows: High-quality remote sensing images are represented as The corresponding low-quality remote sensing image is represented as Simultaneously, the image is fed into the backbone network AlexNet, which extracts image features. After feature extraction, dimensionality reduction is performed through two fully connected layers to generate real-valued hash codes. The real-valued hash code of a high-quality remote sensing image is represented as follows: The real-valued hash code representation of a low-quality remote sensing image is: ,in, and All belong to , The number of bits in the real-valued hash code is given by the following formula: in, The weight parameters represent the backbone network AlexNet, the two fully connected layers, and the hash layer. Indicates a by The mapping function represents the operations, including convolution, pooling, and fully connected operations. Indicates the number of remote sensing image samples; Step 2: Vector Quantization: Input the features extracted in Step 1 into the vector quantization module, quantize them into an independent discrete space, and generate quantized features; Step 3: Hash code generation: The quantized features generated in Step 2 are used to generate hash codes for image retrieval through a deep hashing network; Step 4: Application of loss function: After feature extraction, feature quantization and hash encoding in steps 1 to 3, feature representation is obtained. The feature representation is constrained by applying loss function, which includes Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information preservation, feature distance constraints and collaborative learning of encoder, codebook and decoder.

2. The method for deep hashing retrieval of low-quality remote sensing images based on vector quantization according to claim 1, characterized in that, In step 2, the vector quantization step is as follows: Two learnable codebooks Added at the end of the feature extraction network, it is used to quantify the features of low-quality and high-quality remote sensing images, respectively. The number of codewords in the codebook. For code words, i.e. codebook For a given vector in the dataset, the real-valued hash codes of high-quality and low-quality remote sensing images are quantized using the corresponding codebooks to obtain quantized features. For any high-quality remote sensing image, the real-valued hash code... Its standard quantification process is expressed as follows: in, Indicates will Discretized into a codeword in the codebook. Indicates the first in the codebook Each code character For codebook In and real-valued hash codes The index of the codeword with the smallest Euclidean distance is selected first, followed by the index of the codeword with the smallest Euclidean distance to the real-valued hash code. The codewords are weighted and summed to supplement the information of the quantified features, ultimately improving their representation ability; For any real-valued hash code of a remote sensing image The closest to its European-style front The code is Real-valued hash codes of high-quality remote sensing images The distance is The weights are calculated based on the distance between these codewords and the real-valued hash codes. The final quantization vector The formula is: After vector quantization, the quantization features of a high-quality remote sensing image are represented as follows: The quantization features of low-quality remote sensing images are represented as .

3. The method for deep hashing retrieval of low-quality remote sensing images based on vector quantization according to claim 1, characterized in that, In step 3, the hash encoding generation steps are as follows: After generating the real-valued hash code, it is activated by the function. The value of each dimension of the real-valued hash code is limited to between -1 and 1, as shown in the following formula: Real-value hash codes of high-quality remote sensing images Normalized hash code Represented as: Real-value hash codes of low-quality remote sensing images Normalized hash code Represented as: The quantized features are constrained by cross-entropy loss through a classification layer to inject semantic information. The specific formula is as follows: in, Represents the classification layer. Indicates the first A classification vector of a high-quality remote sensing image. Indicates the first Classification vectors of low-quality remote sensing images; The process of generating hash codes involves using vectors and Binarization is performed to generate the hash code of the final high-quality remote sensing image. Hash codes of low-quality remote sensing images The specific formula is as follows: in, Indicates the first A high-quality remote sensing image hash code, It is its corresponding normalized real-valued hash code; Indicates the first A low-quality remote sensing image hash code, It is the normalized real-valued hash code of its corresponding low-quality remote sensing image, through The function converts the real-valued hash code into binary form, generating the final hash code for retrieval.

4. The method for deep hashing retrieval of low-quality remote sensing images based on vector quantization according to claim 1, characterized in that, In step 4, the loss function is applied as follows: The Pairwise Loss is used in the following three cases: Case 1: Embedding Space Constraints for High-Quality Remote Sensing Images: This constraint is used to constrain the feature vectors of high-quality remote sensing images, ensuring that the feature distances between high-quality remote sensing images of the same class are closer, while the feature distances between high-quality remote sensing images of different classes are farther. The loss function formula is as follows: in, Indicates the number of remote sensing image samples. and These represent the indices of the remote sensing image samples. Indicates the distance threshold. The similarity indicator matrix is ​​defined to be similar to that of high-quality remote sensing images. and These are the feature vectors of low-quality remote sensing images; Case 2: Embedding Spatial Constraints for Low-Quality Remote Sensing Images: This constraint is used to constrain the features of low-quality remote sensing images, making the feature distances between low-quality images of the same class closer and the feature distances between low-quality images of different classes farther. The loss function formula is as follows: in, Indicates the number of remote sensing image samples. and These represent the indices of the remote sensing image samples. Indicates the distance threshold. The similarity indicator matrix is ​​defined to be similar to that of high-quality remote sensing images. and These are the feature vectors of low-quality remote sensing images; Case 3: Pairing constraint for high-quality and low-quality remote sensing image features: This constraint brings high-quality and low-quality remote sensing image features of the same category closer together, while keeping features of different categories further apart. The loss function formula is as follows: in, The similarity indicator matrix represents high-quality and low-quality remote sensing images. When, it indicates the first The first high-quality remote sensing image and the first Several low-quality remote sensing images belong to the same category; when When, it indicates the first The first high-quality remote sensing image and the first The low-quality remote sensing images belong to different categories; The overall Pairwise Loss function for cases 1, 2, and 3 is expressed as follows: The cross-entropy loss function is used to optimize the classification performance of high-quality and low-quality remote sensing images, and the formula is as follows: in, For the first Zhang high-quality remote sensing image about category The true label, For high-quality remote sensing images, regarding categories The predicted probability; For the first Zhang low-quality remote sensing image about category The true label, For low-quality remote sensing images, regarding categories The predicted probability; Number of categories; The reconstruction loss function guides the collaborative learning of the feature extraction network and the codebook. It employs a pass-through estimation method, which bypasses the codebook by passing the gradients from the Pairwise Loss and cross-entropy loss to the preceding feature extraction network, thereby updating the parameters of the feature extraction network. The specific loss function formula is as follows: in, This indicates a stop gradient operation, used to prevent gradients from propagating in the codebook; A dictionary learning method is adopted, which optimizes the codebook learning process by calculating the L2 error between the features output by the feature extraction network and the corresponding quantized codewords. The specific loss function formula is as follows: in, The output of the feature extraction network, For the corresponding codeword; The overall optimization problem of the model is to optimize the learnable parameters in the network framework by jointly using all loss functions. The overall loss function formula is as follows: in, This represents all the learnable parameters in the network framework.

5. A retrieval system for a low-quality remote sensing image deep hash retrieval method based on vector quantization according to any one of claims 1 to 4, characterized in that, include: Image feature extraction module: It takes high-quality and low-quality remote sensing images as input and extracts features from the high-quality and low-quality remote sensing images respectively through a deep convolutional neural network; Vector quantization module: Input the features into the vector quantization module, quantize them into an independent discrete space, and generate quantized features; Hash encoding generation module: Generates hash codes for image retrieval from features using a deep hashing network; Loss function application module: After feature extraction, feature quantization and hash encoding, feature representation is obtained. The feature representation is constrained by applying loss functions, including Pairwise Loss, reconstruction loss and cross-entropy loss, to ensure semantic information preservation, feature distance constraints and collaborative learning of encoder, codebook and decoder.

6. A low-quality remote sensing image depth hashing retrieval device based on vector quantization, characterized in that, include: Memory: for storing a computer program that implements the vector quantization-based deep hash retrieval method for low-quality remote sensing images as described in any one of claims 1 to 4; Processor: Used to implement the low-quality remote sensing image depth hash retrieval method based on vector quantization as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a low-quality remote sensing image depth hash retrieval method based on vector quantization as described in any one of claims 1 to 4.

Images