A self-supervised remote sensing image retrieval method and system
The self-supervised remote sensing image retrieval method using cross-quantization and adaptive error-correcting labels addresses the time and space requirements for training large-scale remote sensing image datasets, improves retrieval efficiency and accuracy, corrects sampling bias during training, and enhances the model's feature generation capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- STATE GRID HUNAN ELECTRIC POWER COMPANY LIMITED
- Filing Date
- 2024-05-24
- Publication Date
- 2026-07-24
AI Technical Summary
Existing self-supervised learning methods require long training times and large storage spaces when processing large-scale remote sensing image data, and suffer from the problem of imbalanced selection of positive and negative samples, resulting in low generalization performance and retrieval accuracy of the model.
By employing cross-quantization and adaptive error correction labels, a model is constructed through feature extraction, cross-quantization, and adaptive error correction labels. This reduces data storage requirements, minimizes sampling bias during training, and improves image retrieval efficiency and accuracy.
It effectively reduces the data processing load, improves retrieval efficiency and accuracy, solves the training time and space requirements of self-supervised learning methods on large-scale remote sensing image datasets, corrects erroneous negative samples in the training process, and enhances the model's feature generation capability and retrieval accuracy.
Smart Images
Figure CN118690042B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image retrieval technology, specifically to a self-supervised remote sensing image retrieval method and system. Background Technology
[0002] With the continuous development of remote sensing technology, a large number of remote sensing devices have been widely used in fields such as power equipment inspection, urban planning, and environmental monitoring, resulting in a massive amount of remote sensing image data. As the application of remote sensing technology continues to expand, the scale of remote sensing image data is also becoming increasingly enormous. Traditional remote sensing image retrieval typically requires a large amount of manually labeled remote sensing image data as a training set. Machine learning algorithms are then used to model and train the data, and the model is used for remote sensing image retrieval. However, existing technologies not only require even larger training sets, but the labeling process for remote sensing images is also time-consuming and costly, making it impractical for real-world applications.
[0003] With the rapid development of artificial intelligence technology, self-supervised learning methods have begun to attract researchers' attention. Compared with traditional supervised learning methods, self-supervised learning methods do not require labeled data and can directly learn feature representations from unlabeled data, exhibiting stronger generalization ability and higher efficiency. However, current self-supervised learning methods require long training times and large storage spaces when processing large-scale datasets, making them unsuitable for remote sensing image retrieval with large datasets. Furthermore, existing self-supervised learning methods may suffer from imbalance and unreasonable selection of positive and negative samples, affecting the model's learning performance and resulting in lower generalization performance and retrieval accuracy. Therefore, there is currently a lack of a method that combines self-supervised learning with remote sensing image retrieval. Summary of the Invention
[0004] One of the objectives of this invention is to provide a self-supervised remote sensing image retrieval method. This method is based on cross-quantization and adaptive error correction labels for modeling, which reduces the need for data storage space, reduces sampling bias during training, and improves image retrieval efficiency and accuracy.
[0005] A second objective of this invention is to provide a system for implementing the self-supervised remote sensing image retrieval method.
[0006] This invention provides a self-supervised remote sensing image retrieval method, comprising the following steps:
[0007] S1. Acquire remote sensing image data;
[0008] S2. Perform data augmentation on the remote sensing image data obtained in step S1 to obtain the training dataset;
[0009] S3. Construct an initial self-supervised remote sensing image retrieval model based on cross-quantization and adaptive error correction labels;
[0010] S4. Using the training dataset obtained in step S2, train the initial self-supervised remote sensing image retrieval model obtained in step S3 to obtain the self-supervised remote sensing image retrieval model.
[0011] S5. Obtain the self-supervised remote sensing image retrieval model through step S4, and perform actual remote sensing image retrieval.
[0012] Step S2 involves performing data augmentation on the remote sensing image data obtained in step S1 to obtain a training dataset. Specifically, this involves taking the remote sensing image data X obtained in step S1 and performing data augmentation on the remote sensing image data X. i Perform random feature augmentation to obtain the training dataset {X} i1 ,X i2 Feature enhancements include random cropping, random rotation, and Gaussian blur.
[0013] The initial self-supervised remote sensing image retrieval model described in step S3 includes a feature extraction module, a cross-quantization module, and an adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model is used as the input of the feature extraction module; the output of the feature extraction module is used as the input of the cross-quantization module; the output of the cross-quantization module is used as the input of the adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model first extracts feature vectors through the feature extraction module, then quantizes and maps the product of the feature vectors to the quantization space through the cross-quantization module, and finally constructs adaptive error correction labels based on the quantization descriptors through the adaptive error correction label construction module.
[0014] The feature extraction module will use the training dataset {X} obtained in step 2. i1 ,X i2 Feature extraction is performed to obtain the feature vector x. i1 ,x i2 ∈R D Let i = 1, 2, 3, ..., N be the batch samples, where D is the original dimension of the feature vector and N is the number of elements in the batch samples.
[0015] The cross-quantization module replaces the mapping layer of contrastive learning with product quantization, transforming the input feature vector x... i1 ∈R D Perform product quantization to obtain the quantization descriptor η. i1 And upload it to the quantization space, including the following steps:
[0016] a. Divide the input feature vector into m independent D / m-dimensional sub-feature vectors x in equal proportions. i1 ∈{x i11 ,x i12 ,...,x i1mThen, a quantization head G∈{g1,g2,...,g} is composed of m independent sub-quantizers. m} Pair of eigenvectors x i1 ∈{x i11 ,x i12 ,...,x i1m} Perform quantization indexing, where gi (i = 1, 2, 3, ..., m) is the sub-quantizer, and the sub-quantizer g i Composed of K different codewords g in (n∈{1,2,...,K}) are composed of g jn The dimension is D / m, g i ∈{g i1 ,g i2 ,...,g iK};
[0017] b. The quantization head G∈{g1,g2,...,g in step a) m Soft quantization is introduced on the}, which quantizes the sub-feature vector into an exponentially weighted sum η of the codewords of the sub-quantizers gi (i = 1, 2, 3, ..., m). i1n As shown in the following formula:
[0018]
[0019] Where, φ g It is a non-negative temperature parameter; The Frobenius norm is used to calculate the distance between a sub-eigenvector and the corresponding codeword in the sub-quantizer; g nk These are trainable parameters, initialized to random values between 0 and 1; x i1n This is the nth sub-feature vector;
[0020] c. Concatenate the obtained codewords together to obtain the feature vector x. i1 ∈R D Quantization descriptor η i1 .
[0021] The adaptive error correction label construction module generates higher confidence through a centralized distribution and assigns adaptive error correction labels to samples based on the relative distribution of the sample feature space, using the relative distribution Q. i The normalized entropy E is weighted by the confidence level to obtain the adaptive error correction label y, which specifically includes the following steps:
[0022] A. Calculate the eigenvector x i1 Quantization descriptor η i1 With batch samples {x 11 ,x 21 ,...,x j1},j=1,2,3,...,N,j≠i quantization descriptor {η 11 ,η 21 ,...,η j1 The cosine similarity S between} ij To construct the relative distribution Q of the batch samples i :
[0023]
[0024] Where, φ a To control the parameters of sample similarity distribution, cosine similarity S ij Calculate using the following formula:
[0025]
[0026] Where, η i1 η is the quantization descriptor for the feature vector. j1 This is the quantization descriptor for the feature vectors in a batch of samples; It is the L2 norm; η i1 T For η i1 transpose;
[0027] B. To quantify the uncertainty of the relative distribution and obtain the confidence levels among different samples in a batch, the relative distribution Q is... i The normalized entropy E, used as the confidence level, is expressed by the following formula:
[0028]
[0029] Among them, H(Q) i ) represents the relative distribution Q i The Shannon entropy, where N is the number of elements in the batch sample;
[0030] C. Based on the relative distribution Q obtained in step A i The relative distribution Q obtained in step B i The normalized entropy E is used to obtain the adaptive error correction label y, which is expressed by the following formula:
[0031]
[0032] Step S4 specifically involves using the training dataset obtained in step S2 to train the initial self-supervised remote sensing image retrieval model according to the total loss function. The parameters are then updated through backpropagation to obtain the self-supervised remote sensing image retrieval model. The total loss function L is expressed by the following formula:
[0033] L = L cq +λL a
[0034] Where λ is the equilibrium L cq and L a hyperparameters; L a The adaptive error correction label loss is expressed using the following formula:
[0035]
[0036] Where, η i1 For the feature vector x i1 ∈R D Quantization descriptor, η j1 This is the quantization descriptor for the feature vectors in a batch of samples;
[0037] L cq The cross-quantization comparison loss is expressed using the following formula:
[0038]
[0039] Where N is the number of elements in the batch sample, L c To quantify the comparative loss, L c This can be expressed by the following formula:
[0040]
[0041] Where S(x) i1 ,η i2 ) is the eigenvector x i1 ∈R D and non-corresponding eigenvectors x i2 ∈R D Quantization descriptor η i2 Cosine similarity between them; φ cq The non-negative temperature parameter is the cross-quantization contrast loss function.
[0042] Step S5 specifically involves inputting the remote sensing image data into the self-supervised remote sensing image retrieval model obtained in step S4, converting the remote sensing image data into quantized features through the model's feature extraction module and cross-quantization module, and then performing a retrieval in the quantization space to obtain the retrieval results.
[0043] The present invention also provides a system for implementing the self-supervised remote sensing image retrieval method, comprising the following modules: a data acquisition module, a data augmentation module, a model building module, a model training module, and a remote sensing image retrieval module;
[0044] The data acquisition module acquires remote sensing image data and uploads the data to the data enhancement module;
[0045] The data augmentation module augments the remote sensing image data based on the received data to obtain a training dataset, and then uploads the data to the model training module.
[0046] The model building module constructs an initial self-supervised remote sensing image retrieval model based on cross-quantization and adaptive error correction labels, and uploads the data to the model training module;
[0047] The model training module trains the initial self-supervised remote sensing image retrieval model using the training dataset based on the received data, obtains the self-supervised remote sensing image retrieval model, and uploads the data to the remote sensing image retrieval module.
[0048] The remote sensing image retrieval module performs actual remote sensing image retrieval based on the received data. The beneficial effects of this invention are:
[0049] (1) In the method model of the present invention, by designing the product quantization in the cross-quantization module, the product quantization is used to compress high-dimensional data, which reduces the load of storing and processing data and effectively improves the retrieval efficiency;
[0050] (2) The method of the present invention can maximize the cross similarity of features before and after image mapping by using a cross learning strategy, thereby enhancing the feature generation capability of the model and improving the retrieval accuracy.
[0051] (3) The method model of the present invention can effectively solve the sampling bias problem in comparative learning by designing an adaptive error correction label construction module, correct the erroneous negative samples in the training process, and further improve the retrieval accuracy. Attached Figure Description
[0052] Figure 1 This is a schematic flowchart of the method of the present invention;
[0053] Figure 2 This is a schematic diagram of the system of the present invention. Detailed Implementation
[0054] This invention provides a self-supervised remote sensing image retrieval method, the schematic diagram of which is shown below. Figure 1 As shown, it includes the following steps:
[0055] S1. Acquire remote sensing image data;
[0056] S2. Perform data augmentation on the remote sensing image data obtained in step S1 to obtain the training dataset. Specifically, this involves taking the remote sensing image data X obtained in step S1... i Perform random feature augmentation to obtain the training dataset {X} i1 ,X i2 Feature enhancements include random cropping, random rotation, and Gaussian blur.
[0057] S3. Based on cross-quantization and adaptive error correction labels, an initial self-supervised remote sensing image retrieval model is constructed. The initial self-supervised remote sensing image retrieval model includes a feature extraction module, a cross-quantization module, and an adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model is used as the input of the feature extraction module; the output of the feature extraction module is used as the input of the cross-quantization module; the output of the cross-quantization module is used as the input of the adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model first extracts feature vectors through the feature extraction module, then quantizes and maps the product of the feature vectors to the quantization space through the cross-quantization module, and finally constructs adaptive error correction labels based on the quantization descriptors through the adaptive error correction label construction module.
[0058] The feature extraction module will use the enhanced training dataset {X} obtained in step 2. i1 ,X i2 Feature extraction is performed to obtain the feature vector x. i1 ,x i2 ∈R D Let i = 1, 2, 3, ..., N be the batch samples, where D is the original dimension of the feature vector and N is the number of elements in the batch samples.
[0059] The cross-quantization module replaces the mapping layer of contrastive learning with product quantization, transforming the input feature vector x... i1 ∈R D Perform product quantization to obtain the quantization descriptor η. i1 And upload it to the quantization space, including the following steps:
[0060] a. Divide the input feature vector into m independent D / m-dimensional sub-feature vectors x in equal proportions. i1 ∈{x i11 ,x i12 ,...,x i1m Then, a quantization head G∈{g1,g2,...,g} is composed of m independent sub-quantizers. m} Pair of eigenvectors x i1 ∈{x i11 ,x i12 ,...,x i1m} Perform quantization indexing, where gi (i = 1, 2, 3, ..., m) is the sub-quantizer, and the sub-quantizer g i Composed of K different codewords g in (n∈{1,2,...,K}) are composed of g jn The dimension is D / m, g i ∈{g i1 ,g i2 ,...,g iK};
[0061] b. The quantization head G∈{g1,g2,...,g in step a) m Soft quantization is introduced on the}, which quantizes the sub-feature vector into an exponentially weighted sum η of the codewords of the sub-quantizers gi (i = 1, 2, 3, ..., m). i1n As shown in the following formula:
[0062]
[0063] Where, φ g The non-negative temperature parameter is set to 5.0; The Frobenius norm is used to calculate the distance between a sub-eigenvector and the corresponding codeword in the sub-quantizer; g nk These are trainable parameters, initialized to random values between 0 and 1; x i1n This is the nth sub-feature vector;
[0064] c. Concatenate the obtained codewords together to obtain the feature vector x. i1 ∈R D Quantization descriptor η i1 .
[0065] The adaptive error correction label construction module generates higher confidence through a centralized distribution and assigns adaptive error correction labels to samples based on the relative distribution of the sample feature space, using the relative distribution Q. i The normalized entropy E is weighted by the confidence level to obtain the adaptive error correction label y, which specifically includes the following steps:
[0066] A. Calculate the eigenvector x i1 Quantization descriptor η i1 With batch samples {x 11 ,x 21 ,...,x j1},j=1,2,3,...,N,j≠i quantization descriptor {η 11 ,η 21 ,...,η j1 The cosine similarity S between} ij To construct the relative distribution Q of the batch samples i :
[0067]
[0068] Where, φ a To control the parameters of sample similarity distribution, cosine similarity S ij Calculate using the following formula:
[0069]
[0070] Where, ηi1 η is the quantization descriptor for the feature vector. j1 This is the quantization descriptor for the feature vectors in a batch of samples; It is the L2 norm; η i1 T For η i1 transpose;
[0071] B. To quantify the uncertainty of the relative distribution and obtain the confidence levels among different samples in a batch, the relative distribution Q is... i The normalized entropy E, used as the confidence level, is expressed by the following formula:
[0072]
[0073] Among them, H(Q) i ) represents the relative distribution Q i The Shannon entropy, where N is the number of elements in the batch sample;
[0074] C. Based on the relative distribution Q obtained in step A i The relative distribution Q obtained in step B i The normalized entropy E is used to obtain the adaptive error correction label y, which is expressed by the following formula:
[0075]
[0076] S4. Using the training dataset obtained in step S2, train the initial self-supervised remote sensing image retrieval model obtained in step S3 to obtain a self-supervised remote sensing image retrieval model. Specifically, using the training dataset obtained in step S2, train the initial self-supervised remote sensing image retrieval model according to the total loss function, update the parameters through backpropagation, and obtain the self-supervised remote sensing image retrieval model. The total loss function L is expressed by the following formula:
[0077] L = L cq +λL a
[0078] Where λ is the equilibrium L cq and L a hyperparameters; L a The adaptive error correction label loss is expressed using the following formula:
[0079]
[0080] Where, η i1 For the feature vector x i1 ∈R D Quantization descriptor, η j1 This is the quantization descriptor for the feature vectors in a batch of samples;
[0081] Lcq The cross-quantization comparison loss is expressed using the following formula:
[0082]
[0083] Where N is the number of elements in the batch sample, L c To quantify the comparative loss, L c This can be expressed by the following formula:
[0084]
[0085] Where S(x) i1 ,η i2 ) is the eigenvector x i1 ∈R D and non-corresponding eigenvectors x i2 ∈R D Quantization descriptor η i2 Cosine similarity between them; φ cq The non-negative temperature parameter is the cross-quantization contrast loss function.
[0086] Step S5 specifically involves inputting the remote sensing image data into the self-supervised remote sensing image retrieval model obtained in step S4, converting the remote sensing image data into quantized features through the model's feature extraction module and cross-quantization module, and then performing a retrieval in the quantization space to obtain the retrieval results.
[0087] The present invention also provides a system for implementing the self-supervised remote sensing image retrieval method, the schematic diagram of which is shown below. Figure 2 As shown, it includes the following modules: data acquisition module, data augmentation module, model building module, model training module, and remote sensing image retrieval module;
[0088] The data acquisition module acquires remote sensing image data and uploads the data to the data enhancement module;
[0089] The data augmentation module augments the remote sensing image data based on the received data to obtain a training dataset, and then uploads the data to the model training module.
[0090] The model building module constructs an initial self-supervised remote sensing image retrieval model based on cross-quantization and adaptive error correction labels, and uploads the data to the model training module;
[0091] The model training module trains the initial self-supervised remote sensing image retrieval model using the training dataset based on the received data, obtains the self-supervised remote sensing image retrieval model, and uploads the data to the remote sensing image retrieval module.
[0092] The remote sensing image retrieval module performs actual remote sensing image retrieval based on the received data.
Claims
1. A self-supervised remote sensing image retrieval method, characterized in that, Includes the following steps: S1. Acquire remote sensing image data; S2. Perform data augmentation on the remote sensing image data obtained in step S1 to obtain the training dataset; S3. Construct an initial self-supervised remote sensing image retrieval model based on cross-quantization and adaptive error correction labels; S4. Using the training dataset obtained in step S2, train the initial self-supervised remote sensing image retrieval model obtained in step S3 to obtain the self-supervised remote sensing image retrieval model. S5. Obtain the self-supervised remote sensing image retrieval model through step S4, and perform actual remote sensing image retrieval; The initial self-supervised remote sensing image retrieval model described in step S3 includes a feature extraction module, a cross-quantization module, and an adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model is used as the input of the feature extraction module. The output of the feature extraction module is used as the input of the cross-quantization module. The output of the cross-quantization module is used as the input of the adaptive error correction label construction module. The input of the initial self-supervised remote sensing image retrieval model first extracts feature vectors through the feature extraction module, then quantizes and maps the product of the feature vectors to the quantization space through the cross-quantization module, and finally constructs adaptive error correction labels based on the quantization descriptors through the adaptive error correction label construction module. The feature extraction module extracts the enhanced remote sensing image data obtained in step S2. Feature extraction is performed to obtain feature vectors. As a batch sample, among which The original dimension of the feature vector. The number of elements in the batch sample; The cross-quantization module replaces the mapping layer of contrastive learning with product quantization, transforming the input feature vector... Perform product quantization to obtain the quantization descriptor. And upload it to the quantization space, including the following steps: a. Divide the input feature vector proportionally into Individual eigenvectors of dimensional sub-elements Then use by The quantization head consists of several independent sub-quantizers. Pair of eigenvectors Perform quantization indexing, where, For subquantizer, subquantizer Depend on Different codes composition, The dimension is , ; b. Quantization head in step a The above introduces soft quantization, which quantizes the sub-feature vector into a sub-quantizer. The index-weighted sum of codewords As shown in the following formula: in, It is a non-negative temperature parameter; is the Frobenius norm, used to calculate the distance between a sub-feature vector and the codeword in the corresponding sub-quantizer; These are trainable parameters, initialized to random values between 0 and 1; For the first Sub-feature vectors; c. Concatenate the obtained codewords together to obtain the feature vector. Quantization descriptor .
2. The self-supervised remote sensing image retrieval method according to claim 1, characterized in that, Step S2 specifically involves: processing the remote sensing image data obtained in step S1. Perform random feature enhancement to obtain the training dataset. Feature enhancements include random cropping, random rotation, and Gaussian blur.
3. The self-supervised remote sensing image retrieval method according to claim 2, characterized in that, The adaptive error correction label construction module generates higher confidence through a centralized distribution and assigns adaptive error correction labels to samples based on the relative distribution of the sample feature space. Normalized entropy The confidence scores are weighted to obtain adaptive error correction labels. Specifically, it includes the following steps: A. Calculate the eigenvectors Quantization descriptor With batch samples Quantization descriptor Cosine similarity between To construct the relative distribution of batch samples : in, Cosine similarity is a parameter used to control the similarity distribution of samples. Calculate using the following formula: in, For feature vectors Quantization descriptor, This is the quantization descriptor for the feature vectors in a batch of samples; It is an L2 norm; for Transpose of; B. To quantify the uncertainty of the relative distribution and obtain the confidence levels among different samples in a batch, the relative distribution is... Normalized entropy The confidence level is expressed by the following formula: in, For relative distribution Shannon entropy, The number of elements in the batch sample; C. Based on the relative distribution obtained in step A The relative distribution obtained in step B Normalized entropy Obtain adaptive error correction labels It can be expressed by the following formula: 。 4. The self-supervised remote sensing image retrieval method according to claim 3, characterized in that, Step S4 specifically involves using the training dataset obtained in step S2 to train the initial self-supervised remote sensing image retrieval model according to the total loss function. Parameters are updated through backpropagation to obtain the self-supervised remote sensing image retrieval model, with the total loss function... Express it using the following formula: in, To balance and hyperparameters; The adaptive error correction label loss is expressed using the following formula: in, For feature vectors Quantization descriptor, This is the quantization descriptor for the feature vectors in a batch of samples; The cross-quantization comparison loss is expressed using the following formula: in The number of elements in the batch sample. To quantify and compare losses, This can be expressed by the following formula: in, For feature vectors and non-corresponding feature vectors Quantization descriptor Cosine similarity between them; The non-negative temperature parameter is the cross-quantization contrast loss function.
5. The self-supervised remote sensing image retrieval method according to claim 4, characterized in that, Step S5 specifically involves inputting the remote sensing image data into the self-supervised remote sensing image retrieval model obtained in step S4, converting the remote sensing image data into quantized features through the model's feature extraction module and cross-quantization module, and then performing a retrieval in the quantization space to obtain the retrieval results.
6. A system for implementing the self-supervised remote sensing image retrieval method according to any one of claims 1 to 5, characterized in that, It includes the following modules: data acquisition module, data augmentation module, model building module, model training module, and remote sensing image retrieval module; The data acquisition module acquires remote sensing image data and uploads the data to the data enhancement module; The data augmentation module augments the remote sensing image data based on the received data to obtain a training dataset, and then uploads the data to the model training module. The model building module constructs an initial self-supervised remote sensing image retrieval model based on cross-quantization and adaptive error correction labels, and uploads the data to the model training module; The model training module trains the initial self-supervised remote sensing image retrieval model using the training dataset based on the received data, obtains the self-supervised remote sensing image retrieval model, and uploads the data to the remote sensing image retrieval module. The remote sensing image retrieval module performs actual remote sensing image retrieval based on the received data.