Hyperspectral image classification method and system based on hierarchical bidirectional contrast learning

Abstract

The application discloses a hyperspectral image classification method and system based on layered bidirectional contrast learning. The method comprises the following steps: acquiring hyperspectral remote sensing image data, and preprocessing the hyperspectral remote sensing image data; data enhancement is performed on the preprocessed hyperspectral remote sensing image data to construct a plurality of data enhancement samples, and online representation and target representation of image data are extracted according to the plurality of data enhancement samples constructed; positive samples and negative samples of the online representation are calculated based on the target representation, and an encoder network model is trained according to the online representation and the positive and negative samples; the hyperspectral remote sensing image data to be classified is classified by using the encoder network model, and a classification result is output. Through the method and system, the information redundancy in the hyperspectral image classification process can be effectively solved, the accuracy of the hyperspectral image classification is improved, and the manpower and material resources consumed for accurate labeling of the hyperspectral image are effectively reduced.

Description

Technical Field

[0001] This invention relates to the technical field of hyperspectral image classification, and in particular to a hyperspectral image classification method and system based on hierarchical bidirectional contrastive learning. Background Technology

[0002] Hyperspectral image (HSI) classification refers to assigning an image to a specific land cover category based on its spatial and spectral characteristics, and it is one of the most active research directions in the field of hyperspectral remote sensing.

[0003] Currently, in the early stages of hyperspectral image classification, most methods focused on exploring the role of spectral information in classification. Many traditional machine learning methods have achieved good results in hyperspectral image classification, such as Support Vector Machine (SVM), Multinomial Logistic Regression (MLR), and Dynamic or Random Subspace (DRS). Furthermore, due to the "curse of dimensionality," some methods have focused on designing effective feature extraction or dimensionality reduction techniques, such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Linear Discriminant Analysis (LDA). While these methods have achieved good results in HSI classification, they share a common drawback: they rely on prior knowledge and parameter settings. This deficiency severely affects the robustness and generalization ability of traditional methods, leading to poor performance under complex conditions.

[0004] In recent years, deep learning has made significant breakthroughs in computer vision tasks such as image classification. Given sufficient training samples, deep learning continuously fits and approximates the sample distribution of the target dataset through multiple layers of linear layers. Then, it uses backpropagation to update the parameters of the linear layers layer by layer from shallow to deep during training, which can effectively extract data features. This also makes deep learning more widely applicable.

[0005] However, the distribution of these land cover categories is obtained by annotating pixels in hyperspectral images using image processing techniques. Although hyperspectral images contain rich spatial-spectral information of land covers that can be used for classification, their spectral dimensions increase geometrically compared to ordinary remote sensing images, thus easily leading to problems such as information redundancy and the curse of dimensionality. Furthermore, while acquiring hyperspectral images has become increasingly easier with technological advancements, achieving accurate annotation of hyperspectral images remains a complex and laborious task. Summary of the Invention

[0006] To overcome the problems of information redundancy during the classification process and the high manpower and material resources required for accurate annotation of hyperspectral images in existing hyperspectral image classification technologies, the present invention aims to propose a hyperspectral image classification method and system based on hierarchical bidirectional contrastive learning. This method can effectively solve the information redundancy problem in the hyperspectral image classification process, improve the accuracy of hyperspectral image classification, and thus effectively reduce the high manpower and material resources required for accurate annotation of hyperspectral images.

[0007] To achieve the objectives of this invention, the following technical solution is adopted:

[0008] A hyperspectral image classification method based on hierarchical bidirectional contrastive learning, the method comprising the following steps:

[0009] Acquire hyperspectral remote sensing image data and preprocess the hyperspectral remote sensing image data;

[0010] Data augmentation is performed on the preprocessed hyperspectral remote sensing image to construct several data augmentation samples, and online representations of the image data and target representations are extracted based on the constructed data augmentation samples.

[0011] The positive and negative samples of the online representation are calculated based on the target representation, and the encoder network model is trained based on the online representation and the positive and negative samples.

[0012] The encoder network model is used to classify hyperspectral remote sensing image data to be classified, and the classification results are output.

[0013] In the above technical solution, preprocessing the acquired hyperspectral remote sensing image data can improve the practicality and reliability of the image data. By performing data augmentation on the preprocessed image data, the features of the image data can be effectively highlighted to effectively extract the online and target representations of the image data. Based on the extracted online and target representations, positive and negative samples of the image data can be further extracted, thereby more finely reflecting the correlation and differences between different types of data in the image data, improving the clustering of similar land features and the separability between different categories of land features in the image data, and effectively solving the information redundancy that occurs in the hyperspectral image classification process. Based on the extracted positive and negative samples, an encoder network model is trained to obtain an unsupervised image classification model that can effectively grasp the changes in hyperspectral remote sensing image data. This model is then used to perform label-free self-supervised classification of the input hyperspectral remote sensing image data, effectively improving the model's discriminability of difficult-to-distinguish hyperspectral samples and the accuracy of hyperspectral image classification, thereby effectively reducing the large amount of manpower and resources consumed in the accurate annotation of hyperspectral images.

[0014] Furthermore, the preprocessing of the hyperspectral remote sensing image data includes:

[0015] The acquired hyperspectral remote sensing image data were subjected to noise reduction and normalization processing.

[0016] In the above technical solution, noise reduction and normalization processing of the acquired hyperspectral remote sensing image data can improve the practicality and reliability of the image data and reduce data redundancy.

[0017] Furthermore, the extracted data augmentation samples are input into the preset online encoder and the target encoder, respectively;

[0018] Online representations of image data are extracted using an online encoder;

[0019] The target representation of the image data is extracted using a target encoder.

[0020] Furthermore, the process of calculating positive and negative samples based on the target representation in online representation includes:

[0021] Design an encoder network based on hierarchical dictionary lookup, expressed as:

[0022]

[0023]

[0024] The encoder network based on hierarchical dictionary lookup maintains two dictionary structures at different depths of the network and updates the dictionary prototype online by receiving gradient updates. It retrieves positive and negative samples at different depths of the online representation by calculating the similarity between the dictionary prototype and the target representation. The dictionary prototype with the maximum similarity value to the target representation is taken as the positive sample, and the dictionary prototype with the minimum similarity value to the target representation is taken as the negative sample.

[0025] Where D1 represents the shallow dictionary of the encoder network, and D2 represents the deep dictionary of the encoder network. * The parameter D2 represents the shallow dictionary. * Parameters representing the deep dictionary. This indicates the parameters of the online encoder. Indicates the positive sample prototype. L represents the remaining prototype. P (x i ) represents the prototype-contrast loss function, z i This represents the online representation extracted by the online encoder.

[0026] Furthermore, the online representations and target representations of the shallow and deep layers are calculated at different depths of the encoder network, respectively, with the following expressions:

[0027]

[0028] in, and These represent the inline encoder φ q (·) and target encoder φ k The s-th convolutional layer is (·), where Avg represents dynamic average pooling, and g(·) represents the mapping between Batch Normalization and ReLU activation function. This represents the online representation of the output of the s-th convolutional layer. x represents the target representation output by the s-th convolutional layer. q and x k These represent the enhanced images input to the in-circuit encoder and the target encoder, respectively;

[0029] The expression for retrieving positive samples of online representations based on the similarity between dictionary prototypes and target representations is:

[0030]

[0031] The expression for retrieving negative samples of an online representation based on the similarity between the dictionary prototype and the target representation is:

[0032]

[0033] in, Indicates input sample x i Target representation, d j This represents the dictionary prototype.

[0034] In the above technical solution, an encoder network based on hierarchical dictionary lookup is designed to store the learned difficult positive and negative samples. At the same time, it is proposed to capture data representations of different scales through dictionary structure in the shallow and deep layers of the deep network for comparative supervision, so as to improve the class perception ability of the network, so as to better extract the positive and negative sample data features of hyperspectral remote sensing images, thereby improving the discriminability of difficult hyperspectral samples and the accuracy of hyperspectral image classification.

[0035] Furthermore, the process of training the encoder network model based on the online representation and its positive and negative samples includes:

[0036] After removing the labels from all samples of the preprocessed hyperspectral remote sensing image data, the data is input into the encoder network model, and several rounds of iterative optimization training are set to optimize the parameters of the encoder network model.

[0037] Based on a hierarchical dictionary structure, positive and negative samples are extracted from different depths of the encoder network.

[0038] Prototype contrastive loss is constructed based on positive and negative samples extracted at different depths for positive contrastive learning, and reverse contrastive loss is constructed based on negative samples for reverse contrastive learning. The two losses are combined to form a bidirectional contrastive loss.

[0039] The encoder network learns the latent representation in hyperspectral remote sensing image data samples, and the bidirectional contrastive loss function is used to optimize the parameters of the encoder network during the iterative training process. When the preset training rounds end or the loss function converges, the trained encoder network model parameters are obtained.

[0040] Initialize the new encoder and fully connected layer, load the pre-trained encoder network model parameters onto the new encoder and freeze them, and divide the hyperspectral remote sensing image data into training and test sets.

[0041] The fully connected layer is trained using the training set, and the encoder with loaded network model parameters is used to classify hyperspectral images on the test set, outputting the classification results.

[0042] In the above technical solution, after removing labels from all samples of the preprocessed hyperspectral remote sensing image data, the data is input into the encoder network model. A hierarchical dictionary structure is used to extract positive and negative samples at different depths. Then, a prototype contrastive loss is constructed based on these different depths for forward contrastive learning, and a reverse contrastive loss is used for backward contrastive learning. This allows for a more refined representation of the correlations and differences between different types of data in the image data, improving the clustering of similar land features and the separability between different categories of land features. The encoder network learns the latent representations in the hyperspectral remote sensing image data samples, and a bidirectional contrastive loss function is used to optimize the parameters of the encoder network during iterative training. This method enables the trained encoder network model to effectively grasp the changes in hyperspectral remote sensing image data, thereby improving the clustering of similar land cover types and the separability of different land cover types under self-supervised conditions. Initializing an encoder and a fully connected layer allows loading the parameters of the trained encoder network model, enabling self-supervised classification of hyperspectral remote sensing images through a division of labor between the encoder network model and the encoder network model. This effectively improves the model's discriminative power for difficult-to-distinguish hyperspectral samples and the accuracy of hyperspectral image classification. Furthermore, when optimizing the parameters of the existing model, it is not necessary to retrain the encoder; only the fully connected layer that processes downstream tasks needs to be trained, thereby effectively reducing the significant human and material resources required for accurate hyperspectral image annotation.

[0043] Furthermore, the process of constructing a back-contrast loss based on negative samples extracted at different depths and performing back-contrast learning includes:

[0044] Based on the negative samples retrieved from the dictionary, a reverse contrastive loss function is constructed, expressed as follows:

[0045]

[0046] in, It is the negative sample prototype, and τ represents the temperature coefficient.

[0047] Furthermore, a hierarchical bidirectional contrastive loss function is set to optimize the parameters of the encoder network model, with the expression as follows:

[0048]

[0049] Among them, L P (x i ) and L N (x i ) represent the prototype contrastive loss function and the reverse contrastive loss function, respectively, which are set in the shallow and deep layers of the encoder network. λ represents the weight term of the reverse contrastive loss, and B represents the batch size.

[0050] Furthermore, the expression for the latent representation learned by the encoder network in the hyperspectral remote sensing image data samples is as follows:

[0051]

[0052] The expression for hyperspectral image classification using the learned latent representation initialized by a fully connected layer is:

[0053] D = FC(L)

[0054] Where x represents the test set sample, and φ(·) represents the encoder network. This represents the model parameters of the encoder network, D represents the classification result, and FC represents the initial fully connected layer.

[0055] In the above technical solution, the inverse contrast loss function and the bidirectional contrast loss function can effectively optimize the parameters of the encoder network model during the iterative training process, so as to improve the discriminativeness of the encoder network model for difficult hyperspectral samples and the accuracy of hyperspectral image classification.

[0056] A hyperspectral image classification system based on hierarchical bidirectional contrastive learning, the system comprising:

[0057] The data acquisition module is used to acquire hyperspectral remote sensing image data;

[0058] The data processing module is used to preprocess the hyperspectral remote sensing image data;

[0059] The characterization extraction module is used to perform data augmentation on the preprocessed hyperspectral remote sensing image to construct several data augmentation samples, and extract online characterization and target characterization of the image data based on the constructed data augmentation samples.

[0060] The model training module is used to calculate positive and negative samples of the online representation based on the target representation, and to train the encoder network model based on the online representation and the positive and negative samples.

[0061] The image classification module is used to classify the hyperspectral remote sensing image data to be classified using the encoder network model and output the classification results.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0063] This invention proposes a hyperspectral image classification method and system based on hierarchical bidirectional contrastive learning. By preprocessing the acquired hyperspectral remote sensing image data, the practicality and reliability of the image data can be improved. Data augmentation of the preprocessed image data effectively highlights its features, enabling the extraction of online and target representations. Based on the extracted online and target representations, positive and negative samples of the image data can be further extracted, thus more precisely reflecting the correlation and differences between different types of data in the image data. This improves the clustering of similar land features and the separability between different categories of land features, effectively solving the information redundancy problem in hyperspectral image classification. An encoder network model is trained based on the extracted positive and negative samples to obtain an unsupervised image classification model that can effectively grasp the changes in hyperspectral remote sensing image data. This model is then used to perform label-free, self-supervised classification of the input hyperspectral remote sensing image data, effectively improving the model's discriminative power for difficult-to-distinguish hyperspectral samples and the accuracy of hyperspectral image classification. This, in turn, effectively reduces the significant manpower and resources required for accurate hyperspectral image annotation. Attached Figure Description

[0064] Figure 1 A flowchart illustrating the steps of a hyperspectral image classification method based on hierarchical bidirectional contrastive learning, provided in this application embodiment;

[0065] Figure 2 A schematic diagram of the hierarchical bidirectional contrastive self-supervised network model structure provided in the embodiments of this application;

[0066] Figure 3 This is a schematic diagram of the dictionary structure provided in an embodiment of this application;

[0067] Figure 4 This is a diagram of the encoder structure in a hierarchical bidirectional contrastive self-supervised network provided in an embodiment of this application.

[0068] Figure 5 A schematic diagram illustrating the process of classifying a given test sample as provided in an embodiment of this application;

[0069] Figure 6 This is a classification result diagram of a hyperspectral remote sensing image provided in an embodiment of this application;

[0070] Figure 7 This is a schematic diagram of the structure of a hyperspectral image classification system based on hierarchical bidirectional contrastive learning, provided in an embodiment of this application. Detailed Implementation

[0071] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0073] Example 1:

[0074] This embodiment provides a hyperspectral image classification method based on hierarchical bidirectional contrastive learning. (See also...) Figure 1 and Figure 2 The method includes the following steps:

[0075] Step S1: Acquire hyperspectral remote sensing image data and preprocess the hyperspectral remote sensing image data;

[0076] Step S2: Perform data augmentation on the preprocessed hyperspectral remote sensing image to construct several data augmentation samples, and extract online representations of the image data and target representations based on the constructed data augmentation samples;

[0077] Step S3: Calculate the positive and negative samples of the online representation based on the target representation, and train the encoder network model based on the online representation and the positive and negative samples;

[0078] Step S4: Use the encoder network model to classify the hyperspectral remote sensing image data to be classified and output the classification results.

[0079] In step S1, the preprocessing of the hyperspectral remote sensing image data includes:

[0080] The acquired hyperspectral remote sensing image data were subjected to noise reduction and normalization processing.

[0081] In the above technical solution, noise reduction and normalization processing of the acquired hyperspectral remote sensing image data can improve the practicality and reliability of the image data and reduce data redundancy.

[0082] In a preferred embodiment, in step S2, several extracted data augmentation samples are respectively input into a preset online encoder and a target encoder;

[0083] Online representations of image data are extracted using an online encoder;

[0084] The target representation of the image data is extracted using a target encoder.

[0085] In this embodiment, preprocessing the acquired hyperspectral remote sensing image data improves its practicality and reliability. Data augmentation of the preprocessed images effectively highlights the features of the image data, enabling the extraction of online and target representations. Based on the extracted online and target representations, positive and negative samples of the image data can be further extracted, thus more precisely reflecting the correlation and differences between different types of data in the image data. This improves the clustering of similar land features and the separability between different categories of land features, effectively addressing information redundancy in hyperspectral image classification. An encoder network model is trained based on the extracted positive and negative samples to obtain a model that effectively grasps changes in hyperspectral remote sensing image data and achieves self-supervised image classification. This model is then used to perform self-supervised classification of subsequent input hyperspectral remote sensing image data, effectively improving the model's generalization, discriminative power for difficult-to-distinguish hyperspectral samples, and the accuracy of hyperspectral image classification. This, in turn, effectively reduces the significant manpower and resources required for precise hyperspectral image annotation.

[0086] Example 2:

[0087] This embodiment further explains steps S3 to S4 in Embodiment 1, as follows:

[0088] In a preferred embodiment, step S3, which involves calculating the positive and negative samples of the online representation based on the target representation, includes:

[0089] See Figure 3 Design an encoder network based on hierarchical dictionary lookup, expressed as:

[0090]

[0091] The encoder network based on hierarchical dictionary lookup maintains two dictionary structures at different depths of the network and updates the dictionary prototype online by receiving gradient updates. It retrieves positive and negative samples of the online representation by calculating the similarity between the dictionary prototype and the target representation. The dictionary prototype with the maximum similarity value to the target representation is taken as the positive sample, and the dictionary prototype with the minimum similarity value to the target representation is taken as the negative sample.

[0092] Where D1 represents the shallow dictionary of the encoder network, and D2 represents the deep dictionary of the encoder network. * The parameter D2 represents the shallow dictionary. * Parameters representing the deep dictionary. This indicates the parameters of the online encoder. Indicates the positive sample prototype. L represents the remaining prototype. P (x i ) represents the prototype-contrast loss function, z i This represents the online representation extracted by the online encoder.

[0093] Specifically, the encoder network based on hierarchical dictionary lookup includes:

[0094] A shallow dictionary D1 and a deep dictionary D2 are maintained by receiving gradient updates.

[0095] Input hyperspectral image sample x and its enhanced image x q and x k ;

[0096] Online encoder φ trained online q and the target encoder φ for momentum update k ;

[0097] The enhanced image is processed by two encoders to output the online representation of the image and the target representation. The target representation is then compared with all prototypes in the dictionary to obtain similarity values. The dictionary prototype with the highest similarity value is retrieved as the positive sample, and the dictionary prototype with the lowest similarity value is retrieved as the negative sample.

[0098] Furthermore, the construction of the shallow dictionary D1 and the deep dictionary D2, and the acquisition of suitable positive samples... Specifically, it includes:

[0099] The dictionary positive prototype is updated by minimizing the contrastive loss function, and the dictionary negative prototype is updated by maximizing the contrastive loss function. The dictionary is updated along with the encoder network as part of the network parameters. Since the representation is normalized on the unit hypersphere, the inner product of the target representation and the dictionary prototype represents the cosine similarity of the samples. The dictionary prototype with the highest similarity to the online representation is retrieved as the positive sample of the online representation.

[0100] In a preferred embodiment, the online representation and target representation of the shallow and deep layers are calculated at different depths of the encoder network, as expressed in the following expression:

[0101]

[0102] in, and These represent the inline encoder φ q (·) and target encoder φ k The s-th convolutional layer is (·), where Avg represents dynamic average pooling, and g(·) represents the mapping between Batch Normalization and ReLU activation function. This represents the online representation of the output of the s-th convolutional layer. This represents the target representation output by the s-th convolutional layer (in this embodiment, s is 1 or 5; s=1 indicates shallow representation extraction, and s=5 indicates deep representation extraction), x q and x k These represent the enhanced images input to the in-circuit encoder and the target encoder, respectively;

[0103] The expression for retrieving positive samples of online representations based on the similarity between dictionary prototypes and target representations is:

[0104]

[0105] The expression for retrieving negative samples of an online representation based on the similarity between the dictionary prototype and the target representation is:

[0106]

[0107] in, Indicates input sample x i Target representation, d j This represents the dictionary prototype.

[0108] Understandably, considering the difficulty in sampling positive and negative samples, a dictionary-based encoder network is designed to store the learned difficult-to-distinguish positive and negative samples. This allows for querying the similarity between multiple batches of samples and online representations, achieving efficient positive and negative sample retrieval, decoupling the model's dependence on large batches, and improving the network's generalization ability. Simultaneously, a comparative supervision method is proposed, capturing data representations at different scales through a dictionary structure in both shallow and deep layers of the encoder network. This addresses some of the issues with contrastive learning's lack of attention to shallow semantic features, achieving more sufficient sample separability and improving the network's ability to preserve the manifold structure and class perception of hyperspectral data. Furthermore, the bidirectional contrastive loss, combined with reverse contrastive learning, mitigates the network's extreme uniformity distribution through bidirectional information flow, preserving the underlying manifold structure of the data and further enhancing the network's semantic representation ability and generalization ability. This ultimately improves the model's ability to discriminate difficult hyperspectral samples and the accuracy of hyperspectral image classification.

[0109] In a preferred embodiment, in step S3, see... Figure 2 and Figure 4 The process of training the encoder network model based on online representations and their positive and negative samples includes:

[0110] Step S31: After removing the labels from all samples of the preprocessed hyperspectral remote sensing image data, input the data into the encoder network model, and set up several rounds of iterative optimization training to optimize the parameters of the encoder network model;

[0111] Step S32: Based on the hierarchical dictionary structure, extract positive and negative samples at different network depths of the encoder;

[0112] Step S33: Construct a prototype contrastive loss based on the extracted positive samples at different depths to perform positive contrastive learning, and construct a reverse contrastive loss based on the negative samples to perform reverse contrastive learning. Combine the two losses to form a bidirectional contrastive loss.

[0113] Step S34: Learn the latent representation in hyperspectral remote sensing image data samples through the encoder network, and optimize the parameters of the encoder network during the iterative training process using a bidirectional contrastive loss function. When the preset training rounds end or the loss function converges, the trained encoder network model parameters are obtained.

[0114] Step S35: Initialize the new encoder and fully connected layer, load the pre-trained encoder network model parameters onto the new encoder and freeze them, and divide the hyperspectral remote sensing image data into training set and test set;

[0115] Step S36: Train the fully connected layer using the training set, and use the encoder with loaded network model parameters and the trained fully connected layer to perform hyperspectral image classification on the test set, and output the classification results.

[0116] Specifically, in step S33, the process of constructing a back-contrast loss based on the extracted negative samples of different depths and performing back-contrast learning includes:

[0117] Based on the negative samples retrieved from the dictionary, a reverse contrastive loss function is constructed, expressed as follows:

[0118]

[0119] in, It is the negative sample prototype, and τ represents the temperature coefficient.

[0120] Specifically, in step S34, a bidirectional contrastive loss function is set to optimize the parameters of the encoder network model, and the expression is:

[0121]

[0122] Among them, L P (x i ) and L N (x i ) represent the prototype contrastive loss function and the reverse contrastive loss function, respectively, which are set in the shallow and deep layers of the encoder network. λ represents the weight term of the reverse contrastive loss, and B represents the batch size.

[0123] The expression for the latent representation in hyperspectral remote sensing image data samples learned through the encoder network is as follows:

[0124]

[0125] In steps S35 and S36, see Figure 5 and Figure 6 The expression for classifying hyperspectral images by initializing the learned latent representations into fully connected layers is as follows:

[0126] D = FC(L)

[0127] Where x represents the test set sample, and φ(·) represents the encoder network. This represents the model parameters of the encoder network, D represents the classification result, and FC represents the initial fully connected layer.

[0128] For example, constructing as Figure 2 The self-supervised network model shown has the encoder structure as follows: Figure 4 As shown, the dictionary structure is as follows: Figure 3 As shown. The encoder consists of an initial Conv 3×3 convolutional layer, a batch normalization (BN) layer, a ReLU activation layer, a MaxPool layer, and basic convolutional blocks repeated N0 times (in this embodiment, repeated twice; the size of N0 can be increased to increase the depth of the network model depending on the task difficulty). Each basic convolutional block contains two Conv 3×3 convolutions, two batch normalization (BN) layers, and two ReLU activation layers. The entire framework uses a Siamese network structure, including two encoder branches φ with identical constructions. q and φ k , where φ q Update φ using slow momentum updates k In the encoder, the convolutional layer extracts local features, the batch normalization layer normalizes the features, and the ReLU layer performs non-linear activation. Image classification is not performed during the network pre-training phase; after pre-training, a new, initialized encoder and fully connected (FC) layer are created for the classification task.

[0129] Without loss of generality, For hyperspectral samples, the For the learned latent representation, L can be represented as:

[0130]

[0131] In the above formula, This represents the parameters of the pre-trained model. A fully connected (FC) layer is initialized. For a given input sample x, it is fed into the pre-trained model φ(·) to obtain the corresponding latent representation L. The latent representation is then passed through the FC layer for classification to obtain the final classification result D.

[0132] The final classification result D is specifically represented as follows:

[0133] D = FC(L)

[0134] In this embodiment, refer to Figure 6 The hyperspectral image classification results are shown in the figure. Figure 6 (a) is a false-color image composed of three bands of the original hyperspectral image. Figure 6 (b) represents the ground truth. Figure 6 (c) shows the classification results of the IndianPines hyperspectral remote sensing image.

[0135] Understandably, the inverse contrast loss function and the bidirectional contrast loss function that combines the prototype contrast loss can effectively optimize the parameters of the encoder network model during the iterative training process, so as to improve the discriminativeness of the encoder network model for difficult hyperspectral samples and the accuracy of hyperspectral image classification.

[0136] In this embodiment, after removing labels from all samples of the preprocessed hyperspectral remote sensing image data, the data is input into the encoder network model. A hierarchical dictionary structure is used to extract positive and negative samples at different depths. Prototype contrastive loss is constructed based on these different depths for forward contrastive learning, and reverse contrastive loss is used for backward contrastive learning. This allows for a more detailed representation of the correlations and differences between different types of data in the image data, improving the clustering of similar land features and the separability between different categories of land features. By learning the latent representations in the hyperspectral remote sensing image data samples through the encoder network and using a bidirectional contrastive loss function to optimize the parameters of the encoder network during iterative training, this approach enables... The trained encoder network model can effectively grasp the changes in hyperspectral remote sensing image data, thereby improving the clustering of similar land features and the separability of different land features under self-supervised conditions. Initializing an encoder and a fully connected layer can load the parameters of the trained encoder network model, enabling self-supervised classification of hyperspectral remote sensing images through a division of labor with the encoder network model. This effectively improves the model's discriminative power for difficult-to-distinguish hyperspectral samples and the accuracy of hyperspectral image classification. Furthermore, when optimizing the parameters of the existing model, it is not necessary to retrain the encoder; only the fully connected layer that processes downstream tasks needs to be trained, thereby effectively reducing the significant human and material resources required for accurate annotation of hyperspectral images.

[0137] Example 3:

[0138] This embodiment provides a hyperspectral image classification system based on hierarchical bidirectional contrastive learning. (See also...) Figure 7 The system includes:

[0139] The data acquisition module is used to acquire hyperspectral remote sensing image data;

[0140] The data processing module is used to preprocess the hyperspectral remote sensing image data;

[0141] The characterization extraction module is used to perform data augmentation on the preprocessed hyperspectral remote sensing images to construct several data augmentation samples, and to extract online characterization and target characterization of the image data based on the constructed data augmentation samples.

[0142] The model training module is used to calculate positive and negative samples of the online representation based on the target representation, and to train the encoder network model based on the online representation and the positive and negative samples.

[0143] The image classification module is used to classify the hyperspectral remote sensing image data to be classified using the encoder network model and output the classification results.

[0144] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A hyperspectral image classification method based on hierarchical bidirectional contrastive learning, characterized in that, The method includes the following steps: Acquire hyperspectral remote sensing image data and preprocess the hyperspectral remote sensing image data; Data augmentation is performed on the preprocessed hyperspectral remote sensing image to construct several data augmentation samples, and online representations of the image data and target representations are extracted based on the constructed data augmentation samples. The positive and negative samples of the online representation are calculated based on the target representation, and the encoder network model is trained based on the online representation and the positive and negative samples. The encoder network model is used to classify the hyperspectral remote sensing image data to be classified, and the classification results are output. The process of calculating positive and negative samples of online representation based on target representation, and training the encoder network model based on the online representation and positive and negative samples, includes: Design an encoder network based on hierarchical dictionary lookup, expressed as: The encoder network based on hierarchical dictionary lookup maintains two dictionary structures at different depths of the network and updates the dictionary prototype online by receiving gradient updates. It retrieves positive and negative samples of the online representation by calculating the similarity between the dictionary prototype and the target representation. The dictionary prototype with the maximum similarity value to the target representation is taken as the positive sample, and the dictionary prototype with the minimum similarity value to the target representation is taken as the negative sample. in, A shallow dictionary representing the encoder network. This represents the deep dictionary of the encoder network. The parameter represents the shallow dictionary. Parameters representing the deep dictionary. This indicates the parameters of the online encoder. Indicates the positive sample prototype. Indicates the remaining prototype. This represents the prototype-contrast loss function. This represents the online representation extracted by the online encoder. B represents the input sample, and B represents the batch size. Based on the negative samples retrieved from the dictionary, a reverse contrastive loss function is constructed, expressed as follows: in, It is a negative sample prototype. This represents the temperature coefficient.

2. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 1, characterized in that, The preprocessing of the hyperspectral remote sensing image data includes: The acquired hyperspectral remote sensing image data were subjected to noise reduction and normalization processing.

3. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 1, characterized in that, Several constructed data-enhanced samples are input into the preset online encoder and the target encoder, respectively; Online representations of image data are extracted using an online encoder; The target representation of the image data is extracted using a target encoder.

4. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 1, characterized in that, Online representations and target representations of shallow and deep layers are extracted at different depths of the encoder network, as expressed in the following formula: in, and These represent inline encoders. With the target encoder The s One convolutional layer, Avg Indicates dynamic average pooling. g (·) represents the mapping between Batch Normalization and ReLU activation function. Indicates the first s Online representation of the output of each convolutional layer Indicates the first s The target representation output by each convolutional layer and These represent the enhanced images input to the in-circuit encoder and the target encoder, respectively; The expression for retrieving positive samples of online representations based on the similarity between dictionary prototypes and target representations is: The expression for retrieving the negative sample prototype of an online representation based on the similarity between the dictionary prototype and the target representation is: in, Indicates input sample The target representation, This represents the dictionary prototype.

5. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 1, characterized in that, The process of training an encoder network model based on online representations and their positive and negative samples includes: After removing the labels from all samples of the preprocessed hyperspectral remote sensing image data, the data is input into the encoder network model, and several rounds of iterative optimization training are set to optimize the parameters of the encoder network model. Based on a hierarchical dictionary structure, positive and negative samples are extracted from different depths of the encoder network. Prototype contrastive loss is constructed based on positive samples extracted at different depths for positive contrastive learning, and reverse contrastive loss is constructed based on negative samples for reverse contrastive learning. The two losses are combined to form a bidirectional contrastive loss. The encoder network learns the latent representation in hyperspectral remote sensing image data samples, and the bidirectional contrastive loss function is used to optimize the parameters of the encoder network during the iterative training process. When the preset training rounds end or the loss function converges, the trained encoder network model parameters are obtained. Initialize the new encoder and fully connected layer, load the pre-trained encoder network model parameters onto the new encoder and freeze them, and divide the hyperspectral remote sensing image data into training and test sets. The fully connected layer is trained using the training set, and the encoder with loaded network model parameters is used to classify hyperspectral images on the test set, outputting the classification results.

6. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 5, characterized in that, A hierarchical bidirectional contrastive loss function is used to optimize the parameters of the encoder network model. The expression is: in, and These represent the prototype contrastive loss function and the reverse contrastive loss function, respectively, which are applied in the shallow and deep layers of the encoder network. λ This represents the weighting term of the reverse comparison loss. B Indicates the batch size.

7. The hyperspectral image classification method based on hierarchical bidirectional contrastive learning according to claim 5, characterized in that, The expression for the latent representation in hyperspectral remote sensing image data samples learned through the encoder network is as follows: The expression for hyperspectral image classification using the learned latent representation initialized by a fully connected layer is: in, This represents the test set samples. Indicates the encoder network. This represents the model parameters of the encoder network. Indicates the classification result. FC This indicates the initialized fully connected layer.

8. A hyperspectral image classification system based on hierarchical bidirectional contrastive learning, the system being based on the method according to any one of claims 1-7, characterized in that, The system includes: The data acquisition module is used to acquire hyperspectral remote sensing image data; The data processing module is used to preprocess the hyperspectral remote sensing image data; The characterization extraction module is used to perform data augmentation on the preprocessed hyperspectral remote sensing image to construct several data augmentation samples, and extract online characterization and target characterization of the image data based on the constructed data augmentation samples. The model training module is used to calculate positive and negative samples of the online representation based on the target representation, and to train the encoder network model based on the online representation and the positive and negative samples. The image classification module is used to classify the hyperspectral remote sensing image data to be classified using the encoder network model and output the classification results.

Images