A hyperspectral classification method based on band erasing and contrastive learning

Abstract

This invention provides a hyperspectral classification method based on band erasure and contrastive learning. First, hyperspectral data is acquired and band erasure is performed, resulting in two band-removed hyperspectral images. These images are then preprocessed to obtain sample patches for two branches of a contrastive learning network. A gradient mask is used to cover all patches in the upper branch, while leaving the patches in the lower branch untouched. Random occlusion is applied to the patches in both branches, followed by data augmentation to obtain positive sample pairs. Hyperparameters are set, and the positive sample pairs are input into the contrastive learning network. After training, the network parameters and features are saved. The extracted features and annotations are used as the training set to train a classifier, achieving hyperspectral classification. This invention combines various data augmentation methods suitable for hyperspectral data with contrastive learning, tapping into the potential of contrastive learning in the hyperspectral field and improving classification accuracy to a new level, effectively achieving the classification of spectral images.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral image classification, and more specifically, relates to a hyperspectral classification method based on band erasure and contrastive learning. Background Technology

[0002] In recent years, hyperspectral images have been increasingly widely used due to their rich spectral and spatial information. Compared to visible light or infrared images, hyperspectral images have hundreds or even thousands of bands, enabling them to distinguish surface materials with diagnostic spectral characteristics with sufficient spectral resolution. They have wide applications in vegetation surveys, atmospheric research, military reconnaissance, and environmental monitoring. Hyperspectral image classification is one of the important research directions in the field of hyperspectral imaging, generally referring to the classification of individual pixels, where the rich spectral information contained in a single pixel is the primary basis for classification. With advancements in hardware technology, the spatial resolution of hyperspectral sensors has also increased, and the spatial information surrounding the target pixel has been utilized. Currently, combining spectral and spatial features has become the mainstream approach for hyperspectral image classification.

[0003] However, the numerous spectral bands in hyperspectral images also present challenges for classification. Due to the massive data volume, direct training without processing would require a huge network size and computer memory. Furthermore, the high spectral resolution can lead to redundancy in spectral information. Dimensionality reduction methods can be used to reduce the data size while retaining key information. Feature extraction is a common method for reducing the dimensionality of hyperspectral images. It involves reducing the dimensionality of the original input data and extracting or organizing effective features for subsequent use. Common feature extraction methods include PCA, ICA, LDA, and MDS. Some of these algorithms are still widely used for hyperspectral data preprocessing due to their simplicity and efficiency. With the increasing maturity of deep learning algorithms, more and more deep learning algorithms are being proposed for extracting features from hyperspectral images. Currently, the commonly used classification method is to first extract spectral features or spatial spectral features using supervised or unsupervised feature extraction algorithms, and then train a classifier using these features.

[0004] Early developments focused on feature extraction algorithms based on supervised deep learning. Within supervised learning, Convolutional Neural Networks (CNNs) played a crucial role, evolving from one-dimensional CNNs that extracted only spectral features to two-dimensional and three-dimensional CNNs that extracted spatial spectral information. Roy et al. combined two-dimensional and three-dimensional convolutions to propose the HybridSN network, further improving classification accuracy. Zhong et al. introduced the classic residual network into the hyperspectral domain, designing the SSRN network. Besides CNNs, DRNNs, DFFNs, and other networks have also achieved some success in hyperspectral classification.

[0005] However, supervised learning often relies heavily on labeled data, requiring a sufficient number of labeled samples to achieve good training results. For hyperspectral images, data acquisition and labeling require significant manpower and time costs. Therefore, in recent years, the focus of feature extraction algorithms has gradually shifted towards unsupervised deep learning. The essential difference between unsupervised and supervised learning is that the training data in unsupervised learning is unlabeled. It classifies samples based on their similarity, narrowing the gap between similar data and widening the gap between dissimilar data. The lack of label restrictions can also stimulate the model's potential, allowing it to delve into the data autonomously for discovery and experience, learning more latent features. Furthermore, well-trained models exhibit better robustness and generalization ability.

[0006] Discriminative learning models conditional probabilities and learns the optimal boundaries between different classes. Contrastive learning is a typical discriminative learning algorithm in deep learning. Its main idea is to learn representations by comparing positive and negative samples in the latent space. Positive samples are pairs of spatially close but spectrally similar patches, while negative samples are pairs of spectrally dissimilar or spatially distant patches. By minimizing the distance between positive samples and maximizing the distance between negative samples, the model learns to encode spatial and spectral information in the latent space. Contrastive learning has developed rapidly in recent years, with many variants proposed, such as Moco, SimCLR, BYOL, SwAV, and SimSiam, and it has gradually been adopted in the hyperspectral field. These methods differ in their choice of contrastive loss, encoder architecture, and training strategy. However, they all share a common goal: to learn representations that capture the underlying structure of hyperspectral data.

[0007] For contrastive learning, while ensuring the model doesn't collapse, we can incorporate more optimization techniques to encourage it to learn more representative features. For the spatial-spectral features of hyperspectral images, we often combine spatial and spectral information into the same sample. While this is simple and compensates for the limitations of relying solely on spectral information, directly inputting an entire sample cube into the model introduces significant redundant information that interferes with feature extraction. Other works place spatial and spectral information in different samples and extract them separately using cross-domain contrastive learning. This approach reduces a large amount of redundant information but may lead to the loss of some valuable, secondary information. The key to improving the efficiency of contrastive learning lies in coordinating the extraction of spatial and spectral information, preserving useful information while minimizing interference from useless information, and increasing the model's focus on key information. Summary of the Invention

[0008] To address the shortcomings of existing deep learning methods in hyperspectral image classification, such as reliance on labeled data and poor classification performance, this invention proposes a hyperspectral classification method based on band erasure and contrastive learning. This invention incorporates band erasure, gradient masking, and random occlusion into a contrastive learning structure to improve the representativeness of extracted features. The extracted features are then input into an SVM classifier for training, achieving high-precision hyperspectral classification.

[0009] This invention first provides a hyperspectral classification method based on band erasure and contrastive learning, which includes the following steps:

[0010] Step 1: Collect hyperspectral data;

[0011] Step 2: Band erasure is performed on the original hyperspectral data to obtain two non-overlapping hyperspectral data X0 and X1. The two hyperspectral data are preprocessed to obtain sample patch groups P0 and P1 for the two branches of the contrastive learning network. Each pixel corresponds to two patches centered on that pixel. These two patches are used as sample pairs.

[0012] The preprocessing of the two hyperspectral data sets is as follows: Principal component analysis is performed on the hyperspectral data X0 and X1 respectively, resulting in d principal component maps. The edges of the principal component maps are filled with 0. Then, an s*s sliding window with a step size of 1 is used to divide the principal component maps into sample patches, resulting in two sample patch groups P0 and P1. Here, d and s are preset values. Patch groups P0 and P1 contain a series of s*s*d three-dimensional data. The patches contain spectral information and spatial information. The spectral information is compressed after principal component analysis, and the category of the center pixel represents the category of the entire patch.

[0013] Step 3: Use a gradient mask to cover the patches that serve as branches on the contrastive learning network, and leave the patches on the lower branches unchanged; where the upper branch on the contrastive learning network is the online network branch, and the lower branch is the target network branch.

[0014] Step 4: Randomly occlude the patches on the two branches processed in Step 3, and perform data augmentation to obtain positive sample pairs (v). i ,v i ');

[0015] Step 5: Set hyperparameters, input all positive sample pairs into the contrastive learning network for network training, and save the network parameters and features after training.

[0016] Step 6: Use the extracted features and labeled categories as the training set to train the classifier.

[0017] Step 7: After processing in steps 2 to 4, the hyperspectral data to be classified is classified using the trained contrastive learning network and classifier.

[0018] Preferably, the band erasure is performed by extracting odd-numbered and even-numbered layers from the original hyperspectral data to obtain two original hyperspectral data with non-overlapping bands. That is, the two original hyperspectral data are obtained by erasing half of the bands from the original hyperspectral data, and the bands are non-overlapping.

[0019] The contrastive learning network is a BYOL network, which has a structure consisting of an upper branch online network and a lower branch target network.

[0020] Put (v i ,v i ') Input to the BYOL network, where v i The input is fed into an online network and encoded by an encoder to obtain the representation u. i =f online (v i The mapping z is obtained after passing through the mapper. i =g online (u i The prediction result k is obtained after passing through the predictor. i =q online (z i );v i The input is fed into the target network and encoded by the encoder to obtain u. i '=f target (v i ), which is obtained by the mapper z i '=g target (u i );

[0021] The computational process of the online network and the target network can be represented by the following formula:

[0022] k i =q online (g online (f online (v i )))

[0023] z i =g target (f target (v i ′))

[0024] The optimization objective of the BYOL network is to make the positive examples of the online network closer to the positive examples of the target network in the representation space. The parameters of the online network are updated using a loss function, and the parameters of the target network are updated using an exponential moving average (EMA) based on the parameters of the online network, with the update step size controlled by the hyperparameter τ. The loss is calculated based on the outputs of the two branches. First, regarding k i and z i Perform L2 regularization. Then take the L2 norm of the difference between the two, as shown in the formula:

[0025]

[0026] Swap the input patches of the two branches, that is, change v i Input to the target network, v i The input is fed into an online network, and the loss function is calculated.

[0027] The final loss function of the BYOL network is:

[0028]

[0029] The parameter update process can be represented as:

[0030]

[0031] W target ←τW target +(1-τ)W online

[0032] Among them W online These are parameters of the online network, W target η represents the parameters of the target network, where η is the learning rate and τ is the weight for parameter updates.

[0033] After training reaches the required number of iterations, the encoder and mapper parameters are saved, and the output of the mapper is used as a feature and input into the classifier.

[0034] In summary, the technical solution conceived in this invention has the following advantages compared with the prior art:

[0035] (1) The method of the present invention includes a band removal strategy to improve the spectral features extracted by contrast learning, and combines random occlusion to improve the extracted spatial features;

[0036] (2) The method of the present invention includes a gradient masking step, which improves the model’s attention to key parts, reduces attention to edge positions, and reduces the interference of useless information on features.

[0037] (3) This invention combines a series of data augmentation techniques suitable for hyperspectral data to improve feature representation and applies them to the BYOL model, thereby raising the accuracy of classification results to a new level. Attached Figure Description

[0038] Figure 1 This is a flowchart of a hyperspectral classification method based on band erasure and contrastive learning provided in an embodiment of the present invention.

[0039] Figure 2 This is a diagram illustrating the algorithm structure of the hyperspectral classification method based on band erasure and contrastive learning provided in this embodiment of the invention.

[0040] Figure 3 This is a visualization of the gradient mask provided in an embodiment of the present invention.

[0041] Figure 4 This is a comparison chart of the features extracted from the IP dataset by the method provided in this embodiment of the invention and the original BYOL dataset.

[0042] Figure 5 This is a comparison chart of the features extracted by the method provided in this embodiment of the invention and the original BYOL on the PU dataset.

[0043] Figure 6 This is a comparison chart of the features extracted by the method provided in this embodiment of the invention and the original BYOL on the SA dataset. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described in detail below with reference to specific embodiments. Specific embodiments are described below to simplify the invention. However, it should be understood that the invention is not limited to the described embodiments, and various modifications are possible without departing from the basic principles; these equivalent forms also fall within the scope defined by the appended claims.

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0046] Example of working conditions: The dataset selected includes three databases: Indian Pines (IP), University of Pavia (PU), and Salina (SA). Step 1: Collect hyperspectral data;

[0047] Step 2: Band erasure is performed on the original hyperspectral data to obtain two non-overlapping hyperspectral data X0 and X1. The two hyperspectral data are preprocessed to obtain sample patch groups P0 and P1 for the two branches of the contrastive learning network. Each pixel corresponds to two patches centered on that pixel. These two patches are used as sample pairs.

[0048] The preprocessing of the two hyperspectral data sets is as follows: Principal component analysis is performed on the hyperspectral data X0 and X1 respectively, resulting in d principal component maps. The edges of the principal component maps are filled with 0. Then, an s*s sliding window with a step size of 1 is used to divide the principal component maps into sample patches, resulting in two sample patch groups P0 and P1. Here, d and s are preset values. Patch groups P0 and P1 contain a series of s*s*d three-dimensional data. The patches contain spectral information and spatial information. The spectral information is compressed after principal component analysis, and the category of the center pixel represents the category of the entire patch.

[0049] Step 3: Use a gradient mask to cover the patches that serve as branches on the contrastive learning network, and leave the patches on the lower branches unchanged; where the upper branch on the contrastive learning network is the online network branch, and the lower branch is the target network branch.

[0050] Step 4: Randomly occlude the patches on the two branches processed in Step 3, and perform data augmentation to obtain positive sample pairs (v). i ,v i ');

[0051] Step 5: Set hyperparameters, input all positive sample pairs into the contrastive learning network for network training, and save the network parameters and features after training.

[0052] Step 6: Use the extracted features and labeled categories as the training set to train the classifier.

[0053] Step 7: After processing in steps 2 to 4, the hyperspectral data to be classified is classified using the trained contrastive learning network and classifier.

[0054] Step 2, the specific implementation method of band erasure, is as follows:

[0055] The essence of band erasure is to divide the original sample into two data points with non-overlapping bands, and extract them at intervals of odd and even layers. That is, the two data points are obtained by erasing half of the bands from the original data, and the bands are non-overlapping.

[0056] Band erasure can improve the representativeness of subsequent features. On the one hand, because hyperspectral data has high band resolution and spectral information has a large degree of redundancy, removing half of the bands will not result in the loss of key spectral information, and can also reduce the interference of redundant information in each branch on feature extraction. On the other hand, the data of the two branches after band erasure have a complementary relationship. In the process of contrastive learning training, in order to make the loss function converge, the model will tend to focus on common information and reduce attention to some unimportant details. However, this common information is often a key feature.

[0057] Two original hyperspectral data sets with non-overlapping bands were obtained by extracting odd-numbered and even-numbered layers from the original hyperspectral data. In other words, both original hyperspectral data sets were obtained by erasing half of the bands from the original hyperspectral data, and the bands were non-overlapping.

[0058] The specific implementation method of preprocessing is as follows:

[0059] Principal component analysis (PCA) was performed on X0 and X1 respectively. The first d PCA images were taken, with the value of d varying depending on the dataset: 30 for the IP dataset and 15 for the PU and SA datasets. Then, an s*s sliding window with a step size of 1 was used to divide the PCA images into sample patches, i.e., s*s*d cubes, where s is 25 and the edges are filled with 0, resulting in two sets of patches, P0 and P1, where N is the number of effective pixels. The patches contain spectral and spatial information, with the spectral information compressed after PCA. The class of the center pixel represents the class of the entire patch; therefore, it can be said that spatial and spectral information closer to the center pixel is more important.

[0060] In step 3, the gradient mask is specifically implemented as follows:

[0061] The gradient mask is essentially a weight matrix with the highest value at the center and decreasing values ​​further away from the center. It has a size of s*s, the same as the patch size. The gradient mask is used to reduce the model's attention to the spatial and spectral information of the patch edges, thereby increasing attention to the key information at the center—essentially a forced attention shift. Specifically, we set the weight at the center to 1, the weights at the four vertices to 0, and the other values ​​are linearly interpolated based on their distance from the center. The weight calculation formula is as follows:

[0062]

[0063] Where mask is a gradient mask matrix, (i, j) is the position of an element in the mask, and (center, center) is the center position of the mask.

[0064] Figure 3This is a visualization of the calculated gradient mask. P0' is obtained by multiplying the gradient mask element-wise with the corresponding element of patch P0; P1 is not processed using the gradient mask. Specifically, the mask is multiplied element-wise with the corresponding element of each layer of the patch to obtain a weighted new patch. This results in smaller differences between the input samples of the upper and lower branches at the center and larger differences at the edges. During contrastive learning training, in order to reduce the loss function of the upper and lower branches, the model gradually reduces its attention to the patch edges, thus focusing more on the key information at the center.

[0065] The random occlusion step in step 4 includes the following steps:

[0066] 4.1) Set the area of ​​the occlusion rectangle, which can be a fixed percentage of the patch area or a random value within a certain range. In this embodiment, the area is set to 10% of the patch area.

[0067] 4.1) Set the aspect ratio, taking a random value between r and 1 / r. In this embodiment, r is 0.3;

[0068] 4.1) Calculate the length and width of the rectangle;

[0069] 4.1) Randomly select the starting point of the rectangle in the patch and replace the original pixel value with the occlusion value.

[0070] The occlusion value can be a fixed value, such as 0, 0.5, or 1, or a random value varying between 0 and 1. In this embodiment, the occlusion value is set to 1. It is important to note that the pixels at the center of the patch must be preserved during occlusion to prevent obscuring critical information. Occlusion is performed simultaneously on each layer of the patch.

[0071] For p 0,i 'and p 1,i After random occlusion, two different patches are obtained. To prevent the two erased areas from overlapping or having too high similarity, which would result in poor network convergence, an auxiliary data augmentation method must be used to make the two patches different.

[0072] Furthermore, the other data augmentation methods are specifically implemented as follows:

[0073] This includes random cropping, color distortion, random flipping, random grayscale, and Gaussian noise.

[0074] After the enhancement, a patch will be applied to p. 0,i 'and p 1,i Generate positive sample pairs (v) i ,v i ').

[0075] The specific implementation method of feature extraction in step 5 is as follows:

[0076] In this embodiment, the BYOL structure is selected as the feature extraction network.

[0077] First, set the network's hyperparameters and structure: learning rate of 0.001; number of iterations of 50; batch size of 128. The BYOL backbone network, from input to output, consists of: 3D convolutional layer, 3D convolutional layer, 3D convolutional layer, 2D convolutional layer, linear layer, linear layer, linear layer, with batch regularization performed after each convolutional layer. The upper branch online network and the lower branch target network use the same backbone network, but the target network does not use the last linear layer; the activation function is the ReLU function.

[0078] The BYOL structure is divided into an upper branch online network and a lower branch target network. We define (v) as follows: i ,v i ') Input into the BYOL structure, where v 0,i The input is fed into an online network, and the encoder outputs the representation u. i =f online (v i The mapping z is obtained after passing through the mapper. i =g online (u i The prediction result k is obtained after passing through the predictor. i =q online (z i );v i The input is fed into the target network, and after passing through the encoder, u is obtained. i '=f target (v i '), after passing through the mapper, z is obtained i '=g target (u i Specifically, the encoder consists of three 3D convolutional layers, one 2D convolutional layer, and one fully connected layer. After convolution, the layers are flattened before being fed into the fully connected layer. Each layer is followed by a batch regularization layer and a ReLU activation function. The mapper consists of a fully connected layer, a regularization layer, and a ReLU activation function. The predictor is a smaller fully connected layer, which is also followed by a batch regularization layer and a ReLU activation function.

[0079] The computational process of the online network and the target network can be represented by the following formula:

[0080] k i =q online (g online (f online (v i )))

[0081] z i =gtarget (f target (v i ′))

[0082] For BYOL, the optimization objective is to make the positive examples in the online network approach the positive examples in the target network in the representation space. Therefore, we update the parameters of the online network using a loss function, and the parameters of the target network are updated using an exponential moving average (EMA) based on the parameters of the online network. The update step size is controlled by the hyperparameter τ. The loss is calculated based on the outputs of the two branches. First, regarding k i and z i Perform L2 regularization. Then take the L2 norm of the difference between the two, as shown in the formula:

[0083]

[0084] Since the BYOL structure is asymmetric, in order to make full use of the data, we swap the input patches of the two branches, that is, we swap v i Input to the target network, v i The input is fed into an online network, and the loss function is calculated. The loss function for BYOL is:

[0085]

[0086] The parameter update process can be represented as:

[0087]

[0088] W target ←τW target +(1-τ)W online

[0089] After training reaches the required number of iterations, we save the encoder and mapper parameters and use the mapper output as features, inputting it into the classifier. To evaluate the feature extraction capability of the contrastive learning model after incorporating novel data augmentation, we use the classic SVM classifier as the downstream task, and use classification accuracy as the metric for evaluating feature performance.

[0090] In this embodiment, overall accuracy (OA) and average accuracy (AA) are used as indicators, and PCA, TPCA, SSAE, EPLS, 3DCAE and basic BYOL are used as comparison methods. UPDA+BYOL is the method proposed in this invention. Figures 4-6 The results were obtained on three public datasets. For the IP and PU datasets, 10% was used as the training set and 90% as the test set. For the SA dataset, 5% was used as the training set and 95% as the test set.

[0091] The comparison method TPCA is derived from Ren, Y.; Liao, L.; Maybank, SJ; Zhang, Y.; Liu, X. Hyperspectral image spectral-spatial feature extraction via tensorprincipal component analysis. IEEE Geoscience and Remote Sensing Letters 2017, 14, 1431–1435.

[0092] Comparative method SSAE from Tao, C.; Pan, H.; Li, Y.; Zou, Z. Unsupervised spectral–spatial feature learning with stacked sparse autoencoder for hyperspectral imaging classification. IEEE Geoscience and remote sensing letters 2015, 12, 2438–2442.

[0093] The comparison method EPLS is from Romero, A.; Gatta, C.; Camps-Valls, G. Unsupervised deep feature extraction for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing 2015, 54, 1349–1362.

[0094] Comparative method 3DCAE from Mei, S.; Ji, J.; Geng, Y.; Zhang, Z.; Li, X.; Du, Q. Unsupervised Spatial–Spectral Feature Learning by 3D ConvolutionalAutoencoder for Hyperspectral Classification. IEEE Transactions on Geoscience and Remote Sensing 2019, pp.6808–6820.

[0095] As shown in Table 1 below, the method of this invention achieved the best OA and AA on the IP dataset, especially in classes 3, 4, 10, 11, 14, and 15. Comparing BYOL and UPDA+BYOL, the introduction of the new data augmentation method in this invention increased OA by 2.29% and AA by 2.84%, which is a significant improvement.

[0096] Table 1

[0097]

[0098] As shown in Table 2 below, the method of the present invention achieved the best OA and AA on the PU dataset, achieving the best accuracy in 7 out of a total of 9 classes. In particular, in classes 1, 2, 3, 6, 7, and 8, the method of the present invention far outperformed other comparative methods. Moreover, the accuracy of the method of the present invention in each class was improved compared to the basic BYOL, with an improvement of 1.51% in OA and 3.46% in AA.

[0099] Table 2

[0100]

[0101] As shown in Table 3 below, the method of the present invention achieves the best OA and AA on the SA dataset, and reaches the highest accuracy in 14 classes (out of 16 classes). The method of the present invention achieves OA and AA close to 100% in all classes, and improves BYOL's OA by 0.47% and AA by 0.46%, demonstrating the superiority of the method of the present invention.

[0102] Table 3

[0103]

[0104] To more intuitively study how the method of this invention affects the features, this invention uses the t-SNE method to visualize the extracted features.

[0105] Figure 4 The image shows a feature visualization of BYOL and the method UPDA+BYOL of this invention on the IP dataset. (b) shows a more concentrated distribution of color patches belonging to the same class, with fewer fragments and a shape that is more blocky than striped.

[0106] Figure 5 This is a feature visualization on the PU dataset. Many categories are partially mixed in (a), but in (b), the overlapping portions of the color patches are smaller, meaning there are larger gaps between classes. Clearly, after introducing the method of this invention, the color patches are more tightly connected internally.

[0107] Figure 6This is a feature visualization on the SA dataset. In (b), the pores in the color patches are smaller, and there are almost no fragmented color patches. In addition, the contact points between different color patches are reduced.

[0108] By comparing the visualization of features, this invention can conclude that the method of this invention can help BYOL reduce intra-class discrepancies, increase inter-class discrepancies, and improve feature representation.

[0109] The method of this invention, compared with existing technologies, realizes a hyperspectral classification method based on band erasure and contrastive learning. It is simple, efficient, low-cost, and achieves high classification accuracy. Addressing the problems of scarce hyperspectral image data and difficult annotation, the contrastive learning algorithm is used for self-supervised feature extraction in hyperspectral classification. Training requires only a small number of labeled samples, resulting in low dependence on labeled data. To address the issue of unsupervised learning's poor performance in hyperspectral classification, band erasure, gradient masking, and random occlusion are incorporated to generate positive sample pairs for the contrastive learning network input. This encourages the model to focus more on information at key locations, reduces interference from non-critical information, and extracts more representative features, effectively improving classification performance.

[0110] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A hyperspectral classification method based on band erasure and contrastive learning, characterized in that, Includes the following steps: Step 1: Collect hyperspectral data; Step 2: Band erasure is performed on the original hyperspectral data to obtain two non-overlapping hyperspectral data X0 and X1. The two hyperspectral data are preprocessed to obtain sample patch groups P0 and P1 for the two branches of the contrastive learning network. Each pixel corresponds to two patches centered on that pixel. These two patches are used as sample pairs. The band erasure mentioned above is: By extracting odd-numbered and even-numbered layers from the original hyperspectral data, two original hyperspectral data sets with non-overlapping bands are obtained. That is, the two original hyperspectral data sets are obtained by erasing half of the bands from the original hyperspectral data, and the bands are non-overlapping. The preprocessing of the two hyperspectral data sets is as follows: Principal component analysis is performed on the hyperspectral data X0 and X1 respectively, resulting in d principal component maps. The edges of the principal component maps are filled with 0. Then, an s*s sliding window with a step size of 1 is used to divide the principal component maps into sample patches, resulting in two sample patch groups P0 and P1. Here, d and s are preset values. Patch groups P0 and P1 contain a series of s*s*d three-dimensional data. The patches contain spectral information and spatial information. The spectral information is compressed after principal component analysis, and the category of the center pixel represents the category of the entire patch. Step 3: Use a gradient mask to cover the patches that serve as branches on the contrastive learning network, and leave the patches on the lower branches unchanged; where the upper branch on the contrastive learning network is the online network branch, and the lower branch is the target network branch. Step 3 specifically includes: The gradient mask is a weight matrix with the highest value at the center and decreasing values ​​further away from the center. It has dimensions s*s, the same as the patch size. The weight at the center is set to 1, the weights at the four vertices are set to 0, and the other values ​​are linearly interpolated based on their distance from the center. The formula is as follows: Where mask is a gradient mask matrix, (i, j) is the position of an element in the mask, and (center, center) is the center position of the mask; The new weighted patch P0' is obtained by multiplying the corresponding element of the patch P0 in the upper branch with the gradient mask; the patch P1 in the lower branch does not need to be processed by the gradient mask. Step 4: Randomly occlude the patches on the two branches processed in Step 3, and perform data augmentation to obtain positive sample pairs (v). i ,v i '); Step 5: Set hyperparameters, input all positive sample pairs into the contrastive learning network for network training, and save the network parameters and features after training. Step 6: Use the extracted features and labeled categories as the training set to train the classifier. Step 7: After processing in steps 2 to 4, the hyperspectral data to be classified is classified using the trained contrastive learning network and classifier.

2. The hyperspectral classification method based on band erasure and contrastive learning according to claim 1, characterized in that, The random occlusion step in step 4 includes the following steps: 4.1) Set the area of ​​the occlusion rectangle, specifically as a fixed percentage of the patch area or a random value within a certain range; 4.2) Set the aspect ratio; 4.3) Calculate the length and width of the rectangle; 4.4) Randomly select the starting point of the rectangle in the patch and replace the original pixel value with the occlusion value; The occlusion value is a set constant. When occluding randomly, the pixels at the center of the patch need to be preserved to prevent key information from being occluded. Occlusion is performed simultaneously on each layer of the patch.

3. The hyperspectral classification method based on band erasure and contrastive learning according to claim 1, characterized in that, The data augmentation in step 4 is one or more of the following: random cropping, color distortion, random flipping, random grayscale, and Gaussian noise.

4. The hyperspectral classification method based on band erasure and contrastive learning according to claim 1, characterized in that, The contrastive learning network is a BYOL network, which has a structure consisting of an upper branch online network and a lower branch target network. Put (v i ,v i ') Input to the BYOL network, where v i The input is fed into an online network and encoded by an encoder to obtain the representation u. i =f online (v i The mapping z is obtained after passing through the mapper. i =g online (u i The prediction result k is obtained after passing through the predictor. i =q online (z i );v i The input is fed into the target network and encoded by the encoder to obtain u. i '=f target (v i '), after passing through the mapper, z is obtained i '=g target (u i '); The computational process for the online network and the target network is represented by the following formula: k i =q online (g online (f online (v i ))) z i '=g target (f target (v i ′)) The optimization objective of the BYOL network is to make the positive examples of the online network closer to the positive examples of the target network in the representation space. The parameters of the online network are updated using a loss function, and the parameters of the target network are updated using an exponential moving average (EMA) based on the parameters of the online network, with the update step size controlled by the hyperparameter τ. The loss is calculated based on the outputs of the two branches. First, regarding k i and z i Perform L2 regularization. Then take the L2 norm of the difference between the two, as shown in the formula: Swap the input patches of the two branches, that is, change v i Input to the target network, v i The input is fed into an online network, and the loss function is calculated. The final loss function of the BYOL network is: The parameter update process can be represented as: W target ←τW target +(1-τ)W online Among them W online These are parameters of the online network, W target η represents the parameters of the target network, where η is the learning rate and τ is the weight for parameter updates. After training reaches the required number of iterations, the encoder and mapper parameters are saved, and the output of the mapper is used as a feature and input into the classifier.

5. The hyperspectral classification method based on band erasure and contrastive learning according to claim 4, characterized in that, The encoder consists of three 3D convolutional layers, one 2D convolutional layer, and one fully connected layer. After convolution, the layers are flattened before being input to the fully connected layer. Each layer is followed by a batch regularization layer and a ReLU activation function. The mapper consists of a fully connected layer, a regularization layer, and a ReLU activation function; the predictor consists of a fully connected layer, a regularization layer, and a ReLU activation function.

Images