A semi-supervised hyperspectral image intelligent classification method

Abstract

The application discloses a kind of semi-supervised hyperspectral image intelligent classification method, accurate ground object identification is carried out using labeled sample and unlabeled sample.The application first constructs a multiscale deformable spectral spatial feature extraction module, can extract features on multiple scales and fusion, the introduction of deformable convolution can extract more in line with the distribution characteristics of sample.Then through the proposed pseudo-label generation strategy, Gaussian function is used to weight, further improve the utilization efficiency of pseudo-label and model performance.In the training process, maintain high quantity and high quality of pseudo-label to effectively use unlabeled sample.Finally, through the transformation, long-range dependencies are captured and effective feature representations of hyperspectral remote sensing images are learned.Experimental results demonstrate the effectiveness and superiority of the application under the condition of limited labeled samples.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and hyperspectral remote sensing image processing technology, specifically to a semi-supervised intelligent classification method for hyperspectral images. Background Technology

[0002] Hyperspectral remote sensing images possess hundreds of continuous bands with very high spectral resolution, providing rich spectral and spatial information for accurate identification of land cover types. Therefore, hyperspectral image classification has been widely applied in fields such as environmental monitoring, precision agriculture, mineral detection, military and national defense security, and natural disaster assessment. However, given the rich spectral and spatial information in hyperspectral images, efficiently extracting and fully utilizing this information is crucial for improving classification accuracy, especially when training samples are limited.

[0003] Traditional classification methods directly use hundreds of spectra in hyperspectral images as pixel features for classification, such as Support Vector Machines (SVM) and Random Forests (RF). While these methods achieve high classification accuracy, they suffer from the curse of dimensionality when dealing with high-dimensional data. Furthermore, these methods struggle to fully explore the correlation between spectral and spatial features.

[0004] In recent years, deep learning technology has made significant progress. Its representative achievement, Convolutional Neural Networks (CNNs), has been widely applied in various fields, including hyperspectral image classification, due to its excellent feature extraction performance. Chen et al. were the first to use two-dimensional CNNs to extract spatial spectral features from hyperspectral data. Two-dimensional CNNs effectively preserve the original data structure through local connectivity, parameter sharing, and shift invariance, but they can only process two-dimensional features and fail to fully utilize the high-dimensional nature of spectral information. Three-dimensional CNNs extend the dimension of the convolutional kernels, enabling them to capture both spectral and spatial features simultaneously. The three-dimensional autoencoder proposed by Mei et al. utilizes a three-dimensional convolutional network to extract joint spectral and spatial features, demonstrating good performance. However, the high computational cost of three-dimensional convolutional kernels limits its practicality. To balance feature extraction efficiency and computational complexity, some methods employ a dual-branch network structure to process spectral and spatial features separately. For example, a one-dimensional CNN or fully connected network is used as the spectral branch, and a two-dimensional CNN as the spatial branch, ultimately fusing the two to generate joint features. This design improves classification accuracy to some extent, but still suffers from insufficient capture of long-range dependency information.

[0005] While CNNs have achieved good results in hyperspectral image classification, they still have some limitations. CNNs struggle to capture long-range information in HSI (Hyperspectral Sequence Indicators). The emergence of Transformers successfully addresses this shortcoming, overcoming the receptive field limitations of CNNs. Unlike CNNs, the Transformer architecture is a self-attention-based structure combined with a multilayer perceptron (MLP), overcoming receptive field limitations and modeling long-range dependencies labeled in the sequence, making it easier to capture long-range information. Vision Transformer (VIT) and its variants have approached or even surpassed CNN-based frameworks, achieving state-of-the-art performance on mainstream computer vision tasks. Hong et al. first applied VIT to hyperspectral image classification, using HSI blocks as sequential data from the spectral dimension, and learning locally detailed spectral representations using group spectral embeddings and a Transformer encoder module. Sun et al. proposed a Spectral Spatial Feature Tagging Transformer (SSFTT), which effectively extracts local features by combining CNN modules and uses a Gaussian weighted tagger to extract high-level semantic features. Although Transformers excel in capturing long-range information, their ability to model short-range relationships is insufficient, leading to inadequate local feature extraction. Furthermore, most current Transformer methods fail to fully utilize multi-scale spectral information and there is still room for improvement.

[0006] The training methods described above belong to supervised learning methods, requiring a sufficient number of training samples to train the model. When the number of training samples is insufficient, the model parameters cannot be updated effectively, and the number of labeled samples is quite limited due to the time-consuming and laborious nature of data labeling. To address this issue, semi-supervised learning methods that combine labeled and unlabeled data for training have received widespread attention. For example, Jia et al. used an autoencoder to extract features from unlabeled data in an unsupervised manner, and then fine-tuned the model using a limited number of labeled samples. Yao et al. proposed an active learning semi-supervised framework that constructs a reliable pseudo-label set by actively selecting samples that generate high label consistency based on SLIC. Wang et al. proposed an Adaptive DropBlock Enhanced Generative Adversarial Network (ADGAN) to address the mode collapse problem and the imbalance of training data. He et al. proposed a semi-supervised generative adversarial network combined with VIT to better preserve spectral sequence information and alleviate class imbalance. Pseudo-label generation methods are widely used due to their simplicity and effectiveness. Pseudo-label generation methods apply algorithms to generate labels (i.e., "pseudo-labels") for unlabeled samples and use these pseudo-labels to further train the network. Pseudo-label generation methods are mainly divided into two types: fixed threshold strategies and adaptive threshold strategies.

[0007] The fixed threshold strategy sets a confidence threshold and selects only high-confidence unlabeled samples for loss calculation. This method effectively ensures the quality of pseudo-labels, but it also discards a large number of low-confidence pseudo-labels that may be correct, resulting in low utilization of unlabeled samples.

[0008] Adaptive thresholding strategies dynamically adjust the confidence threshold based on the current training state of the model, thereby selecting more pseudo-labels for training. Compared to fixed thresholding strategies, adaptive thresholding strategies can more fully tap the potential of unlabeled samples, helping to improve the coverage of training data. However, this strategy inevitably introduces a certain proportion of erroneous pseudo-labels, which may mislead the model training and affect the final performance.

[0009] In summary, although hyperspectral image classification technology has made significant progress in supervised and semi-supervised learning methods, there are still many challenges to be solved in terms of how to efficiently extract spectral and spatial information and how to deal with data scarcity and imbalance. Summary of the Invention

[0010] Objective: To address the limitations of existing technologies, this invention proposes a semi-supervised intelligent classification method for hyperspectral images. This invention utilizes a multi-scale deformable spectral spatial feature extraction module to effectively extract and fuse spectral and spatial features at multiple scales. Simultaneously, deformable convolution is introduced to accurately capture key features consistent with the sample distribution. Furthermore, this invention proposes a Gaussian-weighted pseudo-label generation strategy, further improving model performance by balancing the quantity and quality of pseudo-labels. Combined with the powerful long-range dependency modeling capabilities of Transformer, this invention can efficiently learn multi-scale feature representations of hyperspectral remote sensing images, providing a more accurate and robust solution for hyperspectral image classification.

[0011] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0012] Step 1: First, the data is preprocessed. PCA is used to reduce the dimensionality of the HSI data. While maintaining the spatial structure, the most informative bands in the spectral dimension are extracted. The dimensionality-reduced data is represented as follows: The spatial dimension is represented by H×W, and B represents the number of bands in the spectral dimension;

[0013] Step 2: Divide the dimensionality-reduced data into multiple blocks, each block consisting of the target pixel and its neighboring pixels, represented as follows: Where M×N is the block size;

[0014] Step 3: Randomly divide the samples into labeled datasets, unlabeled datasets, and test sets, with a fixed number of samples in each class. Perform both weak and strong augmentation on the samples in the unlabeled dataset simultaneously, laying the foundation for subsequent consistency regularization training.

[0015] Step 4: Construct a multi-scale deformable convolution module. This module introduces deformable convolution kernels of different scales and extracts feature information of different scales in parallel through a multi-branch structure, effectively enhancing the feature representation capability.

[0016] Step 5: Construct a multi-scale deformable Transformer network, combining the multi-scale deformable convolutional modules built in Step 4 with the Transformer structure. The multi-scale deformable convolutional modules are used to extract local spatial spectral features, while the Transformer structure is used for global feature modeling, thereby achieving more efficient feature extraction and classification. The network simultaneously receives labeled and unlabeled data to facilitate optimization through semi-supervised learning.

[0017] Step 6: For labeled samples, the model performs supervised learning using the cross-entropy loss function to learn the correspondence between data and labels; for unlabeled samples, a consistency regularization strategy is adopted to ensure that the weakly enhanced and strongly enhanced versions of the input image generate consistent prediction results. During training, the proposed pseudo-label weighting strategy is combined to balance the quantity and quality of pseudo-labels, so as to make fuller use of unlabeled samples;

[0018] Step 7: Combine the multi-scale deformable Transformer network with a pseudo-label weighting strategy and perform iterative training until the neural network model converges to obtain the final trained model. Finally, use a fully connected network to post-process the model output to obtain the classification result for each pixel.

[0019] Preferably, in step 3, for the same unlabeled sample, both a weakly enhanced version and a strongly enhanced version are generated simultaneously. Weak enhancement is achieved by randomly horizontally flipping (mirroring) the input image; while strong enhancement includes multiple enhancement methods, specifically: randomly rotating or horizontally flipping the input image, applying a random enhancement module, and adding noise. The random enhancement module includes the following operations: randomly rotating the image by a certain angle, cropping the image along a horizontal or vertical direction (skewing and stretching the image along a certain direction), and translating the image content along a horizontal or vertical direction (moving the image on the canvas).

[0020] Preferably, in step 4, the multi-scale deformable convolution module is constructed as follows:

[0021] In the multi-scale deformable convolution module, we introduce deformable convolution kernels of different scales: 3x3, 5x5, and 7x7. These kernels are computed in parallel on different branches within the same layer, extracting feature information at different scales. The output features of each convolutional layer are processed through batch normalization and ReLU activation layers. Finally, the features output from the convolutional layers of different scales are fused using Concat and 1x1 convolution operations.

[0022] Preferably, deformable convolution dynamically adjusts the position of the convolution kernel by introducing a learnable offset, as implemented below:

[0023] Given an input feature map, the output of a traditional convolution can be represented as:

[0024]

[0025] in It is the sampling range of the convolution kernel. Here, p represents the weights of the convolution kernel, and p0 represents the pixel position on the feature map. This represents the position index within the sampling range, i.e., the offset relative to p0. This indicates that the input feature map is at the location The value on the graph. Deformable convolution adds a learnable offset to each point.

[0026]

[0027] Due to the addition of new sampling points with offset For non-integer values, bilinear interpolation is used to obtain the final pixel value:

[0028]

[0029] Where p represents any non-integer sampling point q lists all integer spatial locations in the feature map x, and G(·,·) is the bilinear interpolation kernel.

[0030] Finally, the complete process can be described by the following formula:

[0031]

[0032] in, This represents the sampling range of the i-th convolutional kernel. This indicates that the i-th convolutional kernel in the l-th layer is at position p. n The weights, Z (l) (p0+p n +Δp i ) indicates that the input feature map is at position p0+p n +Δpi The value at that location, Z represents the bias of the i-th convolutional kernel in the l-th layer. (l+1) (p0) indicates the output feature map value at position p0 of layer l+1.

[0033] Preferably, in step 5, the multi-scale deformable Transformer network is constructed as follows:

[0034] (1) Initial feature extraction: The input is processed by a 3D convolution module for feature extraction. The module includes a 3D convolution layer, a batch normalization layer and a ReLU activation function, which can effectively extract the initial spatial spectral features of the hyperspectral image.

[0035] (2) Feature dimension reorganization: The output of the 3D convolution module is reorganized in the channel and spatial dimensions and flattened into a two-dimensional plane representation to adapt to the input requirements of subsequent multi-scale deformable convolution modules.

[0036] (3) Multi-scale deformable convolution module: Construct a multi-branch deformable convolution module containing deformable convolution kernels of three scales: 3×3, 5×5 and 7×7. Extract local feature information of different scales through parallel operations. Combined with deformable convolution, it can adaptively capture more flexible spatial structure information. The convolution results are spliced ​​in the channel dimension to retain feature information of different scales.

[0037] (4) 2D Convolutional Feature Fusion: The concatenated feature tensors are further processed by a 2D convolution module. This module not only integrates the spatial dimensions of the features but also compresses and fuses the channel-dimensional information to generate a feature map with a unified representation. Simultaneously, batch normalization layers and the ReLU activation function are combined during the fusion process to enhance the discriminative power of the features.

[0038] (5) Location Embedding and Classification Label Addition: Location embedding information is added to the fused feature sequence to preserve the spatial location information of the input data and help the model understand the order of the input sequence. In addition, a classification label is added to the feature sequence for subsequent output results in classification tasks to predict the category.

[0039] (7) Global feature modeling: The feature sequence is modeled using the Transformer structure. The feature dependencies over long distances are captured by the multi-head self-attention mechanism and the multilayer perceptron (MLP). The feature representation is optimized by combining residual connections and layer normalization operations.

[0040] (8) Feature extraction and classification: Extract feature representations of classification labels from the output of Transformer, and complete the final classification prediction through a multilayer perceptron (MLP) containing layer normalization and fully connected layers, outputting the category label of each pixel in the hyperspectral image.

[0041] Preferably, in step 6, the pseudo-label weighting strategy is implemented as follows:

[0042] A truncated Gaussian function is used to fit the sample confidence distribution and to weight the loss function. Lower weights are assigned to potentially correct pseudo-labels, and higher weights are assigned to pseudo-labels with higher confidence. The magnitude of the weight determines the contribution of the sample to the loss function, achieving better generalization performance. The sample weight λ(p) is defined as follows:

[0043]

[0044] Where p is the predicted probability vector of the unlabeled sample, max(p) is the maximum value in p, i.e., the highest confidence of the model for a certain unlabeled sample, and λ max The upper limit of the weighted values ​​is defined when max(p) is greater than or equal to μ. t When the weighted upper limit λ is directly taken, max μ t σ represents the mean confidence level during the current training phase. t The standard deviation of the confidence level during the current training phase is obtained by taking the highest confidence value max(p) of the unlabeled samples, i.e.:

[0045]

[0046]

[0047] Among them, B U For unlabeled samples, the batch size is max(p) i ) represents the highest confidence level of the i-th unlabeled sample. Let be the variance of the confidence level at the current training phase. Then we use an exponential moving average (EMA) with momentum m to summarize the statistics from previous batches to obtain a more stable estimate:

[0048] μ t =mμ t-1 +(1-m)μ b

[0049]

[0050] Here, we initialize μ0 to Initialized to 1.0.

[0051] Adaptively estimating Gaussian parameters based on the confidence distribution improves the model's generalization ability. During training, as the model's learning ability improves and the confidence increases, the tail of the Gaussian function grows exponentially. False labels with low confidence are assigned lower weights, and the confidence level is within μ... t The pseudo-labels on the left and right are being used more effectively.

[0052] Compared with the prior art, the present invention has the following technical effects:

[0053] 1. Robustness of Multi-Scale Feature Extraction and Enhancement. The multi-scale deformable convolution module proposed in this invention utilizes three deformable convolution kernels with different scales and dilation rates to effectively learn discriminative features at different scales. Deformable convolution can dynamically adjust the shape and size of the convolution kernel according to the structure and features of the input data, thereby adapting to complex spatial features, improving the model's ability to capture detailed information in hyperspectral images, and significantly enhancing robustness.

[0054] 2. Optimization of the pseudo-label generation strategy. This invention designs a pseudo-label weighting strategy based on a truncated Gaussian function, which weights unlabeled data by fitting a confidence distribution. During training, this strategy can simultaneously ensure the quantity and quality of pseudo-labels, making full use of unlabeled data and alleviating the limitation of data scarcity on model performance.

[0055] 3. Efficient Spatial Spectral Information Modeling. This invention combines CNN networks and Transformer structures to fully mine the spatial and spectral information in hyperspectral images from shallow to deep. This combination can concisely and efficiently express the multi-level semantic features of hyperspectral images, significantly improving the accuracy and efficiency of classification. Attached Figure Description

[0056] Figure 1 The diagram shows the structure of the semi-supervised hyperspectral image classification model based on multi-scale deformable Transformer and pseudo-label generation proposed in this invention.

[0057] Figure 2 This is a schematic diagram of the multi-scale deformable convolution module proposed in this invention.

[0058] Figure 3 These are pseudo-color images and corresponding ground truth labels of the hyperspectral images from the WHLK dataset used in this embodiment of the invention.

[0059] Figure 4Simulation results are shown for land cover classification using existing supervised and semi-supervised classification methods on the WHLK dataset, and for land cover classification using the present invention. Figure (a) is a real pseudo-color image, Figure (b) is a real land cover label image, Figure (c) is the classification result of the HybridSN model, Figure (d) is the classification result of the HSMSN model, Figure (e) is the classification result of the 3DAES model, Figure (f) is the classification result of the DMSGer model, Figure (g) is the classification result of the S3Net model, Figure (h) is the classification result of the AMGCFN model, and Figure (i) is the classification result of the model of the present invention. Detailed Implementation

[0060] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0061] like Figure 1 As shown in the figure, the semi-supervised intelligent classification method for hyperspectral images disclosed in this invention specifically includes the following steps:

[0062] Step 1: First, preprocess the data by using PCA to reduce the dimensionality of the HSI data. The dimensionality-reduced data is represented as follows: The spatial dimension is represented by H×W, and B represents the number of bands in the spectral dimension.

[0063] Step 2: Divide the dimensionality-reduced data into multiple blocks, each block consisting of the target pixel and its neighboring pixels, represented as follows: Where M×N is the block size.

[0064] Step 3: Randomly divide the samples into labeled, unlabeled, and test sets with a fixed number of samples in each class. Perform both weak and strong augmentation on the samples in the unlabeled dataset simultaneously, laying the foundation for subsequent consistency regularization training. Weak augmentation is achieved by randomly horizontally flipping (mirroring) the input image; strong augmentation includes multiple enhancement methods, specifically: randomly rotating or horizontally flipping the input image, applying a random augmentation module, and adding noise. The random augmentation module includes the following operations: randomly rotating the image by a certain angle, cropping the image along a horizontal or vertical direction (skewing and stretching the image along a certain direction), and translating the image content along a horizontal or vertical direction (moving the image on the canvas).

[0065] Step 4: Construct a multi-scale deformable convolution module. This module introduces deformable convolution kernels of different scales and extracts feature information of different scales in parallel through a multi-branch structure, effectively enhancing the feature representation capability.

[0066] In the multi-scale deformable convolution module, we introduce deformable convolution kernels of different scales: 3x3, 5x5, and 7x7. These kernels are computed in parallel on different branches within the same layer, extracting feature information at different scales. The output features of each convolutional layer are processed through batch normalization and ReLU activation layers. Finally, the features output from the convolutional layers of different scales are fused using Concat and 1x1 convolution operations.

[0067] Deformable convolution dynamically adjusts the position of the convolution kernel by introducing a learnable offset, as implemented below:

[0068] Given an input feature map, the output of a traditional convolution can be represented as:

[0069]

[0070] in It is the sampling range of the convolution kernel. Here, p represents the weights of the convolution kernel, and p0 represents the pixel position on the feature map. This represents the position index within the sampling range, i.e., the offset relative to p0. This indicates that the input feature map is at the location The value on the graph. Deformable convolution adds a learnable offset to each point.

[0071]

[0072] Due to the addition of new sampling points with offset For non-integer values, bilinear interpolation is used to obtain the final pixel value:

[0073]

[0074] Where p represents any non-integer sampling point q lists all integer spatial locations in the feature map x, and G(·,·) is the bilinear interpolation kernel.

[0075] Finally, the complete process can be described by the following formula:

[0076]

[0077] in, This represents the sampling range of the i-th convolutional kernel. This indicates that the i-th convolutional kernel in the l-th layer is at position p. n The weights, Z (l) (p0+p n +Δp i ) indicates that the input feature map is at position p0+p n +Δp iThe value at that location, Z represents the bias of the i-th convolutional kernel in the l-th layer. (l+1) (p0) indicates the output feature map value at position p0 of layer l+1.

[0078] Step 5: Construct a multi-scale deformable Transformer network, combining the multi-scale deformable convolutional modules built in Step 4 with the Transformer structure. The multi-scale deformable convolutional modules are used to extract local spatial spectral features, while the Transformer structure is used for global feature modeling, thereby achieving more efficient feature extraction and classification. The network simultaneously receives labeled and unlabeled data for optimization through semi-supervised learning.

[0079] The multi-scale deformable Transformer network is constructed as follows:

[0080] (1) Initial feature extraction: The input is processed by a 3D convolution module for feature extraction. The module includes a 3D convolution layer, a batch normalization layer and a ReLU activation function, which can effectively extract the initial spatial spectral features of the hyperspectral image.

[0081] (2) Feature dimension reorganization: The output of the 3D convolution module is reorganized in the channel and spatial dimensions and flattened into a two-dimensional plane representation to adapt to the input requirements of subsequent multi-scale deformable convolution modules.

[0082] (3) Multi-scale deformable convolution module: Construct a multi-branch deformable convolution module containing deformable convolution kernels of three scales: 3×3, 5×5 and 7×7. Extract local feature information of different scales through parallel operations. Combined with deformable convolution, it can adaptively capture more flexible spatial structure information. The convolution results are spliced ​​in the channel dimension to retain feature information of different scales.

[0083] (4) 2D Convolutional Feature Fusion: The concatenated feature tensors are further processed by a 2D convolution module. This module not only integrates the spatial dimensions of the features but also compresses and fuses the channel-dimensional information to generate a feature map with a unified representation. Simultaneously, batch normalization layers and the ReLU activation function are combined during the fusion process to enhance the discriminative power of the features.

[0084] (5) Location Embedding and Classification Label Addition: Location embedding information is added to the fused feature sequence to preserve the spatial location information of the input data and help the model understand the order of the input sequence. In addition, a classification label is added to the feature sequence for subsequent output results in classification tasks to predict the category.

[0085] (7) Global feature modeling: The feature sequence is modeled using the Transformer structure. The feature dependencies over long distances are captured by the multi-head selfattention mechanism and the multilayer perceptron (MLP). The feature representation is optimized by combining residual connections and layer normalization operations.

[0086] (8) Feature extraction and classification: Extract feature representations of classification labels from the output of Transformer, and complete the final classification prediction through a multilayer perceptron (MLP) containing layer normalization and fully connected layers, outputting the category label of each pixel in the hyperspectral image.

[0087] Step 6: For labeled samples, the model performs supervised learning using the cross-entropy loss function to learn the correspondence between data and labels. For unlabeled samples, a consistency regularization strategy is adopted to ensure that the weakly and strongly enhanced versions of the input image generate consistent prediction results. During training, the proposed pseudo-label weighting strategy is combined to balance the quantity and quality of pseudo-labels, so as to make fuller use of unlabeled samples. The specific implementation is as follows:

[0088] A truncated Gaussian function is used to fit the sample confidence distribution and to weight the loss function. Lower weights are assigned to potentially correct pseudo-labels, and higher weights are assigned to pseudo-labels with higher confidence. The magnitude of the weight determines the contribution of the sample to the loss function, achieving better generalization performance. The sample weight λ(p) is defined as follows:

[0089]

[0090] Where p is the predicted probability vector of the unlabeled sample, max(p) is the maximum value in p, i.e., the highest confidence of the model for a certain unlabeled sample, and λ max The upper limit of the weighted values ​​is defined when max(p) is greater than or equal to μ. t When the weighted upper limit λ is directly taken, max μ t σ represents the mean confidence level during the current training phase. t The standard deviation of the confidence level during the current training phase is obtained by taking the highest confidence value max(p) of the unlabeled samples, i.e.:

[0091]

[0092]

[0093] Among them, B U For unlabeled samples, the batch size is max(p) i ) represents the highest confidence level of the i-th unlabeled sample. Let be the variance of the confidence level at the current training phase. Then we use an exponential moving average (EMA) with momentum m to summarize the statistics from previous batches to obtain a more stable estimate:

[0094] μ t =mμ t-1 +(1-m) μ b

[0095]

[0096] Here, we initialize μ0 to Initialized to 1.0.

[0097] Adaptively estimating Gaussian parameters based on the confidence distribution improves the model's generalization ability. During training, as the model's learning ability improves and the confidence increases, the tail of the Gaussian function grows exponentially. False labels with low confidence are assigned lower weights, and the confidence level is within μ... t The pseudo-labels on the left and right are being used more effectively.

[0098] Step 7: Combine the multi-scale deformable Transformer network with a pseudo-label weighting strategy and perform iterative training until the neural network model converges to obtain the final trained model. Finally, use a fully connected network to post-process the model output to obtain the classification result for each pixel.

[0099] To verify the effectiveness of the present invention, a simulation experiment was conducted based on the above technical solution. The specific results and analysis are as follows:

[0100] 1. Simulation conditions:

[0101] In this embodiment, the experimental dataset used is the WHU-Hi-LongKou (WHLK) dataset collected over Longkou Town, Hubei Province, China. This data was acquired by a DJI Matrice 600Pro UAV platform equipped with a Headwall Nano-Hyperspec sensor. The dataset images have a pixel size of 550×441, consist of 270 bands, and have a spectral range of 0.4–1.0 μm. The dataset contains 204,542 samples across 9 land cover categories. Figure 3 It provides a visualization of the dataset. Figure 3 (a) shows a pseudo-color image of the true hyperspectral image. Figure 3 (b) is a map showing actual ground features. Figure 3(c) shows the accompanying ground truth image, where different colors represent different land cover categories. Furthermore, this experiment randomly selected 10 labeled samples from each category as the training set, used 10% of the samples as an unlabeled training set for pseudo-label generation, and the remaining samples as the test set. Table 1 lists the category names of all land cover types, the number of samples in each category, and the data partitioning, providing a detailed reference for the experimental analysis.

[0102] Table 1 WHLK Land Cover Categories and Their Data Classification

[0103]

[0104] To comprehensively evaluate the performance of this invention, the simulation experiment of this invention uses the overall accuracy (OA), average accuracy (AA), and Kappa coefficient of the classification results as quantitative evaluation indicators for the classification performance of the hyperspectral image classification task, and plots the classification results of land cover as qualitative evaluation indicators.

[0105] 2. Comparison and Analysis of Simulation Results

[0106] Table 2 Comparison of Ground Cover Classification Accuracy of Existing Techniques and the Method of the Present Invention on Real Hyperspectral Images

[0107]

[0108] As can be seen from the quantitative results in Table 2, the performance of this invention on the dataset outperforms other comparative methods. The multi-scale deformable convolution module proposed in this invention, through flexible feature extraction and effective feature alignment, demonstrates a stronger learning ability for irregularly shaped features, thereby significantly improving the classification accuracy of hyperspectral imagery. Furthermore, the pseudo-label generation strategy generates high-quality predicted labels for unlabeled data, expanding the scale of the training dataset and utilizing these pseudo-labels to help the model learn global relationships between samples. This strategy enables the model to learn from richer samples and deeply understand the complex relationships between samples, thereby improving the model's generalization ability and ultimately achieving better classification results.

Claims

1. A semi-supervised intelligent classification method for hyperspectral images, characterized in that, Includes the following steps: Step 1: First, the data is preprocessed. PCA is used to reduce the dimensionality of the HSI data. While maintaining the spatial structure, the most informative bands in the spectral dimension are extracted. The dimensionality-reduced data is represented as follows: The spatial dimension is represented by H×W, and B represents the number of bands in the spectral dimension; Step 2: Divide the dimensionality-reduced data into multiple blocks, each block consisting of the target pixel and its neighboring pixels, represented as follows: Where M×N is the block size; Step 3: Randomly divide the samples into labeled dataset, unlabeled dataset, and test set according to a fixed number of samples in each class. Perform both weak and strong augmentation on the samples in the unlabeled dataset to lay the foundation for subsequent consistency regularization training. Step 4: Construct a multi-scale deformable convolution module. This module introduces deformable convolution kernels of different scales and extracts feature information of different scales in parallel through a multi-branch structure, effectively enhancing the feature representation capability. Step 5: Construct a multi-scale deformable Transformer network, combining the multi-scale deformable convolutional module constructed in Step 4 with the Transformer structure. The multi-scale deformable convolutional module is used to extract local spatial spectral feature information, while the Transformer structure is used for global feature modeling, thereby achieving more efficient feature extraction and classification. The network simultaneously receives labeled and unlabeled data so that it can be optimized through semi-supervised learning. Step 6: For labeled samples, the model performs supervised learning using the cross-entropy loss function to learn the correspondence between data and labels; For unlabeled samples, a consistency regularization strategy is adopted to ensure that the weak and strong enhancement versions of the input image generate consistent prediction results. During training, the proposed pseudo-label weighting strategy is combined to balance the quantity and quality of pseudo-labels in order to make fuller use of unlabeled samples. Step 7: Combine the multi-scale deformable Transformer network with the pseudo-label weighting strategy and perform iterative training until the neural network model converges to obtain the final trained model. Finally, use a fully connected network to post-process the model output to obtain the classification result of each pixel.

2. The semi-supervised intelligent classification method for hyperspectral images according to claim 1, characterized in that, In step 3, for the same unlabeled sample, both weakly enhanced and strongly enhanced versions are generated. Weak enhancement is achieved by randomly flipping the input image horizontally; while strong enhancement includes multiple enhancement methods, specifically: randomly rotating or horizontally flipping the input image, applying a random enhancement module, and adding noise. The random enhancement module includes the following operations: randomly rotating the image by a certain angle, cropping the image along the horizontal or vertical direction, tilting and stretching the image along a certain direction, and translating the image content along the horizontal or vertical direction.

3. The semi-supervised intelligent classification method for hyperspectral images according to claim 1, characterized in that, In step 4, the multi-scale deformable convolution module is constructed as follows: In the multi-scale deformable convolution module, we introduced deformable convolution kernels of different scales, namely 3x3, 5x5 and 7x7. These convolution kernels are computed in parallel on different branches in the same layer to extract feature information of different scales. The output features of each convolution layer are processed by batch normalization layer and ReLU activation layer, and finally the features output by convolution layers of different scales are fused by Concat and 1x1 convolution operations.

4. The semi-supervised intelligent classification method for hyperspectral images according to claim 3, characterized in that, Deformable convolution dynamically adjusts the position of the convolution kernel by introducing a learnable offset, as implemented below: Given an input feature map, the output of a traditional convolution is represented as: in It is the sampling range of the convolution kernel, w(p) n ) represents the weights of the convolution kernel, p0 represents the pixel position on the feature map, and p n This represents the position index within the sampling range, i.e., the offset relative to p0, x(p0+p n ) indicates that the input feature map is at position p0+p n The deformable convolution adds a learnable offset Δp to each point. n : Due to the addition of a new sampling point p with offset n +Δp n For non-integer values, bilinear interpolation is used to obtain the final pixel value: Where p represents any non-integer sampling point, p = p0 + p n +Δp n q lists all integer spatial locations in the feature map x, and G(·,·) is the bilinear interpolation kernel; Finally, the complete process can be described by the following formula: in, This represents the sampling range of the i-th convolutional kernel. This indicates that the i-th convolutional kernel in the l-th layer is at position p. n The weights, Z (l) (p0+p n +Δp i ) indicates that the input feature map is at position p0+p n +Δp i The value at that location, Z represents the bias of the i-th convolutional kernel in the l-th layer. (t+1) (p0) indicates the output feature map value at position p0 of layer l+1.

5. The semi-supervised intelligent classification method for hyperspectral images according to claim 1, characterized in that, In step 5, the multi-scale deformable Transformer network is constructed as follows: (1) Initial feature extraction: The input is processed by a 3D convolution module for feature extraction. The module includes a 3D convolution layer, a batch normalization layer and a ReLU activation function, which can effectively extract the initial spatial spectral features of the hyperspectral image. (2) Feature dimension reorganization: The output of the 3D convolution module is reorganized in the channel and spatial dimensions and flattened into a two-dimensional plane representation to adapt to the input requirements of the subsequent multi-scale deformable convolution module; (3) Multi-scale deformable convolution module: Construct a multi-branch deformable convolution module, which includes deformable convolution kernels of three scales: 3×3, 5×5 and 7×7. Extract local feature information of different scales through parallel operations. Combined with deformable convolution, it can adaptively capture more flexible spatial structure information. The convolution results are spliced ​​in the channel dimension to retain feature information of different scales. (4) 2D convolution fusion features: The concatenated feature tensors are further processed by the 2D convolution module. This module not only integrates the spatial dimension of the features, but also compresses and fuses the information of the channel dimension to generate a feature map with a unified expression. At the same time, the fusion process combines batch normalization layers and ReLU activation functions to enhance the discriminative ability of the features. (5) Location embedding and classification label addition: Location embedding information is added to the fused feature sequence to preserve the spatial location information of the input data and help the model understand the order of the input sequence. In addition, a classification label is added to the feature sequence for subsequent output results in classification tasks to predict the category. (7) Global feature modeling: The feature sequence is modeled using the Transformer structure. The feature dependencies at a distance are captured by the multi-head self-attention mechanism and the multilayer perceptron (MLP). The feature representation is optimized by combining residual connections and layer normalization operations. (8) Feature extraction and classification: Extract feature representations of classification labels from the output of Transformer, and complete the final classification prediction through a multilayer perceptron (MLP) containing layer normalization and fully connected layers, outputting the category label of each pixel in the hyperspectral image.

6. The semi-supervised intelligent classification method for hyperspectral images according to claim 1, characterized in that, In step 6, the pseudo-label weighting strategy is implemented as follows: A truncated Gaussian function is used to fit the sample confidence distribution and the loss function is weighted. Lower weights are assigned to potentially correct pseudo-labels, and higher weights are assigned to pseudo-labels with higher confidence. The magnitude of the weighting value determines the contribution of the sample to the loss function, thus achieving better generalization performance. The sample weighting value λ(p) is defined as follows: Where p is the predicted probability vector of the unlabeled sample, max(p) is the maximum value in p, i.e., the highest confidence of the model for a certain unlabeled sample, and λ max The upper limit of the weighted value is λ. When max(p) is greater than or equal to μt, the upper limit of the weighted value is directly taken. max μ t σ represents the mean confidence level during the current training phase. t The standard deviation of the confidence level during the current training phase is obtained by taking the highest confidence value max(p) of the unlabeled samples, i.e.: Among them, B U For unlabeled samples, the batch size is max(p) i ) represents the highest confidence level of the i-th unlabeled sample. Let m be the variance of the confidence level at the current training phase. Then, we use an exponential moving average (EMA) with momentum m to summarize the statistics from previous batches to obtain a more stable estimate: m t =mm t-1 +(1-m)m b Here, we initialize μ0 to Initialize to 1.0; Adaptively estimating Gaussian parameters based on the confidence distribution improves the model's generalization ability. During training, as the model's learning ability improves and the confidence increases, the tail of the Gaussian function grows exponentially. False labels with low confidence are assigned lower weights. The confidence level is within μ... t The pseudo-labels on the left and right are being used more effectively.

Images