A hyperspectral remote sensing image classification method based on double-channel mapping

Abstract

The application discloses a hyperspectral remote sensing image classification method based on a double-channel mapping, designs a light-weight space-spectrum joint feature extraction model, fully utilizes space-spectrum joint features under the condition of using only four convolution layers, and in addition, a fixed scale convolution kernel can cause the singleness of feature extraction; in order to obtain different numbers of band combination information and extract more abundant spectral features for classification, a multi-scale feature extraction module is added in the light-weight space-spectrum joint network, three 1x1 convolutions with different scales are used for spectral feature extraction and are connected in series, in view of the problem that the features after the connection are not strongly related, the spectral dimension and the channel dimension of the feature map are integrated, and then a full connection layer is used for mapping; while the multi-scale feature fusion is carried out, the correlation of the features of each scale is enhanced, and the importance of the features of different scales is distinguished through a weighted manner. The design method has very outstanding performances in classification accuracy and the like.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image classification technology, specifically to a hyperspectral remote sensing image classification method based on dual-channel mapping. Background Technology

[0002] Hyperspectral remote sensing imagery contains rich spatial spectral features and, compared to conventional remote sensing imagery, possesses a stronger ability to identify ground features. Therefore, it has been widely applied in fields such as mineral resource development, vegetation analysis, and agricultural production. Furthermore, research based on hyperspectral remote sensing imagery is extensive, including dimensionality reduction and unmixing. Hyperspectral remote sensing imagery classification refers to the process of assigning a corresponding ground feature category to each pixel in the image using a specific classification algorithm. Based on the usage of labeled samples, it can be divided into unsupervised classification, semi-supervised classification, and supervised classification.

[0003] While the high spectral resolution and high dimensionality of hyperspectral remote sensing images bring significant advantages to the detailed identification of ground objects, they also bring many problems and challenges. The main problems are: (1) Redundancy of spectral features: the bands in hyperspectral remote sensing images are continuous, the intervals between adjacent bands are short, the feature similarity is high, and a large number of redundant features are contained. (2) Spatial homogeneity and spatial heterogeneity: the proportion of similar ground objects and dissimilar ground objects in a certain area of ​​hyperspectral remote sensing images is very different, and the spatial distribution between different samples may be different, which will affect the classification work. (3) Different spectra of the same object and different objects with the same spectrum: the spectral features of different ground objects have pseudo-similarity, which interferes with the classification results. (4) Nonlinear structure: hyperspectral remote sensing images are three-dimensional tensor data and belong to nonlinear structure, making it difficult for traditional statistical classification models to classify the original images directly. (5) Difficulty in obtaining labeled samples: the number of labeled samples that can be used in the classification task is small, and more effective classification methods need to be designed when the sample is limited. (6) Data offset: When the sensor acquires ground feature information, it may be affected by factors such as instruments, which may interfere with the acquired information. It needs to be corrected before use.

[0004] In recent years, Convolutional Neural Networks (CNNs), based on their unique convolutional operations, have demonstrated stronger feature extraction capabilities for spatial features and have been widely applied in hyperspectral image classification tasks. Methods such as 1D-CNN, 2D-CNN, and 3D-CNN have been developed. 1D-CNN compresses the image into a one-dimensional vector to extract spectral features; 2D-CNN uses two-dimensional convolutional kernels to extract features in the spatial dimension of the image, enabling modeling of spatial information but neglecting the extraction of spectral features; 3D-CNN extracts feature information from both the spectral and spatial dimensions simultaneously, fully leveraging the unified spectral and spatial characteristics of HSI (Hyperspectral Image Separation), achieving better classification results, but with higher computational complexity. Therefore, how to fully extract the rich spatial-spectral features of hyperspectral images using convolutional neural networks still requires further research.

[0005] Fixed-scale convolutional kernels lead to the uniformity of feature extraction and the loss of some features. Therefore, models need multi-scale receptive fields to obtain features at different scales. Dilated convolution expands the receptive field by increasing the sampling interval while keeping the number of parameters constant. However, when stacked convolutional layers use the same sampling interval, a grid effect occurs, resulting in information loss. Deformable convolution learns an offset for each sampling point through the convolutional layers and uses the offset sampling points to achieve adaptive receptive field sampling. Compared to the fixed-interval sampling method of dilated convolution, the offset of deformable convolution is learnable. Deformable convolution can overcome the limitation of the convolutional kernel neighborhood through offset sampling and achieve adaptive sampling. However, the number of sampling points is fixed and is still limited by the size of the convolutional kernel. Therefore, how to fully utilize features at different scales still requires further research. Summary of the Invention

[0006] The technical problems to be solved by this invention are how to enhance the feature extraction and classification capabilities of the model: 1. How to make full use of different global and local features; 2. How to make full use of the rich spatial and spectral features of hyperspectral images and avoid information loss; 3. How to solve the problem of weak correlation after concatenating multi-scale features.

[0007] To achieve the above functions, this invention designs a hyperspectral remote sensing image classification method based on dual-channel mapping, which executes the following steps S1-S8 to complete the classification of hyperspectral remote sensing images:

[0008] Step S1: For the hyperspectral remote sensing image, divide it into blocks using a 9×9 window size;

[0009] Step S2: Input the segmented hyperspectral remote sensing image into the spectral feature module, and extract spectral features from the hyperspectral remote sensing image based on three-dimensional convolution to obtain a spectral feature map;

[0010] Step S3: Input the spectral feature map into the dual-channel mapping module, extract the spectral feature map using three 1×1 convolutions of different scales, and concatenate the spectral feature maps of each scale. For the concatenated spectral feature map, use a fully connected layer to integrate the information of the spectral feature maps of each scale in the spectral dimension and the channel dimension.

[0011] Step S4: Input the spectral feature map obtained in step S3 into the spectral feature integration module, and use a 1×1 convolution with the same spectral dimension to integrate all the information in the spectral dimension, thereby obtaining the global spectral feature map;

[0012] Step S5: Input the global spectral feature map into the spatial-spectral joint feature extraction module, and perform spatial-spectral joint feature extraction on the global spectral feature map based on the three-dimensional convolutional layer to obtain the spatial-spectral joint feature map;

[0013] Step S6: For the spatial-spectral joint feature map, global average pooling is used to integrate global spatial information;

[0014] Step S7: Flatten the features after global average pooling into a one-dimensional vector and input it into the fully connected layer for category prediction. The output dimension of the fully connected layer has the same number of categories. Finally, the output is a probability vector, and the value in the probability vector represents the probability that the hyperspectral remote sensing image belongs to the corresponding category.

[0015] Step S8: Obtain the corresponding category label based on the probability vector to complete the classification of the hyperspectral remote sensing image.

[0016] As a preferred embodiment of the present invention, the spectral feature module performs the following steps in step S2:

[0017] Step S2.1: Use a 1×1×8 three-dimensional convolution to extract spectral features from the segmented hyperspectral remote sensing image to obtain a spectral feature map, where the three-dimensional convolution form is valid. While extracting spectral features, the hyperspectral remote sensing image is reduced in spectral dimension.

[0018] Step S2.2: Continue to extract spectral features from the spectral feature map using a 1×1×9 three-dimensional convolution. The convolution form is "same". The size of the spectral feature map is not changed during the spectral feature extraction process.

[0019] As a preferred embodiment of the present invention: the dual-channel mapping module in step S3 includes a multi-scale feature fusion module and a feature mapping module; the multi-scale feature fusion module uses three 1×1 convolutions of different scales to extract spectral feature maps, assuming the spectral feature maps at each scale are (X1, X2, ..., X...). n )∈R H×W×S×CThe spectral feature maps at various scales are then concatenated, with a splicing dimension of axis=2. The process is expressed as follows:

[0020] X = concat([X1,X2,...,X...) n ])∈R H×W×nS×C axis=2

[0021] Where X represents the cascaded spectral feature map, the process of merging the spectral dimension and channel dimension of the cascaded spectral feature map is expressed as the following formula:

[0022]

[0023] In the formula, reshape represents the operation of merging the spectral and channel dimensions of the spectral feature map, X ′ This represents the merged spectral feature map;

[0024] The feature mapping module uses fully connected layers to integrate information from spectral feature maps at various scales in both the spectral and channel dimensions. The process is represented by the following equation:

[0025] Y = FC(X) ′ )∈R H×W×q

[0026] Where q represents the scale of the fully connected layer feature map, n represents the number of spectral feature maps at different scales, H represents the height of the spectral feature map, W represents the width of the spectral feature map, S represents the spectral dimension of the spectral feature map, C represents the number of channels of the spectral feature map, Y represents the spectral feature map output by the dual-channel mapping module, and FC represents the fully connected operation, expressed as the following formula:

[0027] Y = wX′ + b

[0028] Where w represents the weight and b represents the bias.

[0029] As a preferred technical solution of the present invention, the three 1×1 convolutions of different scales used in the multi-scale feature fusion module are 1×1×5, 1×1×9, and 1×1×13, respectively.

[0030] As a preferred technical solution of the present invention: the spectral feature integration module in step S4 uses a three-dimensional convolution with a size of 1×1×128 and a form of valid to integrate all information in the spectral dimension, thereby obtaining a global spectral feature map.

[0031] As a preferred technical solution of the present invention: the spatial-spectral joint feature extraction module in step S5 uses a three-dimensional convolutional layer with a size of 5×5×5 to perform spatial-spectral joint feature extraction on the global spectral feature map to obtain a spatial-spectral joint feature map.

[0032] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0033] The original image is divided into 9×9 blocks. Local and global 1×1 convolutions are used to extract local and global spectral features, obtaining more complete spectral features for classification and fully utilizing the rich spectral features of hyperspectral images. Secondly, to avoid the feature extraction uniformity caused by fixed-scale convolution kernels, three 1×1 convolutions of different scales are used to extract spectral features at different scales, and a dual-channel mapping module is used to integrate features from different scales. Finally, a three-dimensional convolution is used to extract joint spatial and spectral features, taking into account the correlation between spectral and spatial features, thereby improving the model's classification ability. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of a spatial-spectral joint feature extraction network based on dual-channel mapping provided in an embodiment of the present invention;

[0035] Figure 2 This is a sensitivity analysis of model parameters based on the convolution kernel scale provided in an embodiment of the present invention;

[0036] Figure 3 This is a sensitivity analysis of SDCNN parameters based on mapping scale provided in an embodiment of the present invention;

[0037] Figure 4 These are classification graphs of different methods provided in embodiments of the present invention on the Indian Pines dataset;

[0038] Figure 5 These are classification graphs on the University of Pavia dataset using different methods provided in embodiments of the present invention;

[0039] Figure 6 This is a classification graph on the University of Houston dataset, showing different methods provided in embodiments of the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0041] In network feature extraction, using a single-size convolutional kernel is insufficient to fully extract targets of varying scales in the feature map. Fixed-scale convolutional kernels only consider single-scale information, resulting in typically fixed extracted features and thus limited feature extraction, impacting the network's final classification performance. To enhance the network's feature extraction capabilities, it's essential to endow the model with the ability to extract features at different scales, broadening the network's width. This involves considering contextual information across multiple scales to obtain feature information from different neighborhoods within the feature map, enriching the diversity of feature extraction and ultimately improving the network's feature representation capabilities. Common multi-scale feature fusion methods primarily include add and concat.

[0042] The Add(Addition) method directly adds feature maps of different scales to generate a fused feature map. This method operates on elements at corresponding positions within the feature maps of different scales, requiring that the size of each dimension of the fused feature map be consistent. Furthermore, the shape of the feature map remains unchanged after fusion; only the information content of each feature point in the feature map increases. While the Add method is an information fusion operation, resulting in stronger correlation between features at different scales in the fused feature map, the direct superposition of elements at corresponding positions alters the feature mapping of the original feature map, potentially causing some feature information to be covered or lost. The Concat method concatenates feature maps of different scales along a certain dimension to generate a new feature map. This method requires that all dimensions of the fused feature map be consistent except for the concatenation dimension. The size of the concatenation dimension after fusion is the sum of the sizes of all fused feature maps in that dimension, while other dimensions remain unchanged. Compared to the Add fusion method, this method has a larger number of parameters and higher computational complexity, and the correlation between the fused features is weaker.

[0043] Therefore, this invention addresses the problems of the concat feature fusion method by proposing a dual-channel mapping module. After concatenating feature maps at three scales, a multi-scale feature mapping module is added. Through a fully connected layer, the spectral dimension of the feature maps and the channel dimension of the 3D convolution are mapped to the same dimension, integrating the feature information of the three feature maps in both the spectral and channel dimensions, and establishing the correlation between the spectral features at the three scales. Furthermore, by training the weight parameters in the fully connected layer, the importance of the features at the three scales is distinguished, enabling the network to extract more discriminative features for classification.

[0044] Reference Figure 1 The present invention provides a hyperspectral remote sensing image classification method based on dual-channel mapping, which performs the following steps S1-S8 to complete the classification of hyperspectral remote sensing images:

[0045] Step S1: For the hyperspectral remote sensing image, divide it into blocks using a 9×9 window size;

[0046] Step S2: Input the segmented hyperspectral remote sensing image into the spectral feature module, and extract spectral features from the hyperspectral remote sensing image based on three-dimensional convolution to obtain a spectral feature map;

[0047] The spectral feature module performs the following steps:

[0048] Step S2.1: Use a 1×1×8 three-dimensional convolution to extract spectral features from the segmented hyperspectral remote sensing image to obtain a spectral feature map, where the three-dimensional convolution form is valid. While extracting spectral features, the hyperspectral remote sensing image is reduced in spectral dimension.

[0049] Step S2.2: Continue to extract spectral features from the spectral feature map using a 1×1×9 three-dimensional convolution. The convolution form is "same". The size of the spectral feature map is not changed during the spectral feature extraction process.

[0050] Step S3: Input the spectral feature map into the dual-channel mapping module, extract the spectral feature map using three 1×1 convolutions at different scales, and concatenate the spectral feature maps at each scale. For the concatenated spectral feature map, use a fully connected layer to integrate the information of the spectral feature maps at each scale in the spectral dimension and the channel dimension to enhance the correlation of features at different scales.

[0051] The dual-channel mapping module includes a multi-scale feature fusion module and a feature mapping module. The multi-scale feature fusion module uses 1×1×5, 1×1×9, and 1×1×13 convolutions to extract spectral feature maps at different scales. Let the spectral feature maps at each scale be (X1, X2, ..., X...). n )∈R H×W×S×C The spectral feature maps at various scales are concatenated along the spectral dimension, with a concatenation dimension of axis=2. The process is expressed as follows:

[0052] X = concat([X1,X2,...,X...) n ])∈R H×W×nS×C axis=2

[0053] Where X represents the concatenated spectral feature map, and concat represents the concatenation operation, which merges the spectral and channel dimensions of the concatenated spectral feature map. The process is expressed as follows:

[0054]

[0055] In the formula, reshape represents the operation of merging the spectral and channel dimensions of the spectral feature map, X ′ This represents the merged spectral feature map;

[0056] The feature mapping module uses fully connected layers to integrate information from spectral feature maps at various scales in both the spectral and channel dimensions. The process is represented by the following equation:

[0057] Y = FC(X) ′ )∈R H×W×q

[0058] Where q represents the scale of the feature mapping of the fully connected layer, determined through parameter sensitivity experiments. The size of this scale determines the amount of feature information contained after feature mapping. If the value of the mapping scale is too small, it cannot fully express the feature information contained in all the fused feature maps, resulting in information loss; if the value of the mapping scale is too large, it will contain too much redundant feature information, increasing the number of additional parameters and computational complexity, affecting the final classification performance of the network. Therefore, the value of the feature mapping scale needs to be set to the most appropriate scale based on parameter sensitivity experiments; n represents the number of spectral feature maps at different scales, H represents the height of the spectral feature map, W represents the width of the spectral feature map, S represents the spectral dimension of the spectral feature map, C represents the number of channels of the spectral feature map, Y represents the spectral feature map output by the dual-channel mapping module, and FC represents the fully connected operation, expressed as the following formula:

[0059] Y = wX′ + b

[0060] Where w represents the weight and b represents the bias, the fully connected layer has the function of feature weighting, and can distinguish the importance of features at different scales through the weights learned during training.

[0061] Step S4: Input the spectral feature map obtained in step S3 into the spectral feature integration module, and use a 1×1 convolution with the same spectral dimension to integrate all the information in the spectral dimension, thereby obtaining the global spectral feature map;

[0062] The spectral feature integration module uses a 1×1×128 three-dimensional convolution with valid form to integrate all information in the spectral dimension, thereby obtaining a global spectral feature map.

[0063] Step S5: Input the global spectral feature map into the spatial-spectral joint feature extraction module, and perform spatial-spectral joint feature extraction on the global spectral feature map based on the three-dimensional convolutional layer to obtain the spatial-spectral joint feature map;

[0064] The spatial-spectral joint feature extraction module uses a 5×5×5 three-dimensional convolutional layer to perform spatial-spectral joint feature extraction on the global spectral feature map, establishes the connection between spectral features and spatial features, solves the problem of information loss when extracting spatial features based on spectral features, and obtains a spatial-spectral joint feature map.

[0065] Step S6: For the spatial-spectral joint feature map, global average pooling is used to integrate global spatial information;

[0066] Step S7: Flatten the features after global average pooling into a one-dimensional vector and input it into the fully connected layer for category prediction. The output dimension of the fully connected layer has the same number of categories. Finally, the output is a probability vector, and the value in the probability vector represents the probability that the hyperspectral remote sensing image belongs to the corresponding category.

[0067] Step S8: Obtain the corresponding category label based on the probability vector to complete the classification of the hyperspectral remote sensing image.

[0068] The following is a hyperspectral remote sensing image classification method based on dual-channel mapping designed in this invention. In practice, the effectiveness of the proposed classification model and its extensions is verified using three internationally recognized hyperspectral remote sensing datasets: Indian Pines, University of Pavia, and University of Houston.

[0069] 1. Experimental setup

[0070] (1) Training sample setup

[0071] On the IP dataset, 5% of the samples from each class are randomly selected as the training set, and the remaining samples are used as the test set; on the PU dataset, 1% of the samples from each class are randomly selected as the training set, and the remaining samples are used as the test set; on the HU dataset, 5% of the samples from each class are selected as the training set, and the remaining samples are used as the test set.

[0072] In experiments with different sample sizes, [0.5%, 1%, ..., 5%] of each class were randomly selected as training samples on the IP dataset; [0.01%, 0.02%, ..., 0.1%] of each class were randomly selected as training samples on the PU dataset; and [0.5%, 1%, ..., 5%] of each class were selected as training samples on the HU dataset, with the remaining samples used as test samples.

[0073] (2) Selection of comparison method

[0074] Two classic types of convolutional neural networks: two-dimensional convolutional neural networks and three-dimensional convolutional neural networks.

[0075] Three methods based on convolutional receptive field enhancement: (1) Dilated convolution: MDR-CNN (Multi-scale Dilated Residual CNN). (2) Deformable convolution: DCNN (Deformable convolutional neural networks) and DHCNet (Deformable hyperspectral image classification networks).

[0076] A method for joint spatial-spectral feature extraction: SSRN (Spectral-Spatial Residual Network).

[0077] (3) Evaluation indicators

[0078] The classification results were quantitatively evaluated by statistically analyzing and comparing overall accuracy (OA), class accuracy (CA), mean accuracy (AA), and the Kappa coefficient (κ). For all classification algorithms used, the evaluation metric was the average of the results from 10 independent runs with randomly initialized training samples.

[0079] 2. Experimental Results

[0080] (1) Effect of kernel size

[0081] The size of the convolution kernel determines the number of feature sampling points. The combination of convolution kernels in the multi-scale modules of the model is one of the key parameters. To determine the optimal combination of convolution kernels in the model, ten different scale combinations were selected: (1,3,5), (1,5,9), (3,5,9), (1,7,13), (3,7,11), (5,7,9), (1,9,17), (3,9,15), (5,9,13), and (7,9,11). A corresponding number of samples were selected from three different datasets for comparison. By observing the model's accuracy performance on the three different datasets, the optimal multi-scale convolution combination was determined. Figure 2 Line graphs showing the relationship between multi-scale convolution kernel combinations and classification accuracy are presented for three datasets.

[0082] Depend on Figure 2 It can be seen that when the convolution kernel combinations are all large or small, such as (1,3,5) and (7,9,11), the model's classification accuracy in the three datasets is lower than in other cases. This is because when three convolution kernels of similar size are all large or small, the extracted features have little difference and are all features of similar scale, which does not sufficiently increase the diversity of spectral features. When the multi-scale convolution combination is (5,9,13), the model achieves the best classification performance in all three datasets. Therefore, the multi-scale convolution combination in the model is set to (5,9,13).

[0083] (2) Impact of mapping scale

[0084] The scale of the feature mapping determines the amount of feature information contained after feature mapping. If the scale is too small, it cannot fully express all the feature information contained in the fused feature map, resulting in information loss. If the scale is too large, it will increase excessive feature redundancy, increasing the number of parameters and computational complexity. In three datasets, the classification accuracy of the model was compared when the mapping scale was {32, 64, 96, 128, 160, 192} to observe the model's performance on the three datasets and determine the optimal mapping scale. Figure 3 Line graphs showing the relationship between mapping scale and classification accuracy in the three datasets are provided.

[0085] Depend on Figure 3 As can be seen, with the increase of the mapping scale, the classification accuracy of the model in this chapter shows a trend of first increasing and then decreasing in all three datasets, which is particularly evident in the Indian Pines and University of Pavia datasets. This is because when the mapping scale is small, the mapped feature map cannot completely represent features at different scales; as the mapping scale increases, the feature map after mapping has stronger feature representation capabilities, containing more complete multi-scale feature information for classification, so the model's classification accuracy improves accordingly; when the mapping scale increases to a certain extent, the mapped feature map can already completely represent multi-scale features, and further increases will result in feature redundancy, increasing the number of model parameters and computational complexity, affecting the model's classification performance, thus leading to a decrease in accuracy. Considering the model's performance in the three datasets, the final mapping scale of the model was set to 128.

[0086] (3) Ablation test

[0087] To compare the classification performance of the two traditional multi-feature fusion methods, add and concat, and to verify the effectiveness of the dual-channel mapping module, a comparison was conducted on three internationally recognized datasets. As shown in Table 1, both traditional multi-feature fusion methods showed some accuracy improvement compared to the base model, with the concat method showing a more significant improvement. This is because both methods utilize features at different scales. However, the add method directly adds features at each scale, which may damage the features; while the concat method concatenates features at each scale, avoiding feature overlap, but the correlation between features at different scales is weak, ignoring the relationships between features at different scales. From the perspective of feature similarity, the spectral information of different neighborhoods differs significantly, making it more suitable for information amplification fusion methods, thus the concat method achieved better results. After adding the multi-scale feature mapping module, the concatenated multi-scale features are connected through fully connected layers, fully utilizing the features at each scale and their correlations. By training the weight parameters in the fully connected layers to distinguish the importance of features at different scales, the model's feature extraction capability is enhanced. The module achieved the best results on the OA, AA, and Kappa datasets, proving the effectiveness of the dual-channel mapping module.

[0088] Table 1

[0089]

[0090] (4) Classification performance analysis of different algorithms on IP dataset

[0091] Table 2 lists the classification accuracy of each comparative method on the Indian Pines dataset. The proposed model achieves the highest classification accuracy compared to the other comparative methods, with an OA of 97.88 ± 0.43%, which is 16.65% and 9.07% higher than 2D-CNN and 3D-CNN methods, respectively; 16.24% higher than MDR-CNN; 2.78% and 2.47% higher than the two spatially deformable convolutional methods DCNNs and DHCNet, respectively; and 3.04% higher than SSRN. The model also achieves the highest AA and Kappa coefficients on the Indian Pines dataset, exceeding the other comparative methods by 1.65%–23.83% and 0.80–19.14%, respectively. Regarding classification accuracy for individual categories, eight out of the 16 categories in the IndianPines dataset achieved the highest accuracy, particularly in categories such as alfalfa, non-plown maize, and maize. The classification accuracy for the haystack category reached 99.98%. Classification diagrams of different methods on the IndianPines dataset are shown below. Figure 4 .

[0092] Table 2

[0093]

[0094] (5) Classification performance analysis of different algorithms on the PU dataset

[0095] Table 3 lists the classification accuracy of each comparative method on the University of Pavia dataset. The proposed model achieves the highest classification accuracy compared to the other comparative methods, with an OA of 99.09 ± 0.33%, which is 6.75% and 4.60% higher than 2D-CNN and 3D-CNN methods, respectively; 7.05% higher than MDR-CNN; 2.26% and 2.76% higher than the two spatially deformable convolutional methods DCNNs and DHCNet, respectively; and 0.91% higher than SSRN. Simultaneously, the AA and Kappa coefficients in the University of Pavia dataset also achieved the highest values, exceeding the other comparative methods by 0.83%–6.42% and 0.61–9.42%, respectively. Regarding the classification accuracy of individual categories, five out of the nine categories in the University of Pavia dataset achieved the highest classification accuracy, including the grass, gravel, and tree categories. The grass category achieved a classification accuracy of 99.97%, and the bare land category achieved a classification accuracy of 99.84%. Figure 5 The corresponding comparison methods are shown in the full-image classification results on the University of Pavia dataset. It can be seen from the classification results of each comparison method that the proposed model has less noise and fewer misclassifications, and the classification results are the best overall.

[0096] Table 3

[0097]

[0098] (6) Classification performance analysis of different algorithms on the HU dataset

[0099] Table 4 lists the classification accuracy of each comparative method on the University of Houston dataset. The proposed model achieves the highest classification accuracy compared to the other comparative methods, with an OA of 97.82 ± 0.46%, which is 5.61% and 3.85% higher than 2D-CNN and 3D-CNN methods, respectively; 5.05% higher than MDR-CNN; 1.56% and 1.46% higher than the two spatially deformable convolutional methods DCNNs and DHCNet, respectively; and 2.27% higher than SSRN. The model also achieves the highest AA and Kappa coefficients on the University of Houston dataset, exceeding the other comparative methods by 1.28%–6.16% and 1.20–6.07%, respectively. Regarding classification accuracy for individual categories, among the 15 categories in the University of Houston dataset, 7 categories achieved the highest classification accuracy, including categories such as forest, water, residential area, and commercial area. Among them, the classification accuracy of the railway category reached 98.68%, and the classification accuracy of the runway category reached 100.0%.

[0100] Figure 6 The corresponding comparison methods are shown in the full-map classification results on the University of Houston dataset. It can be seen from the classification results of each comparison method that the proposed model has less noise, clearer boundaries between categories, less deformation of ground features, and achieves the best overall classification results.

[0101] Table 4

[0102]

[0103] As can be seen from the above embodiments, this invention is a hyperspectral remote sensing image classification method based on dual-channel mapping. In a lightweight spatial-spectral joint network, 1×1 convolution is first used to extract and reduce the dimensionality of spectral features, and then global 1×1 convolution is used to integrate global spectral information, enhancing the completeness of feature extraction. Three-dimensional convolutional layers are used to simultaneously extract spatial-spectral joint features, establishing the correlation between spatial and spectral features. Finally, global average pooling is used to integrate global information. Furthermore, to avoid the problem of feature extraction uniformity caused by fixed-scale convolution kernels, three different scales of 1×1 convolution are used to extract spectral features. To address the problem of weak correlation after concatenating multi-scale features, a dual-channel mapping module is designed. This module integrates information from the spectral and channel dimensions using fully connected layers, thereby establishing the correlation between features at different scales. The method designed in this invention exhibits outstanding performance in terms of classification accuracy, proving the effectiveness of the proposed method.

[0104] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A hyperspectral remote sensing image classification method based on dual-channel mapping, characterized in that, Perform the following steps S1-S8 to complete the classification of the hyperspectral remote sensing image: Step S1: For the hyperspectral remote sensing image, divide it into blocks using a 9×9 window size; Step S2: Input the segmented hyperspectral remote sensing image into the spectral feature module, and extract spectral features from the hyperspectral remote sensing image based on three-dimensional convolution to obtain a spectral feature map; Step S3: Input the spectral feature map into the dual-channel mapping module, extract the spectral feature map using three 1×1 convolutions of different scales, and concatenate the spectral feature maps of each scale in the spectral dimension. For the concatenated spectral feature map, use a fully connected layer to integrate the information of the spectral feature maps of each scale in the spectral dimension and the channel dimension. The dual-channel mapping module includes a multi-scale feature fusion module and a feature mapping module; The multi-scale feature fusion module uses three 1×1 convolutions at different scales to extract spectral feature maps. Let the spectral feature maps at each scale be... The spectral feature maps at various scales are concatenated along the spectral dimension, with a concatenation dimension of axis=2. The process is expressed as follows: in X The concatenated spectral feature map is represented by the following formula, which combines the spectral and channel dimensions of the concatenated spectral feature map: In the formula, reshape This indicates an operation that merges the spectral and channel dimensions of the spectral feature map. This represents the merged spectral feature map; The feature mapping module uses fully connected layers to integrate information from spectral feature maps at various scales in both the spectral and channel dimensions. The process is represented by the following equation: in q This represents the scale of the feature map of the fully connected layer. n This indicates the number of spectral feature maps at different scales. H Indicates the height of the spectral feature map. W Indicates the width of the spectral feature map. S The spectral dimension represents the spectral feature map. C Indicates the number of channels in the spectral feature map. Y This represents the spectral feature map output by the dual-channel mapping module. FC The fully connected operation is represented by the following formula: in w Indicates weight, b Indicates bias; Step S4: Input the spectral feature map obtained in step S3 into the spectral feature integration module, and use a 1×1 convolution with the same spectral dimension to integrate all the information in the spectral dimension, thereby obtaining the global spectral feature map; Step S5: Input the global spectral feature map into the spatial-spectral joint feature extraction module, and perform spatial-spectral joint feature extraction on the global spectral feature map based on the three-dimensional convolutional layer to obtain the spatial-spectral joint feature map; Step S6: For the spatial-spectral joint feature map, global average pooling is used to integrate global spatial information; Step S7: Flatten the features after global average pooling into a one-dimensional vector and input it into the fully connected layer for category prediction. The output dimension of the fully connected layer has the same number of categories. Finally, the output is a probability vector, and the value in the probability vector represents the probability that the hyperspectral remote sensing image belongs to the corresponding category. Step S8: Obtain the corresponding category label based on the probability vector to complete the classification of the hyperspectral remote sensing image.

2. The hyperspectral remote sensing image classification method based on dual-channel mapping according to claim 1, characterized in that, In step S2, the spectral feature module performs the following steps: Step S2.1: Use a 1×1×8 three-dimensional convolution to extract spectral features from the segmented hyperspectral remote sensing image to obtain a spectral feature map, where the three-dimensional convolution form is valid. While extracting spectral features, the hyperspectral remote sensing image is reduced in spectral dimension. Step S2.2: Continue to extract spectral features from the spectral feature map using a 1×1×9 three-dimensional convolution. The convolution form is "same". The size of the spectral feature map is not changed during the spectral feature extraction process.

3. The hyperspectral remote sensing image classification method based on dual-channel mapping according to claim 1, characterized in that, The three 1×1 convolutions of different scales used in the multi-scale feature fusion module are 1×1×5, 1×1×9, and 1×1×13.

4. The hyperspectral remote sensing image classification method based on dual-channel mapping according to claim 1, characterized in that, In step S4, the spectral feature integration module uses a 1×1×128 three-dimensional convolution with the form valid to integrate all information in the spectral dimension, thereby obtaining a global spectral feature map.

5. The hyperspectral remote sensing image classification method based on dual-channel mapping according to claim 1, characterized in that, In step S5, the spatial-spectral joint feature extraction module uses a 5×5×5 three-dimensional convolutional layer to perform spatial-spectral joint feature extraction on the global spectral feature map to obtain the spatial-spectral joint feature map.

Images