Space-spectrum global perception double-branch hyperspectral image classification method

Abstract

The invention discloses a space-spectrum global perception double-branch hyperspectral image classification method, which comprises the following steps of: constructing a space-spectrum global perception double-branch network which comprises a space branch and a spectrum branch, constructing a space branch by using a hierarchical feature transformation structure integrated with a global feature enhancement module, and constructing a hyperspectral image by using the spatial branch; during coding, macroscopic and semantic-rich features formed from low-level local textures, edges and other basic features to level ascending are captured in a progressive mode; a space optimization block of the global feature enhancement module separates and reconstructs an input feature map according to the information richness degree, space redundancy is restrained, and a channel optimization block processes a channel dimension feature map through a segmentation-conversion-fusion strategy, and channel representativeness is enhanced; in the decoding stage, multi-level features are fused in the process of recovering the resolution of the feature map, and the feature map with detail and global information is generated through integration; a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit are used for constructing spectral branches, the single-chain equal-dimensional spectral feature generator performs preliminary extraction of spectral features, and a single-chain equal-dimensional connection mode guarantees the extraction stability; a spectrum channel focusing unit accurately obtains a dependency relationship between channels in the spectrum characteristics; space and spectrum branch features are fused through fractional weighting, so that the network fully combines the advantages of the two features. According to the method, global space context information is extracted through the space-spectrum double-branch global sensing network, and fine spectral features are mined, so that the classification effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image classification, and in particular to a spatial-spectral global perception dual-branch hyperspectral image classification method. Background Art

[0002] Hyperspectral images (HSI) with their unique three-dimensional data structure usually cover hundreds of spectral bands. These rich spectral bands not only accurately record the spectral information of the scene, but also contain valuable spatial information, which makes them powerful in finely distinguishing different materials. They have played an important role in many key fields such as target detection, marine exploration, and land planning, and have become the core force of modern remote sensing technology applications. In today's era of rapid development of big data and artificial intelligence technology, hyperspectral image classification technology has also ushered in rapid changes and innovations, and its importance is self-evident. On the one hand, it provides us with a more accurate cognitive approach to deeply understand various materials and phenomena on the earth's surface, and strongly promotes the progress of scientific research; on the other hand, in practical application scenarios such as agricultural monitoring, environmental assessment, mineral exploration, and military reconnaissance, it shows broad application prospects and huge potential value, providing strong technical support for the development of various industries.

[0003] In the early days, commonly used hyperspectral image classification methods included K nearest neighbor (KNN), extreme learning machine (ELM) and support vector machine (SVM). These traditional methods mainly rely on spectral information for classification, but ignore the spatial consistency characteristics of HSI itself to a certain extent, which limits the further improvement of classification performance to a certain extent. In order to overcome this limitation, many researchers have conducted a series of innovative explorations and attempts. For example, Makantasis et al. designed a model based on 2D convolutional neural network. By filling the surrounding pixels, the intermediate pixels are packed into blocks of fixed size, which are input into the neural network to extract spatial information, and finally the extracted features are sent to the multi-layer perceptron to complete the classification task. However, with the deepening of research, people gradually found that it is often difficult to achieve ideal classification results by simply relying on single-dimensional information for image classification. Therefore, researchers gradually shifted their research focus to the classification experiments of the combination of spatial information and spectral information. Networks such as the spectral-spatial residual network (SSRN) and the end-to-end fast dense spectral-spatial convolutional network (FDSSC) take advantage of dense connections to improve the extraction efficiency and classification accuracy of spatial and spectral features; networks such as the dual-branch deep convolutional neural network, the cascaded dual-scale cross network based on SSRN, and the dual-branch dual attention (DBDA) network extract spatial-spectral features at different scales, further improving the extraction efficiency and classification accuracy of spatial and spectral features.

[0004] The above studies show that the dual-branch structure that extracts spatial and spectral features respectively is an effective framework for HSIC. For the above dual-branch structure, although the spectral branch can learn advanced spectral features, it mainly focuses on the extraction of high-level information, ignoring the capture of macro trends and detail features of spectral features, and cannot fully display the complexity of spectral information. At the same time, the spatial branch usually uses local receptive fields to extract spatial features, which makes it difficult to effectively obtain global spatial relationships and long-distance dependencies. This limitation leads to the model's insufficient expression of spatial and spectral features, affecting the overall performance of the classification task. In addition, in order to balance model performance and complexity, the attention mechanism in these methods compresses a large number of channels or spatial resolutions during training, resulting in a large amount of useful information being discarded. Summary of the invention

[0005] Purpose of the invention: Hyperspectral image classification has the characteristics of multiple spectral bands and complex spatial structure. The data contains a large amount of redundant information. It is difficult for existing classification models to strike a balance between extracting sufficient features and controlling model complexity. Secondly, the size of the receptive field plays a key role in feature extraction. It is necessary to design a model that takes into account both local details and global context to improve classification performance. In addition, the attention mechanism compresses a large number of channels or spatial resolutions during the training process, resulting in a large amount of useful information being discarded. To address these problems, this paper proposes a new network that combines spatial branches and spectral branches. The spatial branch adopts an encoder-decoder structure and embeds a global feature enhancement module to extract global spatial context information; the spectral branch combines dense connections with spectral focusing mechanisms to mine more refined features.

[0006] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0007] A spatial-spectral global perception dual-branch hyperspectral image classification method comprises the following steps:

[0008] S1: constructing an overall network, including a spatial-spectral global perception dual-branch network and a classification module, wherein the spatial-spectral global perception dual-branch network includes a spatial branch module and a spectral branch module;

[0009] S2: The spatial branch of the spatial-spectral global perception dual-branch network is constructed by a hierarchical feature transformation structure. The feature extraction stage progressively captures basic features such as low-level local textures and edges to macroscopic and semantically rich features formed as the hierarchy increases; the spatial optimization block of the global feature enhancement module separates and reconstructs the input feature map according to the information richness, suppresses spatial redundancy, and the channel optimization block uses a segmentation-conversion-fusion strategy to process the channel dimension feature map to enhance the channel representativeness; the feature recovery and fusion stage fuses multi-level features in the process of restoring the resolution of the feature map, and integrates to generate a feature map with both details and global information. This branch generally introduces skip connections and adds channel attention to make the feature expression more complete and enhance shallow features to avoid losing feature information during downsampling; spatial hybrid convolution modules are applied at different levels of the hierarchical feature transformation structure to improve the model's ability to understand long-distance relationships in the image, thereby improving the performance of image classification;

[0010] S3: The spectral branch of the spatial-spectral global perception dual-branch network is constructed using a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit. The single-chain equal-dimensional spectral feature generator uses dense connections to perform preliminary extraction of spectral features. The single-chain equal-dimensional connection method ensures the stability of the extraction. The spectral channel focusing unit is constructed with reference to the NonLocalBlock structure to accurately obtain the dependency between each channel in the spectral feature, so as to mine more refined and higher-quality spectral features and retain higher resolution in the spectral dimension.

[0011] S4: Use score weighting to fuse spatial and spectral branch features so that the network can fully combine the advantages of both, and classify through the Softmax classification module to output the results;

[0012] Furthermore, the process of extracting features by the spatial branch of the spatial-spectral global perception dual-branch network described in step S2 is as follows:

[0013] A1: The overall network structure is a hierarchical feature transformation structure to extract spatial features. Its basic modules are spatial hybrid convolution modules and 2DCNN. The structure of this module is as follows: Figure 5 As shown. In the feature extraction stage, the first layer is a spatial mixed convolution module, which uses 1×1 point-by-point convolution to reduce the number of channels of the feature map, thereby reducing the amount of calculation, and then uses a dilated convolution with a dilation rate of 2 to increase the interval within the receptive field and expand the range of the receptive field. Then, maximum pooling is used for downsampling. The second layer is a spatial mixed convolution module. Unlike the first layer, the dilation rate of this layer is set to 4. Then, maximum pooling is used for downsampling. The third layer is a 2DCNN, which uses a 3×3 convolution kernel;

[0014] A2: After preliminary feature extraction, the feature map already has certain high-level semantic information. At this time, the obtained feature map is input into the global feature enhancement module. The schematic diagram of this module is as follows: Figure 4 As shown. First, the feature map is input into the spatial optimization block, and the input feature map is divided into two parts, one with rich information and one with less information, through the separation operation. Then, the feature maps of these two parts are combined through the reconstruction operation to enhance the feature expression and suppress the redundancy in the spatial dimension. The output feature map is used as the input of the channel optimization block, and the split-conversion-fusion strategy is adopted to split the feature map into two parts in the channel dimension. First, 1×1 convolution operations are performed on each part. One part is extracted through efficient convolution operations, and the other part is directly used. Finally, the feature maps of the two parts are merged through an adaptive fusion strategy to reduce the redundancy in the channel dimension and improve the representativeness of the features.

[0015] A3: The feature recovery and fusion module is symmetrical to the feature extraction module. The only difference is that this module uses deconvolution for upsampling. This part uses skip connections to concatenate the features before downsampling with the data after upsampling in the channel dimension. At the same time, the periphery of the upsampled features is filled with 0 to avoid mismatch in spatial dimensions.

[0016] The rules for filling are as follows:

[0017] H p =(H down -H up ) / 2

[0018] W p =(W down -W up ) / 2

[0019] The above formulas represent the padding H above and below the upsampled feature map in the spatial dimension. p 0, fill W on the left and right p 0. Among them, H down and W down They represent the size before downsampling, H up and W up Respectively represent the sizes after upsampling;

[0020] A4: When using skip connections to concatenate features before downsampling and features after upsampling, channel attention is used to capture the relationship between channels, refine channel features, and reduce redundant information of channels;

[0021] A5: Before the second upsampling and the feature map output by the feature recovery and fusion module, the feature is enhanced using spatial attention. At the same time, spatial attention is calculated to reduce spatial redundant information and assign different weights to different pixels.

[0022] A6: After the feature recovery and fusion module, a fully connected layer is used to reduce the channel dimension to the number of categories L, and the output F of the spatial sub-network is obtained. spa ∈R H×W×L ;

[0023] A7: Flatten the obtained spatial subnetwork in the spatial dimension and record the results of the sample pixels according to the index. The indexed output is F spa ∈R d×L .

[0024] Furthermore, the process of extracting features from the spectral branch of the spatial-spectral global perception dual-branch network described in step S3 is as follows:

[0025] B1: Randomly select a 7×7×103 3-D patch cube from the dataset as network input;

[0026] B2: The input data is fed into a single-chain equal-dimensional spectral feature generator, which has a 3-D convolutional layer with batch normalization (BN) and Mish activation function. The convolution kernel size of the convolution layer is 1×1×7, the number of filters is 24, the stride is (1, 1, 2) and there is no padding operation, resulting in a feature map of size (7×7×49, 24);

[0027] B3: The above feature map is sent to the one-time connection part of the single-chain equal-dimensional spectral feature generator. The convolution kernel size of the 3-D convolution layer in this part is 1×1×7, the number of filters is 12, the step size is (1, 1, 1), and the padding is (0, 0, 3) to obtain the intermediate feature map;

[0028] B4: Connect the intermediate feature maps in the channel dimension to obtain a feature map of size (7×7×49, 60);

[0029] B5: The above feature map is passed through a 1×1×1 3-D convolution layer to reduce the channel dimension from 60 to 24, and is added to the last feature map of the single-chain equal-dimensional spectral feature generator to achieve residual connection;

[0030] B6: Send the residual-connected feature map to a 3-D convolution layer with a convolution kernel size of 1×1×49 to obtain a spectral feature map of (7×7×1, 24);

[0031] B7: The feature map of (7×7×1, 24) is used as the input for constructing the spectral channel focusing unit. The convolution kernel size of the 3-D convolution layer of this module is 3×3×1, and the other hyperparameters are the same as those of the single-chain equal-dimensional spectral feature generator to obtain the dependency between each channel in the spectral feature.

[0032] Furthermore, the operation described in step S4 includes:

[0033] C1: Set a learnable positive parameter γ in the network, and the activation function f(·) is a Sigmoid function. The two sub-networks are fused by adaptive weighting and output through the Softmax layer to obtain the final feature representation F:

[0034] f(γ)∈(0,1)γ∈(0,1)

[0035] F = Softmax(f(γ)F spa +(1-f(γ))F spe )

[0036] C2: Compare the model's prediction results with the actual sample labels, and feed back the predicted error loss to the model, which updates the network parameters during the feedback process. After multiple training and network updates, the final network classification results are close to the sample labels. After completing the above training process, the validation data is tested, and the test results are used to determine whether the current network weights should be retained as the best result.

[0037] In the present invention, the spatial branch of the spatial-spectral global perception dual-branch network is first constructed by a hierarchical feature transformation structure. The feature extraction stage captures the basic features from low-level local textures, edges, etc. to the macroscopic and semantically rich features formed with the ascending levels in a progressive manner; the spatial optimization block of the global feature enhancement module separates and reconstructs the input feature map according to the information richness, suppresses spatial redundancy, and the channel optimization block uses a segmentation-conversion-fusion strategy to process the channel dimension feature map to enhance the channel representativeness; the feature recovery and fusion stage fuses multi-level features in the process of restoring the resolution of the feature map, and integrates to generate a feature map with both details and global information. This branch generally introduces skip connections and adds channel attention to make the feature expression more complete and enhance shallow features to avoid the loss of feature information during downsampling; spatial hybrid convolution modules are applied at different levels of the hierarchical feature transformation structure to improve the model's ability to understand long-distance relationships in the image, thereby improving the performance of image classification;

[0038] Secondly, the spectral branch of the spatial-spectral global perception dual-branch network is constructed using a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit. The single-chain equal-dimensional spectral feature generator uses dense connections to perform preliminary extraction of spectral features. The single-chain equal-dimensional connection method ensures the stability of the extraction. The spectral channel focusing unit is constructed with reference to the NonLocalBlock structure to accurately obtain the dependency between each channel in the spectral feature, so as to mine more refined and higher-quality spectral features and retain higher resolution in the spectral dimension.

[0039] Finally, the spatial and spectral branch features are fused by score weighting so that the network can fully combine the advantages of both. The Softmax classification module is used to classify and output the results.

[0040] Beneficial effects: Compared with the prior art, the present invention constructs a spatial-spectral global perception dual-branch network, constructs a spatial branch with a hierarchical feature transformation structure, and introduces a spatial mixed convolution module, a feature enhancement module, spatial attention, and channel attention to obtain and enhance spatial features; in the spectral branch, a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit are used to extract spectral features, ensure the stability of the extraction, and retain a higher spectral resolution, thereby improving classification accuracy and ensuring a good classification effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a network model diagram for the implementation of the present invention.

[0042] Figure 2 Schematic diagram of the spatial branch of the spatial-spectral global perception dual-branch network.

[0043] Figure 3 Schematic diagram of the spectral branch of the spatial-spectral global perception dual-branch network.

[0044] Figure 4 Schematic diagram of the spatial mixed convolution module.

[0045] Figure 5 Schematic diagram of feature enhancement module.

[0046] Figure 6 Schematic diagram of channel attention.

[0047] Figure 7 Schematic diagram of spatial attention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a spatial-spectral global perception dual-branch network hyperspectral image classification method, firstly constructing an overall network, such as Figure 1 As shown, it includes two branches, space and spectral. The spatial branch is constructed with a hierarchical feature transformation structure, in which a spatial mixed convolution module, a feature enhancement module, spatial attention, and channel attention are introduced to obtain and enhance spatial features; in the spectral branch, a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit are used to extract spectral features, ensure the stability of the extraction, and retain a higher spectral resolution.

[0050] Based on the above method, this embodiment will be applied to hyperspectral image classification. The specific process is as follows:

[0051] like Figure 1 As shown, a spatial-spectral global perception dual-branch hyperspectral image classification method disclosed in an embodiment of the present invention specifically comprises the following steps:

[0052] S1: The original hyperspectral image is represented as X∈R H×W×B , using principal component analysis (PCA, Pri n cipalComponents Analysis) to reduce the dimension of the original hyperspectral image, where H, W, and B represent the height, width, and number of bands of the hyperspectral image, respectively. The hyperspectral image after dimensionality reduction is represented as X∈R H×W×b , where b represents the number of bands after dimensionality reduction;

[0053] S2: Input the hyperspectral data after dimensionality reduction into the spatial branch, such as Figure 2 As shown, specifically including:

[0054] A1: The overall network structure is a hierarchical feature transformation structure to extract spatial features. Its basic modules are spatial hybrid convolution modules and 2DCNN. The structure of this module is as follows: Figure 4 As shown. In the feature extraction stage, the first layer is a spatial mixed convolution module, which uses 1×1 point-by-point convolution to reduce the number of channels of the feature map, thereby reducing the amount of calculation, and then uses a dilated convolution with a dilation rate of 2 to increase the interval within the receptive field and expand the range of the receptive field. Then, maximum pooling is used for downsampling. The second layer is a spatial mixed convolution module. Unlike the first layer, the dilation rate of this layer is set to 4. Then, maximum pooling is used for downsampling. The third layer is a 2DCNN, which uses a 3×3 convolution kernel;

[0055] A2: After preliminary feature extraction, the feature map already has certain high-level semantic information. At this time, the obtained feature map is input into the global feature enhancement module. The schematic diagram of this module is as follows: Figure 5 As shown. First, the feature map is input into the spatial optimization block, and the input feature map is divided into two parts, one with rich information and one with less information, through the separation operation. Then, the feature maps of these two parts are combined through the reconstruction operation to enhance the feature expression and suppress the redundancy in the spatial dimension. The output feature map is used as the input of the channel optimization block, and the split-conversion-fusion strategy is adopted to split the feature map into two parts in the channel dimension. First, 1×1 convolution operations are performed on each part. One part is extracted through efficient convolution operations, and the other part is directly used. Finally, the feature maps of the two parts are merged through an adaptive fusion strategy to reduce the redundancy in the channel dimension and improve the representativeness of the features.

[0056] A3: The feature recovery and fusion module is symmetrical to the feature extraction module. The only difference is that this module uses deconvolution for upsampling. This part uses skip connections to concatenate the features before downsampling with the data after upsampling in the channel dimension. At the same time, the periphery of the upsampled features is filled with 0 to avoid mismatch in spatial dimensions.

[0057] The rules for filling are as follows:

[0058] H p =(H down -H up ) / 2

[0059] W p =(W down -W up ) / 2

[0060] The above formulas represent the padding H above and below the upsampled feature map in the spatial dimension. p 0, fill W on the left and right p 0. Among them, H down and W down They represent the size before downsampling, H up and W up Respectively represent the sizes after upsampling;

[0061] A4: When using skip connections to concatenate features before downsampling and features after upsampling, channel attention is used to capture the relationship between channels, refine channel features, and reduce redundant information of channels. Channel attention is as follows Figure 6 As shown;

[0062] A5: Before the second upsampling and the feature map output by the feature recovery and fusion module, the features are enhanced using spatial attention. At the same time, spatial attention is calculated to reduce spatial redundant information and assign different weights to different pixels. Figure 7 As shown;

[0063] A6: After the feature recovery and fusion module, a fully connected layer is used to reduce the channel dimension to the number of categories L, and the output F of the spatial sub-network is obtained. spa ∈R H×W×L ;

[0064] A7: Flatten the obtained spatial subnetwork in the spatial dimension and record the results of the sample pixels according to the index. The indexed output is F spa ∈R d×L .

[0065] S3: The hyperspectral data after dimensionality reduction is input into the spectral branch again, such as Figure 3 As shown, specifically including:

[0066] B1: Randomly select a 7×7×103 3-D patch cube from the dataset as network input;

[0067] B2: The input data is fed into a single-chain equal-dimensional spectral feature generator, which has a 3-D convolutional layer with batch normalization (BN) and Mish activation function. The convolution kernel size of the convolution layer is 1×1×7, the number of filters is 24, the stride is (1, 1, 2) and there is no padding operation, resulting in a feature map of size (7×7×49, 24);

[0068] B3: The above feature map is sent to the one-time connection part of the single-chain equal-dimensional spectral feature generator. The convolution kernel size of the 3-D convolution layer in this part is 1×1×7, the number of filters is 12, the step size is (1, 1, 1), and the padding is (0, 0, 3) to obtain the intermediate feature map;

[0069] B4: Connect the intermediate feature maps in the channel dimension to obtain a feature map of size (7×7×49, 60);

[0070] B5: The above feature map is passed through a 1×1×1 3-D convolution layer to reduce the channel dimension from 60 to 24, and is added to the last feature map of the single-chain equal-dimensional spectral feature generator to achieve residual connection;

[0071] B6: Send the residual-connected feature map to a 3-D convolution layer with a convolution kernel size of 1×1×49 to obtain a spectral feature map of (7×7×1, 24);

[0072] B7: The feature map of (7×7×1, 24) is used as the input for constructing the spectral channel focusing unit. The convolution kernel size of the 3-D convolution layer of this module is 3×3×1, and the other hyperparameters are the same as those of the single-chain equal-dimensional spectral feature generator to obtain the dependency between each channel in the spectral feature.

[0073] S4: Use score weighting to fuse spatial and spectral branch features so that the network can fully combine the advantages of both, and classify through the Softmax classification module to output the results, including:

[0074] C1: Set a learnable positive parameter γ in the network, and the activation function f(·) is a Sigmoid function. The two sub-networks are fused by adaptive weighting and output through the Softmax layer to obtain the final feature representation F:

[0075] f(γ)∈(0,1)γ∈(0,1)

[0076] F = Softmax(f(γ)F spa +(1-f(γ))F spe )

[0077] C2: Compare the model's prediction results with the actual sample labels, and feed back the predicted error loss to the model, which updates the network parameters during the feedback process. After multiple training and network updates, the final network classification results are close to the sample labels. After completing the above training process, the validation data is tested, and the test results are used to determine whether the current network weights should be retained as the best result.

[0078] In order to verify the effect of the method of the present invention, based on the above technical solution, this embodiment carried out a simulation experiment, and the specific results and analysis are as follows:

[0079] 1. Experimental Data

[0080] In this embodiment, the proposed spatial-spectral global perception dual-branch classification network will be tested on three benchmark datasets: IP (Indian Pines), PU (Pavia University), and SA (Salinas) to verify the effectiveness and reliability of the model.

[0081] The Indiana Pines (IP) dataset is a commonly used test dataset for hyperspectral image classification. It was acquired by the American Infrared Imaging Spectrometer (AVIRIS) in northwest Indiana. The wavelength range of this dataset is 0.4-2.5μm, the spatial size is 145×145 pixels, and the spatial resolution is about 20m. Generally, the 200 bands remaining after excluding 20 bands are used as the research objects. It contains a total of 10,249 ground object pixels, which contain 16 types of ground objects.

[0082] The Pavia University (PU) dataset was collected by the German Airborne Reflection Optical Spectroscopic Imager (ROSIS). The wavelength range of the dataset is 0.43-0.86μm, the spatial size is 610×340 pixels, and the spatial resolution is 13m. Generally, the image composed of 103 spectral bands remaining after removing 12 frequency bands with strong noise and water vapor interference is used as the research object. It contains a total of 42,776 ground object pixels, which contain 9 types of ground objects.

[0083] The Salinas (SA) dataset was taken by the AVIRIS imaging spectrometer in the Salinas Valley of California, USA. The wavelength range of the dataset is 0.4-2.5μm, the spatial size is 512×217 pixels, and the spatial resolution is about 3.7m. Generally, the 204 bands remaining after excluding 20 bands are used as the research objects, and a total of 54,129 ground object pixels are included, which contain 16 types of ground objects.

[0084] Table 1. Category of ground feature information in IP dataset, SA dataset and PU dataset

[0085]

[0086]

[0087] 2. Experimental methods and related parameter settings

[0088] To verify the performance, a series of experiments were conducted on three datasets. The classification performance was measured using three evaluation indicators: overall accuracy (OA), average accuracy (AA), and kappa coefficient (KC). OA is the ratio of correct labels to the total number of labels, AA is the average accuracy of all categories, and the kappa coefficient measures the consistency between the predicted label and the true value. All experiments were implemented on the NVIDIA RTX 3080 Ti GPU. In this experiment, 1%, 1%, and 98% of the samples of each class were randomly selected for training, verification, and testing on the IP dataset, SA dataset, and PU dataset, respectively.

[0089] 3. Comparison of experimental results

[0090] Table-2 IP dataset test results

[0091]

[0092] Table-3 PU dataset test results

[0093]

[0094]

[0095] Table-3 SA dataset test results

[0096]

[0097] Among the selected comparison networks, SVM is a classic machine learning method, and CDCNN only extracts spatial information. RSSAN, SSRN and DSGSF are all deep learning methods based on the fusion of spatial and spectral features. Among them, RSSAN is a serial network for extracting spatial spectral features; SSRN is a method for jointly extracting spatial spectral features based on 3-D-CNN; DSGSF is a parallel dual-branch network, each branch extracts spatial and spectral features. It can be seen from the table that the model proposed in the present invention achieves the highest classification accuracy by learning deep spatial-spectral features, wherein the feature representativeness is enhanced through the spatial mixed convolution module, the global feature enhancement module, and the attention mechanism, the dependency between channels, and retaining higher resolution, thereby improving the classification effect.

Claims

1. A spatial-spectral global perception dual-branch hyperspectral image classification method, characterized in that: The steps include: S1: constructing an overall network, including a spatial-spectral global perception dual-branch network and a classification module, wherein the spatial-spectral global perception dual-branch network includes a spatial branch module and a spectral branch module; S2: The spatial branch of the spatial-spectral global perception dual-branch network is constructed by a hierarchical feature transformation structure. The feature extraction stage captures the basic features such as low-level local textures and edges in a progressive manner to the macroscopic and semantically rich features formed as the hierarchy increases. The spatial optimization block of the global feature enhancement module separates and reconstructs the input feature map according to the information richness to suppress spatial redundancy. The channel optimization block uses a segmentation-conversion-fusion strategy to process the channel dimension feature map to enhance channel representativeness. The feature recovery and fusion stage integrates multi-level features in the process of restoring the resolution of the feature map, and integrates to generate a feature map with both details and global information. This branch generally introduces skip connections and adds channel attention to make the feature expression more complete, enhance shallow features, and avoid losing feature information during downsampling; spatial hybrid convolution modules are applied at different levels of the hierarchical feature transformation structure to improve the model's ability to understand long-distance relationships in the image, thereby improving the performance of image classification; S3: The spectral branch of the spatial-spectral global perception dual-branch network is constructed using a single-chain equal-dimensional spectral feature generator and a spectral channel focusing unit. The single-chain equal-dimensional spectral feature generator uses dense connections to perform preliminary extraction of spectral features. The single-chain equal-dimensional connection method ensures the stability of the extraction. The spectral channel focusing unit refers to the NonLocalBlock structure to accurately obtain the dependency between each channel in the spectral feature, so as to mine more refined and higher-quality spectral features and retain higher resolution in the spectral dimension. S4: Use score weighting to fuse the spatial and spectral branch features so that the network can fully combine the advantages of both, and classify through the Softmax classification module to output the results.

2. A spatial-spectral global perception dual-branch hyperspectral image classification method according to claim 1, characterized in that: The process of extracting features from the spatial branch of the spatial-spectral global perception dual-branch network in step S2 is as follows: A1: The overall network structure is a hierarchical feature transformation structure to extract spatial features. Its basic modules are the spatial mixed convolution module and 2DCNN. The structure of this module is shown in Figure 5. In the feature extraction stage, the first layer is the spatial mixed convolution module, which uses 1×1 point-by-point convolution to reduce the number of channels of the feature map, thereby reducing the amount of calculation, and then uses a dilated convolution with a dilated rate of 2 to increase the interval within the receptive field and expand the range of the receptive field. Subsequently, maximum pooling is used for downsampling. The second layer is a spatial mixed convolution module. Unlike the first layer, the dilated rate of this layer is set to 4. Subsequently, maximum pooling is used for downsampling. The third layer is a 2DCNN, which uses a 3×3 convolution kernel; A2: After the initial feature extraction, the feature map has a certain amount of high-level semantic information. At this time, the obtained feature map is input into the global feature enhancement module. The schematic diagram of this module is shown in Figure 4. First, the feature map is input into the spatial optimization block. The input feature map is divided into two parts, one with rich information and one with less information, through the separation operation. Then, the feature maps of these two parts are combined through the reconstruction operation to enhance the feature expression and suppress the redundancy in the spatial dimension. The output feature map is used as the input of the channel optimization block. The split-conversion-fusion strategy is adopted to split the feature map into two parts in the channel dimension. First, 1×1 convolution operations are performed separately. One part is extracted through efficient convolution operations, and the other part is directly used. Finally, the feature maps of the two parts are merged through an adaptive fusion strategy to reduce the redundancy in the channel dimension and improve the representativeness of the features. A3: The feature recovery and fusion module is symmetrical to the feature extraction module. The only difference is that this module uses deconvolution for upsampling. This part uses skip connections to concatenate the features before downsampling with the data after upsampling in the channel dimension. At the same time, the periphery of the upsampled features is filled with 0 to avoid mismatch in spatial dimensions. The rules for filling are as follows: H p =(H down -H up ) / 2 IN p =(In down -IN up ) / 2 The above formulas represent the padding H above and below the upsampled feature map in the spatial dimension. p 0, fill W on the left and right p 0. Among them, H down and W down They represent the size before downsampling, H up and W up Respectively represent the sizes after upsampling; A4: When using skip connections to concatenate features before downsampling and features after upsampling, channel attention is used to capture the relationship between channels, refine channel features, and reduce redundant information of channels; A5: Before the second upsampling and the feature map output by the feature recovery and fusion module, the feature is enhanced using spatial attention. At the same time, spatial attention is calculated to reduce spatial redundant information and assign different weights to different pixels. A6: After the feature recovery and fusion module, a fully connected layer is used to reduce the channel dimension to the number of categories L, and the output F of the spatial sub-network is obtained. spa ∈R H×W×L ; A7: Flatten the obtained spatial subnetwork in the spatial dimension and record the results of the sample pixels according to the index. The indexed output is F spa ∈R d×L .

3. The spatial-spectral global perception dual-branch hyperspectral image classification method according to claim 1 is characterized in that: The process of extracting features from the spectral branches of the spatial-spectral global perception dual-branch network in step S3 is as follows: B1: Randomly select a 7×7×103 3-D patch cube from the dataset as network input; B2: The input data is fed into a 3-D convolutional layer with batch normalization (BN) and Mish activation function. The convolution kernel size is 1×1×7, the number of filters is 24, the stride is (1, 1, 2) and there is no padding operation, resulting in a feature map of size (7×7×49, 24); B3: The above feature map is fed into the one-time connection part of the spectral one-time dense block. The convolution kernel size of the 3-D convolution layer in this part is 1×1×7, the number of filters is 12, the stride is (1, 1, 1), and the padding is (0, 0, 3) to obtain the intermediate feature map; B4: Connect the intermediate feature maps in the channel dimension to obtain a feature map of size (7×7×49, 60); B5: The feature map is passed through a 1×1×1 3-D convolutional layer to reduce the channel dimension from 60 to 24, and is added to the last feature map of the one-time connection part to achieve residual connection; B6: Send the residual-connected feature map to a 3-D convolution layer with a convolution kernel size of 1×1×49 to obtain a spectral feature map of (7×7×1,24); B7: The feature map of (7×7×1, 24) is used as the input of the spatial one-time dense block. The convolution kernel size of the 3-D convolution layer of this block is 3×3×1, and other hyperparameters are the same as those of the spectral one-time dense block for spatial feature extraction.

4. The spatial-spectral global perception dual-branch hyperspectral image classification method according to claim 1 is characterized in that: The process of extracting features by the multi-scale channel attention mechanism module in step S4 is as follows: C1: Set a learnable positive parameter γ in the network, and the activation function f(·) is a Sigmoid function. The two sub-networks are fused by adaptive weighting and output through the Softmax layer to obtain the final feature representation F: f(y)∈(0,1)γ∈(0,1) F=Softmax(f(γ)F spa +(1-f(γ))F spe ) C2: Compare the model's prediction results with the actual sample labels, and feed back the predicted error loss to the model, which updates the network parameters during the feedback process. After multiple training and network updates, the final network classification results are close to the sample labels. After completing the above training process, the validation data is tested, and the test results are used to determine whether the current network weights should be retained as the best result.

Images