Hyperspectral image classification method based on BoostFormer double-view perceptual feature fusion

Abstract

The invention relates to a hyperspectral image classification method based on BoostFormer double-view perceptual feature fusion, and belongs to the technical field of information processing. The method comprises the following steps: S1, hyperspectral image acquisition and data preprocessing; s2, inputting the training sample into a BoostFormer dual-view perceptual feature fusion network for learning; s3, calculating loss and updating model parameters; s4, after training is completed, the test samples are classified, and a test result is obtained. The performance of the method provided by the invention on a hyperspectral image classification task is superior to that of other hyperspectral image classification algorithms.

Description

Technical Field

[0001] This invention belongs to the field of information processing technology and relates to a hyperspectral image classification method based on BoostFormer dual-view perception feature fusion. Background Technology

[0002] Hyperspectral images are high-resolution digital images containing hundreds of continuous spectral bands. The rich spectral spatial information significantly enhances feature extraction capabilities. Therefore, hyperspectral imaging has been widely applied in various fields such as environmental monitoring, agricultural development, geological exploration, and natural resource management. Hyperspectral image classification has become a research hotspot in the field of remote sensing image interpretation. The goal is to assign a unique label to each pixel to provide high-precision image analysis.

[0003] In the course of technological development, traditional classification methods were once the mainstream solution, with representative techniques including support vector machines, principal component analysis, random forests, and linear discriminant analysis. However, these methods often rely on manual design or shallow learning to extract features. When faced with the high-dimensionality characteristics of hyperspectral data and complex ground cover scenarios, they often suffer from insufficient feature representation capabilities and unstable classification performance, making it difficult to meet the requirements of high-precision applications.

[0004] In recent years, the rise of deep learning technology has brought breakthroughs to hyperspectral image classification. Among them, Convolutional Neural Networks (CNNs) have become commonly used models in this field due to their powerful local feature exploration and spatial correlation modeling capabilities, effectively improving the recognition accuracy of complex ground feature details. However, the performance of CNNs is highly dependent on the design of the convolutional kernel size. Fixed-size convolutional kernels are difficult to flexibly capture contextual information at different scales, and have inherent limitations in long-distance feature correlation modeling. To compensate for this deficiency, the Transformer model, with its self-attention mechanism, achieves efficient capture of long-distance dependencies in hyperspectral data, further optimizing the fusion effect of global features. However, the Transformer still has shortcomings in the accuracy of extracting local fine-grained features.

[0005] Furthermore, most current research still focuses on mining the original spectral and spatial features of hyperspectral images, with relatively little exploration of frequency domain analysis. In fact, frequency domain information of spectral features can reveal subtle changes between spectral bands, while frequency domain information of spatial features can intuitively reflect the high-frequency structure and low-frequency distribution of ground objects. Therefore, how to construct a joint feature modeling framework that integrates local details and global correlations, while deeply exploring the complementary value of multi-dimensional features in the spectral, spatial, and frequency domains, has become a key direction for breaking through the bottleneck of hyperspectral image classification accuracy. Summary of the Invention

[0006] This invention addresses the unique characteristics of hyperspectral image data by providing a hyperspectral image classification method and system based on BoostFormer dual-view perceptual feature fusion. It employs Spatial Feature Awareness FA (SpaFA) and Spectral Feature Awareness FA (SpeFA) modules to enhance the model's perception of spatial and spectral features. Spatial Enhancement Transformer (SpaBF) and Spectral Enhancement Transformer (SpeBF) are used to establish nonlinear representations of spatial and spectral features. A Collaborative Enhancement Dual-View Feature Fusion (CoDF) mechanism is employed to effectively fuse spatial and spectral information, further improving model performance. This method can better extract local and global information from both spatial and spectral perspectives and exhibits superior performance in ground feature classification compared to other methods.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A hyperspectral image classification method and system based on BoostFormer dual-view perceptual feature fusion includes the following steps: S1: hyperspectral image acquisition and data preprocessing; S2: inputting training samples into the BoostFormer dual-view perceptual feature fusion network for learning; S3: calculating the loss and updating the model parameters; S4: classifying test samples after training to obtain test results.

[0008] Furthermore, in step S1, the hyperspectral image is preprocessed, specifically including: S11, data dimensionality reduction and block segmentation, firstly dividing the data into blocks centered on each pixel using a window of a certain size. Then, principal component analysis was used to reduce the spectral dimensions from... Down to To obtain the image The data blocks containing the original spectrum and the dimensionality-reduced data blocks are used for spatial feature extraction. The data blocks containing the original spectrum and the center pixels of the data blocks containing the original spectrum are... Used for spectral sensing feature extraction; S12, data partitioning, using data blocks containing the original spectrum as model input, and selecting 30 sample points for each land cover category as the training set, 10 sample points as the validation set, and the remaining sample points as the test set.

[0009] Further, in step S2, the BoostFormer-based dual-view perceptual feature fusion network extracts spatial and spectral features from spatial and spectral perspectives. The spatial perspective includes a Spatial Feature Awareness (SpaFA) module and a Spatial Enhancement Transformer (SpaBF). The spectral perspective includes a Spectral Feature Awareness (SpeFA) module and a Spectral Enhancement Transformer (SpeBF). The SpeFA module consists of a Spatial Wavelet Representation (SpaWR) module and a Spatial Information Aggregation (SpaIA) module. The SpeFA module is composed of a Spectral Wavelet Representation (SpeWR) module and a Spectral Information Aggregation (SpeIA) module. First, the dimensionality-reduced data blocks... The SpaWR module is used to extract multi-scale spatial features with nonlinear mapping capabilities; the original data blocks are then processed. After three convolutional layers, the data is input into the SpaIA module to extract information focused on spatial structure; the fused features from the SpaWR and SpaIA modules are input into the SpaBF module to enhance the network's ability to express both local and global spatial features; then, the center pixels of the data blocks are... Inputting SpeWR enables precise extraction of cross-band nonlocal related information, transforming the original data blocks... The input is processed through three convolutional layers and then fed into the SpeIA module to focus on the important information of each spectral band. The fused features of the SpeWR and SpeIA modules are then fed into the SpeBF module to enhance the nonlinear expressive power of the features. Finally, the CoDF module is used to fuse spatial and spectral features in a collaborative enhancement dual-view feature fusion process.

[0010] Furthermore, wavelet convolution WTConv is used in the SpaWR module to expand the sensing range and achieve multi-scale extraction of spatial features. WTConv uses wavelet transform to convert the input... The system is decomposed into multi-scale frequency components, and convolution operations are performed in parallel on each frequency band. Then, inverse wavelet transform is applied to convert the components to spatial features. Finally, the spatial features are passed to channel-wise convolution to adjust the number of channels, resulting in spatial wavelet features. The formula is as follows:

[0011] in, This represents channel-wise convolution. Represents wavelet transform, This represents the inverse wavelet transform. This represents a 2D convolution with a kernel of 3. It is a weight tensor. In this invention, the "Haar" wavelet basis is used, and a cascaded wavelet decomposition strategy is adopted.

[0012] Furthermore, in the SpaIA module, the input is first processed... Multi-layer convolution is performed to extract hierarchical features. Adaptive average pooling and adaptive max pooling are used to capture statistical features. The two pooled statistical features are then concatenated along the channels to obtain the pooled features. Secondly, the pooling features are transformed into a two-dimensional tensor, and the scrambled features are obtained by randomly shuffling the space. Enhance the stability of spatial location perturbations; finally, splice pooling features. and random shuffling features Convolution is then applied to achieve cross-channel fusion, resulting in fused spatial aggregation information. The formula can be expressed as:

[0013]

[0014] in, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. This represents a two-dimensional convolution with a kernel size of 3.

[0015] Furthermore, integrate spatial aggregation information and spatial wavelet features To obtain spatial perception features The input is fed into the Spatial Augmentation Transformer (SpaBF) to extract spatial augmentation features, specifically spatial awareness features. Features are obtained through layer normalization, multi-scale multi-head attention, and residual connections. These features are then further normalized and processed by convolutional gated linear units to obtain the final spatially enhanced features, expressed by the formula:

[0016] In the formula, Representation layer normalization, This indicates multi-scale, multi-head attention operations. This represents a convolution-gated linear unit operation; Furthermore, in the multi-scale multi-head attention module (MHMSA), firstly, spatially perceived features are... Linear transformation to generate query vector Key vector Value vector and projection vector Secondly, by using convolutional layers with different kernel sizes... Multi-scale feature extraction is performed, and the results are concatenated to obtain a multi-scale feature representation. Then, by pressing and The attention score is calculated by performing a proportional dot product operation, and then the attention weights are obtained through the Softmax function. and combine this weight with Multiply; finally, multiply the attention output with the projection vector. and Adding the Adama products together yields the multi-scale multi-head attention features. The formula is expressed as:

[0017]

[0018]

[0019] In the formula, This represents a convolution with a kernel size of 3. This represents a convolution with a kernel size of 5. This represents a convolution with a kernel size of 7. Represents the feature concatenation function. This represents the Softmax function. Indicates the shrinkage coefficient. Indicates attention weight; Furthermore, the Convolutionally Gated Linear Unit (CGLU) contains two parallel linear projection branches. One branch further integrates a convolutionally gated attention mechanism, which consists of linear projection, depthwise convolution, and the GELU activation function in sequence to generate dynamic gating weights. Finally, the output of this branch is element-wise multiplied with the projected features of the other branch, thereby achieving feature recalibration and filtering to obtain the final feature. The formula is expressed as:

[0020] In the formula, Representation layer normalization, Represents a linear projection function. Represents depthwise convolution. This represents the GELU activation function.

[0021] Furthermore, in the SpeWR module, wavelet multi-scale decomposition technology is used to effectively capture the long-term correlation between different frequency bands. WTConv uses wavelet transform to convert the input... The system is decomposed into multi-scale frequency components, and convolution operations are performed in parallel on each frequency band. Then, inverse wavelet transform is applied to convert the components to spatial features. Finally, these features are passed to channel-wise convolution to adjust the number of channels, resulting in spectral wavelet features. The formula is as follows:

[0022] In the formula, This represents channel-wise convolution. Represents wavelet transform, This represents the inverse wavelet transform. Indicates the kernel size as convolution, It is a weight tensor; Furthermore, in the SpeIA module, the input is first processed... Multi-layer convolution is performed to extract hierarchical features. Adaptive average pooling and adaptive max pooling are used to capture statistical features. The two pooled statistical features are then concatenated along the channels to obtain the pooled features. Secondly, the features are randomly shuffled to obtain the shuffled features. Finally, pooling features are spliced ​​along the width direction. and the characteristics after scrambling Convolution is applied to achieve feature fusion and obtain spectral aggregate information. The formula is expressed as:

[0023]

[0024] In the above formula, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. Indicates the kernel size as The convolution.

[0025] Furthermore, fusion of spectral aggregation information and spectral wavelet features Spectral sensing features were obtained. The input is fed into the SpeBF spectral enhancement converter to extract spectral enhancement features, specifically spectral sensing features. Features are obtained through layer normalization, local-global self-attention, and residual connections. These features are then further normalized through layer normalization and multilayer perceptron units to obtain the final spectral enhancement features, expressed by the formula:

[0026] In the formula, Representation layer normalization, This represents a local-to-global self-attention operation. This indicates a unit operation of a multilayer sensor.

[0027] Furthermore, Local-Global Self-Attention LGSA comprises two parts: one part extracts local information using two-dimensional convolution, and the other part is a self-attention module used to model global spectral information. Specifically, firstly, spectral sensing features are... After layer normalization, the data is reshaped into a two-dimensional feature tensor. Then, the features are processed through convolutional layers and a self-attention module to obtain spectral local and global attention features. The formula is expressed as:

[0028] in, Represents the two-dimensional reshaping of tensors. This represents the left and right fill cells in a dimension, and the fill method is to copy the edge values. Representation layer normalization, This represents the convolutional layer operation. This indicates a self-attention operation.

[0029] Furthermore, in the CoDF module, firstly, the spectral view features... Spatial view features Element-wise multiplication produces an enhanced spatial representation. At the same time, calculation The average value along the channel direction is then fused together using matrix multiplication. In this context, enhanced spectra are represented. .

[0030]

[0031]

[0032] In the above formula, This represents matrix multiplication. This represents the average value along the channel direction. Then, the initial fused feature is obtained by adding and fusing the spatial and spectral enhancement features. :

[0033] Finally, global average pooling, one-dimensional convolution, and sigmoid are applied to obtain spatial attention weights. and spectral attention weight and the initial fusion features Adding them together yields the final fusion characteristic. The formula is expressed as:

[0034]

[0035]

[0036] In the formula, Indicates global average pooling. Represents one-dimensional convolution. This represents the Sigmoid activation function.

[0037] Furthermore, in step S3, the fused features are passed to the classifier to obtain the estimated classification probability of the corresponding pixel. The classification loss is calculated by comparing the predicted probabilities with the ground truth labels. In this invention, cross-entropy loss is chosen as the selected loss function to train and update the model parameters. The formula is expressed as:

[0038] in, The output represents the cross-entropy loss. Indicates the number of categories. Indicates the number of samples. It represents the estimated classification probability for the corresponding pixel. This represents the ground truth label. After the model is trained, the test set is input into the trained network to predict the labels, and the predicted results are output.

[0039] Furthermore, in step S4, the predicted samples are input into the trained teacher model to obtain the final test results.

[0040] The beneficial effects of this invention are as follows: This invention proposes a hyperspectral image dual-view perceptual feature fusion classification method and system based on BoostFormer to extract joint spatial-spectral features. The spatial feature perception module effectively captures local details and global context, while the spatial enhancement transformer establishes local-global hierarchical associations to enhance the discriminative power of spatial features. The spectral feature perception module extracts local and global interaction information from the spectrum, and the spectral enhancement transformer further models the interaction between local details and long-term dependencies to improve the robustness of spectral features. The proposed collaborative enhancement dual-view feature fusion module obtains joint spatial-spectral features through spatial-spectral view feature extraction and adaptive weighted fusion, further enhancing the model's representational power. The classification network proposed in this invention significantly outperforms state-of-the-art hyperspectral image classification methods.

[0041] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is an overall framework diagram of the present invention; Figure 3 Detailed structural diagram of the SpaFA module for spatial feature perception; Figure 4 This is a structural diagram of the Spatial Enhancement Converter SpaBF of the present invention. Figure 5 Detailed structure of the SpeFA module for spectral feature sensing; Figure 6 This is a structural diagram of the SpeBF spectral enhancement converter of the present invention; Figure 7 Structure diagram of CoDF for joint enhancement of dual-view feature fusion; Figure 8 Visualizations of different methods on the PaviaU dataset are shown, where (a) is a true-color image, (b) is a ground truth image, (c) is SSAtt, (d) is SSSAN, (e) is CVSSN, (f) is CLOLN, (g) is SSFTT, (h) is GSC-ViT, (i) is LRDTN, and (j) is the result of this invention. Detailed Implementation

[0043] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0044] Figure 1 This is a flowchart of the method of the present invention. The present invention provides a hyperspectral image classification method based on BoostFormer dual-view perceptual feature fusion. Figure 1 As shown, in the data preprocessing stage after acquiring the hyperspectral image, spatial data is obtained through data segmentation and dimensionality reduction using principal component analysis. Data blocks containing the original spectrum and their center pixels are used as spectral data. The network diagram for hyperspectral image classification is shown below. Figure 2 As shown, it can obtain fine and representative spatial-spectral joint features. The network consists of two perspectives, spatial and spectral, and a fusion branch. First, to capture sufficient spatial information, this invention designs a spatial feature-aware SpaFA structure, which includes a spatial wavelet representation (SpaWR) and a spatial information aggregation (SpaIA) module. Spatial data is input into the SpaFA module to mine sufficient and fine spatial information. Simultaneously, to capture sufficient spectral information, this invention designs a spectral feature-aware SpeFA structure, which includes a spectral wavelet representation (SpeWR) and a spectral information aggregation (SpeIA) module. Spectral data is input into the SpeFA module to extract sufficient local-global spectral information. Then, to enhance the nonlinear representation capability of spatial and spectral information, this invention designs a spatial enhancement converter (SpaBF) and a spectral enhancement converter (SpeBF). Finally, this invention designs a collaboratively enhanced dual-view feature fusion (CoDF) module, which efficiently fuses spatial and spectral branch features and further enhances the overall feature representation capability.

[0045] Specifically, the technical solution of the present invention includes the following: 1. Hyperspectral Image Acquisition and Data Preprocessing: Hyperspectral image acquisition, acquiring hyperspectral images of the target scene containing c types of land cover. ,in and Indicates the size of the image. The spectral dimension is the primary dimension; data preprocessing involves preprocessing the hyperspectral image. In this method, the data is first divided into blocks for dimensionality reduction, using a 23×23 block as a sliding window centered on each pixel to obtain data blocks for information extraction. Then, principal component analysis is used to reduce the spectral dimension from... The value was reduced to 6, which was used as the data input for spatial information extraction. The raw spectrum of this data block and its center pixel was used as the data input for spectral information extraction. Then, 30 samples, 10 samples, and the remaining samples were randomly selected for each land cover category to divide the data into training set, validation set, and test set.

[0046] 2. Input the training samples into a BoostFormer dual-view perceptual feature fusion network for learning, such as... Figure 2 As shown, firstly, the preprocessed spatial data is input into the spatial perspective branch to extract spatial features and enhance its nonlinear expressive power. Then, the preprocessed spectral data is input into the spectral perspective branch to extract spectral features and fully enhance the local and global dependencies of the spectrum. Finally, the spatial and spectral features are input into the joint enhancement dual-perspective feature fusion module to obtain joint spatial-spectral features.

[0047] 3. The spatial feature perception module in the spatial perspective branch can make full use of spatial context information, such as... Figure 3 As shown. Block data is input into convolutional blocks to extract hierarchical features. Adaptive average pooling and adaptive max pooling are combined to capture statistical features, which are then spatially randomized. The pooling features from the previous two steps are then concatenated. and random shuffling features And convolution is applied to achieve cross-channel feature fusion. The formula can be expressed as:

[0048]

[0049] in, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. This represents a two-dimensional convolution with a kernel size of 3; The dimensionality-reduced block data is input into a 3×3 level Haar wavelet convolution to extract multi-scale spatial features, and these features are then passed to a channel-wise convolution to adjust the number of channels. The formula is expressed as:

[0050] in, It is a weight tensor. Represents wavelet transform, This represents the inverse wavelet transform. This represents a 2D convolution with a kernel of 3. This represents pointwise convolution.

[0051] 4. Spatial augmentation transformers in the spatial branch can further enhance the network's ability to express local and global spatial features, such as... Figure 4 As shown. Integrating spatial aggregation information. and spatial wavelet features To obtain spatial perception features The input is fed into the Spatial Augmentation Transformer (SpaBF) to extract spatial augmentation features, specifically spatial awareness features. Features are obtained through layer normalization, multi-scale multi-head attention, and residual connections. These features are then further normalized and processed by convolutional gated linear units to obtain the final spatially enhanced features, expressed by the formula:

[0052] In the formula, Representation layer normalization, This indicates multi-scale, multi-head attention operations. This represents a convolution-gated linear unit operation; In the Multi-Scale Multi-Head Attention Module (MHMSA), the features after layer normalization are first subjected to a linear transformation to generate the query vector. Key vector Value vector and projection vector Secondly, by using convolutional layers with different kernel sizes... Multi-scale feature extraction is performed, and the results are concatenated to obtain a multi-scale feature representation. Then, by pressing and The attention score is calculated by performing a proportional dot product operation, and then the attention weights are obtained through the Softmax function. and combine this weight with Multiply; finally, multiply the attention output with the projection vector. and Adding the Adama products together yields the multi-scale multi-head attention features. The formula is expressed as:

[0053]

[0054]

[0055] In the formula, This represents a convolution with a kernel size of 3. This represents a convolution with a kernel size of 5. This represents a convolution with a kernel size of 7. Represents the feature concatenation function. This represents the Softmax function. Indicates the coefficient of shrinkage; The Convolutionally Gated Linear Unit (CGLU) comprises two parallel linear projection branches. One branch further integrates a convolutionally gated attention mechanism, which consists of linear projection, depthwise convolution, and the GELU activation function in sequence to generate dynamic gating weights. Finally, the output of this branch is element-wise multiplied with the projected features of the other branch, thereby recalibrating and filtering the features to obtain the final feature set. The formula is expressed as:

[0056] In the formula, Representation layer normalization, Represents a linear projection function. Represents depthwise convolution. This represents the GELU activation function.

[0057] 5. The spectral feature perception module in the spectral perspective branch can fully identify spectral features, such as... Figure 5 As shown. Block data is input into convolutional blocks to extract hierarchical features. Adaptive average pooling and adaptive max pooling are combined to capture statistical features, which are then randomly shuffled by channel. The pooling features from the previous two steps are then concatenated. and random shuffling features And convolution is applied to achieve cross-channel feature fusion. The formula can be expressed as:

[0058]

[0059] in, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. Indicates the kernel size as The convolution.

[0060] The original spectral data of the center pixel is input into a 4-level Haar wavelet convolution with a kernel size of 1×5 to capture the local and global correlations between different bands, and its features are then passed to a channel-wise convolution to adjust the number of channels. The formula is as follows:

[0061] In the formula, This represents channel-wise convolution. Represents wavelet transform, This represents the inverse wavelet transform. Indicates the kernel size as convolution, It is a weight tensor; 6. Spectral enhancement converters in the spectral branch can fully utilize both local and non-local spectral information to enhance the network's expressive power, such as... Figure 6 As shown. Fusion spectral aggregation information. and spectral wavelet features Spectral sensing features were obtained. The input is fed into the SpeBF spectral enhancement converter to extract spectral enhancement features, specifically spectral sensing features. Features are obtained through layer normalization, local-global self-attention, and residual connections. These features are then further normalized through layer normalization and multilayer perceptron units to obtain the final spectral enhancement features, expressed by the formula:

[0062] In the formula, Representation layer normalization, This represents a local-to-global self-attention operation. This represents the unit operation of a multilayer perceptron; Local-Global Self-Attention LGSA comprises two parts: one part extracts local information using 2D convolution, and the other part is a self-attention module used to model global spectral information. Specifically, firstly, spectral sensing features are... After layer normalization, the data is reshaped into a two-dimensional feature tensor. Then, the features are processed through convolutional layers and a self-attention module to obtain spectral local and global attention features. The formula is expressed as:

[0063] in, Represents the two-dimensional reshaping of tensors. This represents the left and right fill cells in a dimension, and the fill method is to copy the edge values. Representation layer normalization, This represents the convolutional layer operation. This indicates a self-attention operation.

[0064] 7. Based on the above extraction of spatial and spectral features, the jointly enhanced dual-view feature fusion module can effectively fuse spatial and spectral features, such as... Figure 8 As shown. First, spectral view features. Spatial view features Element-wise multiplication produces an enhanced spatial representation. At the same time, calculation The average value along the channel direction is then fused together using matrix multiplication. In this context, enhanced spectra are represented. .

[0065]

[0066]

[0067] in, Represents matrix multiplication. This indicates that the average value is taken along the channel direction. Then, the initial fused features are obtained by adding and fusing the spatial and spectral enhancement features:

[0068] Finally, global average pooling, one-dimensional convolution, and sigmoid are applied to obtain spatial attention weights. and spectral attention weight and the initial fusion features Adding them together yields the final fusion characteristic. The formula is expressed as:

[0069]

[0070]

[0071] In the formula, Indicates global average pooling. Represents one-dimensional convolution. This represents the Sigmoid activation function.

[0072] 8. Calculate the loss and update the model parameters: The fused features are passed to the classifier to obtain the estimated classification probability of the corresponding pixel. The classification loss is calculated by comparing the predicted probabilities with the ground truth labels. In this invention, cross-entropy loss is chosen as the selected loss function to train and update the model parameters. The formula is expressed as: in, The output represents the cross-entropy loss. Indicates the number of categories. Indicates the number of samples. It represents the estimated classification probability for the corresponding pixel. This indicates the actual ground label.

[0073] 9. After the model training is complete, input the test set into the trained network to predict the labels and output the prediction results.

[0074] The classification performance of this invention can be further illustrated through comparative experiments. On the Pavia University dataset, the method of this invention was compared with other classification methods SSATT, SSSAN, CVSSN, CLOLN, SSFTT, GSC-ViT, and LRDTN. Overall Accuracy (OA), Average Accuracy (AA), Kappa coefficient (KC), and classification accuracy for each class were calculated, with higher values ​​for all metrics being better. Table 1 shows the values ​​of each metric for the classification results of different methods: Table 1. Values ​​of various indicators for classification results from different methods:

[0075] It can be seen that the method of this invention achieves the best accuracy on this dataset. For example... Figure 8 The visualization results of the above methods are presented. It can be seen that the types of land features are well classified, with few misclassified areas and clear boundaries.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A hyperspectral image classification method based on BoostFormer dual-view perceptual feature fusion, characterized by: The method includes the following steps: S1: Hyperspectral image acquisition and data preprocessing; S2: Input the training samples into the BoostFormer dual-view perceptual feature fusion network for learning; S3: Calculate the loss and update the model parameters; S4: After training is complete, classify the test samples to obtain the test results; In step S1, a hyperspectral image of the target scene containing c types of land cover is acquired. ,in and Indicates the size of the image. The spectral dimension is used for preprocessing and partitioning the acquired hyperspectral images. First, the data is divided into blocks and dimensionality reduced. A 23×23 block is used as a sliding window centered on each pixel to obtain data blocks for information extraction. Then, principal component analysis is used to reduce the spectral dimension from... The value was reduced to 6, which was used as the data input for spatial information extraction. The original spectrum of the data block and the center pixel was used as the data input for spectral information extraction. Finally, 30 samples were randomly selected for each land cover category as the training set, 10 samples as the validation set, and the remaining samples as the test set. In step S2, the BoostFormer-based dual-view perceptual feature fusion network extracts spatial and spectral features from spatial and spectral perspectives. The spatial perspective includes a Spatial Feature Awareness (SpaFA) module and a Spatial Enhancement Transformer (SpaBF). The spectral perspective includes a Spectral Feature Awareness (SpeFA) module and a Spectral Enhancement Transformer (SpeBF). The SpaFA module consists of Spatial Wavelet Representation (SpaWR) and Spatial Information Aggregation (SpaIA). The SpeFA module consists of Spectral Wavelet Representation (SpeWR) and Spectral Information Aggregation (SpeIA). First, the dimensionality-reduced data blocks... The input SpaWR module is used to extract multi-scale spatial features with nonlinear mapping capabilities from the original data blocks. After three convolutional layers, the data is input into the SpaIA module to extract information focused on spatial structure. The fused features from the SpaWR and SpaIA modules are then input into the SpaBF module to enhance the network's ability to express both local and global spatial features. Finally, the center pixels of the data blocks are... Inputting SpeWR enables precise extraction of cross-band nonlocal related information, transforming the original data blocks... After three convolutional layers, the input is fed into the SpeIA module to focus on the important information of each spectral band. The fused features of the SpeWR and SpeIA modules are fed into the SpeBF module to enhance the nonlinear expressive power of the features. Finally, the Co-Enhanced Dual-View Feature Fusion Module (CoDF) is used to fuse spatial and spectral features. In step S3, the fused features are passed to the classifier to obtain the estimated classification probability of the corresponding pixel. The classification loss is calculated by comparing the predicted probability with the ground truth label. In this invention, the cross-entropy loss function is selected to train and update the model parameters, and the formula is expressed as: ,in, The output represents the cross-entropy loss. Indicates the number of categories. Indicates the number of samples. It represents the estimated classification probability for the corresponding pixel. This represents the ground truth labels. After the model training is completed, the trained model is obtained. In step S4, based on the trained model, the test set is input into the trained network to predict labels and the prediction results are output.

2. The hyperspectral image classification method based on BoostFormer dual-view perceptual feature fusion according to claim 1, characterized in that: In step S2, the SpaWR module uses wavelet convolution to expand the sensing range and extract multi-scale spatial wavelet features. Wavelet convolution transforms the input... The system is decomposed into multi-scale frequency components, and convolution operations are performed on each frequency band. Then, inverse wavelet transform is applied to convert the system to the spatial domain. Finally, the spatial features are passed to channel-wise convolution to adjust the number of channels, resulting in spatial wavelet features. The formula is as follows: ,in, This represents pointwise convolution. Represents wavelet transform, This represents the inverse wavelet transform. This represents a two-dimensional convolution function with a kernel of 3. It is a weight tensor. In this invention, the "Haar" wavelet basis is used, and a cascaded wavelet decomposition strategy is employed. Furthermore, in the SpaIA module, the input is first processed... Multi-layer convolution is performed to extract hierarchical features. Adaptive average pooling and adaptive max pooling are used to capture statistical features. The two pooled statistical features are then concatenated along the channels to obtain the pooled features. Secondly, the pooling features are transformed into a two-dimensional tensor, and then subjected to a spatial random shuffling operation to obtain the shuffled features. Enhance the stability of spatial location perturbations; finally, splice pooling features. and random shuffling features Convolution is then applied to achieve cross-channel fusion, resulting in fused spatial aggregation information. The formula can be expressed as: , In the above formula, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. This represents a two-dimensional convolution with a kernel size of 3; Furthermore, integrate spatial aggregation information and spatial wavelet features To obtain spatial perception features The input is fed into the Spatial Augmentation Transformer (SpaBF) to extract spatial augmentation features, specifically spatial awareness features. Features are obtained through layer normalization, multi-scale multi-head attention, and residual connections. These features are then further normalized and processed by convolutional gated linear units to obtain the final spatially enhanced features. The formula is expressed as: , in the formula, Representation layer normalization, This indicates multi-scale, multi-head attention operations. This represents a convolution-gated linear unit operation; Furthermore, in the multi-scale multi-head attention module (MHMSA), the features after layer normalization are first subjected to a linear transformation to generate the query vector. Key vector Value vector and projection vector Secondly, by using convolutional layers with different kernel sizes... Multi-scale feature extraction is performed, and the results are concatenated to obtain a multi-scale feature representation. ;Then, and The attention score is calculated by performing a proportional dot product operation, and then the attention weights are obtained through the Softmax function. and combine this weight with Multiply; finally, multiply the attention output with the projection vector. and Adding the Adama products together yields the multi-scale multi-head attention features. The formula is expressed as: , , In the above formula, This represents a convolution with a kernel size of 3. This represents a convolution with a kernel size of 5. This represents a convolution with a kernel size of 7. Represents the feature concatenation function. This represents the Softmax function. Indicates the shrinkage coefficient. Indicates attention weight; Furthermore, the Convolutionally Gated Linear Unit (CGLU) contains two parallel linear projection branches. One branch further integrates a convolutionally gated attention mechanism, which consists of linear projection, depthwise convolution (DWConv), and the GELU activation function in sequence to generate dynamic gating weights. Finally, the output of this branch is element-wise multiplied with the projected features of the other branch, thereby achieving feature recalibration and filtering to obtain the final feature. The formula is expressed as: , in the formula, Representation layer normalization, Represents a linear projection function. Represents depthwise convolution. This represents the GELU activation function.

3. The hyperspectral image classification method based on BoostFormer dual-view perceptual feature fusion according to claim 1, characterized in that: In step S2, within the SpeWR module, wavelet multi-scale decomposition technology is used to effectively capture the long-term correlation between different frequency bands. Specifically, wavelet transform is used to transform the input... The system is decomposed into multi-scale frequency components, and convolution operations are performed on each frequency band. Then, inverse wavelet transform is applied to convert the system to the spatial domain. Finally, the spatial domain features are passed to channel-wise convolution to adjust the number of channels, resulting in spectral wavelet features. The formula is as follows: , in the formula, This represents channel-wise convolution. Represents wavelet transform, This represents the inverse wavelet transform. Indicates the kernel size as convolution, It is a weight tensor; Furthermore, in the SpeIA module, the input is first processed... Multi-layer convolution is performed to extract hierarchical features. Adaptive average pooling and adaptive max pooling are used to capture statistical features. The two pooled statistical features are then concatenated along the channels to obtain the pooled features. Secondly, the features are randomly shuffled to obtain the shuffled features. Finally, pooling features are spliced ​​along the width direction. and the characteristics after scrambling Convolution is applied to achieve feature fusion and obtain spectral aggregate information. The formula is expressed as: , ,in, This indicates a multi-layer convolution operation. This indicates adaptive max pooling. Indicates adaptive average pooling. This indicates a feature concatenation operation. Indicates the kernel size as Convolution; Furthermore, fusion of spectral aggregation information and spectral wavelet features Spectral sensing features were obtained. The input is fed into the SpeBF spectral enhancement converter to extract spectral enhancement features, specifically spectral sensing features. Features are obtained through layer normalization, local-global self-attention, and residual connections. These features are then further normalized through layer normalization and multilayer perceptron units to obtain the final spectral enhancement features, expressed by the formula: , in the formula, Representation layer normalization, This represents a local-to-global self-attention operation. This indicates the operation of a multilayer perceptron unit; Furthermore, Local-Global Self-Attention LGSA comprises two parts: one part extracts local information using two-dimensional convolution, and the other part is a self-attention module used to model global spectral information. Specifically, firstly, spectral sensing features are... After layer normalization, the data is reshaped into a two-dimensional feature tensor. Then, the features are processed through convolutional layers and a self-attention module to obtain spectral local and global attention features. The formula is expressed as: ,in, Represents the two-dimensional reshaping of tensors. This represents the left and right fill cells in a dimension, and the fill method is to copy the edge values. Representation layer normalization, This represents the convolutional layer operation. This indicates a self-attention operation.

4. The hyperspectral image classification method based on BoostFormer dual-view perceptual feature fusion according to claim 1, characterized in that: In step S2, the CoDF module first identifies the spectral view features. Spatial view features Element-wise multiplication produces an enhanced spatial representation. Simultaneously, calculation The average value along the channel direction is then fused together using matrix multiplication. In this context, enhanced spectra are represented. The formula is expressed as: , ,in, Represents matrix multiplication. This indicates that the average value is taken along the channel direction; then, the initial fused feature is obtained by adding and fusing the spatial and spectral enhancement features. : Finally, global average pooling, one-dimensional convolution, and sigmoid are applied to obtain spatial attention weights. and spectral attention weight and the initial fusion features Adding them together yields the final fusion characteristic. The formula is expressed as: , , , in the formula, Indicates global average pooling. Represents one-dimensional convolution. This represents the Sigmoid activation function.

Images