Lightweight hyperspectral image classification method based on multi-scale convolution

Abstract

The application discloses a lightweight hyperspectral image classification method based on multi-scale convolution, comprising the following steps: dividing hyperspectral image data into a training set and a test set according to a preset ratio; constructing a lightweight hyperspectral image classification model based on convolution kernel decomposition and double perception channel attention cooperation; training the constructed lightweight hyperspectral image classification model using the training set to obtain a trained lightweight hyperspectral image classification model; inputting hyperspectral image data in the test set into the trained lightweight hyperspectral image classification model, inferring to obtain a predicted category, and evaluating the obtained predicted category through an evaluation index. The application fuses a multi-scale feature extraction module and a double perception channel attention module, significantly reduces the model complexity while ensuring the classification accuracy, and is suitable for high-precision hyperspectral image classification tasks in resource-limited scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a lightweight hyperspectral image classification method based on multi-scale convolution. Background Technology

[0002] With the continuous advancement of observation technology, the acquisition capabilities of hyperspectral images (HSI) have significantly improved. Compared to traditional RGB images, HSI covers a wider spectral range, including visible, near-infrared, mid-infrared, and even far-infrared regions, typically possessing tens to hundreds of consecutive bands. Its rich spectral information makes fine-grained differentiation of ground features possible, thus HSI is widely used in fields such as agricultural conservation, geological mapping, resource exploration, and environmental monitoring. As the foundation for these applications, HSI classification aims to assign a semantic category label to each pixel in the image.

[0003] High-quality HSI classification relies on effective feature representation. Traditional methods often depend on handcrafted features. In recent years, deep learning techniques, especially convolutional neural networks (CNNs), have been widely used in hyperspectral image classification tasks due to their powerful feature learning capabilities. Although existing methods are continuously improving in classification accuracy, their model structures are becoming increasingly complex, leading to high computational and storage costs. In practical scenarios such as real-time impurity detection on conveyor belts or UAV forest monitoring, efficient model deployment and low resource consumption are crucial. Therefore, achieving high-precision hyperspectral image classification with limited resources remains a key research challenge. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a lightweight hyperspectral image classification method based on multi-scale convolution that features a simple algorithm and high recognition accuracy.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is: a lightweight hyperspectral image classification method based on multi-scale convolution, comprising the following steps:

[0006] Step 1, Data Preprocessing: Divide the hyperspectral image data into training and test sets according to a preset ratio, and perform preprocessing.

[0007] Step 2, Model Building: Constructing a lightweight hyperspectral image classification model ELS based on convolutional kernel decomposition and dual-sensory channel attention collaboration. 2 C-Net, ELS 2 C-Net consists of a multi-scale feature extraction module CMSF, a dual-sensor attention module DAE, and a classification head connected in sequence.

[0008] Step 3, Model Training: The lightweight hyperspectral image classification model ELS, constructed using the training set, is trained.2 The C-Net model is trained by calculating the cross-entropy loss between the predicted results and the actual labels during training. The model parameters are optimized iteratively until the model converges. The optimized model parameters are recorded and stored, resulting in a trained lightweight hyperspectral image classification model, ELS. 2 C-Net;

[0009] Step 4, Model Inference: Input the hyperspectral image data from the test set into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, inference yields predicted categories, and these predicted categories are evaluated using evaluation metrics.

[0010] The specific process of step 1 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0011] Step 11, Dataset Partitioning: Divide the original hyperspectral dataset... The data is randomly divided into training sets according to a set ratio. and test set The training set accounts for 10% of the total. The proportion of the test set is , It is a constant. The stratified sampling process ensures that the distribution of samples in each category is consistent.

[0012] Step 12, Spatial Context Window Sampling: with target pixels Centered on, is the x-coordinate of the target pixel. Using the ordinate of the target pixel, the processed dataset is cropped pixel by pixel, with a window size of [value missing]. , This represents the size of the spatial pixel block obtained by cropping, resulting in a sample set of spatial pixel blocks centered on the pixel. Spatial pixel block sample set The spatial pixel block in , , The size is , This represents the number of spectral channels. Represented as:

[0013] ;

[0014] in, This represents the clipping operation function. Indicates target pixels The spatial pixel block centered on;

[0015] Step 13, Normalization and Data Augmentation: First, normalize the spatial pixel blocks. Perform normalization processing, The value is standardized to the [0,1] interval, and the processing method is as follows:

[0016] ;

[0017] in, For normalized spatial pixel blocks, The minimum value in the spatial pixel block. It is the maximum value in the spatial pixel block;

[0018] Then, data augmentation operations are performed; random rotation and flipping operations are used to... Data augmentation is performed, resulting in a new set of spatial pixel block samples. New spatial pixel block sample set The new spatial pixel block sample is , , The size is , This represents the number of spectral channels. Represented as:

[0019] ;

[0020] in, Indicates a data augmentation function;

[0021] Step 14, Construct the training sample pair set: The Middle Match each new spatial pixel block sample with its corresponding real class label to establish a training sample pair. , for The Middle A new spatial pixel block sample for The corresponding real category label, , The total number of categories is used to obtain the final training sample pair set. Training sample set Represented as:

[0022] ;

[0023] in, for The total number of samples in the sample.

[0024] The specific process of step 2 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0025] Step 21: Construct a multi-scale feature extraction module (CMSF) based on convolution kernel decomposition. CMSF adopts a lightweight architecture of 1×1 pointwise convolution, lightweight extraction of multi-scale spectral-spatial features, and channel shuffling fusion. The 1×1 pointwise convolution is used to reduce the dimensionality of the hyperspectral image. Lightweight extraction of multi-scale spectral-spatial features is achieved through convolution kernel decomposition and a dual-branch architecture, which includes a spatial branch and a spectral branch. A channel shuffling operation is designed to allow features from different scale branches to be mixed across groups and redistributed.

[0026] Step 22: Construct a dual-sensory channel attention module (DAE). The dual-sensory channel attention module (DAE) works in collaboration between the channel-sensory branch and the spatial-sensory branch. In the DAE, the original feature map is first modeled using one-dimensional convolution and global average pooling, and then spatial information is modeled using two-dimensional depthwise separable convolution and mean square. Next, the output weights of the channel-sensory branch and the spatial-sensory branch are fused by element-wise multiplication and then normalized using the Sigmoid function to form a unified attention weight vector containing dual perception. Finally, the attention weight vector is multiplied with the original feature map to output the enhanced feature map.

[0027] Step 23: Construct a classification head consisting of a global pooling layer and a fully connected layer to interpret the enhanced feature map, predict the category of the original pixel, and obtain a classification prediction map.

[0028] The specific process of step 21 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0029] Step 211, dimensionality reduction of the input channel;

[0030] First, use 1×1 pointwise convolution to... Spectral dimensions from Reduce to :

[0031] ;

[0032] in, This represents the number of spectral channels. This represents a 1×1 pointwise convolution operation. The pointwise convolution weight matrix is... The feature map after dimensionality reduction. , This represents the number of spectral channels after dimensionality reduction. For the real number field;

[0033] Step 212, spatial feature extraction;

[0034] The reduced feature map Input spatial branches, perform multi-scale spatial feature extraction, and obtain spatial branch feature maps. , , The number of spectral channels after spatial feature extraction. = ;

[0035] Step 213, spectral feature extraction;

[0036] The reduced feature map Input the spectral branch, perform spectral feature extraction, and obtain the spectral branch feature map. , ;

[0037] Step 214, the multi-scale feature extraction module CMSF outputs;

[0038] Spatial branch feature map With spectral branching feature map Stitching along the channel dimension:

[0039] ;

[0040] in, This represents the channel splicing function. This is the final output of the multi-scale feature extraction module CMSF. .

[0041] The specific process of step 212 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0042] Step 2121: Spatial branching decomposes CKD+ using convolutional kernels and designs CKD conv-n using multi-scale fusion. CKD conv-n represents the convolutional kernel decomposition and dual-branch architecture, where n represents the convolutional kernel size. First, the dimensionality-reduced feature map is... Feed into CKD conv-n: Set two parallel sub-branches, with n set to 3 and 5 respectively, to obtain CKD conv-3 convolution sub-branch and CKD conv-5 convolution sub-branch, and replace the standard two-dimensional convolution with a decomposition method of horizontal one-dimensional convolution + vertical one-dimensional convolution;

[0043] The CKD conv-3 convolutional sub-branch is used for small-scale feature extraction. First, a horizontal one-dimensional convolution is performed, using a 1×3 convolution kernel that slides horizontally to calculate the weighted sum of the current position and its horizontal left and right adjacent pixels:

[0044] ;

[0045] in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-3 convolution sub-branch;

[0046] Then, a vertical one-dimensional convolution is performed, using a 3×1 convolution kernel that slides along the vertical direction to calculate the weighted sum of the current position and its upper and lower adjacent pixels:

[0047] ;

[0048] in, The first vertical one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the output of the CKD conv-3 convolutional sub-branch;

[0049] The CKD conv-5 convolutional sub-branch is used for large-scale feature extraction. It first performs a horizontal one-dimensional convolution, using a 1×5 convolution kernel that slides horizontally.

[0050] ;

[0051] in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-5 convolution sub-branch;

[0052] Then perform a vertical one-dimensional convolution, using a 5×1 convolution kernel that slides along the vertical direction:

[0053] ;

[0054] in, The first vertical one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the output of the CKD conv-5 convolutional sub-branch;

[0055] Step 2122: Perform feature splicing and channel shuffling;

[0056] First, and The layers are stitched together along the channel dimension, and then the channel separation between scales is broken through channel shuffling:

[0057] ;

[0058] in, This represents the channel splicing function. It is a channel shuffling function. This is a spatial branch feature map.

[0059] The specific operation of step 213 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0060] The spectral branch focuses on extracting discriminative features in the spectral dimension, employing a lightweight design using one-dimensional depthwise separable convolution; firstly, the dimensionality-reduced feature map... Each spectral channel is individually applied a one-dimensional depthwise separable convolution to capture only the spectral dependence within a single channel, as shown in the formula:

[0061] ;

[0062] in, The kernel size is [size]. For depthwise convolution weights, for The output features obtained by applying one-dimensional depthwise separable convolution, Indicates the kernel size as Depth-separable convolution;

[0063] Then, a one-dimensional pointwise convolution is performed to adjust the number of channels, thus aligning it with the number of channels output by the spatial branch. The formula is as follows:

[0064] ;

[0065] in, For one-dimensional pointwise convolution weights, This is a spectral branching characteristic map.

[0066] The specific process of step 22 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0067] Step 221, DAE input definition;

[0068] The input feature map of DAE is the feature map output by the multi-scale feature extraction module CMSF. The input feature map of DAE is denoted as... ,Right now , This is the final output of the multi-scale feature extraction module CMSF;

[0069] Step 222, calculate the weights of the dual-sensor branches;

[0070] Calculate the channel-aware branch attention weights: First, process the input feature map of the DAE. Global average pooling is performed on the spatial dimension to compress it into a one-dimensional channel feature vector:

[0071] ;

[0072] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. , The total number of channels. This is the global channel vector. This indicates the size of the resulting pixel block;

[0073] Then to Applying one-dimensional convolution with a kernel size of 3:

[0074] ;

[0075] in, This represents a one-dimensional convolution operation where the convolution kernel slides only along the channel dimension. It is a one-dimensional convolution weight matrix. To The output after applying one-dimensional convolution;

[0076] Calculate the attention weights for the spatial awareness branch: First, process the input feature map of the DAE. Extracting local spatial features using two-dimensional depthwise separable convolution:

[0077] ;

[0078] in, For depthwise convolution weights, This indicates a depthwise separable convolution with a kernel size of 3×3. It is a local feature in space;

[0079] Then perform a spatial dimensional averaging operation:

[0080] ;

[0081] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. For spatial perception vectors;

[0082] Step 223, Attention Fusion and Feature Calibration;

[0083] First, attention fusion is performed, and Element-by-element multiplication:

[0084] ;

[0085] in, This represents element-wise multiplication. for and The result of element-wise multiplication;

[0086] Then a normalization operation is performed, using the Sigmoid function to... Normalize to the [0,1] interval:

[0087] ;

[0088] in, It is the Sigmoid function. for The result after normalization; Represented by natural constant An exponential function with base 0;

[0089] Finally, feature calibration is performed. Input feature map of DAE Channel-by-channel multiplication:

[0090] ;

[0091] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. for and The result after multiplying each channel.

[0092] The specific process of step 23 in the above-mentioned lightweight hyperspectral image classification method based on multi-scale convolution is as follows:

[0093] Step 231, global average pooling;

[0094] right Global average pooling is performed to compress two-dimensional spatial features into one-dimensional feature vectors, thus eliminating the influence of spatial dimension on classification.

[0095] ;

[0096] in, This indicates a global average pooling operation. for The feature vector obtained after global average pooling;

[0097] Step 232, interpret the fully connected layer category;

[0098] Through the fully connected layer Mapped to the land cover category space, the category probability distribution is obtained through linear transformation and Softmax normalization, as shown in the formula:

[0099] ;

[0100] in, This is the weight matrix of the fully connected layer. , The total number of channels. This represents the total number of land cover categories in the hyperspectral data. For the real number field, For the bias vector of the fully connected layer, , For normalization function, This indicates the probability that the current pixel belongs to a land cover category;

[0101] Step 233: Generate the classification prediction map;

[0102] For each pixel in the test set, take The category corresponding to the maximum value in the image is used as the predicted label for that pixel. After traversing all pixels, all predicted labels are stitched together according to the spatial location of the original hyperspectral image to obtain a classification prediction map with the same size as the input image.

[0103] In the aforementioned lightweight hyperspectral image classification method based on multi-scale convolution, step 3 involves the following steps during model training: First, the training set data is input into the constructed lightweight hyperspectral image classification model for forward propagation calculation to obtain the predicted category probability distribution. Then, the cross-entropy loss function is used to calculate the loss between the predicted result and the true label. The loss function Loss is:

[0104] ;

[0105] in, The number of training samples. For sample index, For category indexing, The total number of categories, Indicates the first The sample belongs to the first The true label value of the category, The model predicts the first... The sample belongs to the first The probability value of the category, It is a logarithmic function with base 10;

[0106] The Adam optimizer is used for parameter optimization, with the initial learning rate set to a constant. Cosine annealing is then used to adjust the learning rate.

[0107] ;

[0108] in, Based on the learning rate, The adjusted learning rate, For the current training round, Pi This is the maximum number of training rounds;

[0109] After all training rounds are completed, save the globally optimal model parameters.

[0110] In the aforementioned lightweight hyperspectral image classification method based on multi-scale convolution, the specific process of reasoning to obtain the predicted category in step 4 is as follows:

[0111] The hyperspectral image data to be predicted in the test set are input batch by batch into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, two multi-scale feature extraction modules (CMSF) and two dual-sensor efficient channel attention modules (DAE) are cascaded to form a feature extraction process of CMSF→CMSF→DAE→DAE, generating a calibrated feature map. The calibrated feature map is then subjected to global average pooling to obtain a global feature vector, which is then passed through a fully connected layer to generate a class probability distribution. Finally, the class index with the largest class probability distribution is used as the predicted class of the pixel to generate a classification prediction map.

[0112] The beneficial effects of this invention are as follows:

[0113] 1. The lightweight hyperspectral image classification model ELS of the present invention 2 C-Net has a simple and efficient structure. By integrating the multi-scale feature extraction module CMSF and the dual-sensory channel attention module DAE, it significantly reduces model complexity while ensuring classification accuracy, making it suitable for high-precision hyperspectral image classification tasks in resource-constrained scenarios.

[0114] 2. The multi-scale feature extraction module CMSF of the present invention decomposes the standard convolution operation into a series of efficient convolution processes by decomposing the convolution kernel, thereby realizing lightweight extraction of multi-scale spectral-spatial features. CMSF maintains the ability to perceive complex spectral-spatial features while significantly reducing parameters and computational complexity.

[0115] 3. This invention sets up a dual-sensory channel attention module (DAE), which overcomes the limitation of traditional channel attention relying solely on global information. By fusing global channel dependence with local spatial context, it achieves adaptive calibration of feature responses with lower computational cost, thereby improving the model's ability to learn discriminative features. Attached Figure Description

[0116] Figure 1 This is the overall flowchart of the present invention.

[0117] Figure 2 The following are visualization results of the model comparison experiment of the present invention, wherein (a) is the truth graph, (b) is the inference result graph of Ghostnet, (c) is the inference result graph of Res-LS2CM, (d) is the inference result graph of LMSS-NAS, (e) is the inference result graph of Lite-HCNet, (f) is the inference result graph of LPCN, (g) is the inference result graph of ARC, (h) is the inference result graph of CSM-DSPCss-Ghost, (i) is the inference result graph of LMAFN, and (j) is the inference result graph of the present invention. Detailed Implementation

[0118] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0119] like Figure 1 As shown, the lightweight hyperspectral image classification method based on multi-scale convolution includes the following steps:

[0120] Step 1, Data Preprocessing: Divide the hyperspectral image data into training set and test set according to a preset ratio, and perform preprocessing.

[0121] This invention utilizes the Pavia University dataset, which was collected by the ROSIS-3 imaging spectrometer sensor over Pavia, Italy. The dataset contains 610×340 pixels and 103 spectral bands. The images are divided into 9 categories: asphalt, grass, gravel, trees, metal plates, bare soil, asphalt, bricks, and shadows, totaling 42,776 labeled samples.

[0122] The specific process of step 1 is as follows:

[0123] Step 11, Dataset Partitioning: Divide the original hyperspectral dataset... The data were randomly divided into training sets at a ratio of 1:9. and test set The training set accounts for 10% of the total. The proportion of the test set is , It is a constant. The stratified sampling process was used to ensure that the distribution of samples in each category was consistent.

[0124] Step 12, Spatial Context Window Sampling: with target pixels Centered on, is the x-coordinate of the target pixel. Using the ordinate of the target pixel, the processed dataset is cropped pixel by pixel, with a window size of [value missing]. , This represents the size of the spatial pixel block obtained by cropping, resulting in a sample set of spatial pixel blocks centered on the pixel. Spatial pixel block sample set The spatial pixel block in , , The size is , This represents the number of spectral channels. Represented as:

[0125] ;

[0126] in, This represents the clipping operation function. Indicates target pixels The spatial pixel block centered on the center.

[0127] Step 13, Normalization and Data Augmentation: First, normalize the spatial pixel blocks. Perform normalization processing, The value is standardized to the [0,1] interval, and the processing method is as follows:

[0128] ;

[0129] in, For normalized spatial pixel blocks, The minimum value in the spatial pixel block. It is the maximum value in the spatial pixel block;

[0130] Then, data augmentation operations are performed; random rotation and flipping operations are used to... Data augmentation is performed, resulting in a new set of spatial pixel block samples. New spatial pixel block sample set The new spatial pixel block sample is , , The size is , This represents the number of spectral channels. Represented as:

[0131] ;

[0132] in, This represents a data augmentation function.

[0133] Step 14, Construct the training sample pair set: The Middle Match each new spatial pixel block sample with its corresponding real class label to establish a training sample pair. , for The Middle A new spatial pixel block sample for The corresponding real category label, , The total number of categories is used to obtain the final training sample pair set. Training sample set Represented as:

[0134] ;

[0135] in, for The total number of samples in the sample.

[0136] Step 2, Model Building: Constructing a lightweight hyperspectral image classification model ELS based on convolutional kernel decomposition and dual-sensory channel attention collaboration. 2 C-Net, ELS 2 C-Net consists of a multi-scale feature extraction module (CMSF), a dual-sensor attention module (DAE), and a classification head, which are connected in sequence.

[0137] The specific process of step 2 is as follows:

[0138] Step 21: Construct a multi-scale feature extraction module CMSF based on convolution kernel decomposition. CMSF adopts a lightweight architecture of 1×1 pointwise convolution, lightweight extraction of multi-scale spectral-spatial features, and channel shuffling fusion. 1×1 pointwise convolution is used to reduce the dimensionality of hyperspectral images. Lightweight extraction of multi-scale spectral-spatial features is achieved through convolution kernel decomposition and a dual-branch architecture, which includes a spatial branch and a spectral branch. Channel shuffling operation is designed to allow features from different scale branches to be mixed across groups and redistributed.

[0139] The specific process of step 21 is as follows:

[0140] Step 211, dimensionality reduction of the input channel;

[0141] First, use 1×1 pointwise convolution to... Spectral dimensions from Reduce to :

[0142] ;

[0143] in, This represents the number of spectral channels. This represents a 1×1 pointwise convolution operation. The pointwise convolution weight matrix is... The feature map after dimensionality reduction. , This represents the number of spectral channels after dimensionality reduction. It is the field of real numbers.

[0144] Step 212, spatial feature extraction;

[0145] The reduced feature map Input spatial branches, perform multi-scale spatial feature extraction, and obtain spatial branch feature maps. , , The number of spectral channels after spatial feature extraction. = .

[0146] The specific process of step 212 is as follows:

[0147] Step 2121: Spatial branching uses convolutional kernel decomposition and multi-scale fusion design (CKD conv-n) to capture spatial features under different receptive fields. CKD conv-n represents convolutional kernel decomposition and a dual-branch architecture, where n represents the convolutional kernel size. First, the dimensionality-reduced feature map... Feed into CKD conv-n: Set two parallel sub-branches, with n set to 3 and 5 respectively, to obtain CKD conv-3 convolution sub-branch and CKD conv-5 convolution sub-branch. Replace the standard two-dimensional convolution with a decomposition method of horizontal one-dimensional convolution + vertical one-dimensional convolution to reduce the number of parameters and computation.

[0148] The CKD conv-3 convolutional sub-branch is used for small-scale feature extraction. First, a horizontal one-dimensional convolution is performed, using a 1×3 convolution kernel that slides horizontally to calculate the weighted sum of the current position and its horizontal left and right adjacent pixels, capturing local dependencies in the horizontal direction.

[0149] ;

[0150] in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-3 convolution sub-branch;

[0151] Then, a vertical one-dimensional convolution is performed, using a 3×1 convolution kernel that slides along the vertical direction to calculate the weighted sum of the current position and its upper and lower adjacent pixels, capturing local dependencies in the vertical direction:

[0152] ;

[0153] in, The first vertical one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the output of the CKD conv-3 convolutional sub-branch;

[0154] The CKD conv-5 convolutional sub-branch is used for large-scale feature extraction. It first performs a horizontal one-dimensional convolution, then uses a 1×5 convolutional kernel to slide along the horizontal direction to capture a wider range of horizontal spatial dependencies.

[0155] ;

[0156] in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-5 convolution sub-branch;

[0157] Then, a vertical one-dimensional convolution is performed, using a 5×1 convolution kernel that slides along the vertical direction to capture a wider range of vertical spatial dependencies:

[0158] ;

[0159] in, The first vertical one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the output of the CKD conv-5 convolutional sub-branch.

[0160] Step 2122: Perform feature splicing and channel shuffling;

[0161] First, and The layers are stitched together along the channel dimension, and then the channel separation between scales is broken through channel shuffling:

[0162] ;

[0163] in, This represents the channel splicing function. It is a channel shuffling function. This is a spatial branch feature map.

[0164] Step 213, spectral feature extraction;

[0165] The reduced feature map Input the spectral branch, perform spectral feature extraction, and obtain the spectral branch feature map. , ;

[0166] The specific operation of step 213 is as follows:

[0167] The spectral branch focuses on extracting discriminative features in the spectral dimension, employing a lightweight design using one-dimensional depthwise separable convolution; firstly, the dimensionality-reduced feature map... Each spectral channel is individually applied a one-dimensional depthwise separable convolution to capture only the spectral dependence within a single channel, as shown in the formula:

[0168] ;

[0169] in, The kernel size is [size]. For depthwise convolution weights, for The output features obtained by applying one-dimensional depthwise separable convolution, Indicates the kernel size as Depth-separable convolution;

[0170] Then, a one-dimensional pointwise convolution is performed to adjust the number of channels, thus aligning it with the number of channels output by the spatial branch. The formula is as follows:

[0171] ;

[0172] in, For one-dimensional pointwise convolution weights, This is a spectral branching characteristic map.

[0173] Step 214, the multi-scale feature extraction module CMSF outputs;

[0174] Spatial branch feature map With spectral branching feature map Stitching along the channel dimension:

[0175] ;

[0176] in, This represents the channel splicing function. This is the final output of the multi-scale feature extraction module CMSF. .

[0177] Step 22: Construct a dual-sensory channel attention module (DAE). The DAE uses a dual-sensory branch, namely a channel-sensory branch and a spatial-sensory branch, for collaboration. In the DAE, the original feature map is first modeled using one-dimensional convolution and global average pooling, and then spatial information is modeled using two-dimensional depthwise separable convolution and mean square. Next, the output weights of the channel-sensory branch and the spatial-sensory branch are fused by element-wise multiplication and then normalized using the Sigmoid function to form a unified attention weight vector that includes dual perception. Finally, the attention weight vector is multiplied with the original feature map to output the enhanced feature map.

[0178] The specific process of step 22 is as follows:

[0179] Step 221, DAE input definition;

[0180] The input feature map of DAE is the feature map output by the multi-scale feature extraction module CMSF. The input feature map of DAE is denoted as... ,Right now , This is the final output of the multi-scale feature extraction module CMSF.

[0181] Step 222, calculate the weights of the dual-sensor branches;

[0182] Calculate the channel-aware branch attention weights: First, process the input feature map of the DAE. Global average pooling is performed on the spatial dimension to compress it into a one-dimensional channel feature vector:

[0183] ;

[0184] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. , The total number of channels. This is the global channel vector. This indicates the size of the resulting pixel block;

[0185] Then to Applying one-dimensional convolution with a kernel size of 3:

[0186] ;

[0187] in, This represents a one-dimensional convolution operation, where the convolution kernel slides only along the channel dimension, with stride = 1 and padding = 1 (to ensure that the number of input and output channels is consistent). It is a one-dimensional convolution weight matrix, which captures the local dependencies between adjacent channels through convolution operations, avoiding the information loss caused by the traditional "dimensionality reduction-dimensionality increase" process; To The output after applying one-dimensional convolution;

[0188] Calculate the attention weights for the spatial awareness branch: First, process the input feature map of the DAE. Extracting local spatial features using two-dimensional depthwise separable convolution:

[0189] ;

[0190] in, For depthwise convolution weights, This indicates a depthwise separable convolution with a kernel size of 3×3. It is a local feature in space;

[0191] Then perform a spatial dimensional averaging operation:

[0192] ;

[0193] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. This is a spatial perception vector.

[0194] Step 223, Attention Fusion and Feature Calibration;

[0195] First, attention fusion is performed, and Element-by-element multiplication:

[0196] ;

[0197] in, This represents element-wise multiplication. for and The result of element-wise multiplication;

[0198] Then a normalization operation is performed, using the Sigmoid function to... Normalize to the [0,1] interval:

[0199] ;

[0200] in, It is the Sigmoid function. for The result after normalization; Represented by natural constant An exponential function with base 0;

[0201] Finally, feature calibration is performed. Input feature map of DAE Channel-by-channel multiplication:

[0202] ;

[0203] in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. for and The result after multiplying each channel.

[0204] Step 23: Construct a classification head consisting of a global pooling layer and a fully connected layer to interpret the enhanced feature map, predict the category of the original pixel, and obtain a classification prediction map.

[0205] The specific process of step 23 is as follows:

[0206] Step 231, global average pooling;

[0207] right Global average pooling is performed to compress two-dimensional spatial features into one-dimensional feature vectors, thus eliminating the influence of spatial dimension on classification.

[0208] ;

[0209] in, This indicates a global average pooling operation. for The feature vector obtained after global average pooling;

[0210] Step 232, interpret the fully connected layer category;

[0211] Through the fully connected layer Mapped to the land cover category space, the category probability distribution is obtained through linear transformation and Softmax normalization, as shown in the formula:

[0212] ;

[0213] in, This is the weight matrix of the fully connected layer. , The total number of channels. This represents the total number of land cover categories in the hyperspectral data. For the real number field, For the bias vector of the fully connected layer, , For normalization function, This indicates the probability that the current pixel belongs to a land cover category;

[0214] Step 233: Generate the classification prediction map;

[0215] For each pixel in the test set, take The category corresponding to the maximum value in the image is used as the predicted label for that pixel. After traversing all pixels, all predicted labels are stitched together according to the spatial location of the original hyperspectral image to obtain a classification prediction map with the same size as the input image.

[0216] Step 3, Model Training: The lightweight hyperspectral image classification model ELS, constructed using the training set, is trained. 2 The C-Net model is trained by calculating the cross-entropy loss between the predicted results and the actual labels during training. The model parameters are optimized iteratively until the model converges. The optimized model parameters are recorded and stored, resulting in a trained lightweight hyperspectral image classification model, ELS. 2 C-Net.

[0217] In step 3, during the model training phase, the training set data is first input into the constructed lightweight hyperspectral image classification model for forward propagation calculation to obtain the predicted category probability distribution; the cross-entropy loss function is used to calculate the loss value between the predicted result and the true label, and the loss function Loss is:

[0218] ;

[0219] in, The number of training samples. For sample index, For category indexing, The total number of categories, Indicates the first The sample belongs to the first The true label value of the category, The model predicts the first... The sample belongs to the first The probability value of the category, It is a logarithmic function with base 10;

[0220] The Adam optimizer is used for parameter optimization, with the initial learning rate set to a constant. Cosine annealing is then used to adjust the learning rate.

[0221] ;

[0222] in, Based on the learning rate, The adjusted learning rate, For the current training round, Pi This is the maximum number of training rounds;

[0223] After all training rounds are completed, save the globally optimal model parameters.

[0224] Step 4, Model Inference: Input the hyperspectral image data from the test set into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, inference yields predicted categories, and these predicted categories are evaluated using evaluation metrics.

[0225] The specific process of reasoning to obtain the predicted category is as follows:

[0226] The hyperspectral image data to be predicted in the test set are input batch by batch into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, two multi-scale feature extraction modules (CMSF) and two dual-sensor efficient channel attention modules (DAE) are cascaded to form a feature extraction process of CMSF→CMSF→DAE→DAE, generating a calibrated feature map. The calibrated feature map is then subjected to global average pooling to obtain a global feature vector, which is then passed through a fully connected layer to generate a class probability distribution. Finally, the class index with the largest class probability distribution is used as the predicted class of the pixel to generate a classification prediction map.

[0227] Evaluation metrics include overall accuracy. Average accuracy and coefficient, , and The formula for calculating the coefficient is as follows:

[0228]

[0229]

[0230]

[0231] in, It is a random consistency error. Indicates the total number of categories. Indicates the number of correctly classified samples. This represents the total number of samples. Indicates the first The number of correctly classified samples in the class. Indicates the first The total number of samples in the class.

[0232] Table 1

[0233]

[0234] Table 1 describes the evaluation metrics of the method of the present invention and the comparison method on the Pavia University dataset. Ours represents the lightweight hyperspectral image classification model ELS provided by the present invention. 2 C-Net, comparison methods include:

[0235] Ghostnet: Ghost Network;

[0236] Res-LS 2 CM: Residual-Lightweight Spatial Spectral Convolutional Module Network;

[0237] LMSS-NAS: Lightweight multi-scale search neural architecture search network;

[0238] Lite-HCNet: Lightweight Hybrid Convolutional Neural Network;

[0239] LPCN: A lightweight and accurate classification network for hyperspectral remote sensing images based on multi-objective optimization;

[0240] ARC: Attention-based reservoir computing network;

[0241] CSM-DSPCss-Ghost: Channel Switching - Dynamic Splitting Pointwise Convolution Compresses Excitation Space - Ghost Network;

[0242] LMAFN: Lightweight spectral-spatial attention feature fusion network;

[0243] Using overall accuracy Average accuracy and The coefficients are used to evaluate the accuracy of hyperspectral image classification. As shown in Table 1, the classification results of the model of this invention are improved compared with other methods in the table.

[0244] like Figure 2 As shown, the input is the original hyperspectral image and its ground truth value, and the trained ELS... 2C-Net outputs the classification prediction values ​​for the image, where (a) is the ground truth map, (b) is the inference result map of Ghostnet, (c) is the inference result map of Res-LS2CM, (d) is the inference result map of LMSS-NAS, (e) is the inference result map of Lite-HCNet, (f) is the inference result map of LPCN, (g) is the inference result map of ARC, (h) is the inference result map of CSM-DSPCss-Ghost, (i) is the inference result map of LMAFN, and (j) is the inference result map of this invention. Figure 2 It can be seen that the model recognition accuracy of the present invention is higher; the model testing and reasoning operation is now complete.

Claims

1. A lightweight hyperspectral image classification method based on multi-scale convolution, characterized in that, Includes the following steps: Step 1, Data Preprocessing: Divide the hyperspectral image data into training and test sets according to a preset ratio, and perform preprocessing. Step 2, Model Building: Constructing a lightweight hyperspectral image classification model ELS based on convolutional kernel decomposition and dual-sensory channel attention collaboration. 2 C-Net, ELS 2 C-Net consists of a multi-scale feature extraction module CMSF, a dual-sensor attention module DAE, and a classification head connected in sequence. Step 3, Model Training: The lightweight hyperspectral image classification model ELS, constructed using the training set, is trained. 2 The C-Net model is trained by calculating the cross-entropy loss between the predicted results and the actual labels during training. The model parameters are optimized iteratively until the model converges. The optimized model parameters are recorded and stored, resulting in a trained lightweight hyperspectral image classification model, ELS. 2 C-Net; Step 4, Model Inference: Input the hyperspectral image data from the test set into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, inference yields predicted categories, and these predicted categories are evaluated using evaluation metrics.

2. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 11, Dataset Partitioning: Divide the original hyperspectral dataset... The data is randomly divided into training sets according to a set ratio. and test set The training set accounts for 10% of the total. The proportion of the test set is , It is a constant. The stratified sampling process ensures that the distribution of samples in each category is consistent. Step 12, Spatial Context Window Sampling: with target pixels Centered on, is the x-coordinate of the target pixel. Using the ordinate of the target pixel, the processed dataset is cropped pixel by pixel, with a window size of [value missing]. , This represents the size of the spatial pixel block obtained by cropping, resulting in a sample set of spatial pixel blocks centered on the pixel. Spatial pixel block sample set The spatial pixel block in , , The size is , This represents the number of spectral channels. Represented as: ； in, This represents the clipping operation function. Indicates target pixels The spatial pixel block centered on; Step 13, Normalization and Data Augmentation: First, normalize the spatial pixel blocks. Perform normalization processing, The value is standardized to the [0,1] interval, and the processing method is as follows: ； in, For normalized spatial pixel blocks, The minimum value in the spatial pixel block. It is the maximum value in the spatial pixel block; Then, data augmentation operations are performed; random rotation and flipping operations are used to... Data augmentation is performed, resulting in a new set of spatial pixel block samples. New spatial pixel block sample set The new spatial pixel block sample is , , The size is , This represents the number of spectral channels. Represented as: ； in, Indicates a data augmentation function; Step 14, Construct the training sample pair set: The Middle Match each new spatial pixel block sample with its corresponding real class label to establish a training sample pair. , for The Middle A new spatial pixel block sample for The corresponding real category label, , The total number of categories is used to obtain the final training sample pair set. Training sample set Represented as: ； in, for The total number of samples in the sample.

3. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 2, characterized in that, The specific process of step 2 is as follows: Step 21: Construct a multi-scale feature extraction module (CMSF) based on convolution kernel decomposition. CMSF adopts a lightweight architecture of 1×1 pointwise convolution, lightweight extraction of multi-scale spectral-spatial features, and channel shuffling fusion. The 1×1 pointwise convolution is used to reduce the dimensionality of the hyperspectral image. Lightweight extraction of multi-scale spectral-spatial features is achieved through convolution kernel decomposition and a dual-branch architecture, which includes a spatial branch and a spectral branch. A channel shuffling operation is designed to allow features from different scale branches to be mixed across groups and redistributed. Step 22: Construct a dual-sensory channel attention module (DAE). The dual-sensory channel attention module (DAE) works in collaboration between the channel-sensory branch and the spatial-sensory branch. In the DAE, the original feature map is first modeled using one-dimensional convolution and global average pooling, and then spatial information is modeled using two-dimensional depthwise separable convolution and mean square. Next, the output weights of the channel-sensory branch and the spatial-sensory branch are fused by element-wise multiplication and then normalized using the Sigmoid function to form a unified attention weight vector containing dual perception. Finally, the attention weight vector is multiplied with the original feature map to output the enhanced feature map. Step 23: Construct a classification head consisting of a global pooling layer and a fully connected layer to interpret the enhanced feature map, predict the category of the original pixel, and obtain a classification prediction map.

4. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 3, characterized in that, The specific process of step 21 is as follows: Step 211, input channel dimensionality reduction; First, use 1×1 pointwise convolution to... Spectral dimensions from Reduce to : ； in, This represents the number of spectral channels. This represents a 1×1 pointwise convolution operation. The pointwise convolution weight matrix is... The feature map after dimensionality reduction. , This represents the number of spectral channels after dimensionality reduction. For the real number field; Step 212, Spatial feature extraction; The reduced feature map Input spatial branches, perform multi-scale spatial feature extraction, and obtain spatial branch feature maps. , , The number of spectral channels after spatial feature extraction. = ; Step 213, spectral feature extraction; The reduced feature map Input the spectral branch, perform spectral feature extraction, and obtain the spectral branch feature map. , ; Step 214, the multi-scale feature extraction module CMSF outputs; Spatial branch feature map With spectral branching feature map Stitching along the channel dimension: ； in, This represents the channel splicing function. This is the final output of the multi-scale feature extraction module CMSF. .

5. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 4, characterized in that, The specific process of step 212 is as follows: Step 2121: Spatial branching decomposes CKD+ using convolutional kernels and designs CKD conv-n using multi-scale fusion. CKD conv-n represents the convolutional kernel decomposition and dual-branch architecture, where n represents the convolutional kernel size. First, the dimensionality-reduced feature map is... Feed into CKD conv-n: Set two parallel sub-branches, with n set to 3 and 5 respectively, to obtain CKD conv-3 convolution sub-branch and CKDconv-5 convolution sub-branch, and replace the standard two-dimensional convolution with a decomposition method of horizontal one-dimensional convolution + vertical one-dimensional convolution. The CKD conv-3 convolutional sub-branch is used for small-scale feature extraction. First, a horizontal one-dimensional convolution is performed, using a 1×3 convolution kernel that slides horizontally to calculate the weighted sum of the current position and its horizontal left and right adjacent pixels: ； in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-3 convolution sub-branch; Then, a vertical one-dimensional convolution is performed, using a 3×1 convolution kernel that slides along the vertical direction to calculate the weighted sum of the current position and its upper and lower adjacent pixels: ； in, The first vertical one-dimensional convolution kernel after decomposing a 3×3 convolution kernel. Each weight, This is the output of the CKDconv-3 convolutional sub-branch; The CKD conv-5 convolutional sub-branch is used for large-scale feature extraction. It first performs a horizontal one-dimensional convolution, using a 1×5 convolution kernel that slides horizontally. ； in, For row index, For column indexes, For channel indexing, For weighted indexes, The first horizontal one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the feature map after horizontal convolution in the CKD conv-5 convolution sub-branch; Then perform a vertical one-dimensional convolution, using a 5×1 convolution kernel that slides along the vertical direction: ； in, The first vertical one-dimensional convolution kernel after decomposing a 5×5 convolution kernel. Each weight, This is the output of the CKDconv-5 convolutional sub-branch; Step 2122: Perform feature splicing and channel shuffling; First, and The layers are stitched together along the channel dimension, and then the channel separation between scales is broken through channel shuffling: ； in, This represents the channel splicing function. It is a channel shuffling function. This is a spatial branch feature map.

6. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 5, characterized in that, The specific operation of step 213 is as follows: The spectral branch focuses on extracting discriminative features along the spectral dimension, employing a lightweight design using one-dimensional depthwise separable convolution. First, the dimensionality-reduced feature map... Each spectral channel is individually applied a one-dimensional depthwise separable convolution to capture only the spectral dependence within a single channel, as shown in the formula: ； in, The kernel size is [size]. For depthwise convolution weights, for The output features obtained by applying one-dimensional depthwise separable convolution, Indicates the kernel size as Depth-separable convolution; Then, a one-dimensional pointwise convolution is performed to adjust the number of channels, thus aligning it with the number of channels output by the spatial branch. The formula is as follows: ； in, For one-dimensional pointwise convolution weights, This is a spectral branching characteristic map.

7. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 6, characterized in that, The specific process of step 22 is as follows: Step 221, DAE input definition; The input feature map of DAE is the feature map output by the multi-scale feature extraction module CMSF. The input feature map of DAE is denoted as... ,Right now , This is the final output of the multi-scale feature extraction module CMSF; Step 222, calculate the weights of the dual-sensor branches; Calculate the channel-aware branch attention weights: First, process the input feature map of the DAE. Global average pooling is performed on the spatial dimension to compress it into a one-dimensional channel feature vector: ； in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. , The total number of channels. This is the global channel vector. This indicates the size of the resulting pixel block; Then to Applying one-dimensional convolution with a kernel size of 3: ； in, This represents a one-dimensional convolution operation where the convolution kernel slides only along the channel dimension. It is a one-dimensional convolution weight matrix. To The output after applying one-dimensional convolution; Calculate the attention weights for the spatial awareness branch: First, process the input feature map of the DAE. Extracting local spatial features using two-dimensional depthwise separable convolution: ； in, For depthwise convolution weights, This indicates a depthwise separable convolution with a kernel size of 3×3. It is a local feature in space; Then perform a spatial dimensional averaging operation: ； in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. For spatial perception vectors; Step 223, Attention Fusion and Feature Calibration; First, attention fusion is performed, and Element-by-element multiplication: ； in, This represents element-wise multiplication. for and The result of element-wise multiplication; Then a normalization operation is performed, using the Sigmoid function to... Normalize to the [0,1] interval: ； in, It is the Sigmoid function. for The result after normalization; Represented by natural constant An exponential function with base 0; Finally, feature calibration is performed. Input feature map of DAE Channel-by-channel multiplication: ； in, Represents the row index of the feature map. The column index representing the feature map. Indicates the channel index. for and The result after multiplying each channel.

8. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 7, characterized in that, The specific process of step 23 is as follows: Step 231, global average pooling; right Global average pooling is performed to compress two-dimensional spatial features into one-dimensional feature vectors, thus eliminating the influence of spatial dimension on classification. ； in, This indicates a global average pooling operation. for The feature vector obtained after global average pooling; Step 232, interpret the fully connected layer category; Through the fully connected layer Mapped to the land cover category space, the category probability distribution is obtained through linear transformation and Softmax normalization, as shown in the formula: ； in, This is the weight matrix of the fully connected layer. , The total number of channels. This represents the total number of land cover categories in the hyperspectral data. For the real number field, For the bias vector of the fully connected layer, , For normalization function, This indicates the probability that the current pixel belongs to a land cover category; Step 233: Generate the classification prediction map; For each pixel in the test set, take The category corresponding to the maximum value in the image is used as the predicted label for that pixel. After traversing all pixels, all predicted labels are stitched together according to the spatial location of the original hyperspectral image to obtain a classification prediction map with the same size as the input image.

9. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 8, characterized in that, In step 3, during the model training phase, the training set data is first input into the constructed lightweight hyperspectral image classification model for forward propagation calculation to obtain the predicted category probability distribution; the cross-entropy loss function is used to calculate the loss value between the predicted result and the true label, and the loss function Loss is: ； in, The number of training samples. For sample index, For category indexing, The total number of categories, Indicates the first The sample belongs to the first The true label value of the category, The model predicts the first... The sample belongs to the first The probability value of the category, It is a logarithmic function with base 10; The Adam optimizer is used for parameter optimization, with the initial learning rate set to a constant. Cosine annealing is then used to adjust the learning rate. ； in, Based on the learning rate, The adjusted learning rate, For the current training round, Pi This is the maximum number of training rounds; After all training rounds are completed, save the globally optimal model parameters.

10. The lightweight hyperspectral image classification method based on multi-scale convolution according to claim 9, characterized in that, In step 4, the specific process of reasoning to obtain the predicted category is as follows: The hyperspectral image data to be predicted in the test set are input batch by batch into the trained lightweight hyperspectral image classification model ELS. 2 In C-Net, two multi-scale feature extraction modules (CMSF) and two dual-sensor efficient channel attention modules (DAE) are cascaded to form a feature extraction process of CMSF→CMSF→DAE→DAE, generating a calibrated feature map. The calibrated feature map is then subjected to global average pooling to obtain a global feature vector, which is then passed through a fully connected layer to generate a class probability distribution. Finally, the class index with the largest class probability distribution is used as the predicted class of the pixel to generate a classification prediction map.

Images