Hyperspectral image classification method based on light spectrum hybrid adaptive waveband selection

Abstract

The invention discloses a hyperspectral image classification method based on light spectrum hybrid adaptive band selection, which belongs to the technical field of remote sensing image processing, and comprises the following steps: based on a data cube sample set and label vectors, carrying out hierarchical random sampling in proportion, and dividing a training set and a test set; inputting the training set into a hyperspectral image classification model based on light spectrum hybrid adaptive band selection, and performing forward propagation to record an optimal weight; inputting a test set into the model, performing forward propagation by using the optimal weight, outputting a pixel-level category probability through a convolution integral category head, and determining a prediction label; the forward propagation comprises the steps of establishing learnable weight vectors for all spectral bands at the first layer of a data loader, performing band-by-band weighting on each cube sample, extracting spatial spectral features by using a light spectrum mixed structure combined with a local convolution global converter, performing cross attention band screening on high-level spatial spectral tensor, and performing data processing on the high-level spatial spectral tensor. And dynamically calculating the importance weight of the spectral band, and updating the learnable band weight vector by using the importance weight of the spectral band.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a hyperspectral image classification method based on adaptive band selection of light spectrum mixing. Background Technology

[0002] In recent years, deep learning techniques have been widely adopted in the field of hyperspectral image classification. Convolutional neural networks, with their ability to extract local spatial-spectral features, have achieved excellent classification accuracy in various remote sensing scenarios; converter structures, due to their global modeling advantages, have also been introduced into hyperspectral analysis; and hybrid frameworks combining the two have further enriched the spatial-spectral representations. Simultaneously, researchers have begun to focus on the synergistic optimization of band selection and classification, hoping to reduce redundant bands and improve network efficiency through attention mechanisms or sparse regularization. These explorations have collectively driven the development of intelligent hyperspectral image processing.

[0003] However, existing methods still have the following shortcomings: The model parameter scale and computational cost are too large; mainstream converters rely on deep self-attention stacking, resulting in significantly higher parameter count and memory usage compared to convolutional networks; overfitting is prone to occur and inference speed is slowed down when training samples are limited or hardware resources are constrained. Band selection is disconnected from classification tasks. Traditional band selection is often done offline outside the network or achieves "soft selection" through simple weight constraints, lacking direct feedback of classification information, making it difficult to guarantee that the selected bands are truly effective for the current task. The band weight update mechanism is rigid. Existing differentiable weighted methods often treat band attention as static trainable parameters, lacking dynamic adjustment based on task performance, making it difficult to adapt to data distribution and training progress. Local-global feature fusion is insufficient. Convolution focuses on neighborhood texture, while converters focus on long-range dependencies. If an effective fusion layer is lacking, spatial-spectral information may still be separated, affecting discrimination ability.

[0004] In the Chinese invention patent application CN115410085A, published on November 29, 2022, entitled "A Hyperspectral Image Classification Method Based on Spectral Enhancement and Dense Connection Transformer," a "spectral enhancement module" is added to the front end of the classification network, and a "dense connection transformer module" is used in the backbone. Spectral enhancement enriches the input features by expanding and transforming the original spectral curve; the dense connection transformer establishes multi-level jump connections between layers, which can simultaneously capture local details and long-range dependencies, enhancing the consistency of classification regions and overall accuracy. However, some key issues still remain: Spectral enhancement employs fixed or preset spectral domain operations, lacking learnable band weight adjustment during training and failing to dynamically select effective bands based on task feedback. Band compression and classification objectives are independent of each other, lacking joint loss constraints, and redundant bands cannot be explicitly reduced during training. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of existing hyperspectral image classification methods, such as large and easily overfitting models, separation of band selection and classification, lack of dynamic adjustment of band weights, and single loss target. In response, this invention proposes a hyperspectral image classification method based on light spectrum hybrid adaptive band selection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a hyperspectral image classification method based on adaptive band selection of light spectrum mixing, comprising the following steps: A data cube sample set and label vector are constructed using the original hyperspectral image. Based on the data cube sample set and label vector, stratified random sampling is performed to divide the data cube sample set and the test set. The training set is input into a hyperspectral image classification model based on light spectrum hybrid adaptive band selection. After forward propagation, the joint optimization loss is calculated and backpropagation is performed to update the network parameters and band weight vector and record the optimal weight. The test set is input into the hyperspectral image classification model based on light spectrum mixing adaptive band selection. The model performs forward propagation using the optimal weights, outputs pixel-level class probabilities and determines the predicted labels through the convolution class head, and maps the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

[0007] Furthermore, a data cube sample set and label vector are constructed using the original hyperspectral image, specifically as follows: For each pixel coordinate of the original hyperspectral image, a data cube is constructed according to the data cube expression. By traversing all pixels of the original hyperspectral image, a data cube sample set and label vector are obtained. The data cube expression is as follows:

[0008] c =( k -1) / 2 in, Representing a data cube, iIndicates the row index of the local window. j The column index representing the local window. b Indicates the spectral band index. x Represents the x-coordinate of pixels. y Represents the pixel ordinate. c Indicates the window radius. k Indicates the side length of the window; when x+i-c or y+j-c When the image exceeds the boundary, a mirror fill method is used to obtain the pixel values ​​at the symmetrical position.

[0009] Furthermore, based on the data cube sample set and label vectors, stratified random sampling is performed proportionally to divide the data into training and test sets, specifically as follows: Predefined category numbers serve as the global sequence numbers for samples extracted from each category, resulting in a set of category sample indices; For each category's sample index set, sample numbers are randomly selected proportionally to obtain the test sample index set for each category. The test sample index sets for each category are then merged to obtain the test set. The remaining sample indices in each category's sample index set are grouped into training sample index sets for each category. The training sample index sets for each category are then merged to obtain the training set.

[0010] Furthermore, the formula for calculating the joint optimization loss is as follows:

[0011] in, Indicates the joint optimization loss. Represents classification loss, This indicates band canonical loss.

[0012] Furthermore, the weighting in the band-by-band weighting of each cube sample in the training or test set using the learnable weight vector is obtained using the following formula:

[0013]

[0014] in, Indicates the first b Initial weights for each spectral band, Let represent any cube sample in the training set. This represents the weighted training set cube tensor. Represents any cube sample in the test set. This represents the weighted test set cube tensor. i Indicates the row index of the local window. jThe column index representing the local window. b Indicates the spectral band index.

[0015] Furthermore, a light spectral mixing structure combining local convolutional global transforms is used to extract spatial spectral features from the weighted cubic tensor to obtain a high-level spatial spectral tensor, specifically: A 3D convolution is applied to each weighted cubic tensor to extract local convolutional features; The local convolutional features are divided into non-overlapping image blocks in the spatial dimension. Each block is expanded into a vector of uniform length in row-major order and stacked along the first dimension to form a token sequence. A lightweight multi-head self-attention-feedforward structure is applied to the token sequence to obtain the globally modeled token sequence. The token sequence after global modeling is reassembled back into the spatial grid to obtain a new tensor; The new tensor is concatenated with local convolutional features in the channel dimension, and a convolution kernel is applied for linear fusion to obtain the high-level spatial spectrum tensor. The importance weights of spectral bands are dynamically calculated, and the learnable band weight vector is updated using these weights. Specifically: The mid-level feature map of the high-level spatial spectrum tensor is selected as the query tensor, and the local convolutional features are used as the key tensor and value tensor. After flattening the query tensor, key tensor, and value tensor in spatial dimension, perform multi-head dot product attention on the query tensor and key tensor to obtain the cross-attention matrix. The average of the cross-attention matrix along the spatial dimension is used to obtain the sample-level band attention vector. The importance of a batch of bands is calculated by averaging the sample-level band importance vectors over a small batch. Based on the importance of batch bands, momentum is used to update and normalize the learnable band weights.

[0016] Furthermore, the pixel-level class probabilities are output through the convolution class header, and the predicted labels are determined, including: A convolution kernel is applied to the high-level spatial spectrum tensor to obtain the class log score. Softmax is then applied to the class log score in the class dimension to obtain the probability vector of the cube. The probability vector of the center pixel of the cube is selected from the probability vector of the cube as the output pixel-level class probability and the predicted label is determined.

[0017] Secondly, the present invention provides a hyperspectral image classification system based on light spectrum hybrid adaptive band selection, comprising: The dataset partitioning module is used to construct a data cube sample set and label vector using the original hyperspectral image. Based on the data cube sample set and label vector, stratified random sampling is performed to partition the dataset into training and test sets. The parameter and weight update module is used to input the training set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation, calculate the joint optimization loss and backpropagate, update the network parameters and band weight vector and record the optimal weights. The probability and predicted label output module is used to input the test set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation using the optimal weights, output pixel-level class probabilities and determine the predicted labels through the convolution class head, and map the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

[0018] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a hyperspectral image classification method based on light spectrum mixing adaptive band selection.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, provides a hyperspectral image classification method based on light spectrum mixing adaptive band selection.

[0020] Compared with the prior art, the present invention has the following beneficial technical effects: This invention proposes a hyperspectral image classification method based on lightweight spectral hybrid adaptive band selection. During forward propagation, a set of real-valued learnable weight parameters are introduced for all spectral bands at the data loading entry point, and each band is weighted channel-by-channel during cube reading. This overcomes the limitations of traditional methods where band weighting is static or requires offline preprocessing and cannot be adaptively adjusted with task feedback. The lightweight spectral hybrid structure combining local convolution and global transformer maintains and even improves feature discrimination and regional consistency while significantly reducing network parameter count and memory usage. This alleviates the problems of traditional deep transformer models being large, having long inference times, and being prone to overfitting in small sample scenarios. Cross-attention band selection solves the problems of decoupling band selection and classification, low correlation between selected bands and specific tasks, and difficulty in adjusting with the training phase. Joint optimization of loss overcomes the shortcomings of simply minimizing classification error, which leads to redundant band residues and poor model interpretability. This solves the problems of existing hyperspectral image classification methods, such as large models prone to overfitting, separation of band selection and classification, lack of dynamic adjustment of band weights, and single loss objective. Attached Figure Description

[0021] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the invention in any way. Furthermore, the shapes and proportions of the components in the drawings are merely schematic to aid in understanding the invention and do not specifically limit the shapes and proportions of the components. In the drawings: Figure 1 This is a flowchart of the hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to the present invention.

[0022] Figure 2 This is a structural diagram of the hyperspectral image classification system based on light spectrum hybrid adaptive band selection according to the present invention.

[0023] Figure 3 This is an electronic device diagram of the hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to the present invention.

[0024] Figure 4 This is a flowchart of a hyperspectral image classification method based on adaptive band selection of light spectrum mixing in an embodiment of the present invention.

[0025] Figure 5 This is a classification visual diagram of the Salinas dataset in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] Example 1 See Figure 1 A hyperspectral image classification method based on light spectrum mixing adaptive band selection includes the following steps: A data cube sample set and label vector are constructed using the original hyperspectral image. Based on the data cube sample set and label vector, stratified random sampling is performed to divide the data cube sample set and the test set. The training set is input into a hyperspectral image classification model based on light spectrum hybrid adaptive band selection. After forward propagation, the joint optimization loss is calculated and backpropagation is performed to update the network parameters and band weight vector and record the optimal weight. The test set is input into the hyperspectral image classification model based on light spectrum mixing adaptive band selection. The model performs forward propagation using the optimal weights, outputs pixel-level class probabilities and determines the predicted labels through the convolution class head, and maps the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

[0028] In this embodiment, data processing is handled by stratified random sampling to divide the training and test sets, ensuring a reasonable data distribution and improving the model's generalization ability. During training, the joint optimization loss is calculated and backpropagated to update network parameters and band weight vectors, enabling the model to adaptively adjust band weights and optimize performance. In the feature extraction stage, learnable weight vectors are first established for each spectral band and weighted band by band to highlight important bands. Then, a lightweight spectral hybrid structure combining local convolution and global transform is used to fully mine spatial spectral information and extract high-level spatial spectral features. Finally, cross-attention band filtering is used to reduce redundant information. In the testing phase, the optimal weights recorded during training are used to ensure the model runs with optimal parameters. This method ultimately outputs pixel-level class probabilities and predicted labels, achieving accurate classification of hyperspectral image pixels. It provides more detailed and accurate information for fields such as remote sensing image analysis, demonstrating excellent performance in improving classification accuracy and efficiency, and providing reliable technical support for hyperspectral image applications.

[0029] The data cube sample set and label vector are constructed using the original hyperspectral image, specifically as follows: For each pixel coordinate of the original hyperspectral image, a data cube is constructed according to the data cube expression. By traversing all pixels of the original hyperspectral image, a data cube sample set and label vector are obtained. The data cube expression is as follows:

[0030] c =( k -1) / 2 in, Representing a data cube, i Indicates the row index of the local window. j The column index representing the local window. b Indicates the spectral band index. x Represents the x-coordinate of pixels. y Represents the pixel ordinate. c Indicates the window radius. k Indicates the side length of the window; when x+i-c or y+j-c When the image exceeds the boundary, a mirror fill method is used to obtain the pixel values ​​at the symmetrical position.

[0031] Constructing a data cube centered on each pixel using an expression fully integrates local neighborhood information, capturing the spatial continuity and structural features of ground features. This provides richer spatial information for the classification model, improving classification accuracy. Furthermore, when a local window extends beyond the image boundary, mirroring is used to obtain pixel values ​​at symmetrical locations, effectively avoiding data loss. This ensures that even pixels at image edges can generate complete data cubes, allowing the model to fully utilize all data during training and testing, maintaining data integrity and consistency, and thus enhancing the model's generalization ability and classification performance.

[0032] Based on the data cube sample set and label vectors, stratified random sampling is performed proportionally to obtain the training set and the test set, specifically: Predefined category numbers serve as the global sequence numbers for samples extracted from each category, resulting in a set of category sample indices; For each category's sample index set, sample numbers are randomly selected proportionally to obtain the test sample index set for each category. The test sample index sets for each category are then merged to obtain the test set. The remaining sample indices in each category's sample index set are grouped into training sample index sets for each category. The training sample index sets for each category are then merged to obtain the training set.

[0033] By predefining category numbers and extracting global sample indices to form a category sample index set, different categories of samples can be clearly distinguished. Sample indices are randomly selected proportionally from each category sample index set to form the test set, ensuring that the proportion of samples from each category in the test set is consistent with the original dataset and avoiding category distribution bias. The remaining sample indices form the training set, similarly maintaining the proportion of each category. This partitioning ensures a reasonable distribution of training and test set data, allowing the model to be exposed to comprehensive category features during training and accurately evaluating the model's performance on real-world data distributions during testing, effectively improving the model's generalization ability and classification results.

[0034] The formula for calculating the joint optimization loss is as follows:

[0035] in, Indicates the joint optimization loss. Represents classification loss, This indicates band canonical loss.

[0036] The joint optimization loss consists of classification loss and band regularization loss. Classification loss directly measures the model's performance on classification tasks, prompting continuous optimization to improve classification accuracy and ensure effective differentiation of different categories in hyperspectral images. Band regularization loss, on the other hand, constrains band weights, preventing the model from over-relying on certain bands or exhibiting unreasonable weights, thus improving the rationality and stability of band selection. The combination of these two methods addresses both classification performance and band selection optimization, enabling the model to find a balance between classification accuracy and band selection rationality during training, thereby enhancing its generalization ability and practical application effectiveness.

[0037] The weighted cube tensor is obtained by performing band-wise weighting on each cube sample in the training or test set using a learnable weight vector, and the weighting is performed using the following formula:

[0038]

[0039] in, Indicates the first b Initial weights for each spectral band, Let represent any cube sample in the training set. This represents the weighted training set cube tensor. Represents any cube sample in the test set. This represents the weighted test set cube tensor. i Indicates the row index of the local window. j The column index representing the local window. b Indicates the spectral band index.

[0040] By setting initial weights for each spectral band, the model can autonomously adjust the level of attention given to different bands. During the weighting process, each cube sample in the training or test set is processed band by band according to a formula, highlighting the information of bands that contribute significantly to the classification task and suppressing the influence of noise or unimportant bands. This helps improve the quality of feature extraction, allowing the model to focus on more discriminative band features. Moreover, the learnable weight vectors are continuously optimized during training, making the weighting process more closely match the data characteristics, thereby improving the model's classification accuracy for hyperspectral images and enhancing its performance and robustness in complex data environments.

[0041] A lightweight spectral mixing structure combining local convolution and global transform is used to extract spatial spectral features from a weighted cubic tensor to obtain a high-level spatial spectral tensor, specifically: A 3D convolution is applied to each weighted cubic tensor to extract local convolutional features; The local convolutional features are divided into non-overlapping image blocks in the spatial dimension. Each block is expanded into a vector of uniform length in row-major order and stacked along the first dimension to form a token sequence. A lightweight multi-head self-attention-feedforward structure is applied to the token sequence to obtain the globally modeled token sequence. The token sequence after global modeling is reassembled back into the spatial grid to obtain a new tensor; The new tensor is concatenated with local convolutional features in the channel dimension, and a convolution kernel is applied for linear fusion to obtain the high-level spatial spectrum tensor. The importance weights of spectral bands are dynamically calculated, and the learnable band weight vector is updated using these weights. Specifically: The mid-level feature map of the high-level spatial spectrum tensor is selected as the query tensor, and the local convolutional features are used as the key tensor and value tensor. After flattening the query tensor, key tensor, and value tensor in spatial dimension, perform multi-head dot product attention on the query tensor and key tensor to obtain the cross-attention matrix. The average of the cross-attention matrix along the spatial dimension is used to obtain the sample-level band attention vector. The importance of a batch of bands is calculated by averaging the sample-level band importance vectors over a small batch. Based on the importance of batch bands, momentum is used to update and normalize the learnable band weights.

[0042] 3D convolution extracts local convolutional features, capturing both local spatial and spectral information in hyperspectral images. These local convolutional features are then formed into a token sequence and globally modeled to uncover spatial and spectral relationships from a global perspective. Reassembling the token sequence and cascading it with the local convolutional features integrates both local and global features, resulting in a more discriminative high-level spatial-spectral tensor. In the cross-attention band selection process, the importance weights of spectral bands are dynamically calculated and the learnable band weight vector is updated, adaptively highlighting important bands and suppressing redundant bands. This entire process effectively integrates local and global information, optimizes band selection, and improves the quality of feature extraction, contributing to enhanced accuracy and robustness in hyperspectral image classification.

[0043] The pixel-level class probabilities are output through the convolution class header, and the predicted labels are determined, including: A convolution kernel is applied to the high-level spatial spectrum tensor to obtain the class log score. Softmax is then applied to the class log score in the class dimension to obtain the probability vector of the cube. The probability vector of the center pixel of the cube is selected from the probability vector of the cube as the output pixel-level class probability and the predicted label is determined.

[0044] Applying a convolution kernel to the high-level spatial spectral tensor yields class logarithmic scores, which can be further extracted and integrated to provide a valid basis for classification. Using Softmax, the class logarithmic scores are converted into probability vectors, normalizing the output values ​​to between 0 and 1, with the sum of probabilities for each class being 1, intuitively reflecting the likelihood of each pixel belonging to each class. The probability vector of the center pixel of the cube is selected as the output, precisely focusing on each pixel location to achieve pixel-level classification. This method fully utilizes the extracted high-level spatial spectral features, accurately outputs pixel-level class probabilities, and thus determines the predicted label, effectively improving the accuracy and precision of hyperspectral image classification and meeting practical application needs.

[0045] Example 2 See Figure 2 A hyperspectral image classification system based on light spectrum hybrid adaptive band selection includes: The dataset partitioning module is used to construct a data cube sample set and label vector using the original hyperspectral image. Based on the data cube sample set and label vector, stratified random sampling is performed to partition the dataset into training and test sets. The parameter and weight update module is used to input the training set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation, calculate the joint optimization loss and backpropagate, update the network parameters and band weight vector and record the optimal weights. The probability and predicted label output module is used to input the test set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation using the optimal weights, output pixel-level class probabilities and determine the predicted labels through the convolution class head, and map the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

[0046] Example 3 See Figure 3 An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a hyperspectral image classification method based on light spectrum mixing adaptive band selection.

[0047] Example 4 A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, is a hyperspectral image classification method based on light spectrum mixing adaptive band selection.

[0048] Example 5 This embodiment provides a hyperspectral image classification method based on lightweight spectral hybrid adaptive band selection, addressing the following pain points in existing technologies: Large and prone to overfitting models; existing hyperspectral converters rely on deep self-attention stacking, resulting in large parameter counts, high inference time, and unstable performance under limited sample conditions. Disconnect between band selection and classification; traditional band selection is mostly offline or statically weighted, lacking classification feedback, resulting in low relevance and high redundancy of selected bands to specific tasks. Lack of dynamic adjustment of band weights; existing differentiable weighting methods typically fix the weight shape only in the early stages of training, making it difficult to adaptively update with data and training progress. Single loss objective; mainstream methods only focus on classification accuracy, failing to simultaneously constrain band sparsity and weight determinism, leading to redundant information and poor model interpretability.

[0049] See Figure 4 The specific steps are as follows: S1, Data Augmentation Cube Construction This embodiment uses the Salinas dataset, and the hyperspectral image is denoted as... It has 512 rows, 217 columns, and 204 spectral bands; the pixel category is denoted as... This step expands the 15×15 spatial neighborhood of each pixel in the dataset with all 204 bands into a three-dimensional data cube of uniform size, enabling subsequent models to utilize both local spatial texture and complete spectral information simultaneously.

[0050] S101, Space Window Settings Set the window side length k =15, window radius c =( k -1) / 2=7; Construct a data cube for any pixel coordinates ( x , y ), cube elements are ordered as follows:

[0051] in i , j , b These represent the row index, column index, and spectral band index of the local window, respectively. If... x+i-c or y+j-c If the image exceeds the image boundary, mirror filling is used to obtain the pixel values ​​at symmetrical positions to avoid loss of edge information.

[0052] Using the above formula, each center pixel yields a data cube containing local spatial texture and a complete spectral curve. .

[0053] S102, Aggregated Sample Set Applying the S101 operation iteratively to all pixels of the image yields the sample set:

[0054] in N This represents the total number of samples.

[0055] The corresponding label vector is:

[0056] in Indicates the first n The category number of the center pixel of each cube sample.

[0057] The final data format is as follows: .

[0058] S2, Divide the training-test sample set The sample set obtained in step S1 and its label vector Based on this, all samples are divided into training and test sets according to the 9:1 stratified random sampling principle to ensure that the proportion of samples of each category in the two sets is consistent, which facilitates subsequent model training and fair evaluation.

[0059] S201, Calculate the category index set The category number is .

[0060] Extract the global index of the corresponding sample for each category, and you will get:

[0061] in n Indicates that the cube sample is in the sample set The serial number in (1≤ n ≤ N ), For category c The sample index set.

[0062] S202, Randomly select test samples For each category c proportional ρ =0.1 Randomly select test sample number:

[0063] The remaining sample numbers are assigned to the training set:

[0064] This operation ensures that the test set and training set maintain a 9:1 sample ratio at each category level.

[0065] S203, Generate training-test dataset Merge training and test indexes of different categories:

[0066] Therefore, we can conclude that:

[0067]

[0068] , These are the training cube set and the test cube set, respectively. , These are the corresponding category label vectors. After the partitioning, the training set accounts for approximately 90%, and the test set accounts for approximately 10%. This provides the data foundation for the subsequent training and evaluation phases of the differentiable spectral attention-weighted and light spectral hybrid network.

[0069] S3, Microwaveable Band Weight Initialization This step establishes a learnable weight vector for all 204 spectral bands in the first layer of the data loader, and uses this vector to perform band-by-band weighting on each cube sample in the training and test sets, thereby generating an input tensor that has both differentiability and initial balance, providing an end-to-end updatable band selection channel for subsequent networks.

[0070] S301. Initialize the band weight vector set up The initial weight vector is defined as follows:

[0071] in Indicates the first b The initial weights of each spectral band are all set to the same value of 1 / 204 to ensure that each band contributes equally to the model at the beginning of training; this vector is then automatically updated through backpropagation.

[0072] S302, Calculate the weighted cube input For the training set Arbitrary cube sample The weighted input is obtained using the following formula:

[0073] This is the weighted cube tensor.

[0074] Similarly, for the test set samples The corresponding result can also be obtained By multiplying each band by learnable weights, the network can dynamically amplify or suppress information in each band during training without disrupting gradient propagation.

[0075] S4, Light Spectrum Mixture Feature Extraction This step utilizes a light spectral hybrid structure combining local convolution and global transform to process the weighted cubic samples obtained in S3. Spatial-spectral feature extraction is performed to obtain a joint feature tensor containing both local texture and long-range dependence, providing discriminative information for subsequent band selection and classification.

[0076] S401. Extracting local convolutional features For each weighted sample Applying 3D convolution:

[0077] in This represents the convolution operation. A set of trainable convolutional kernels of size 3×3×204, BN(·) is the batch normalization. This is the SiLU activation function.

[0078] result , d This represents the number of output channels for the convolution. The convolution kernel only slides within a 3×3 spatial range, achieving local spatial noise reduction and feature extraction, while simultaneously performing a 204→ operation in the spectral dimension. d Dimensional reduction of the channel.

[0079] S402, Divide the image into blocks Will With step size p =5 Perform non-overlapping block division in the spatial dimension:

[0080] Divide into blocks Size p × p × d Small pieces.

[0081] Expand each block in row-major order to its length. The vectors are stacked along the first dimension to form a token sequence:

[0082] This operation transforms local features into a sequence, creating conditions for subsequent global modeling.

[0083] S403, Modeling Global Spatial-Spectral Relationships For token sequences Employs a lightweight multi-head self-attention-feedforward structure:

[0084] Where MSA(·) is a multi-head self-attention network with depth separable projection; FFN(·) is a linear-SiLU-linear lightweight feedforward network; LN(·) is layer normalization; The token sequence is modeled globally. This structure captures long-range spatial spectral dependencies across blocks while maintaining a low parameter count.

[0085] S404, Fusion Spatial Spectral Features Will Reassemble the spatial mesh to obtain the tensor:

[0086] Subsequently, it is combined with the local convolution features of S401. Concatenate along the channel dimension and apply a 1×1 convolution kernel. Perform linear fusion:

[0087] Output ,in Indicates channel splicing. This represents the number of channels after merging.

[0088] The fusion operation integrates local convolution and global attention features to form a high-level representation containing multi-scale information of the spatial spectrum. This is used for subsequent cross-attention band selection and classification head.

[0089] S5, Cross-band Filtering High-level spatial spectral features generated in S4 Based on this, a classification-guided cross-attention mechanism is introduced to dynamically calculate the importance weights of spectral bands, which are used to update the learnable band weight vector, thereby realizing task-driven soft band selection.

[0090] S501, Constructing Queries and Key Values Select Mid-layer feature map (Obtained through a single 1×1 convolution) as the query tensor Extract S401 convolutional features As a key tensor Sum tensor .

[0091] Flatten both in space:

[0092] in L =15×15.

[0093] S502, Calculate the cross-attention matrix right and Points to note when doing long positions:

[0094] in , For trainable projection matrices, The key dimension for each attention head.

[0095] S503, Importance of Generation Batch Bands Will Averaging along the spatial dimension yields the sample-level band importance vector:

[0096] Where the function c(v) Back button location v Spectral band index of the pixel For indicator functions. Matrix elements. Indicates the query position u Key position v attention.

[0097] For batch size B The importance of the batch band is calculated by averaging the samples from a small batch. : .

[0098] S504, Update Learnable Band Weights Momentum update and normalization are used:

[0099] in This is the momentum coefficient.

[0100] Updated The data is then applied to the next batch of inputs to achieve a classification-band selection closed loop.

[0101] S6. Output Prediction and Label Determination High-level spatial spectrum tensor obtained in S4 After being filtered by the S5 cross-attention band, the shape remains unchanged. Now, the pixel-level category probability is output through the classification head to determine the predicted label.

[0102] S601, Generate categorical logarithmic scores right Apply a 1×1 convolution kernel ( C =16 is the number of categories), resulting in:

[0103] in pixels within the window ( i,j ) belongs to category C Logarithmic fractions.

[0104] This convolution does not downsample and maintains the same spatial size as the input; Conv is the corresponding convolutional layer.

[0105] S602, Calculating Category Probability and Predicting Labels First, in terms of category dimension... Applying Softmax:

[0106] in .

[0107] Then only the center pixel of the cube is taken. probability vector:

[0108] And determine the predicted labels:

[0109] At this point, the class probability of the center pixel for each sample has been obtained. With predictive labels This provides input for the next step of joint loss calculation and model training cycle.

[0110] S7, Joint Loss of Band Selection and Classification Tasks Batch size is B After one forward pass, the model generates a value for each sample. n Obtain the class probability vector of the center pixel (See S6), and the learnable band weight vector has been obtained. This step provides the specific formulas for classification loss and band regularization loss, and combines them into a joint optimization objective.

[0111] S701, Calculate classification loss Let the true category number of each sample be . The loss is calculated using the cross-entropy method. :

[0112] in Indicates the first n Each sample is predicted to be its true class. The probability. This measure is the classification accuracy of the model in the current batch.

[0113] S702, Calculate band canonical loss The band canonical loss consists of two parts: a sparsity term and an entropy term, used to encourage... Both sparse and distinct:

[0114] The first item Equivalent to This causes the overall weight vector to become smaller and minimizes the selection of wavebands; the second term As an entropy constraint, it drives each Keep values ​​close to 0 or 1 to avoid ambiguous intermediate values; and This is the loss weight hyperparameter.

[0115] S703, Combined Loss Combining the two parts, we arrive at the overall optimization objective for this batch:

[0116] The model then followed Backpropagation synchronously updates network parameters and band weight vectors, achieving a synergistic improvement in classification performance and band selection effectiveness.

[0117] S8, Model Training and Testing Process S801, Training from Random sampling without replacement B Each index forms a batch Perform forward propagation according to S3–S5 to obtain the joint loss. The loss is backpropagated, the parameters are updated using the AdamW optimization rule, and the optimal weights are saved.

[0118] S802, Test Reasoning Load the optimal weights and index the test set. Repeat the forward steps S3–S6 for all samples to output predicted labels. With probability vector This allows us to obtain pixel-level classified images.

[0119] In this embodiment, S3, the microband weight initialization introduces a set of real-valued learnable weight parameters for all spectral bands at the data loading entry point, and performs channel-by-channel weighting for each band during cube reading. The weights are initially set to the same constant and then entered into the backpropagation channel along with network convolution, transformer, and other parameters. This allows the band weights to be trained synchronously with the classification network, enabling the network to "learn and filter" early in the training process—gradually amplifying bands with high information content and suppressing redundant bands. This overcomes the limitations of traditional methods where band weighting is static or requires offline preprocessing and cannot adaptively adjust to task feedback.

[0120] S4 lightweight spectral hybrid feature extraction employs a hybrid structure of "local convolution + lightweight global transformer + one-to-one convolution fusion": convolution handles neighborhood texture denoising and channel compression, lightweight multi-head self-attention captures long-range spatial spectral dependencies after block segmentation, and finally, one-to-one convolution integrates local and global information. While significantly reducing the number of network parameters and memory usage, it maintains and even improves feature discriminative power and region consistency. This alleviates the problems of traditional deep transformer models being large in size, having long inference time, and being prone to overfitting in small sample scenarios.

[0121] The S5 cross-attention band selection method utilizes mid-level features from the classification branch as queries, constructing cross-attention with shallow convolutional features. It statistically analyzes the contribution of each band to the current classification task in batches and employs a momentum smoothing strategy to write back learnable weights in real time. This forms a closed loop of "classification feedback → attention evaluation → weight write-back," with weights dynamically updated according to sample distribution, achieving true band adaptive selection and continuously improving the utilization rate of effective bands. It solves the problems of decoupling band selection from classification, low correlation between selected bands and specific tasks, and difficulty in adjusting bands during training.

[0122] The S6 band selection and classification task uses a joint loss mechanism. In addition to the cross-entropy classification loss, it incorporates dual constraints of sparsity regularization and entropy regularization to simultaneously compress the number of bands and improve weight determinism. Both types of losses are backpropagated along with the classification error. The network automatically finds the optimal balance between "highest accuracy" and "fewest bands," resulting in a clear and interpretable weight distribution and further reducing computational cost during inference. This overcomes the shortcomings of simply minimizing classification error, which leads to redundant bands and poor model interpretability.

[0123] The effects of this invention will be further illustrated below with simulation experiments: This study was conducted in a Windows 11 environment using Python 3.12. The server hardware configuration used in the experiment was as follows: GPU: NVIDIA RTX 3090 24 GB; CPU: Intel Core i9-13900K; Memory: 128 GB DDR5. All simulation experiments were completed.

[0124] The experimental data used publicly available Salinas hyperspectral images. Acquired by the AVIRIS sensor in the Salinas Valley, California, these images possess fine spatial resolution and rich spectral dimensions, comprising 224 bands and covering 16 typical land cover categories, including crops, vegetation, and soil. The file format is .mat. Due to its abundant information and clear categories, this dataset is often used as a standard test set for hyperspectral image classification and object detection algorithms.

[0125] To verify the effectiveness of the algorithm, a hyperspectral image classification method based on light spectrum hybrid adaptive band selection (LSH-BC) was used. Three methods with good band selection and classification performance were also selected for comparison: Deep Joint Optical Spectral Band Selection and Classification Learning (DJ-OSBCL), Learning-Based Optimization of Hyperspectral Band Selection for Classification (LBO-HBSC), and LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Classification (LG-CAF). Pixel sets were classified on the experimental dataset, and the classification results are shown below. Figure 5 As shown, Figure 5 (a) is the actual label image. Figure 5 (b) is the LG-CAF classification diagram. Figure 5 (c) is the LBO-HBSC classification diagram. Figure 5 (d) is the DJ-OSBCL classification diagram. Figure 5 (e) is the LSH-BC classification diagram. Experimental results show that the LSH-BC method yields the best classification results.

[0126] Calculate the total accuracy OA, average accuracy AA, and classification accuracy for the 16 land cover classes using the formulas below. Plot all the calculation results in Table 1.

[0127]

[0128]

[0129]

[0130] Table 1. Classification results of the LSH-BC method and the contrast algorithm on the Salinas dataset. .

[0131] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, read-only optical discs, optical storage, etc.) containing computer-usable program code.

[0132] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A hyperspectral image classification method based on light spectrum hybrid adaptive band selection, characterized in that, Includes the following steps: A data cube sample set and label vector are constructed using the original hyperspectral image. Based on the data cube sample set and label vector, stratified random sampling is performed to divide the data cube sample set and the test set. The training set is input into a hyperspectral image classification model based on light spectrum hybrid adaptive band selection. After forward propagation, the joint optimization loss is calculated and backpropagation is performed to update the network parameters and band weight vector and record the optimal weight. The test set is input into the hyperspectral image classification model based on light spectrum mixing adaptive band selection. The model performs forward propagation using the optimal weights, outputs pixel-level class probabilities and determines the predicted labels through the convolution class head, and maps the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

2. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The process of constructing a data cube sample set and label vector using the original hyperspectral image is as follows: For each pixel coordinate of the original hyperspectral image, a data cube is constructed according to the data cube expression. By traversing all pixels of the original hyperspectral image, a data cube sample set and label vector are obtained. The data cube expression is as follows: c =( k -1) / 2 in, Representing a data cube, i Indicates the row index of the local window. j The column index representing the local window. b Indicates the spectral band index. x Represents the x-coordinate of pixels. y Represents the pixel ordinate. c Indicates the window radius. k Indicates the side length of the window; when x+i-c or y+j-c When the image exceeds the boundary, a mirror fill method is used to obtain the pixel values ​​at the symmetrical position.

3. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The method of dividing the data cube sample set and label vector into training and test sets by proportional stratified random sampling is as follows: Predefined category numbers serve as the global sequence numbers for samples extracted from each category, resulting in a set of category sample indices; For each category's sample index set, sample numbers are randomly selected proportionally to obtain the test sample index set for each category. The test sample index sets for each category are then merged to obtain the test set. The remaining sample indices in each category's sample index set are grouped into training sample index sets for each category. The training sample index sets for each category are then merged to obtain the training set.

4. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The formula for calculating the joint optimization loss is as follows: in, Indicates the joint optimization loss. Represents classification loss, This indicates band canonical loss.

5. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The weighting in the band-by-band weighting of each cube sample in the training or test set using learnable weight vectors is obtained using the following formula: in, Indicates the b Initial weights for each spectral band, Let represent any cube sample in the training set. This represents the weighted training set cube tensor. Represents any cube sample in the test set. This represents the weighted test set cube tensor. i Indicates the row index of the local window. j The column index representing the local window. b Indicates the spectral band index.

6. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The light spectral mixing structure, which combines local convolution and global transform, is used to extract spatial spectral features from the weighted cubic tensor to obtain a high-level spatial spectral tensor. Specifically: A 3D convolution is applied to each weighted cubic tensor to extract local convolutional features; The local convolutional features are divided into non-overlapping image blocks in the spatial dimension. Each block is expanded into a vector of uniform length in row-major order and stacked along the first dimension to form a token sequence. A lightweight multi-head self-attention-feedforward structure is applied to the token sequence to obtain the globally modeled token sequence. The token sequence after global modeling is reassembled back into the spatial grid to obtain a new tensor; The new tensor is concatenated with local convolutional features in the channel dimension, and a convolution kernel is applied for linear fusion to obtain the high-level spatial spectrum tensor. The dynamic calculation of spectral band importance weights, and the updating of the learnable band weight vector using these spectral band importance weights, specifically involves: The mid-level feature map of the high-level spatial spectrum tensor is selected as the query tensor, and the local convolutional features are used as the key tensor and value tensor. After flattening the query tensor, key tensor, and value tensor in spatial dimension, perform multi-head dot product attention on the query tensor and key tensor to obtain the cross-attention matrix. The average of the cross-attention matrix along the spatial dimension is used to obtain the sample-level band attention vector. The importance of a batch of bands is calculated by averaging the sample-level band importance vectors over a small batch. Based on the importance of batch bands, momentum is used to update and normalize the learnable band weights.

7. The hyperspectral image classification method based on light spectrum hybrid adaptive band selection according to claim 1, characterized in that, The step of outputting pixel-level class probabilities and determining predicted labels through the convolutional class header includes: A convolution kernel is applied to the high-level spatial spectrum tensor to obtain the class log score. Softmax is then applied to the class log score in the class dimension to obtain the probability vector of the cube. The probability vector of the center pixel of the cube is selected from the probability vector of the cube as the output pixel-level class probability and the predicted label is determined.

8. A hyperspectral image classification system based on light spectrum hybrid adaptive band selection, characterized in that, include: The dataset partitioning module is used to construct a data cube sample set and label vector using the hyperspectral image dataset. Based on the data cube sample set and label vector, stratified random sampling is performed to partition the dataset into training and test sets. The parameter and weight update module is used to input the training set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation, calculate the joint optimization loss and backpropagate, update the network parameters and band weight vector and record the optimal weights. The probability and predicted label output module is used to input the test set into the hyperspectral image classification model based on light spectrum mixing adaptive band selection, perform forward propagation using the optimal weights, output pixel-level class probabilities and determine the predicted labels through the convolution class head, and map the predicted labels to the original hyperspectral image to generate pixel-level hyperspectral classification images. The forward propagation includes: In the first layer of the data loader, a learnable weight vector is established for all spectral bands. The learnable weight vector is then used to perform band-by-band weighting on each cube sample in the training or test set to generate a weighted cube tensor. By utilizing a light spectral mixing structure combining local convolution and global transform, spatial spectral features are extracted from the weighted cubic tensor to obtain the high-level spatial spectral tensor. Cross-attention band selection is performed on the high-level spatial spectrum tensor, the importance weight of spectral bands is dynamically calculated, and the learnable band weight vector is updated using the importance weight of spectral bands.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the hyperspectral image classification method based on light spectrum mixing adaptive band selection as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the hyperspectral image classification method based on light spectrum hybrid adaptive band selection as described in any one of claims 1-7.

Images