Hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention

Abstract

The application discloses a hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention, relates to the technical field of hyperspectral band selection, and comprises the following steps: acquiring a hyperspectral image dataset and a LiDAR dataset of a target object; constructing a pre-trained band selection network, taking the hyperspectral image dataset and the LiDAR dataset of the target object as inputs of the pre-trained band selection network, and acquiring attention weights of each band of the hyperspectral image; performing descending order sorting according to the weights of each band of the hyperspectral image, and selecting the first N hyperspectral bands in the sorting result as a band subset. Through intelligent guidance of LiDAR features and deep refinement of a StarG module, the application can adaptively and intelligently filter out a band subset with the most discriminative power and information quantity from original HSI data, thereby effectively eliminating redundant and noise information, and being beneficial to subsequent hyperspectral image processing tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral band selection, and particularly relates to a hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention. BACKGROUND

[0002] Remote sensing technology plays a key role in high-precision ground object classification, environmental monitoring and other fields. In particular, hyperspectral images (HSI) provide fine spectral response curves of ground objects in hundreds of continuous narrow bands, are known as "fingerprint maps", and have strong potential to identify subtle differences in ground objects. However, the "curse of dimensionality" problem of hyperspectral data has always been a core challenge in its application. Hundreds of bands not only result in massive data, increasing storage and computing burden, but more importantly, the high correlation between adjacent bands causes serious information redundancy, while introducing noise, making the classifier prone to overfitting under limited samples, greatly limiting the efficiency and accuracy of hyperspectral data analysis.

[0003] To address the "curse of dimensionality", band selection technology has emerged, aiming to intelligently select a subset of the most representative, discriminative and least redundant bands from the original hyperspectral bands, thereby reducing data dimensionality, improving computational efficiency, and improving model classification performance. Traditional band selection methods, whether based on statistical features, information theory or optimization algorithms, are mostly independent of the final application task (such as classification), and often rely on time-consuming human experience or heuristic search, making it difficult to adaptively capture the best band combination required for different ground object classes in complex scenes. More importantly, these methods are usually limited to hyperspectral data itself and fail to fully utilize the external guiding information that other auxiliary data sources may bring.

[0004] At the same time, with the development of remote sensing technology, multi-modal data fusion has become a trend. LiDAR (Light Detection and Ranging) data provides high-precision elevation and vertical structure details with its unique three-dimensional structure information, complementing the spectral information of hyperspectral data. However, existing technologies still have significant deficiencies in effectively integrating heterogeneous multi-modal data such as LiDAR into the band selection process in order to achieve more intelligent and robust band selection. Simple feature-level fusion cannot achieve deep-level collaborative guidance between modalities, especially how to allow LiDAR's structural information to directly and dynamically affect and optimize the weight distribution and selection of hyperspectral bands, which remains a difficult problem to be solved. In addition, existing methods may also lack fine modeling of the internal spatial context information of features and stability constraints on the band weight learning process when performing band selection and feature fusion, thereby affecting the final performance and generalization ability.

[0005] In summary, the current hyperspectral remote sensing field faces key bottlenecks in efficient, intelligent and robust band selection, especially when combined with multi-modal data for collaborative optimization, there is a lack of innovative methods that can fully utilize the complementary nature of heterogeneous information to achieve data-driven adaptive band optimization. SUMMARY

[0006] In view of the above shortcomings in the prior art, the hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention provided by the present application solves the problems that the prior art is difficult to adaptively capture the optimal band combination required by different ground object categories in complex scenes, and is difficult to realize deep-level collaborative guidance between modalities.

[0007] In order to achieve the above invention purpose, the technical scheme adopted by the present application is: a hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention, comprising:

[0008] Obtain a hyperspectral image dataset and a LiDAR dataset of a target object;

[0009] Construct a pre-trained band selection network, input the hyperspectral image dataset and the LiDAR dataset of the target object into the pre-trained band selection network as inputs, and obtain attention weights of each band of the hyperspectral image;

[0010] According to the weights of each band of the hyperspectral image, descending order sorting is performed, and the first N hyperspectral bands in the sorting result are selected as a band subset.

[0011] The present application has the following advantages:

[0012] 1. By using the structure and height information provided by the LiDAR data, through the attention mechanism in the LiDAR guided StarG band selection and feature weight fusion module, the weights of the hyperspectral bands are directly calculated and guided. This enables the model to learn more discriminative spectral features from complementary modalities, thereby selecting a band subset with more information and less redundancy, and improving the accuracy of band selection.

[0013] 2. Through the bidirectional cross-modal attention interaction module, bidirectional information flow from LiDAR features to hyperspectral features and from hyperspectral features to LiDAR features is realized. This bidirectional interaction enables the two modalities to learn and enhance each other, capturing more rich joint spatial-spectral features, overcoming the limitations of single modalities or single-direction fusion, and further improving the final classification performance.

[0014] 3. By deeply fusing LiDAR and HSI data, and taking LiDAR information as the core, the optimal hyperspectral band subset is intelligently and dynamically selected, and the selected features are enhanced, so as to significantly improve the accuracy and efficiency of remote sensing ground object classification. Attached Figure Description

[0015] Figure 1 A flowchart of a hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention is provided for an embodiment.

[0016] Figure 2 A schematic diagram of the network structure for band selection;

[0017] Figure 3 for Module structural diagram;

[0018] Figure 4 A schematic diagram of the LiDAR-guided StarG band selection and feature enhancement module structure;

[0019] Figure 5 This is a schematic diagram of the StarG module structure;

[0020] Figure 6 The example shows the changing trends of the OA, AA, and Kappa coefficients on the Houston 2013 dataset when the model selects different numbers of bands. Detailed Implementation

[0021] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0022] like Figure 1 As shown, in one embodiment of the present invention, a hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention includes the following steps:

[0023] S1. Obtain the hyperspectral image dataset and LiDAR dataset of the target object.

[0024] The data used in this embodiment comes from the hyperspectral remote sensing dataset used in the IEEE GRSS 2013 data fusion competition, covering the University of Houston and the surrounding area. The data contains 144 spectral bands (380-1050nm), a spatial resolution of 2.5 meters, an image size of 349×1905 pixels, and labels 15 types of ground objects (such as grassland, roads, buildings, etc.).

[0025] S2, construct a pre-trained band selection network, input the hyperspectral image dataset and the LiDAR dataset of the target object into the pre-trained band selection network to obtain attention weights of each band of the hyperspectral image.

[0026] As shown in Figure 2 , the band selection network comprises a modal feature embedding module, a bidirectional cross-modal attention interaction module, a LiDAR guided StarG band selection and feature enhancement module, a multi-scale feature extraction module and a linear classifier connected in sequence.

[0027] The input of the modal feature embedding module is the hyperspectral image dataset and the LiDAR dataset of the target object, the patches of the LiDAR and the hyperspectral image are embedded into the respective patch embedding modules, the patches are projected into a unified embedding space through a 3D convolution layer, and layer normalization is performed, and the embedded LiDAR feature sequence (LiDAR embedded feature) and the embedded hyperspectral feature sequence (hyperspectral embedded feature) are output.

[0028] The bidirectional cross-modal attention interaction module comprises a parallel module and a module; as shown in Figure 3 , the module comprises a first input layer, a first normalization layer, a first linear layer, an attention calculation layer, an activation function layer, a feature output layer, a linear mapping layer, a first residual layer and a first output layer connected in sequence; The module further comprises a second input layer, a second normalization layer and a second linear layer connected in sequence; wherein the output of the second linear layer is connected to the input of the attention calculation layer; The module further comprises a third linear layer; wherein the input of the third linear layer is the output of the second normalization layer, and the output of the third linear layer is connected to the input of the attention calculation layer;

[0029] The structure of the module is the same as that of the module; The input of the first input layer of the module is the embedded LiDAR feature sequence, The input of the second input layer of the module is the embedded hyperspectral feature sequence; The output of the module is the LiDAR guided hyperspectral feature;

[0030] The output of the For example, first, the input LiDAR and HIS (hyperspectral) features are normalized by independent LayerNorm layers to unify the numerical distribution of different modal features and improve training stability; then, the LiDAR features are mapped to query vectors Q through a linear projection layer, while the HSI features are mapped to key vectors K and value vectors V through two independent linear layers; this process ensures that the three vectors required for attention mechanism calculation come from two different modalities but are in a unified vector space;

[0031] In the attention calculation stage, the module adopts the typical Scaled Dot-Product Attention form to measure the similarity between LiDAR and HSI features, and its calculation formula is:

[0032]

[0033] where, represents the similarity matrix of query and key, is the dimension of the key vector for scaling, is an optional spatial position bias term to introduce spatial structure information, under this mechanism, each LiDAR query position will dynamically focus on the position of all HSI features to learn their most relevant hyperspectral response patterns;

[0034] Subsequently, the calculated attention weights are normalized by the Softmax function and used for weighted summation of the Value vector, thereby generating the fused output feature representation. The output is then adjusted in shape by a linear mapping layer (Linear & Reshape) operation, and added to the original feature through a residual connection to enhance the information transmission capability of the model and improve the stability of training deep networks; finally, the module outputs two parts of information: one is the fused cross-modal feature representation, which can be used for downstream tasks; the other is the attention map (Attention Map), which explicitly reflects the semantic association between LiDAR and HSI positions;

[0035] Similarly, The module is the same as the above, taking the hyperspectral embedding feature as the query (Q), and the LiDAR embedding feature as the key (K) and value (V), calculating the attention of hyperspectral to LiDAR, and generating hyperspectral guided LiDAR features Each cross-attention module includes linear projection of query, key, and value, layer normalization, output projection, residual connection, and position bias to improve stability.

[0036] For example, Figure 4As shown, the LiDAR-guided StarG band selection and feature enhancement module includes a spatial attention unit, a first global average pooling layer, a multi-scale band attention unit, a first weighted sum layer, a StarG module, a second global average pooling layer, a second residual layer, a fusion network, and a third residual layer connected in sequence; wherein the input of the spatial attention unit is also the input of the third residual layer; the output of the first global average pooling layer is connected to the input of the second residual layer; the input of the spatial attention unit is hyperspectral-guided LiDAR features; the input of the first weighted sum layer also includes LiDAR-guided hyperspectral features (HIS features); and the output of the LiDAR-guided StarG band selection and feature enhancement module is enhanced LiDAR features.

[0037] The LiDAR-guided StarG band selection and feature enhancement module takes as input LiDAR features and hyperspectral features that have undergone cross-attention interaction; the spatial importance information is extracted through a spatial attention module, and then global context information is generated through global average pooling; then, the global LiDAR features are sent into a multi-scale band attention module as a guide signal, which processes the input features through three different scale MLPs (multi-layer perceptron), extracts band features at different scales, and generates three attention weights (weight 1, weight 2, and weight 3) respectively; then, these weights are fused by weighting and input into a fusion MLP to generate a fused band attention weight, which will be used for subsequent weighted processing of HSI features, thereby achieving attention degree adjustment at the band level; In this process, the module also introduces a band consistency loss to constrain the smoothness of the band weight in the training process to enhance the robustness and stability of band selection; then, the generated band weight is multiplied element by element with the original HSI feature to achieve dynamic weighting and selection of HSI bands by LiDAR, obtaining a weighted HSI feature, which will be input into the StarG module for deep refinement; in another parallel branch, the LiDAR features are directly processed through a fusion network, which deep-fuses the LiDAR features with the HSI global features obtained through the global average pooling after the StarG module processing; finally, the output of the fusion network is residual-connected (through an addition operation) with the original LiDAR features to generate the final enhanced LiDAR features.

[0038] The multi-scale band attention unit in the LiDAR guided StarG band selection and feature enhancement module comprises three parallel MLP layers of different scales and a fusion MLP layer; the band features of different scales in the output of the first global average pooling layer are extracted through the three parallel MLP layers of different scales, and the three attention weights are fused after weighting; the fused weights are taken as the input of the fusion MLP layer to obtain the fusion band attention weight as the output of the multi-scale band attention unit.

[0039] In addition, the StarG module comprises an input feature layer, a first bottleneck convolution layer, a second bottleneck convolution layer, a second weighted summation layer, a fourth bottleneck convolution layer and a fourth residual layer connected in sequence; the StarG module further comprises a third bottleneck convolution layer and a Psi gating unit connected in sequence; wherein the outputs of the input feature layer are connected to the inputs of the third bottleneck convolution layer and the fourth residual layer respectively; the output of the Psi gating unit is connected to the input of the second weighted summation layer.

[0040] As shown in Figure 5 , the StarG module adopts a double-branch parallel structure; branch 1 first performs continuous feature extraction and transformation on the input feature through the first bottleneck convolution layer and the second bottleneck convolution layer, while branch 2 parallelly inputs the input feature into the third bottleneck convolution layer to independently learn the features from the input feature and serve as the basis features for gating control; then, the features are sent to the Psi gating unit to calculate the gating weights; in the Psi gating unit, a gating map (i.e. gating feature) between 0 and 1 is usually generated through a series of activation functions (such as 1x1 convolution, normalization layer and Sigmoid activation function) to measure the importance of each channel or each spatial position; further, the gating feature generated by the Psi gating unit is multiplied element by element with the output of the second bottleneck convolution layer in the path of branch 1, and the features after the gating operation are further refined through the fourth bottleneck convolution layer; finally, the output of the fourth bottleneck convolution layer is fused with the original input feature through residual connection (element by element addition), which helps to alleviate the gradient vanishing problem and allows the network to learn incremental improvement of the input feature, thereby generating enhanced output features.

[0041] The fusion band attention weight and the enhanced LiDAR feature obtained are linearly transformed and nonlinearly activated by the multi-scale feature extraction module to obtain depth feature enhancement; the depth feature enhancement is normalized and subjected to Dropout processing to extract classification ; the extracted classification is classified by a linear classifier to make classification prediction and output a classification result.

[0042] The training process of the band selection network is specifically as follows:

[0043] Load hyperspectral image data, LiDAR data and ground truth labels, and extract local regions of size 9x9 from hyperspectral and LiDAR data as samples, and by randomly shuffling the coordinates of the ground truth labels, ensure the randomness and diversity of the training samples, and avoid sample overlap between the training set and the test set; thereafter, the input data is preprocessed, including data augmentation, data normalization, label processing and data division and the like; obtain a hyperspectral image and LiDAR training sample subset;

[0044] The training sample subset after data augmentation is normalized and the dimension is adjusted to obtain a preprocessed training sample subset;

[0045] The preprocessed training sample subset is input into the band selection network for class prediction until the loss function converges, and a pre-trained band selection network is obtained.

[0046] To enhance the feature expression ability and network generalization ability in the band selection process, the present application introduces a multi-task learning mechanism, in addition to the main task of band selection, adds a spectral feature reconstruction task and a modal consistency constraint as an auxiliary target. Specifically, the network needs to minimize the classification loss, the reconstruction loss and the modal consistency loss at the same time, so as to ensure that the band subset retains important information while having explainability and stability. The expression of the joint loss is:

[0047]

[0048] Among them, denotes the joint loss, denotes the main classification loss, denotes the reconstruction loss, denotes the modal consistency loss; denotes the i-th real value, denotes the i-th predicted value; denotes the weight parameter of the reconstruction loss; denotes the weight coefficient of the KL divergence loss; denotes the reconstructed complete hyperspectral image; denotes the decoder; denotes the band selection mask, used for weighted selection of hyperspectral bands; denotes the original hyperspectral image input; denotes the square of the Euclidean norm; denotes the KL divergence; denotes the feature representation of the hyperspectral image; denotes the feature representation of the LiDAR data; denotes "relative to" or "between", used for comparison of two distributions in the KL divergence.

[0049] The model parameter is optimized by using an AdamW optimizer, and weight decay is set; a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate; mixed precision training is used to improve training efficiency and reduce memory consumption; and an early stopping mechanism is used to monitor model convergence according to the accuracy of the verification set, to prevent overfitting.

[0050] In view of the problems of high model complexity and high computing resource consumption in the hyperspectral band selection task, the application provides a knowledge distillation mechanism: first, a teacher model with the best performance is trained, then the band weight information is migrated to a lightweight student model through a distillation loss, so that the latter can maintain high classification performance while significantly reducing the computing cost; the mechanism is particularly suitable for resource-constrained environments such as mobile platforms and edge computing; the process is summarized as follows:

[0051] A pre-trained Teacher model (namely a pre-trained band selection network) is used to provide band selection guidance, and then the band weight information is migrated to a lightweight student model (the student model is a band selection network that only needs to output band weights) through a distillation loss, so that the latter can maintain high classification performance while significantly reducing the computing cost, and the Student model outputs the band weights; when training the Student model, the distillation loss and the main classification loss are combined to optimize the Student model, and the expression is:

[0052]

[0053]

[0054] wherein, L denotes the loss of the Student model, L denotes the main classification loss of the Student model, is a weight coefficient, is a distillation loss, denotes a band selection mask generated by the Teacher model; denotes a band selection mask generated by the Student model.

[0055] S3, according to the weights of each band of the hyperspectral image, performing descending order sorting, and selecting the first N hyperspectral bands in the sorting result as a band subset.

[0056] The band weight value of each sample is obtained through forward propagation of the model. The band weight values ​​of all test samples are collected, their average value is calculated, and all hyperspectral bands are sorted in descending order according to the average band weight. The top N bands (e.g., N=5) are selected as the final band subset. These bands have the highest attention scores, indicating that they contain more information and have less redundancy. The selected band indices and corresponding attention values ​​are recorded and saved.

[0057] The band selection network employs a category-aware mechanism to dynamically adjust the generation strategy of band attention weights based on category error; the implementation steps of the category-aware mechanism are as follows:

[0058] A new auxiliary category control branch is added, which introduces category prediction error information and defines a category sensitivity factor; the category sensitivity factor is used as the category importance weight to guide the update of the band attention weight.

[0059] The expression for the category sensitivity factor is:

[0060]

[0061] In the formula, Indicates category sensitivity factor, The number of samples in category C. Cross-entropy loss;

[0062] The expression for band attention weights is:

[0063]

[0064] In the formula, Indicates the first j Attention weights for each band, Indicates category , Indicates the first j Attention weights for each band to category C.

[0065] S4. Test the classification performance of the selected band subset.

[0066] A new dataset is constructed using a selected subset of bands to test the performance of the band subset on the classification task and to verify the effectiveness of the selected bands.

[0067] To further verify the superiority of the present invention, the method of the present invention was compared with a number of representative hyperspectral classification / fusion algorithms on the Houston 2013 dataset, including: SVM, CNN-HSI, CoupledCNN, FusAtNet, CCL and S2ENet. All comparison algorithms were run under their respective optimal settings, and the results are shown in Table 1.

[0068] As can be seen from Table 1, the method of the present application is significantly better than all the comparative algorithms in various evaluation indicators. Specifically, the overall accuracy (OA), average accuracy (AA) and Kappa coefficient of the present method are 0.9570, 0.9643 and 0.9534 respectively. Compared with S2ENet, the present method improves the OA by about 1.53 percentage points and the Kappa coefficient by about 1.67 percentage points, which fully proves that the LiDAR guided mechanism and the bidirectional cross-modal attention mechanism proposed in the present method can effectively fuse heterogeneous data and accurately select key bands, thereby greatly improving the performance of hyperspectral image classification.

[0069] Table 1

[0070]

[0071] In addition, in order to explore the effectiveness of the band selection mechanism in the proposed method, we systematically experiment the influence of different numbers of selected bands (N) on the classification performance of the model. Figure 6 The OA, AA and Kappa coefficient trends when the model selects different numbers of bands on the Houston 2013 dataset are shown.

[0072] As Figure 6 shown, with the increase of the number of selected bands N, the classification accuracy of the model presents a trend of first rapid increase, then tends to be stable, and finally slightly decreases. When the number of bands is small (for example, N is less than 15), the classification accuracy of the model is relatively low, which indicates that only a small number of bands is not enough to fully capture the spectral difference information of the ground object class; with the gradual increase of the number of bands, the model can integrate more effective spectral information, and the classification accuracy rapidly increases and enters a performance platform period when N reaches about 15 bands, which indicates that most of the key spectral features have been captured at this time; it is worth noting that when the number of selected bands is 45, the present application achieves the best classification performance, with an OA of 0.9570, an AA of 0.9643 and a Kappa coefficient of 0.9534, and when the number of bands is further increased to 50, the accuracy indicators appear a slight decrease.

[0073] This phenomenon just verifies the effectiveness and necessity of the LiDAR guided band selection mechanism in the present application. It shows that not all bands in the original hyperspectral data are beneficial to the classification task. In some cases, too many bands may introduce spectral redundancy, noise interference, and even increase the complexity of the model and the risk of overfitting.

[0074] To sum up, through the intelligent guidance of LiDAR features and the depth refinement of StarG module, the most discriminative and informative subset of bands can be adaptively and intelligently filtered from the original HSI data, so as to effectively eliminate redundant and noise information. Therefore, the selection of 45 optimal bands not only enables the model to achieve the best classification accuracy, but also highlights the unique advantage of the application in efficient utilization of spectral information.

Claims

1. A hyperspectral band selection method based on LiDAR guidance and bidirectional cross-modal attention, characterized in that, include: Obtain the hyperspectral image dataset and LiDAR dataset of the target object; A pre-trained band selection network is constructed, and the hyperspectral image dataset and LiDAR dataset of the target object are used as inputs to the pre-trained band selection network to obtain the attention weights of each band of the hyperspectral image. The hyperspectral images are sorted in descending order based on the weights of each band, and the top N hyperspectral bands in the sorting results are selected as a band subset. The band selection network consists of a modal feature embedding module, a bidirectional cross-modal attention interaction module, a LiDAR-guided StarG band selection and feature enhancement module, a multi-scale feature extraction module, and a linear classifier, which are connected in sequence. The input to the modal feature embedding module is the hyperspectral image dataset and the LiDAR dataset of the target object, and the output of the modal feature embedding module is the embedded LiDAR feature sequence and the embedded hyperspectral feature sequence. The LiDAR-guided StarG band selection and feature enhancement module comprises, in sequence, a spatial attention unit, a first global average pooling layer, a multi-scale band attention unit, a first weighted summation layer, a StarG module, a second global average pooling layer, a second residual layer, a fusion network, and a third residual layer. The input to the spatial attention unit also serves as the input to the third residual layer. The output of the first global average pooling layer is connected to the input of the second residual layer. The input to the spatial attention unit is hyperspectral-guided LiDAR features. The input to the first weighted summation layer also includes LiDAR-guided hyperspectral features. The output of the LiDAR-guided StarG band selection and feature enhancement module is enhanced LiDAR features. The StarG module includes an input feature layer, a first bottleneck convolutional layer, a second bottleneck convolutional layer, a second weighted summation layer, a fourth bottleneck convolutional layer, and a fourth residual layer connected in sequence. The StarG module also includes a third bottleneck convolutional layer and a Psi gated unit connected in sequence. The output of the input feature layer is connected to the input of the third bottleneck convolutional layer and the fourth residual layer, respectively. The output of the Psi gated unit is connected to the input of the second weighted summation layer. The Psi gate unit is used to calculate the gate weights and generate gated features; the Psi gate unit includes a 1×1 convolutional layer, a normalization layer and a sigmoid activation function connected in sequence.

2. The method according to claim 1, characterized in that, The bidirectional cross-modal attention interaction module includes parallel... Modules and Module; in, The module includes a first input layer, a first normalization layer, a first linear layer, an attention calculation layer, an activation function layer, a feature output layer, a linear mapping layer, a first residual layer, and a first output layer connected in sequence. The module also includes a second input layer, a second normalization layer, and a second linear layer connected in sequence; wherein the output of the second linear layer is connected to the input of the attention calculation layer. The module also includes a third linear layer; wherein the input of the third linear layer is the output of the second normalization layer, and the output of the third linear layer is connected to the input of the attention calculation layer; Modules and The modules have the same structure; The input to the first input layer of the module is the embedded LiDAR feature sequence. The input to the second input layer of the module is the embedded hyperspectral feature sequence; The inputs of the two input layers of the module and The modules are the opposite; The module outputs LiDAR-guided hyperspectral features; The module outputs hyperspectral guided LiDAR features.

3. The method according to claim 1, characterized in that, The multi-scale band attention unit consists of three parallel MLP layers of different scales and a fused MLP layer. Band features at different scales are extracted from the output of the first global average pooling layer through the three parallel MLP layers of different scales, and three attention weights are generated and then weighted and fused. The fused weights are used as the input of the fused MLP layer to obtain the fused band attention weights, which are then used as the output of the multi-scale band attention unit.

4. The method according to claim 3, characterized in that, The output of the multi-scale band attention unit is linearly transformed and nonlinearly activated by the multi-scale feature extraction module to obtain deep feature enhancement. The deep feature enhancement is then normalized and Dropout processed to extract classification data. ; Based on the extracted classification using a linear classifier Perform classification prediction and output the classification results.

5. The method according to claim 4, characterized in that, The training process of the band selection network is as follows: Obtain the raw hyperspectral image dataset and LiDAR dataset of the target object; Extracting pixels from the original hyperspectral image dataset and the LiDAR dataset of size [number missing] From the neighboring region, hyperspectral images and a subset of LiDAR training samples were obtained. Indicates the side length of the adjacent region; Data augmentation is performed on the subset of training samples; The augmented training sample subset is normalized and its dimensions are adjusted to obtain the preprocessed training sample subset. The pre-processed training sample subset is input into the band selection network for category prediction until the loss function converges, thus obtaining the pre-trained band selection network. The band selection network is optimized by minimizing the joint loss. The expression for the joint loss is: in, Indicates joint loss, Indicates the primary classification loss, Represents reconstruction loss, This represents the modal consistency loss; Indicates the first i A true value, Indicates the first i One predicted value; The weight parameters represent the reconstruction loss; The weighting coefficients represent the KL divergence loss. This represents the reconstructed complete hyperspectral image; Indicates decoder; This represents the band selection mask, used for weighted selection of hyperspectral bands; This represents the original hyperspectral image as input; Represents the square of the Euclidean norm; Indicates KL divergence; Feature representation of hyperspectral images; Characteristic representation of LiDAR data; Indicates "relative to" or "between", used for comparing two distributions in KL divergence.

6. The method according to claim 5, characterized in that, In the hyperspectral band selection process, a pre-trained Teacher model is used to provide band selection guidance, while a Student model is used to output band weights. When training the Student model, the distillation loss and main classification loss are combined to optimize the Student model; its expression is as follows: in, This represents the loss of the Student model. This represents the main classification loss of the Student model. These are the weighting coefficients. For distillation loss, This represents the band selection mask generated by the Teacher model; This represents the band selection mask generated by the student model.

7. The method according to claim 5, characterized in that, The band selection network employs a category-aware mechanism to dynamically adjust the generation strategy of band attention weights based on category error; the implementation steps of the category-aware mechanism are as follows: A new auxiliary category control branch is added, which incorporates category prediction error information and defines a category sensitivity factor. The category sensitivity factor is used as the category importance weight to guide the update of the band attention weight; The expression for the category sensitivity factor is: In the formula, Indicates category sensitivity factor, The number of samples in category C. Cross-entropy loss; The expression for band attention weights is: In the formula, Indicates the first j Attention weights for each band, Indicate category , Indicates the first j Attention weights for each band to category C.

Images