A hyperspectral lidar collaborative classification method based on central spiral scanning

Abstract

The application belongs to the technical field of remote sensing image processing, and particularly relates to a hyperspectral lidar collaborative classification method based on central spiral scanning. The method comprises dimension reduction, normalization preprocessing of acquired hyperspectral and lidar images, and cutting to generate heterogeneous image block sample pairs; a double-branch feature extraction network extracts spatial-spectral cascaded features of hyperspectral and spatial elevation structure features of lidar by using four-path multi-scale central spiral space scanning and a state space model; single-mode features are subjected to feature enhancement and weighted optimization by a multi-scale feature enhancement module and a dual-domain adaptive attention module; the enhanced double-mode features are input into a cross-modal fusion module for channel splicing and dimension reduction interaction, and finally a collaborative classification result of ground objects is output by a classifier. The application greatly reduces the consumption of computing resources, maintains linear computational complexity, and realizes high-precision and high-robustness multi-modal remote sensing ground object classification in a complex ground object scene.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and remote sensing image processing technology, specifically to a collaborative classification method for hyperspectral lidar based on central spiral scanning. Background Technology

[0002] With the rapid development of remote sensing technology, multi-source remote sensing data has become a crucial foundation for Earth observation. As a core task of remote sensing applications, land use and land cover classification directly impacts key decision-making processes in environmental monitoring, resource management, and urban development. Among various remote sensing data sources, hyperspectral imagery and lidar are two key data sources. Hyperspectral imagery, due to its high-dimensional spectral data, exhibits powerful material identification capabilities. Lidar data can accurately measure the three-dimensional structure of terrain and objects, providing crucial elevation information.

[0003] Multi-source remote sensing data fusion is a key method for land cover classification. Current mainstream classification methods are mainly based on convolutional neural networks (CNNs) and Transformers. However, CNNs are limited by local receptive fields and have difficulty capturing long-distance dependencies; although Transformers have global modeling capabilities, the computational complexity of their self-attention mechanism increases quadratically with the sequence length, thus facing a huge computational burden and memory consumption.

[0004] State-space models (SSMs) offer linear computational complexity and long-range modeling capabilities, but traditional line-by-line or end-to-end scanning methods struggle to capture core features in remote sensing images and neglect the uniqueness and importance of the patch's center pixel as the target for classification, leading to the dilution of core features during sequence evolution. Mamba is primarily used for single-modal information processing; however, the same ground object exhibits significant differences across different sensors, and Mamba's ability to effectively process information across different modalities needs improvement. Furthermore, Mamba has limitations in local feature extraction and multi-scale information processing, lacking an efficient method for handling complex features and comprehensive representations in remote sensing data. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a collaborative classification method for hyperspectral lidar based on central spiral scanning. This method solves the problems of forgetting central target features due to the scanning method and overfitting due to hyperspectral noise caused by a lack of local detail perception in existing state-space model-based classification systems. Furthermore, this invention overcomes the high computational resource consumption of traditional Transformer architectures, enabling deep feature fusion and high-precision ground feature classification of hyperspectral and lidar data while maintaining computational efficiency.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides a collaborative classification method for hyperspectral lidar based on central spiral scanning, comprising the following steps: S1: Preprocess the hyperspectral image and lidar image of the same scene: perform dimensionality reduction on the hyperspectral data, and perform normalization on the dimensionality-reduced hyperspectral data and lidar data respectively to generate an image patch containing the center pixel and its neighborhood as input sample. S2: The image patch is fed into a dual-stream feature extraction network, which includes a hyperspectral feature extraction branch and a lidar feature extraction branch; For hyperspectral image patches, a spatial-spectral cascade approach is adopted. First, spatial features are extracted through four-path multi-scale central spiral spatial scanning and selective state-space model. Then, spectral features are extracted through sequential scanning and selective state-space model to generate hyperspectral spatial-spectral cascade features. For lidar image blocks, the four-path multi-scale central spiral spatial scanning and selective state-space model (SSM), which is exactly the same as the spatial feature extraction in the hyperspectral feature extraction branch, is used to extract the spatial elevation structure features of lidar. The generated hyperspectral spatial-spectral cascade features and lidar spatial elevation structure features are respectively enhanced by a post-multi-scale feature enhancement module to obtain hyperspectral enhanced features and lidar enhanced features. The post-multi-scale feature enhancement module first extracts multi-scale features through a parallel multi-scale convolutional fusion unit, and then enhances the features through a series of channel-spatial dual-domain attention units to output refined features. The four-path multi-scale central spiral spatial scan includes: First path and second path: Taking the geometric center of the image patch as the starting anchor point, the two-dimensional feature map is rearranged into a one-dimensional spiral sequence in clockwise and counterclockwise directions respectively, so that the central target pixel to be classified is located at the beginning of the sequence. After the sequence is processed by deep convolution, it is input into the selective state space model to capture the fine-grained spatial dependence that diffuses from the center to the periphery. The third and fourth paths: Spatial downsampling of the feature map is performed to expand the receptive field. Then, reverse scanning and spatial transpose reverse scanning are performed respectively and input into the selective state space model. The outputs of the third and fourth paths are upsampled through transpose convolution to restore the original resolution and are multi-scale stitched and fused with the output features of the first and second paths, and then mapped back to the two-dimensional space. S3: Input the output hyperspectral enhancement features and lidar enhancement features into the cross-modal fusion module, and generate the final multimodal fusion features through dimension alignment and interaction; S4: Input the multimodal fusion features into the classifier, calculate the probability of land cover categories, calculate the loss function in combination with sample labels and update the network parameters to complete the model training; input the test samples into the trained model and output the land cover classification results and classification accuracy index.

[0007] Preferably, the preprocessing in step S1 specifically includes: Principal component analysis was used to reduce the dimensionality of the original hyperspectral image, retaining 30 principal components. Min-Max normalization was performed on the dimensionality-reduced hyperspectral data and the single-channel lidar data. Using each pixel in the image as the geometric center, a sliding window technique was used to crop an image patch with a spatial size of 11×11 pixels as the input sample pair.

[0008] Preferably, step S2, extracting hyperspectral spatial-spectral cascade features, specifically includes: Spatial feature extraction stage: First, spatial features are extracted from the input hyperspectral image patch using a central spiral spatial module, with a size of [missing information]. ,in The number of spectral channels after PCA dimensionality reduction. This represents the number of spectral channels after PCA dimensionality reduction. Indicates the spatial size of an image block; Input hyperspectral features First, a 3D convolutional layer is used to process the data, simultaneously reducing the spectral and spatial dimensions and extracting preliminary features. The 3D convolution operation is represented as follows: CONV3D uses kernel size, , The layer index is represented with a stride of 1 and padding of 0; the output features are... ,in It is the number of output channels. , Batch normalization stabilizes the training dynamics, and ReLU activation introduces a nonlinear response. The convolutional features are first reshaped by merging the channel and spectral dimensions. The reshaped features are then processed by layer normalization and linear projection: , ,in Used for state-space modeling As a gating signal; Next, Transform to the spatial domain and process using depthwise separable convolution: ,in Before convolution Transpose DWConv uses The convolution kernel has a group size equal to the channel dimension. right Perform the four-path multi-scale central spiral spatial scan as described in claim 1: the original resolution features are processed through the spiral scan path, and the feature map is obtained from... Remodeling ,in Simultaneously, downsampling is generated for complementary scanning: , , The step size Depth-separable convolutions reduce spatial resolution to Flatten flattens the two-dimensional feature map into a one-dimensional sequence, Flip reverses the sequence dimension, and T represents the spatial dimension transpose, that is, the columns are generated first and then the rows to generate two downsampling scan paths in the forward and reverse directions; The four scan sequences are divided into two groups: Spiral scan including original resolution. Complementary scans including downsampling; two sets of sequences are input into a selective state-space model for long-range dependency modeling: , The dynamic parameters of the selective state-space model The input sequence is adaptively generated through linear projection; the output of the downsampled path is upsampled by transposed convolution to restore resolution, then aggregated with the output of the spiral path, and finally processed... Gated modulation yields spatial enhancement features ; Spectral feature extraction stage: Perform spectral sequence modeling: First, through The convolution kernel performs spatial neighborhood aggregation independently for each spectral channel: This incorporates local spatial context before spectral scanning; features are stabilized through layer normalization (LayerNorm) and mapped to feature dimensions via an input linear projection layer (Linear). Then, a one-dimensional depthwise separable convolution and SiLU activation function are introduced to nonlinearly enhance the sequence, maintaining spectral channel independence while strengthening fine-grained interactions between adjacent channels. Finally, the high-dimensional spectral channel sequence is grouped according to a set number of groups, and a forward sequence maintaining the original band order is independently constructed within each subgroup. and the reverse sequence after reversal These two sequences are input in parallel into a one-dimensional selective state-space model to capture long-range nonlinear dependencies between bands; finally, the bidirectional outputs are summed, normalized, and gated to produce spectral enhancement features. The hyperspectral spatial-spectral cascade features are obtained by fusing them with spatial enhancement features.

[0009] Preferably, when processing the input sequence, the state-space model uses the zero-order preservation technique to discretize the continuous time-scale parameters; the time-scale parameters, input matrix, and output matrix are all adaptively generated based on the current input features through a linear projection layer, and the state dimension of the model is set to 16.

[0010] Preferably, the extraction of spatial elevation structure features from the lidar in step S2 specifically includes: A two-dimensional convolutional layer is used to map single-channel LiDAR data to a high-dimensional feature space and initially aggregate the elevation information of the local neighborhood. Then, a four-path multi-scale central spiral spatial scanning strategy is applied to extract the global and local multi-scale structural features of the LiDAR data, ensuring strict alignment of heterogeneous modes in spatial perception logic.

[0011] Preferably, the processing of the post-multi-scale feature enhancement module in step S2 specifically includes: For input features Dimensional replacement and reshaping are carried out, among which For batch size, For the number of channels, This represents the number of spectral slices. and For spatial dimensions; by swapping the channel dimension and spectral dimension, and merging the batch dimension and spectral dimension, the 5D features are converted into 4D tensors suitable for 2D operations. ; For the reconstructed features Parallel branch networks are employed for multi-scale feature extraction and fusion: 3×3 convolutions (padding=1) are used to extract small-scale local details, while 5×5 convolutions (padding=2) are used to extract medium-scale contextual information; the two outputs are concatenated along the channel dimension, and then... Convolution dimensionality reduction to The channel is processed and nonlinearly mapped using batch normalization (BatchNorm) and the ReLU activation function; finally, a residual connection is introduced to connect the mapping result with the input features. Element-wise addition, outputting multi-scale corrected features. The calculation process can be expressed as follows: ;feature Then it enters the cascaded channel - spatial attention unit; Among them, channel attention units Spatial global average pooling and global max pooling are performed separately, followed by the introduction of multi-compression-ratio asymmetric MLP branches: average pooling features are input to the MLP branch with a compression ratio of 16:1, and max pooling features are input to the MLP branch with a compression ratio of 8:1; the outputs of the two branches are summed, and then channel adaptive weights are generated using the Sigmoid function, and then... Channel-by-channel multiplication to enhance output channel features ; Next to The average and maximum values ​​are calculated along the channel dimension and concatenated. This concatenation is then fed into a multi-kernel convolution branch, where spatial responses are extracted using convolutional kernels with three different receptive fields: 3×3, 7×7, and 11×11. The three responses are stacked along the channel dimension and then fused through a convolution to generate a single-channel spatial response map. Spatial adaptive weights are then generated using the Sigmoid function and applied to the target channel. At the corresponding position, output attention enhancement features ; Finally, the 4D features are enhanced in multiple dimensions. By re-unfolding and permuting the dimensions, the original 5D tensor structure is restored, ultimately outputting refined features with a high signal-to-noise ratio. .

[0012] Preferably, the cross-modal fusion in step S3 specifically includes: concatenating the enhanced hyperspectral refined features and lidar refined features in the channel dimension, and then using a 1×1 convolutional layer for dimensionality reduction and cross-channel interaction to generate joint multimodal features.

[0013] Preferably, in step S4, the joint multimodal features are flattened into a one-dimensional vector by global average pooling, then input into a fully connected layer, and the probability distribution of land cover categories is calculated using the Softmax activation function. The category corresponding to the maximum probability is used as the predicted label of the pixel.

[0014] (III) Beneficial Effects This invention provides a collaborative classification method for hyperspectral lidar based on central spiral scanning. It has the following beneficial effects: This invention effectively addresses the problem of core features being forgotten during long-sequence evolution in patch-based classification tasks by designing a central spiral scanning strategy, significantly improving classification accuracy. Simultaneously, this invention utilizes a cascaded spatial-spectral Mamba architecture and a post-processed multi-scale correction module to achieve complementarity from global long-range dependency modeling to fine-grained local detail recovery, effectively overcoming band redundancy in hyperspectral images and background noise interference from LiDAR. Compared to traditional locally constrained CNN-based models or computationally expensive Transformer-based models, this invention achieves high-precision and robust multimodal remote sensing land cover classification in complex terrain scenarios while significantly reducing computational resource consumption and maintaining linear computational complexity. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the overall framework of the hyperspectral lidar collaborative classification method based on central spiral scanning according to the present invention. Figure 2 This is a structural diagram of the four-path multi-scale collaborative scanning mechanism of the spatial state module in this invention; Figure 3 This is a structural diagram of the grouped bidirectional sequence scanning mechanism of the spectral state module in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1-3 The present invention provides a technical solution: such as Figure 1 As shown, feature extraction and land cover classification of multimodal remote sensing images are mainly completed in the following steps: The first step is multimodal remote sensing data preprocessing: To address the issue of high band redundancy in hyperspectral images, principal component analysis (PCA) is used to compress the original spectral dimensions to 30 principal components, preserving the main spectral information and removing noise. Subsequently, Min-Max normalization is performed on both the hyperspectral and lidar data. Finally, using each pixel in the image as the geometric center, a sliding window technique is used to crop an image patch of size 11×11 as the input sample.

[0018] The second step is optimal network training: A dual-stream feature extraction network architecture based on central spiral scanning and post-enhancement is constructed. This network integrates a hyperspectral image spatial-spectral cascade extraction module and a lidar isomorphic feature extraction module, with post-multi-scale correction and enhancement modules connected to the ends of each branch. The preprocessed training set is input into this network, and iterative training and parameter optimization are performed based on the Mamba algorithm to ultimately achieve high-precision land cover classification.

[0019] The specific steps for training the network structure module are as follows: (1) Constructing a central spiral scan: The input image patch size is Define the geometric center coordinates of the image patch as To achieve comprehensive spatial context awareness, two sets of one-dimensional index sequences are pre-constructed. The first set is a clockwise spiral index. From the anchor point Starting from the beginning, traverse layer by layer in a clockwise direction from the inside out, and the resulting coordinate sequence is denoted as... ,in This represents the total length of the sequence. The second group uses a counter-clockwise spiral index. From the anchor point Starting from the beginning, traverse layer by layer counterclockwise from the inside out. These two sets of indices are used to subsequently map the two-dimensional feature map into a one-dimensional sequence, ensuring that the beginning of the sequence always corresponds to the center pixel feature to be classified.

[0020] (2) Constructing a cascaded feature extraction module for the hyperspectral branch: First, spatial features are extracted from the input hyperspectral image patch using a central spiral spatial module, the size of which is... ,in The number of spectral channels after PCA dimensionality reduction. This represents the number of spectral channels after PCA dimensionality reduction. This represents the spatial size of the image patch. Input hyperspectral features. First, a 3D convolutional layer is used to process the data, simultaneously reducing spectral and spatial dimensions and extracting preliminary features. The 3D convolution operation is represented as follows: CONV3D uses kernel size, ( (Represents the layer index), with a step of 1 and padding of 0. The output is then: ,in It is the number of output channels. , Batch normalization (BN) stabilizes the training dynamics, while ReLU activation introduces a non-linear response. The convolutional features are first reshaped by merging the channel and spectral dimensions. The reshaped features are then subjected to layer normalization and linear projection. , ,in Used for state-space modeling As a gating signal. Next, Transform to the spatial domain and process using depthwise separable convolution: ,in Before convolution Transpose DWConv uses The convolution kernel has a group size equal to the channel dimension. Then, Perform multi-scale spatial scanning. The original resolution features are processed via a spiral scan path: , , ,in and Representing the clockwise and counterclockwise spiral indices pre-calculated in (1), respectively, the feature map from Remodeling ,in Simultaneously, downsampling is generated for complementary scanning: , , The step size Depth-separable convolutions reduce spatial resolution to Flatten flattens a two-dimensional feature map into a one-dimensional sequence, and Flip reverses the sequence along its dimensions. This indicates a spatial dimension transpose (column first, then row), generating two downsampling scan paths, one forward and one reverse.

[0021] like Figure 2 As shown, the four scan sequences are divided into two groups: Spiral scan including original resolution. Complementary scans including downsampling. Two sets of sequences are input into a selective state-space model for long-range dependency modeling: Similarly, this applies to downsampling paths: The dynamic parameters of the selective state-space model The input sequence is adaptively generated through linear projection. The output of the downsampled path is upsampled by transposed convolution to restore resolution, then aggregated with the output of the spiral path, and finally processed... Gated modulation yields spatial enhancement features .

[0022] Next to To perform spectral sequence modeling, firstly by... The convolution kernel performs spatial neighborhood aggregation independently for each spectral channel: This allows for the incorporation of local spatial context before spectral scanning.

[0023] like Figure 3 As shown, the features are stabilized through layer normalization (LayerNorm) and mapped to the feature dimensions through an input linear projection layer. Then, a one-dimensional depthwise separable convolution and the SiLU activation function are introduced to nonlinearly enhance the sequence, strengthening fine-grained interactions between adjacent channels while maintaining spectral channel independence. The high-dimensional spectral channel sequence is then grouped according to a predetermined number of groups, and a forward sequence maintaining the original band order is independently constructed within each subgroup. and the reverse sequence after reversal These two sequences are input in parallel into a one-dimensional SSM model to capture long-range nonlinear dependencies between bands. Finally, the bidirectional outputs are summed, normalized, and gated, resulting in enhanced spectral features. .

[0024] (3) Constructing a homogeneous feature extraction module for the lidar branch: This branch aims to extract the elevation geometry of ground features, employing a four-path multi-scale collaborative architecture identical to that of the hyperspectral branch's spatial extraction section. First, a two-dimensional convolutional layer is used to map single-channel lidar data to a high-dimensional feature space, and the elevation information of the local neighborhood is initially aggregated. Subsequently, the same four-path spatial scanning strategy as the hyperspectral branch is applied: on the one hand, a clockwise and counterclockwise central spiral scanning mechanism is used to capture fine elevation undulations centered on the target pixel; on the other hand, downsampling reverse and spatial transpose scanning are used to capture the global elevation geometry at the macro scale. After upsampling and fusion of the outputs of each path, the global and local multi-scale structural features of the lidar data are extracted, thereby ensuring strict alignment of heterogeneous modalities in spatial perception logic.

[0025] (4) Constructing a post-processing multi-scale correction and enhancement module: to facilitate independent processing using 2D convolution and attention mechanisms in the future. First, different spectral slices were analyzed. Dimensional replacement and reshaping are carried out, among which For batch size, For the number of channels, This represents the number of spectral slices. and For spatial dimensions, by swapping the channel dimension and spectral dimension, and merging the batch dimension and spectral dimension, the 5D features are converted into 4D tensors suitable for 2D operations. For the reconstructed features A parallel branch network is employed for multi-scale feature extraction and fusion: a 3×3 convolution (padding=1) is used to extract small-scale local details, while a 5×5 convolution (padding=2) is used to extract medium-scale contextual information. The two outputs are then concatenated along the channel dimension. Convolution dimensionality reduction to The channels are nonlinearly mapped using batch normalization (BatchNorm) and the ReLU activation function. Finally, residual connections are introduced to connect the mapping results to the input features. Element-wise addition, outputting multi-scale corrected features. The calculation process can be expressed as follows: .feature This is then followed by the cascaded channel-space attention units. The channel attention units... Spatial global average pooling and global max pooling are performed separately, followed by the introduction of multi-compression-ratio asymmetric MLP branches: average pooling features are input to an MLP branch with a compression ratio of 16:1, and max pooling features are input to an MLP branch with a compression ratio of 8:1. The outputs of the two branches are summed and then used to generate channel adaptive weights via a sigmoid function, which are then compared with... Channel-by-channel multiplication to enhance output channel features Then on The average and maximum values ​​are calculated along the channel dimension and concatenated. This concatenation is then fed into a multi-kernel convolution branch, where spatial responses are extracted using convolutional kernels with three different receptive fields: 3×3, 7×7, and 11×11. The three responses are stacked along the channel dimension and then fused through a convolution to generate a single-channel spatial response map. Spatial adaptive weights are then generated using the Sigmoid function and applied to the target channel. At the corresponding position, output attention enhancement features Finally, the 4D features, enhanced in multiple dimensions, will be... By re-unfolding and permuting the dimensions, the original 5D tensor structure is restored, ultimately outputting refined features with a high signal-to-noise ratio. This provides high-quality data input for subsequent cross-modal feature alignment and fusion.

[0026] The third step is cross-modal feature fusion and collaborative classification output: The test set samples are input into the optimal dual-stream feature extraction network trained in the second step. First, the enhanced hyperspectral refined features and lidar refined features are received, concatenated along the channel dimension, and then dimensionality reduction and cross-channel interaction are performed using a 1×1 convolutional layer to generate joint multimodal features. Subsequently, the joint multimodal features are flattened into a one-dimensional vector through global average pooling, and then input into a fully connected layer. The Softmax activation function is used to calculate the probability distribution of land cover categories, and the category corresponding to the maximum probability is used as the predicted label for that pixel. Finally, the overall accuracy (OA), average accuracy (AA), and Kappa coefficient of all test samples are calculated, and the classification effect map and evaluation index data are printed out in the display window, completing the land cover classification task for the target area.

[0027] Based on the above steps, the present invention provides a collaborative classification method for hyperspectral lidar based on central spiral scanning, the specific operation steps of which are as follows: S1: Preprocess the hyperspectral image and lidar image of the same scene, including dimensionality reduction of the hyperspectral data, normalization of the dual-modal data, and generating an image patch containing the center pixel and its neighborhood as input sample using the sliding window technique. S2: The image patch is fed into the dual-stream feature extraction network, and the spatial-spectral cascade features of hyperspectral data and the spatial elevation structure features of lidar are extracted by using four-path multi-scale central spiral spatial scanning and state space model respectively. The features are then enhanced by a post-multi-scale feature enhancement module. S3: Input the output hyperspectral enhancement features and lidar enhancement features into the cross-modal fusion module, and generate the final multimodal fusion features through dimension alignment and interaction; S4: Input the multimodal fusion features into the classifier to calculate the probability of land cover categories, combine the labels to calculate the loss function and update the network parameters, and finally print out the classification accuracy results of the test set in the display window.

[0028] In the optimization process of this invention, the network gradually learns the nonlinear dependencies between complex spatial textures and spectral semantics in multimodal data, masters the ability to represent global context with the central pixel as the core, and learns how to effectively suppress hyperspectral redundancy and lidar noise. This process can be seen as a remote sensing interpretation expert continuously strengthening its understanding of the microstructure and macroscopic material properties of ground objects. When the model finds the optimal network weights through continuous optimization, it essentially possesses high-precision multimodal ground object identification capabilities, enabling it to quickly and accurately classify remote sensing images in complex scenes.

[0029] To further verify the effectiveness and technical advantages of this invention, we conducted detailed comparative and ablation experiments on the publicly available remote sensing dataset Houston2013. This invention uses overall classification accuracy (OA), average classification accuracy (AA), and the Kappa coefficient as evaluation metrics.

[0030] (1) Comparative Experiment: The performance of the present invention was compared with that of existing mainstream remote sensing image classification algorithms. As shown in Table 1, under the same hardware and software testing environment, the method of the present invention achieved the best performance in the three core evaluation indicators of overall classification accuracy (OA), average classification accuracy (AA), and Kappa coefficient, which significantly improved the classification accuracy.

[0031] Table 1. Classification performance of different methods on the Houston 2013 dataset.

[0032] (2) Ablation Experiment: To verify the independent contribution of each core module in this invention, an ablation experiment was conducted based on the baseline model (Module A: Central Helical Spatial Scan, Module B: Spectral State Module, Module C: Multi-scale Correction and Enhancement Module). The experimental results are shown in Table 2. The overall classification accuracy of the baseline model was only 90.90%. After introducing each module, the performance was significantly improved: After introducing Module A, the OA increased to 92.59%, proving the effectiveness of the four-path multi-scale scanning; after introducing Module B, the OA increased to 91.61%, indicating the effectiveness of the grouped bidirectional sequence scanning mechanism; after introducing Module C, the OA reached 93.64%, proving the key role of this module in suppressing hyperspectral redundancy and restoring edge details.

[0033] Table 2 Comparison of ablation experimental performance of core modules

[0034] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A collaborative classification method for hyperspectral lidar based on central spiral scanning, characterized in that: Includes the following steps: S1: Preprocess the hyperspectral image and lidar image of the same scene: perform dimensionality reduction on the hyperspectral data, and perform normalization on the dimensionality-reduced hyperspectral data and lidar data respectively to generate an image patch containing the center pixel and its neighborhood as input sample. S2: The image patch is fed into a dual-stream feature extraction network, which includes a hyperspectral feature extraction branch and a lidar feature extraction branch; For hyperspectral image patches, a spatial-spectral cascade approach is adopted. First, spatial features are extracted through four-path multi-scale central spiral spatial scanning and selective state-space model. Then, spectral features are extracted through sequential scanning and selective state-space model to generate hyperspectral spatial-spectral cascade features. For lidar image blocks, the four-path multi-scale central spiral spatial scanning and selective state-space model (SSM), which is exactly the same as the spatial feature extraction in the hyperspectral feature extraction branch, is used to extract the spatial elevation structure features of lidar. The generated hyperspectral spatial-spectral cascade features and lidar spatial elevation structure features are respectively enhanced by a post-multi-scale feature enhancement module to obtain hyperspectral enhanced features and lidar enhanced features. The post-multi-scale feature enhancement module first extracts multi-scale features through a parallel multi-scale convolutional fusion unit, and then enhances the features through a series of channel-spatial dual-domain attention units to output refined features. The four-path multi-scale central spiral spatial scan includes: First path and second path: Taking the geometric center of the image patch as the starting anchor point, the two-dimensional feature map is rearranged into a one-dimensional spiral sequence in clockwise and counterclockwise directions respectively, so that the central target pixel to be classified is located at the beginning of the sequence. After the sequence is processed by deep convolution, it is input into the selective state space model to capture the fine-grained spatial dependence that diffuses from the center to the periphery. The third and fourth paths: Spatial downsampling of the feature map is performed to expand the receptive field. Then, reverse scanning and spatial transpose reverse scanning are performed respectively and input into the selective state space model. The outputs of the third and fourth paths are upsampled through transpose convolution to restore the original resolution and are multi-scale stitched and fused with the output features of the first and second paths, and then mapped back to the two-dimensional space. S3: Input the output hyperspectral enhancement features and lidar enhancement features into the cross-modal fusion module, and generate the final multimodal fusion features through dimension alignment and interaction; S4: Input the multimodal fusion features into the classifier, calculate the probability of land cover categories, calculate the loss function in combination with sample labels and update the network parameters to complete the model training; input the test samples into the trained model and output the land cover classification results and classification accuracy index.

2. The hyperspectral lidar collaborative classification method based on central spiral scanning according to claim 1, characterized in that: The preprocessing in step S1 specifically includes: Principal component analysis was used to reduce the dimensionality of the original hyperspectral image, retaining 30 principal components. Min-Max normalization was performed on the dimensionality-reduced hyperspectral data and the single-channel lidar data. Using each pixel in the image as the geometric center, a sliding window technique was used to crop an image patch with a spatial size of 11×11 pixels as the input sample pair.

3. The hyperspectral lidar collaborative classification method based on central spiral scanning according to claim 1, characterized in that: Step S2, which extracts hyperspectral spatial-spectral cascade features, specifically includes: Spatial feature extraction stage: First, spatial features are extracted from the input hyperspectral image patch using a central spiral spatial module, with a size of [missing information]. ,in The number of spectral channels after PCA dimensionality reduction. This represents the number of spectral channels after PCA dimensionality reduction. Indicates the spatial size of an image block; Input hyperspectral features First, a 3D convolutional layer is used to process the data, simultaneously reducing the spectral and spatial dimensions and extracting preliminary features. The 3D convolution operation is represented as follows: CONV3D uses kernel size, , The layer index is represented with a stride of 1 and padding of 0; the output features are... ,in It is the number of output channels. , Batch normalization stabilizes the training dynamics, and ReLU activation introduces a nonlinear response. The convolutional features are first reshaped by merging the channel and spectral dimensions. The reshaped features are then processed by layer normalization and linear projection: , ,in Used for state-space modeling As a gating signal; Next, Transform to the spatial domain and process using depthwise separable convolution: ,in Before convolution Transpose DWConv uses The convolution kernel has a group size equal to the channel dimension. right Perform the four-path multi-scale central spiral spatial scan as described in claim 1: the original resolution features are processed through the spiral scan path, and the feature map is obtained from... Remodeling ,in Simultaneously, downsampling is generated for complementary scanning: , , The step size Depth-separable convolutions reduce spatial resolution to Flatten flattens the two-dimensional feature map into a one-dimensional sequence, Flip reverses the sequence dimension, and T represents the spatial dimension transpose, that is, the columns are generated first and then the rows to generate two downsampling scan paths in the forward and reverse directions; The four scan sequences are divided into two groups: Spiral scan including original resolution. Complementary scans including downsampling; two sets of sequences are input into a selective state-space model for long-range dependency modeling: , The dynamic parameters of the selective state-space model The input sequence is adaptively generated through linear projection; the output of the downsampled path is upsampled by transposed convolution to restore resolution, then aggregated with the output of the spiral path, and finally processed... Gated modulation yields spatial enhancement features ; Spectral feature extraction stage: Perform spectral sequence modeling: First, through The convolution kernel performs spatial neighborhood aggregation independently for each spectral channel: This incorporates local spatial context before spectral scanning; features are stabilized through layer normalization (LayerNorm) and mapped to feature dimensions via an input linear projection layer (Linear). Then, a one-dimensional depthwise separable convolution and SiLU activation function are introduced to nonlinearly enhance the sequence, maintaining spectral channel independence while strengthening fine-grained interactions between adjacent channels. Finally, the high-dimensional spectral channel sequence is grouped according to a set number of groups, and a forward sequence maintaining the original band order is independently constructed within each subgroup. and the reverse sequence after reversal These two sequences are input in parallel into a one-dimensional selective state-space model to capture long-range nonlinear dependencies between bands; finally, the bidirectional outputs are summed, normalized, and gated to produce spectral enhancement features. The hyperspectral spatial-spectral cascade features are obtained by fusing them with spatial enhancement features.

4. A hyperspectral lidar cooperative classification method based on central spiral scanning according to claim 1 or 3, characterized in that: When processing the input sequence, the state-space model uses the zero-order preservation technique to discretize the continuous time-scale parameters; the time-scale parameters, input matrix and output matrix are all adaptively generated based on the current input features through a linear projection layer, and the state dimension of the model is set to 16.

5. The hyperspectral lidar collaborative classification method based on central spiral scanning according to claim 1, characterized in that: Step S2, which extracts the spatial elevation structure features of the lidar, specifically includes: A two-dimensional convolutional layer is used to map single-channel LiDAR data to a high-dimensional feature space and initially aggregate the elevation information of the local neighborhood. Then, a four-path multi-scale central spiral spatial scanning strategy is applied to extract the global and local multi-scale structural features of the LiDAR data, ensuring strict alignment of heterogeneous modes in spatial perception logic.

6. The hyperspectral lidar collaborative classification method based on central spiral scanning according to claim 1, characterized in that: The processing in step S2 of the post-multi-scale feature enhancement module specifically includes: For input features Dimensional replacement and reshaping are carried out, among which For batch size, For the number of channels, This represents the number of spectral slices. and For spatial dimensions; by swapping the channel dimension and spectral dimension, and merging the batch dimension and spectral dimension, the 5D features are converted into 4D tensors suitable for 2D operations. ; For the reconstructed features Parallel branch networks are employed for multi-scale feature extraction and fusion: 3×3 convolutions (padding=1) are used to extract small-scale local details, while 5×5 convolutions (padding=2) are used to extract medium-scale contextual information; the two outputs are concatenated along the channel dimension, and then... Convolution dimensionality reduction to The channel is processed and nonlinearly mapped using batch normalization (BatchNorm) and the ReLU activation function; finally, a residual connection is introduced to connect the mapping result with the input features. Element-wise addition, outputting multi-scale corrected features. The calculation process can be expressed as follows: ;feature Then it enters the cascaded channel - spatial attention unit; Among them, channel attention units Spatial global average pooling and global max pooling are performed separately, followed by the introduction of multi-compression-ratio asymmetric MLP branches: average pooling features are input to the MLP branch with a compression ratio of 16:1, and max pooling features are input to the MLP branch with a compression ratio of 8:1; the outputs of the two branches are summed, and then channel adaptive weights are generated using the Sigmoid function, and then... Channel-by-channel multiplication to enhance output channel features ; Next to The average and maximum values ​​are calculated along the channel dimension and concatenated. This concatenation is then fed into a multi-kernel convolution branch, where spatial responses are extracted using convolutional kernels with three different receptive fields: 3×3, 7×7, and 11×11. The three responses are stacked along the channel dimension and then fused through a convolution to generate a single-channel spatial response map. Spatial adaptive weights are then generated using the Sigmoid function and applied to the target channel. At the corresponding position, output attention enhancement features ; Finally, the 4D features are enhanced in multiple dimensions. By re-unfolding and permuting dimensions, the original 5D tensor structure is restored, ultimately outputting refined features with a high signal-to-noise ratio. .

7. The hyperspectral lidar cooperative classification method based on central spiral scanning according to claim 1, characterized in that: The cross-modal fusion in step S3 specifically includes: concatenating the enhanced hyperspectral refined features and lidar refined features in the channel dimension, and then using a 1×1 convolutional layer for dimensionality reduction and cross-channel interaction to generate joint multimodal features.

8. The hyperspectral lidar cooperative classification method based on central spiral scanning according to claim 1, characterized in that: In step S4, the joint multimodal features are flattened into a one-dimensional vector by global average pooling, then input into a fully connected layer, and the probability distribution of land cover categories is calculated using the Softmax activation function. The category corresponding to the maximum probability is used as the predicted label of the pixel.

Images