A hyperspectral image classification method, system, electronic device and storage medium

Abstract

This invention belongs to the field of computer vision and image processing, specifically providing a hyperspectral image classification method, system, electronic device, and storage medium. The method includes: preprocessing hyperspectral images of the same type of land cover to obtain multiple hyperspectral image patches; constructing a Grassman principal margin hyperspectral image classification model for open scenes; and classifying the hyperspectral images to obtain classification detection results for known or unknown categories. The Grassman principal margin hyperspectral image classification model in this invention introduces the perspective of manifold geometry into the HSI classification scenario under open sets for the first time. Through manifold structure analysis and geodesic distance measurement, it accurately distinguishes between known and unknown categories, providing a novel solution to the HSI classification problem in open scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing, and relates to a hyperspectral image classification method, system, electronic device and storage medium, specifically to an open scene hyperspectral image classification method, system, electronic device and storage medium based on Grassmann principal angle margin. Background Technology

[0002] Hyperspectral images (HSI) contain rich spectral and spatial information, playing an irreplaceable role in precision agriculture, urban planning, mineral exploration, and environmental monitoring. Early hyperspectral image classification (HSIC) algorithms primarily focused on addressing the high dimensionality and nonlinearity of spectral features. They typically transformed highly disordered spectral features into a structured feature space and then used linear methods for classification. While these methods possess comprehensive theoretical interpretability, their assumptions were overly idealistic, making it difficult to generate reliable classification results. In recent years, the development of deep learning has provided new possibilities for hyperspectral image classification. Deep learning techniques, represented by Convolutional Neural Networks (CNNs) and Transformers, have achieved great success in HSIC tasks. CNN-based methods effectively extract texture and detail features using local receptive fields. The Transformer mechanism exhibits superior characteristics in capturing long-range correlations. However, most existing hyperspectral classification models are based on the strict closed-set assumption, meaning that all land cover categories appearing during the testing phase are labeled and visible during the training phase. However, in real-world applications, hyperspectral image classification models often face the challenge of the open world, meaning they must handle samples of unknown categories outside the training set distribution. Because traditional models often follow the closed-set assumption, they tend to mistakenly classify these unknown samples as predefined known categories with high confidence. This overfitting to a known distribution not only limits the model's generalization ability in complex remote sensing environments but can also produce misleading results, thus affecting subsequent decision-making.

[0003] To address this challenge, researchers have recently proposed some preliminary solutions for open-scene hyperspectral image classification (OS-HSIC). Probabilistic methods primarily mine the confidence response of the network output. For example, inspired by extremum theory, some studies have abandoned the traditional SoftMax and instead used the OpenMax function to fit the Weibull distribution of known classes, thereby calibrating the threshold of boundary samples. Reconstruction-based methods, based on the prior assumption that unknown samples deviate from the known observation distribution, utilize generative architectures such as spatial-spectral latent representations to filter unknown targets by evaluating the abnormally high reconstruction error. In contrast, prototypal metric-based methods are favored due to their explicit geometric meaning. These methods typically compress known classes into mean vectors in the feature space and explicitly amplify the distinction between known and unknown classes using specific metrics (such as Euclidean distance). Although these methods alleviate the overconfidence problem to some extent, they still face two major challenges when dealing with the highly challenging feature distribution of HSI: ① Euclidean Metrics Limitations Due to Manifold Topology Disruption: Existing OS-HSIC feature extraction paradigms are generally constrained by the traditional Euclidean metric assumption. However, from the perspective of the physical mechanism of hyperspectral imaging, due to interference from factors such as illumination conditions and terrain undulations, the spectral features of similar land features do not exhibit an ideal Gaussian distribution in high-dimensional space, but are embedded within nonlinear manifolds. Under this complex topological structure, feature coordinates that are close in Euclidean space may have significantly different geodesic distances along the real manifold surface. This disruption of topological connectivity causes existing models to be unable to separate overlapping manifold boundaries when characterizing fine-grained land features with extremely weak inter-class differences, thus leading to feature confusion.

[0004] ② Open-space risk and lack of geometric rejection boundaries: Existing open-set classifiers typically model known classes as single isolated centroids in the feature space. Due to intra-class differences in HSI spectra within the same class, this single-point representation makes the decision boundary very loose. When faced with a large number of unknown class samples, the model lacks sufficient ability to distinguish them, resulting in low accuracy in identifying unknown classes. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an open-scene hyperspectral image classification method, system, electronic device, and storage medium based on the Grassmann Principal Margin Hyperspectral Image Classification Model (GPAM-Net). It is the first to introduce nonlinear manifold geometry and Grassmann subspace theory into the open-set hyperspectral classification task. Furthermore, to overcome the limitations of traditional Euclidean metrics, a decoupled dual-stream geodesic transformer (DDGT) module is designed.

[0006] This invention provides a hyperspectral image classification method, comprising the following steps: Step 1: Acquire hyperspectral images of the same type of land cover and preprocess the hyperspectral images to obtain multiple hyperspectral image patches after PCA dimensionality reduction. ,in, This represents the number of spectral bands after dimensionality reduction. For the height of the image, The width of the image; A Grassman principal angle margin hyperspectral image classification model for open scenarios is constructed. The Grassman principal angle margin hyperspectral image classification model includes a 3D-2D spatial-spectral joint feature extractor, a dual-stream geodesic attention module, and a Grassman principal angle margin framework. Step 2: Extract each hyperspectral image patch The input is fed into a 3D-2D spatial-spectral joint feature extractor to obtain the feature sequence. ; Step 3: Extract the feature sequence The input is fed into the dual-stream geodesic attention module to obtain multiple feature samples. Hyperspectral image classification results; Step 4: Extract each feature sample The input is fed into the Grassman principal angle margin framework to detect whether the current feature sample belongs to a known category or an unknown category.

[0007] Furthermore, the specific process of step two is as follows: Step 2.1: Apply 3D convolution kernels to each hyperspectral image patch. Feature extraction is performed to obtain a 3D feature tensor; Step 2.2: Reshape the extracted 3D feature tensor into a 2D feature tensor, and map the channels of the 2D feature tensor to a fixed feature manifold dimension using a 2D convolution kernel. The above yields local spatial-spectral joint features. ; Step 2.3: Combine local spatial and spectral features Flattening the spatial dimensions to form a feature sequence ;in, The number of space tokens.

[0008] Furthermore, 3D feature tensors The expression is as follows: ; in, The weight tensor of the 3D convolution kernel has its receptive field size set to (7,3,3).

[0009] Furthermore, local spatial-spectral joint features The expression is as follows: ; in, For the weight tensor of the 2D convolution kernel, To reshape.

[0010] Furthermore, the specific process of step three is as follows: Step 3.1: The dual-stream geodesic attention module receives the feature sequence. For the feature sequence After normalization, the query matrix is ​​obtained through linear projection. Key matrix Sum matrix ; Step 3.2: Query matrix Key matrix Sum matrix The output is fed into the Euclidean topological flow branch and has spatial smoothness. ; Step 3.3: Query matrix Key matrix Sum matrix The output is fed into the manifold characteristic flow branch to obtain a nonlinear manifold-dependent output. ; Step 3.4: Output with spatial smoothing characteristics and nonlinear manifold dependent output Multi-head adaptive fusion is performed at the output feature level of the dual-stream geodesic attention module to obtain the fused output. ; Step 3.5, Fusion Output through The data is then input into the MLP classification head for classification, including multiple feature samples. Hyperspectral image classification results.

[0011] Furthermore, the specific process of step 3.2 is as follows: Step 3.2.1: Convert the value matrix Reconstruct it into a 2D tensor and introduce a 3×3 depthwise separable convolution to extract local features. Then, through multi-head adaptive gating parameters Injecting local spatial information yields the processed value matrix. ; Step 3.2.1: Analyze the query matrix. Bond matrix Perform separately Normalization process yields the normalized query matrix. and normalized bond matrix ; Normalize the query matrix Normalized bond matrix and the processed value matrix Multiplying yields an output with spatial smoothness. .

[0012] Furthermore, the specific process of step 3.3 is as follows: Step 3.3.1: Analyze the query matrix. Bond matrix Perform separately Normalization is performed to obtain the normalized query matrix. and normalized bond matrix ; Step 3.3.2: Normalize the query matrix. and normalized bond matrix After spherical geodesic transformation, the sum-value matrix Performing the product yields a nonlinear manifold-dependent output. .

[0013] Furthermore, the normalized query matrix and normalized bond matrix The method for spherical geodesic conversion is as follows: In the unit hypersphere The normalized query matrix is ​​captured using geodesic distance. and normalized bond matrix The global manifold dependency feature.

[0014] Furthermore, the Grassman principal angle margin framework consists of three progressive mechanisms connected in sequence: Grassman subspace modeling, principal angle margin loss, and subspace orthogonal discretization constraint.

[0015] Furthermore, the specific process of step four is as follows: Step 4.1: Construct the Grassman subspace: With feature space For the bottom space; feature space Modeled as a Grassmanifold one above A linear subspace of dimension , and the basis matrix of this subspace is . ,in, It is any one of the known categories, and ; Step 4.2: Calculate feature samples In all Projection length on each known category subspace ; Step 4.3: Define the detection score for the unknown category as... The cutoff threshold is ;like If it is an unknown category, then it is determined to be an unknown category; if If so, it is determined to be a known category.

[0016] As a further aspect of the present invention, the present invention also provides a hyperspectral image classification system for implementing the hyperspectral image classification method as described above, comprising a data acquisition module, a 3D-2D spatial-spectral joint feature extractor, a dual-stream geodesic attention module, and a Grassmann principal angle margin framework connected in sequence. The data acquisition module is used to acquire hyperspectral images of the same type of land cover and preprocess the acquired hyperspectral images to obtain multiple hyperspectral image patches after PCA dimensionality reduction. The 3D-2D spatial-spectral joint feature extractor is used to extract feature sequences from hyperspectral image patches; The dual-stream geodesic attention module achieves a balance between smoothing of structural features and preservation of features of unknown features by constructing Euclidean topological flow and manifold feature flow in parallel. The Grassman principal angle margin framework is used to map samples belonging to known land cover categories to Grassman manifolds. An orthogonal subspace is constructed on the , and an independent subspace is built for each known category. Feature samples are then calculated. The principal angle margin loss of each category subspace enables classification tasks for known categories and detection tasks for unknown categories.

[0017] As a further aspect of the present invention, the present invention also provides an electronic device, characterized in that it includes a memory, one or more processing units, and one or more programs stored in the memory, said one or more programs including instructions for performing the hyperspectral image classification method as described above.

[0018] As a further aspect of the present invention, the present invention also provides a storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the hyperspectral image classification method as described above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention provides a hyperspectral image classification method for open scenes using the Grassman principal margin hyperspectral image classification model (GPAM-Net). The Grassman principal margin hyperspectral image classification model introduces the perspective of manifold geometry into the HSI classification scenario under open sets for the first time. Through manifold structure analysis and geodesic distance measurement, it accurately distinguishes between known and unknown categories, providing a brand-new solution for the HSI classification problem in open scenes.

[0020] (2) This invention, by designing a decoupled Dual-Stream Geodesic Transformer (DDGT) module, abandons simple spatial smoothing. It constructs "Euclidean topological flow" and "geodesic manifold feature flow" in parallel and adaptively decouples them under a multi-head mechanism. While preserving the spectral features of unknown classes, it brings known similar features closer along the nonlinear manifold surface. Furthermore, to accurately distinguish unknown categories in the feature space, the Grassmannian Principal Angle Margin Framework (GPAM) is proposed. This framework abandons single-point representation and models each known category as a high-dimensional orthogonal subspace on the Grassmannian manifold. By introducing a strict subspace orthogonality penalty and principal angle margin, different known categories are forced to be geometrically mutually exclusive, thereby achieving an "angular vacuum zone" between subspaces. During the testing phase, any feature that falls into this region and is almost orthogonal to all known categories will be determined as an unknown category.

[0021] (3) This invention proposes a novel OS-HSIC geometric perspective framework (GPAM-Net), which breaks the limitation of traditional models relying on Euclidean space probability thresholds. For the first time, it provides a physically interpretable solution to the problem of open set recognition of hyperspectral images from the dimensions of manifold topology and non-Euclidean geometric subspace projection.

[0022] (4) The present invention designs a decoupled dual-stream geodesic transformer (DDGT), which perfectly resolves the contradiction between smoothing of continuous known class space and specific preservation of isolated unknown class by introducing Taylor approximation of geodesic distance and feature-level multi-head adaptive gating, and significantly improves the discrimination power for complex similar ground objects.

[0023] (5) This invention constructs the Grassman principal angle margin decision mechanism (GPAM), which innovatively represents the known class as a multi-dimensional linear subspace. By applying margin constraints and cross-class orthogonal penalties on the cosine principal angle, it actively constructs an angle vacuum region that excludes unknown classes in the deep feature space. It has achieved breakthroughs in the classification accuracy of known classes and the recognition accuracy of unknown classes on the Houston2013, WhuHi-HongHu, and WhuHi-HanChuan datasets.

[0024] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0025] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating a hyperspectral image classification method according to an embodiment of the present invention; Figure 2(a) is Figure 1 A schematic diagram of the overall architecture of the Glassman principal angle margin hyperspectral image classification model for open scenarios. Figure 2(b) is Figure 1 A schematic diagram of the Grassman principal margin framework for a Grassman principal margin hyperspectral image classification model oriented towards open scenes. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of the present invention clearer and easier to understand, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the accompanying drawings of the present invention are all in a simplified form and use non-precise proportions, and are only used to facilitate and clearly illustrate the implementation of the present invention.

[0027] Example 1: See Figure 1 As shown in Figures 2(a) and 2(b), the hyperspectral image classification method provided by the present invention includes the following steps: Step 1: Acquire hyperspectral images of the same type of land cover and preprocess the hyperspectral images to obtain multiple hyperspectral image patches after PCA dimensionality reduction. ,in, The number of spectral bands after dimensionality reduction (specifically, in this embodiment, the number of spectral bands after dimensionality reduction). Set to 30). For the height of the image, The width of the image; A Grassmann Principal Angle Margin hyperspectral image classification model (GPAM-Net) for open scenes is constructed. The Grassmann Principal Angle Margin hyperspectral image classification model includes a 3D-2D spatial-spectral joint feature extractor (Hybrid 3-Dimensional and 2-Dimensional Spatial-Spectral Tokenizer), a decoupled dual-stream geodesic attention (DDGT) module, and a Grassmannian Principal Angle Margin Framework (GPAM). Step 2: Extract each hyperspectral image patch The input is fed into a 3D-2D spatial-spectral joint feature extractor to obtain the feature sequence. ; Step 3: Extract the feature sequence The input is fed into the dual-stream geodesic attention module to obtain multiple feature samples. Hyperspectral image classification results; Step 4: Extract each feature sample The input is fed into the Grassman principal angle margin framework to detect whether the current feature sample belongs to a known category or an unknown category.

[0028] As a further solution in this embodiment, based on the core characteristics of hyperspectral images (their continuous spectral bands contain unique physical absorption and reflection properties of ground objects), in open set classification tasks, GPAM-Net not only needs to identify known classes but also needs to maintain high sensitivity to unknown anomalous ground objects. Therefore, a feature extraction mechanism that can ensure spectral physical continuity, effectively integrate local spatial topology, and is computationally efficient is needed to provide high-quality underlying features for subsequent manifold construction. Thus, this invention designs a 3D-2D spatial-spectral joint feature extractor and uses it to extract hyperspectral image patches after PCA dimensionality reduction. Feature extraction is performed to obtain local spatial-spectral joint features; these features are then flattened in space to form a feature sequence. .

[0029] Preferably, the specific process of step two is as follows: Step 2.1: Apply 3D convolution kernels to the hyperspectral image patches after PCA dimensionality reduction. Feature extraction is performed to obtain a 3D feature tensor; Step 2.2: To adapt to the sequence input requirements of Transformer and reduce computational complexity, the extracted 3D feature tensor is reshaped into a 2D feature tensor, and the channels of the 2D feature tensor are mapped to a fixed feature manifold dimension using a 2D convolution kernel. The above yields local spatial-spectral joint features. ; Step 2.3: Combine local spatial and spectral features Flattening the spatial dimensions to form a feature sequence Attention is directed to the dual-flow geodesic; among which, The number of space tokens.

[0030] Further optimized, 3D feature tensor The expression is as follows: ; in, The weight tensor of the 3D convolution kernel has a receptive field size of (7,3,3); (7,3,3) means that a larger kernel size of 7 is used in the spectral dimension to capture continuous spectral features, while a 3×3 size is used in the spatial dimension to preserve local spatial information.

[0031] Further preferred local spatial-spectral joint features The expression is as follows: ; in, For the weight tensor of the 2D convolution kernel, To reshape.

[0032] As a further solution in this embodiment, in order to break the contradiction between spatial smoothing and spectral feature preservation, this invention proposes a dual-flow geodesic attention (DDGT) module based on multi-head decoupling. By constructing Euclidean topological flow and manifold feature flow in parallel, a balance is achieved between smoothing of structural landforms and preservation of features of unknown landforms.

[0033] Preferably, the specific process of step three is as follows: Step 3.1: The dual-stream geodesic attention module receives the feature sequence. The query matrix is ​​obtained through linear projection. Key matrix Sum matrix ; Step 3.2: Query matrix Key matrix Sum matrix The output is fed into the Euclidean topological flow branch and has spatial smoothness. ; The specific process is as follows: Step 3.2.1: Convert the value matrix It is reduced to a two-dimensional tensor, and a 3×3 depthwise separable convolution is introduced to extract local features. Then, through multi-head adaptive gating parameters Injecting local spatial information yields the processed value matrix. ; Step 3.2.1: Analyze the query matrix. Bond matrix Perform separately Normalization is performed to obtain the normalized query matrix. and normalized bond matrix ; Normalize the query matrix Normalized bond matrix and the processed value matrix Multiplying yields an output with spatial smoothness. ; Step 3.3: Query matrix Key matrix Sum matrix The output is fed into the manifold characteristic flow branch to obtain a nonlinear manifold-dependent output. ; The specific process is as follows: Step 3.3.1: Analyze the query matrix. Bond matrix Perform separately Normalization is performed to obtain the normalized query matrix. and normalized bond matrix ; Step 3.3.2: Normalize the query matrix. and normalized bond matrix After spherical geodesic transformation, the sum-value matrix Performing the product yields a nonlinear manifold-dependent output. ; Step 3.4: Output with spatial smoothing characteristics and nonlinear manifold dependent output Multi-head adaptive fusion is performed at the output feature level of the dual-stream geodesic attention module; and multi-head learnable fusion weights are introduced. To coordinate the information from the two branches and obtain a fused output. ; Step 3.5, Fusion Output through The data is then input into the MLP classification header for classification, and the classification results are obtained.

[0034] This decoupled design allows the GPAM-Net model to flexibly adjust the preferences of each head: some heads enhance the spatial topology of continuous features through Euclidean flow, while other heads maintain accurate detection of unknown class samples through manifold flow, thereby enhancing the ability to distinguish unknown classes while maintaining the accuracy of known classes.

[0035] Further preferred, the Euclidean topological flow branch aims to enhance the identification ability of features with continuous structures (such as roads) by utilizing linear topology and local spatial neighborhood information in Euclidean space. The manifold feature flow branch focuses on capturing nonlinear manifold dependencies and ensuring the spectral purity of unknown pixels.

[0036] Further preferred output with spatial smoothing characteristics The expression is as follows: ; ; ; in, The basic attention weights for branches of Euclidean topological flow. This represents the feature dimension of each attention head. The value is enhanced by local spatial information. This is the transpose operator.

[0037] Further preferred, nonlinear manifold dependent output The expression is as follows: ; ; in, The attention weights for the characteristic flow branches of the manifold. is the manifold temperature factor.

[0038] Further optimized, fused output The expression is as follows: ; in, This is the Sigmoid activation function.

[0039] Further optimization involves normalizing the query matrix. and normalized bond matrix The method for spherical geodesic conversion is as follows: In the unit hypersphere The normalized query matrix is ​​captured using geodesic distance. and normalized bond matrix The global manifold dependency feature.

[0040] As a further solution in this embodiment, existing HSIC methods for identifying open sets typically use a classification model based on closed sets to obtain predicted probabilities, and then use a threshold to distinguish unknown classes. However, this method only represents the class as an isolated weight vector in Euclidean space, which makes the classification model prone to overfitting in cases with few samples. Furthermore, the overly tight decision boundary cannot leave sufficient exclusive space for unknown classes, leading to unseen features being easily forced into the nearest known class. To completely overcome this geometric limitation, this invention proposes a GPAM framework, which innovatively upgrades the classification paradigm from traditional "point-to-point" Euclidean distance matching to "point-to-subspace" manifold projection. The core design concept of this framework is to allocate an independent subspace with a certain dimension to each known class, thereby accommodating spectral variations of similar features and alleviating overfitting in cases with few samples. Simultaneously, the geometric topological relationships of these subspaces are strictly constrained throughout the entire space, thus constructing an angular vacuum region that is extremely exclusive to unknown classes. Based on the GPAM framework, feature samples... The specific method for detection is as follows: samples belonging to known land cover categories are mapped to Grassman manifolds. We construct orthogonal subspaces on the matrix, and then construct subspaces for all known categories, by calculating feature samples. The principal angle margin loss of each category subspace enables classification tasks for known categories and detection tasks for unknown categories.

[0041] Preferably, the theoretical derivation and implementation of the GPAM framework consists of three progressive mechanisms: Grassman subspace modeling, principal angle margin loss, and subspace orthogonal discretization constraint.

[0042] The following is a further preferred method for constructing the Grassmann subspace: feature space Consider it as a base space; , feature space Modeled as a Grassmanifold one above 3D linear subspace (This invention sets the subspace dimension) ); Let the basis matrix of this subspace be... ,in, It is any one of the known categories, and ; For those already conducted Normalized feature samples Its relationship with known categories The similarity of the corresponding Grassman subspace is determined by the feature samples. To the principal angle of that subspace. If determined, then the feature sample belongs to a known category. The projected length of the corresponding Grassman subspace That is: ; in, Basis matrix The first in Row vectors; Let be the dimension of the subspace.

[0043] Spectral variations of similar land features can be found in this Free movement within the dimensional subspace ensures generalization, while unknown features are likely orthogonal to all known subspaces to ensure exclusivity.

[0044] Further optimization involves calculating feature samples. The specific process of the principal angle margin loss with each category subspace is as follows: To further push open the angular vacuum region between known subspaces to expose unknown classes, we assume feature samples... The true known labels are In the real category An angle margin is forcibly introduced on the principal angle. ; Due to direct calculation This can easily lead to gradient explosion, so in the implementation, trigonometric identities are used to avoid it, resulting in: ; ; ; in, The scaling factor is used to control the smoothness of the probability distribution. This represents the index of known categories other than the actual labels. For the current sample and the first angular distance between subspaces To add angular margin to the target category similarity, This is the loss of the Grassmann principal angle margin; To prevent overlapping of subspaces of different classes from leaving redundant representation spaces for unknown classes, a global orthogonal constraint is applied to all basis matrices. Let To concatenate all known classes The global basis matrix after the orthogonality penalty term Defined as: ;

[0045] in, It is the identity matrix. It is the Frobenius norm.

[0046] Therefore, the total joint training loss of the GPAM-Net model is .

[0047] A further preferred open set identification strategy based on Grassmann orthogonal projection is as follows: Thanks to the rigorous geometric modeling from the open set perspective described above, the boundary delineation between known and unknown categories during the testing and inference phases is clearly defined. Since unknown category samples do not participate in the construction of any Grassman subspace, based on the geometric properties of high-dimensional space, they are highly likely to be approximately orthogonal to all known subspaces. During the inference and testing phase, for any input hyperspectral test sample, the feature samples... and calculate its value in all Projection length on each known category subspace Since the unknown class samples do not participate in the construction of any Grassman subspace during training, according to the geometric properties of high-dimensional space, this feature must necessarily be approximately orthogonal to all known subspaces with a high probability (i.e., ...). Therefore, this feature sample is defined. Detection score for: ; Detection score The higher the value, the greater the deviation of the sample from the known manifold system. During evaluation, a cutoff threshold is given. ;like Then the feature sample Samples classified as unknown categories; if If so, it is determined to be a known category.

[0048] As a further aspect of this embodiment, in order to fully verify the robustness of the proposed method in open scenarios with different spatial resolutions, different spectral coverages, and different land cover complexities, this invention selected three widely used public hyperspectral benchmark datasets for experiments.

[0049] To strictly simulate the open set environment, the last three categories in the labeled categories of each dataset are uniformly set as "Unknown" categories, which are not visible during the training phase, while the remaining categories are set as "Known" categories.

[0050] 1) Houston 2013 Dataset: This dataset was collected by the ITRES CASI-1500 hyperspectral imager over the University of Houston campus and surrounding area. Its spatial resolution is 2.5m, and it contains... Each pixel. The raw data contains 144 bands in the spectral range of 380nm to 1050nm. The dataset is labeled with 15 typical urban land features. Under the open set setting of this invention, the first 12 categories are used as known classes for network training, while the last 3 categories (i.e., Crosswalks, Major thoroughfares, Highways) are classified as unknown classes to test the GPAM-Net model's ability to reject unseen man-made land features.

[0051] 2) WHU-Hi-HongHu Dataset: This dataset was acquired using a Headwall Nano-Hyperspec sensor mounted on a drone in a complex agricultural area of ​​Honghu City, Hubei Province, China. It boasts extremely high spatial resolution (0.043 m) and image size... The dataset contains 274 spectral bands within a wavelength range of 400nm to 1000nm. This region encompasses up to 22 fine-grained crop and land cover categories. In the experiments of this invention, the first 19 crop categories were designated as known classes, while the last three categories (i.e., Bare soil, Water, and Unknown cover) were designated as unknown classes. The dataset contains a large number of fine-grained crops with highly similar spectra (such as different varieties of Chinese cabbage), posing a significant challenge to the manifold feature differentiation and open set boundary construction of the GPAM-Net model.

[0052] 3) WHU-Hi-HanChuan Dataset: Similar to the HongHu dataset, this dataset was also acquired by a UAV hyperspectral platform in a mixed agricultural and suburban area of ​​Hanchuan City, Hubei Province. The image size is 1217×303 pixels, with a spatial resolution of 0.109m, and contains 274 bands. A total of 16 land cover categories were labeled in this scene. Based on the open set partitioning method, the first 13 categories were selected as known categories, while the last 3 categories (i.e., Water, Buildings, and Non-vegetated area) were hidden as unknown categories.

[0053] During the data input phase, considering the high redundancy among hyperspectral image bands, Principal Component Analysis (PCA) was first applied to reduce the dimensionality of all hyperspectral images to 30 dimensions. Then, [further details are needed]. Spatial windows are used to extract image patches as the local spectral-spatial input for the network. Regarding sample partitioning, to simultaneously examine the model's generalization ability in both limited labeled samples and open scenes, the partitioning method of the state-of-the-art model DSFormer in 2025 was adopted: for selected "known classes," 50 samples were randomly selected to form the training set; for a very small number of classes with scarce samples, the number of training samples was further reduced to 15. All known class samples outside the training set, as well as all unknown class samples, together constituted the test set used for the final evaluation.

[0054] All deep learning models in this invention are implemented on a hardware platform equipped with an NVIDIA GeForce RTX 4060 Laptop GPU (16GB VRAM). For network optimization, the AdamW optimizer is used for parameter updates. The training epochs of the GPAM-Net model are uniformly set to 300, and the batch size is set to 256. The global initial learning rate is set to... and apply Weight decay is used to effectively mitigate overfitting under small sample conditions. To ensure absolute fairness in the comparative experiments, all baseline methods used in the comparison employed the hyperparameter configurations that achieved optimal performance in their original papers.

[0055] The performance of the GPAM-Net model is evaluated using overall accuracy (OA), average accuracy (AA), and the Kappa coefficient. For unknown categories, the accuracy of the Unknown category is used for evaluation. All three metrics are expressed as percentages (%), with higher values ​​indicating better performance.

[0056] To comprehensively validate the performance of the proposed method in open-set hyperspectral image classification (HSIC), seven influential methods were selected for comparison. These methods can be divided into two categories: traditional closed-set HSIC methods and open-set classification methods. For the former, representative HSIC methods were selected: DSFormer, the state-of-the-art attention model of 2025; SpectralFormer, the first model to apply Transformer to HSIC; GSCViT, a global spatial channel vision Transformer method; MorphFormer, a morphology-based attention model; DBDA, a two-branch attention network; and SSFTT, a spectral-spatial feature labeling Transformer model. Regarding open-set classification methods, since there are currently few dedicated methods for open-set HSIC, and most models lack open-source code, MDL4OW, a multi-task deep learning method for open-world applications, was chosen. This study draws on conventional solutions to open-set problems and uniformly modifies closed-set HSIC models by deploying an additional SoftMax function after the classification layer. In terms of the judgment logic, in order to ensure fair evaluation, a uniform fixed threshold is not used for all methods. Instead, a percentile search is performed on the score distribution of known class samples to automatically select an adaptive threshold that optimizes the average precision (AA) for each method.

[0057] Table 1: Classification results on the Houston2013 dataset

[0058] Table 2: Classification results on the WhuHi-HongHu dataset

[0059] Table 3: Classification results on the WhuHi-HanChuan dataset

[0060] Tables 1-3 show the specific quantitative metrics for all comparison methods on the Houston2013, WHU-Hi-HongHu, and WHU-Hi-HanChuan datasets, respectively.

[0061] Experimental results on the Houston2013 dataset show that closed-set models modified with Softmax thresholds possess some feature extraction capabilities when handling known-class samples, but their generalization performance in open-set environments is severely limited. Specifically, SpectralViT and DBDA achieved OA of 76.68% and 84.34%, respectively, but their unknown-class accuracies were only 1.57% and 0.20%. This confirms that traditional models have a serious tendency to overconfide in single-point prototype representations and cannot effectively isolate out-of-distribution (OOD) samples in the feature space. MDL4OW, designed for open-world applications, only achieved an unknown-class accuracy of 46.76%. In contrast, the method of this invention effectively addresses the rejection task in open sets while maintaining high-precision classification in closed sets. Its OA (94.42%), AA (94.97%), and Kappa (93.92%) are all superior to the second-best performing SSFTT model, and its unknown-class accuracy reaches 91.96% (an improvement of 14.73% over SSFTT). This significant improvement validates the rationality and superiority of the Grassmann principal angle margin mechanism in constructing effective geometric rejection boundaries.

[0062] On the WHU-Hi-HongHu dataset, the inclusion of numerous fine-grained crop categories with extremely similar spectra leads to manifold overlap in the deep feature space. As a result, the unknown class accuracy of all comparative models significantly declines compared to the Houston2013 dataset. For example, the unknown class accuracy of morphFormer and SpectralViT drops to 5.33% and 6.34%, respectively, while MDL4OW drops to 0.00%. Despite the dual challenges of high inter-class similarity and strong manifold overlap, the method of this invention still exhibits the strongest feature discrimination robustness: it not only achieves the globally best OA (90.06%) in the fine-grained partitioning of known classes but also maintains the highest unknown class accuracy (36.13%). This is mainly attributed to the decoupled dual-manifold geodesic mechanism (DDGT) designed in this invention. This mechanism effectively amplifies the inter-class differences of fine-grained land cover by modeling the geodesic distance of the nonlinear manifold, mitigating the problem of unknown class features being misabsorbed by the known class subspace.

[0063] On the WHU-Hi-HanChuan dataset, characterized by scattered spatial features, high heterogeneity between farmland and man-made structures, and imbalanced sample size, the classification accuracy of most baseline models drops significantly in this complex mixed scenario. SpectralViT's OA is only 49.71% (unknown class accuracy is only 0.11%), DBDA's OA is 56.17% (unknown class accuracy is 0.13%), and MDL4OW's OA drops to 26.27%. In contrast, the method of this invention exhibits extremely strong scene adaptability and stability. Regarding the closing set metric, the GPAM-Net model of this invention achieves an OA of 79.15%, significantly outperforming the second-best model, DSFormer (72.48%). In terms of unknown class accuracy, the GPAM-Net model of this invention demonstrates a strong advantage with an unknown class accuracy of 63.26% (most baseline methods have unknown class accuracy below 2%). This set of experiments fully demonstrates that in complex hyperspectral scenarios, orthogonal modeling based on high-dimensional Grassman subspaces has greater generalization potential than traditional distance-based single-point representations, and can provide sufficient topological exclusion space for unknown anomalous samples while compacting known class distributions.

[0064] To thoroughly verify the effectiveness of each core component in the proposed network, detailed ablation experiments were conducted on three datasets: Houston2013, WHU-Hi-HongHu, and WHU-Hi-HanChuan. In the experimental setup, the baseline model was defined as follows: the DDGT module in the model was replaced with a standard Transformer architecture, and the traditional Softmax function was used for classification and unknown class discrimination. Based on this, the model was then replaced with the proposed DDGT (replacing the standard Transformer) and GPAM (replacing the standard Softmax) frameworks, respectively, and the classification performance under different component combinations was tested. The quantitative evaluation results of the ablation experiments are listed in Tables 4, 5, and 6.

[0065] While the standard Transformer can capture global dependencies, it often fails to fully extract deep spectral-spatial joint features when processing hyperspectral data containing complex spatial structures and rich spectral details. Comparing the baseline in Table 4-6 with the model that only adds DDGT shows that introducing DDGT improves the overall classification accuracy (OA) across all datasets. For example, on the WHU-Hi-HanChuan dataset (Table 6), replacing the standard Transformer with DDGT increases the OA from 70.95% to 73.92%, and the Kappa coefficient from 66.30 to 69.54. This clearly demonstrates the significant advantages of the DDGT module in enhancing feature representation, suppressing background noise, and improving the separability of known categories. However, because it still relies on the traditional Softmax mechanism, the network's recognition rate (UA) for unknown categories remains low (e.g., only 8.17% on the WHU-Hi-HongHu dataset), indicating that simply improving feature extraction capabilities cannot fundamentally solve the problem of open set recognition.

[0066] Traditional Softmax-based methods tend to divide the feature space into extremely compact closed regions, which leads to the tendency to forcibly classify unknown features into the nearest known class in open-set environments. To overcome this limitation, this invention proposes a GPAM framework based on manifold projection. As shown in the table, introducing GPAM into the baseline model alone can trigger a qualitative change in the unknown class identification accuracy (UA). On the Houston2013 dataset (Table 4), after introducing GPAM, the UA index jumped significantly from 61.35% to 91.16%; on the WHU-Hi-HongHu and WHU-Hi-HanChuan datasets, the UA also achieved significant increases from 8.19% to 24.49% and from 46.03% to 56.48%, respectively. This improvement is attributed to the synergistic effect of the subspace orthogonal discretization constraint and the principal angle margin loss in the GPAM framework, which successfully preserves a highly exclusive angular vacuum region for the unknown class, thus avoiding overfitting and forced classification. In particular, it was observed that reverting to ordinary attention with a standard learning rate leads to gradient explosion. In contrast, the proposed geodesic attention naturally constrains feature variance on the manifold, requires no complex learning rate warm-up, and ensures convergence.

[0067] When DDGT and GPAM are introduced into the network simultaneously, they exhibit excellent synergistic effects, achieving globally optimal overall performance on all three datasets. On the most challenging WHU-Hi-HongHu dataset (Table 5), the complete GPAM-Net model not only achieves the highest overall accuracy (OA) of 90.06%, but its unknown class recognition rate (UA) also further climbs to 36.13%. The underlying logic of this synergistic effect lies in the fact that the high-quality, high-discrimination spectral-spatial features extracted by DDGT provide a robust base space for the GPAM framework to model subspaces on the Grassmann manifold; conversely, the geometric orthogonality constraints of GPAM further enable DDGT to learn features with greater inter-class differences and more compact intra-class features. The combination of the two completely breaks the limitations of closed-set feature representation, achieving robust land cover classification in complex open environments.

[0068] Table 4: Ablation studies of different components on the Houston2013 dataset

[0069] Table 5: Ablation studies of different components on the WhuHi-HongHu dataset

[0070] Table 6: Ablation studies of different components on the WhuHi-HanChuan dataset

[0071] This invention addresses the challenges of open set classification (OS-HSIC) in hyperspectral images (HSI) used in practical remote sensing applications, proposing a Grassman principal angle margin hyperspectral image classification model (GPAM-Net) for open scenes. To address the pain point of traditional deep learning models misclassifying unknown features as known categories with high confidence under the strict closed-set assumption, this invention fundamentally improves upon these aspects from two dimensions: feature extraction and classification decision. First, a DDGT module is designed to effectively suppress background noise and significantly enhance the compactness and separability of known category features by deeply mining deep spectral-spatial joint features. Second, to solve the open space risk inherent in the softmax function of traditional learning paradigms, this invention constructs a GPAM framework based on manifold projection. This framework maps the feature space to a Grassman manifold and utilizes subspace orthogonal discretization constraints and principal angle margin loss to geometrically preserve a highly exclusive angular vacuum region for unknown categories, effectively avoiding forced classification of unknown samples. Extensive qualitative and quantitative experiments on three benchmark datasets—Houston2013, WHU-Hi-HongHu, and WHU-Hi-HanChuan—fully validated the effectiveness of the proposed method. The results show that DDGT and GPAM exhibit excellent synergistic effects. This model not only maintains excellent overall accuracy (OA) for the classification of known land features but also achieves a significant improvement in the recognition rate (UA) for unknown categories, successfully realizing robust land feature classification in complex open environments. Future work will further explore the classification potential of this framework in scenarios with extremely small sample open sets and attempt to introduce lightweight design to reduce the computational complexity of manifold projection and feature measurement, thereby promoting the practical application of hyperspectral open set recognition technology in large-scale real-time remote sensing monitoring.

[0072] Example 2: As a further embodiment of the present invention, the present invention also provides an electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0073] In practical use, users can interact with servers, which are also electronic devices, via a network to receive or send messages. Terminal devices are generally various electronic devices equipped with a display and used through a human-computer interface, including but not limited to smartphones, tablets, laptops, and desktop computers. Various specific application software can be installed on these terminal devices as needed, including but not limited to web browsers, instant messaging software, social media platforms, and shopping apps.

[0074] Furthermore, the server is a network service that provides various services, such as a backend server that provides corresponding calculation services for hyperspectral images of the same type of land cover transmitted from the terminal device, so as to realize the processing of hyperspectral image classification methods, calculate the hyperspectral image classification results, and finally return them to the terminal device.

[0075] Example 3: As a further embodiment of the present invention, the present invention also provides a storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the hyperspectral image classification method as described above.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of hyperspectral image classification, characterized in that, Includes the following steps: Step one, collect hyperspectral images of the same category of ground objects, and pre-process the hyperspectral images to obtain a plurality of hyperspectral image blocks after PCA dimension reduction wherein, is the number of spectral bands after dimension reduction, is the height of the image, is the width of the image, is the feature space; A Grassman principal angle margin hyperspectral image classification model for open scenarios is constructed. The Grassman principal angle margin hyperspectral image classification model includes a 3D-2D spatial-spectral joint feature extractor, a dual-stream geodesic attention module, and a Grassman principal angle margin framework. Step two, input each hyperspectral image block to the 3D-2D spectral joint feature extractor to obtain a feature sequence ; Step three, inputting the feature sequence into a double-stream geodesic attention module to obtain a hyperspectral image classification result including a plurality of feature samples ​ Step four, inputting each feature sample into the Grassmannian principal angle margin framework to detect the classification detection result of whether the current feature sample belongs to a known class or an unknown class. The specific process of step three is as follows: Step 3.1, the dual-stream geodesic attention module receives a sequence of features , normalizes the sequence of features , and obtains a query matrix , a key matrix , and a value matrix via linear projection Step 3.2, send the query matrix , key matrix and value matrix into the Euclidean structure topology flow branch to obtain the output with spatial smoothing characteristics ; Step 3.3, send the query matrix , key matrix and value matrix into the manifold feature flow branch to get the nonlinear manifold dependent output ; Step 3.4: Output with spatial smoothing characteristics and nonlinear manifold dependent output Multi-head adaptive fusion is performed at the output feature level of the dual-stream geodesic attention module to obtain the fused output. ; Step 3.5, Fusion Output through The samples are then input into the MLP classification head for classification, resulting in samples containing multiple features. Hyperspectral image classification results; The Grassman principal angle margin framework consists of three progressive mechanisms connected in sequence: Grassman subspace modeling, principal angle margin loss, and subspace orthogonal discretization constraint. The Grassman principal angle margin framework is used to map samples belonging to known land cover categories to Grassman manifolds. An orthogonal subspace is constructed on the , and a corresponding independent subspace is built for each known category. Feature samples are then calculated. The principal angle margin loss of each category subspace enables classification tasks for known categories and detection tasks for unknown categories.

2. The hyperspectral image classification method according to claim 1, characterized in that, The specific process of step two is as follows: Step 2.1: Apply 3D convolution kernels to each hyperspectral image patch. Feature extraction is performed to obtain a 3D feature tensor; Step 2.2: Reshape the extracted 3D feature tensor into a 2D feature tensor, and map the channels of the 2D feature tensor to a fixed feature manifold dimension using a 2D convolution kernel. The above yields local spatial-spectral joint features. ; Step 2.3: Combine local spatial and spectral features Flattening the spatial dimensions to form a feature sequence ;in, The number of space tokens.

3. The hyperspectral image classification method according to claim 2, characterized in that, 3D feature tensor The expression is as follows: ； in, The weight tensor of the 3D convolution kernel has its receptive field size set to (7,3,3).

4. The hyperspectral image classification method according to claim 3, characterized in that, Local spatial-spectral joint features The expression is as follows: ； in, For the weight tensor of the 2D convolution kernel, To reshape.

5. The hyperspectral image classification method according to any one of claims 1-4, characterized in that, The specific process of step 3.2 is as follows: Step 3.2.1: Convert the value matrix Reconstruct it into a 2D tensor and introduce a 3×3 depthwise separable convolution to extract local features. Then, through multi-head adaptive gating parameters Injecting local spatial information yields the processed value matrix. ; Step 3.2.1: Analyze the query matrix. Bond matrix Perform separately Normalization is performed to obtain the normalized query matrix. and normalized bond matrix ; Normalize the query matrix Normalized bond matrix and the processed value matrix Multiplying yields an output with spatial smoothness. .

6. The hyperspectral image classification method according to claim 5, characterized in that, The specific process of step 3.3 is as follows: Step 3.3.1: Analyze the query matrix. Bond matrix Perform separately Normalization is performed to obtain the normalized query matrix. and normalized bond matrix ; Step 3.3.2: Normalize the query matrix. and normalized bond matrix After spherical geodesic transformation, the sum-value matrix Performing the product yields a nonlinear manifold-dependent output. .

7. The hyperspectral image classification method according to claim 6, characterized in that, Normalize the query matrix and normalized bond matrix The method for spherical geodesic conversion is as follows: In the unit hypersphere The normalized query matrix is ​​captured using geodesic distance. and normalized bond matrix The global manifold dependency feature.

8. The hyperspectral image classification method according to claim 7, characterized in that, The specific process of step four is as follows: Step 4.1: Construct the Grassman subspace: With feature space For the bottom space; feature space Modeled as a Grassmanifold one above A linear subspace of dimension , and the basis matrix of this subspace is . ,in, It is any one of the known categories, and , The total number of known categories; Step 4.2: Calculate feature samples In all Projection length on each known category subspace ; Step 4.3: Define the feature sample Detection score for: ; Define the detection score for the unknown category as The cutoff threshold is ;like If it is an unknown category, then it is determined to be an unknown category; if If so, it is determined to be a known category.

9. A hyperspectral image classification system, characterized in that, The method for implementing the hyperspectral image classification method as described in any one of claims 1-8 includes a data acquisition module, a 3D-2D spatial-spectral joint feature extractor, a dual-stream geodesic attention module, and a Grassman principal angle margin framework connected in sequence. The data acquisition module is used to acquire hyperspectral images of the same type of land cover and preprocess the acquired hyperspectral images to obtain multiple hyperspectral image patches after PCA dimensionality reduction. The 3D-2D spatial-spectral joint feature extractor is used to extract feature sequences from hyperspectral image patches; The dual-flow geodesic attention module achieves a balance between smoothing of structural features and preservation of features of unknown features by constructing Euclidean topological flow and manifold feature flow in parallel.

10. An electronic device, characterized in that, It includes a memory, one or more processes, and one or more programs stored in the memory, said one or more programs including instructions for performing the hyperspectral image classification method as described in any one of claims 1-8.

11. A storage medium, characterized in that, It includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including instructions for performing the hyperspectral image classification method as described in any one of claims 1-8.

Images