Hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction

Abstract

The application discloses a hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction. The method optimizes the hyperparameters of the feature extraction network through the meta-learning mechanism, enabling it to dynamically adjust with the task and improve the adaptability and generalization ability of the model in complex spectral scenarios. This method learns the internal normal prototype from known categories and uses it to guide feature reconstruction, strengthening the feature cohesion of known categories while significantly amplifying the reconstruction deviation of unknown samples. On this basis, a token-based feature aggregation strategy is used to compress high-dimensional features into a compact discriminative representation, preserving key information while improving computational efficiency and discriminative stability. Experimental verification shows that the proposed method achieves better classification accuracy and unknown sample recognition performance than existing techniques on PaviaU and other datasets, with an overall classification accuracy of 97.17% and an unknown category recognition accuracy of 85.04%, effectively improving the overall performance of hyperspectral open set classification.

Description

Technical Field

[0001] This invention relates to a hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, belonging to the field of image processing technology, specifically the field of hyperspectral open set image classification technology. Background Technology

[0002] Hyperspectral imaging technology acquires reflection or radiation information of ground objects within a continuous and narrow spectral range, ensuring that each pixel contains rich spectral features. This allows for the precise characterization of differences in spectral and spatial dimensions among various ground materials. Based on these characteristics, hyperspectral image classification technology has been widely applied in fields such as remote sensing monitoring, precision agriculture, environmental assessment, urban planning, and resource exploration.

[0003] Most existing hyperspectral image classification methods are based on the closed-set classification assumption, meaning that the set of categories used in the training phase is completely consistent with that used in the testing phase. However, in real-world applications, the land cover types covered by hyperspectral data are often highly complex and dynamically changing, and new or unknown land cover categories inevitably appear during testing or actual operation. When faced with unseen category samples, traditional closed-set classification models often force them to be incorrectly classified into one of the known categories, resulting in unreliable classification results and severely impacting the practical application value of the system.

[0004] To address these challenges, open set classification has gradually become an important research direction in the field of hyperspectral image analysis. Open set classification requires models not only to accurately identify known categories but also to recognize, reject, or distinguish samples of unknown categories. However, the high dimensionality, high redundancy, severe spectral aliasing, and sensitivity to noise inherent in hyperspectral data make open set hyperspectral classification even more difficult.

[0005] While some existing open-set hyperspectral classification methods have mitigated the misclassification problem of unknown categories to some extent through methods such as reconstruction error, distance metrics, or uncertainty modeling, they still generally suffer from the following shortcomings:

[0006] First, the feature extraction stage has limited adaptability to different tasks and scenarios, making it difficult to simultaneously consider the discriminative power of known categories and the separability of unknown categories. Second, the lack of stable modeling of the "intrinsic structure of normal (known) categories" leads to loose intra-class distribution and blurred inter-class boundaries in the feature space. Third, directly using high-dimensional space-spectral features for discrimination results in high computational complexity and sensitivity to noise, which is not conducive to robust detection of unknown categories.

[0007] Therefore, there is an urgent need for a hyperspectral open set classification method that can introduce an adaptive mechanism in the feature extraction stage, build a stable category reference pattern in the intermediate representation stage, and achieve efficient compression and differentiation in the discrimination stage, so as to improve the reliability and generalization ability of the system in complex real-world scenarios. Summary of the Invention

[0008] To address the aforementioned issues, this patent proposes an end-to-end learning framework for hyperspectral open set classification. This framework introduces a meta-learning mechanism, a prototype modeling strategy, and a tokenized compression structure to construct a complete processing flow from adaptive feature extraction to robust discrimination, thereby achieving accurate classification of known categories and effective detection of unknown categories.

[0009] This invention provides a hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction (Meta-Guided Intrinsic Prototype Network, MGIPNet). Its core lies in constructing a processing flow that works collaboratively with adaptive feature extraction, normal prototype modeling, prototype-guided reconstruction, and tokenized compression discrimination, so as to simultaneously achieve accurate classification of known categories and effective identification of unknown categories.

[0010] Specifically, in the feature extraction stage, this invention employs a spectral-spatial residual encoder to perform preliminary feature modeling on the input hyperspectral data. Unlike traditional methods that use pre-set fixed hyperparameters, this invention introduces a trainable hyperparameter mechanism guided by meta-learning, converting some key hyperparameters into parameters that can be dynamically adjusted according to the training task. This mechanism, through task-level training that includes inner-loop updates and outer-loop optimization, enables the encoder to adaptively adjust its feature extraction behavior based on the spectral distribution and spatial structure characteristics of different hyperspectral scenes, thereby obtaining a more generalized adaptive feature representation.

[0011] Based on the obtained adaptive features, this invention further employs an intrinsic normal prototype multi-head attention extractor to learn a set of intrinsic normal prototypes representing the core structure of known categories from the feature space. This extractor utilizes learnable tokens and a multi-head attention mechanism to perform weighted fusion of spectral-spatial features, allowing different attention heads to focus on different combinations of spectral bands and spatial relationships, thereby extracting stable patterns of known categories in high-dimensional features. The extracted intrinsic normal prototypes serve as reference benchmarks for known categories, enhancing the compactness of intra-class features and providing a basis for distinguishing unknown categories. This invention explicitly introduces these prototypes into the feature reconstruction process through a decoding and token generation mechanism based on intrinsic normal prototypes. In this mechanism, the intrinsic normal prototypes are used as attention keys and values ​​to reconstruct the features generated by the encoder under prototype guidance. Through this constraint, the reconstructed features of known category samples will approach their corresponding intrinsic normal prototypes, while the reconstructed features of unknown category samples, lacking matching prototypes, will exhibit significant deviations, thus amplifying the differences between known and unknown categories at the feature representation level.

[0012] To reduce feature redundancy and improve processing efficiency, this invention further employs a learning-based cross-attention spectral token prototype generation module. This module uses a set of learnable spectral tokens to perform cross-attention aggregation on the reconstructed high-dimensional spectral-spatial features, thereby compressing the dispersed spatial information into a compact and highly discriminative spectral token prototype representation. This tokenization compression process effectively reduces redundant information interference while enhancing the sensitivity of features to unknown categories. The generated spectral token prototype is fed into a classification and discrimination module to complete classification decisions for known categories and detection of unknown categories. Since the above modules constitute a collaborative processing chain from "adaptive feature extraction - normal prototype modeling - prototype-guided reconstruction - tokenization compression discrimination," this invention can significantly reduce the risk of unknown categories being misclassified as known categories while maintaining high classification accuracy for known categories.

[0013] This invention, through the collaborative design of multi-stage model structure and learning mechanism, systematically improves the generalization ability, prototype representation stability and high-dimensional feature discrimination efficiency in hyperspectral open set classification tasks, and has good practical application value.

[0014] The hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction described in this invention includes:

[0015] First, the acquired hyperspectral sample data is reclassified according to its category attributes. At least one category is selected from the multiple categories corresponding to the sample as the unknown category, and the remaining categories are designated as known categories. For the known category samples, a training sample set and a test sample set are constructed according to a pre-defined partitioning strategy. During the training phase, the hyperspectral classification model is trained using only the training sample set without introducing the unknown category samples. During the testing phase, the unknown category samples and the test sample set are input into the hyperspectral classification model to determine the known categories and detect or distinguish the unknown category samples.

[0016] When constructing a feature extraction network for hyperspectral open set classification, the spectral-spatial residual network SSRN is used as the encoder backbone. A meta-learning framework is introduced to transform at least one class of hyperparameters in the encoder from "pre-fixed" to a set of parameters that are "learnable and adaptively optimized". This enables the encoder to dynamically adjust the feature extraction process under the feedback of different tasks / scenes, improving its adaptability to changes in spectral-spatial complexity and its cross-scene generalization ability.

[0017] Specifically, the encoder employs an episodeic training mechanism during the training phase. Each episode constructs a simulated task from known categories and divides the task samples into a support set and a query set. The support set is used for fast adaptive updates of the inner loop, while the query set is used for evaluation of the outer loop and meta-optimization updates, so that the learned trainable hyperparameters have "cross-task transferable" optimization behavior.

[0018] After completing the initial processing of the hyperspectral sample data and obtaining the adaptive feature representation, this patent further introduces an Intrinsic Normal Prototype Multi-HeadAttention Extractor (INP-MHAE) to extract intrinsic normal prototypes that can stably characterize the structure of known categories from the adaptive spectral-spatial features. This step aims to provide a reliable reference basis for the subsequent differentiation and detection of unknown categories by explicitly modeling the intrinsic feature patterns of known categories.

[0019] Specifically, INP-MHAE receives the spectral-spatial features output from the preceding steps as input and introduces a set of learnable tokens into the feature space. These learnable tokens do not depend on specific category labels but serve as query carriers, actively aggregating representative pattern information from the input features through an attention mechanism. Unlike directly averaging or pooling all features, this approach adaptively selects the spectral-spatial structure most critical for discriminating known categories globally.

[0020] During feature modeling, INP-MHAE employs a multi-head attention structure, enabling different attention heads to focus on different aspects of the input features in parallel. For example, different attention heads can focus on different band combinations, local spatial relationships, or cross-regional spectral correlations, thereby avoiding the problem of insufficient modeling capability of a single attention mechanism for complex hyperspectral structures. Through the synergistic effect of multi-head attention, the generated prototype can more comprehensively reflect the inherent distribution characteristics of known categories in both spectral and spatial dimensions.

[0021] Through the aforementioned attention aggregation process, INP-MHAE learns a set of intrinsic normal prototypes from the input features. These intrinsic normal prototypes are not simple category centers or statistical means, but rather abstract representations of the "normal patterns" of known categories, capturing their stable and discriminative spectral-spatial feature structures. By normalizing and constraining the prototypes, the impact of noise, outliers, and local perturbations on prototype quality can be further reduced, thereby improving the stability of the prototypes across different samples and scenarios.

[0022] In open-set classification scenarios, the intrinsic normal prototype acts as a reference anchor in the feature space. For samples of known categories, their feature representations are more likely to cluster towards the corresponding intrinsic normal prototype under prototype constraints, thereby enhancing intra-class compactness. However, for samples of unknown categories, since their feature patterns cannot form a good match with the intrinsic normal prototype, they often exhibit more significant deviations in the feature space. This deviation effect is further amplified in subsequent decoding, reconstruction, or discrimination processes, thus facilitating the identification or differentiation of unknown categories.

[0023] By introducing INP-MHAE after the feature extraction stage, this patent realizes the transformation from adaptive feature representation to prototype-level representation, which not only improves the stability and consistency of known category feature modeling, but also provides a reliable structured prior for subsequent prototype-guided decoding and discrimination steps, effectively supporting the detection needs of unknown categories in hyperspectral open set classification tasks.

[0024] After obtaining the intrinsic normal prototype in INP-MHAE, this patent further introduces a decoding and reconstruction step based on the intrinsic normal prototype. Specifically, it uses an Intrinsic Normal Prototype Decoder Token Generation (IDTG) module to perform prototype-guided decoding processing on the features output from the previous steps. This step aims to explicitly introduce the intrinsic normal prototype into the feature reconstruction process, so that the feature representation is structurally constrained on known class samples and produces a distinguishable bias response on unknown class samples.

[0025] Specifically, IDTG uses the intrinsic normal prototypes learned in INP-MHAE as reference information and embeds them into the feature modeling process during the decoding stage. Unlike traditional methods that rely solely on encoded features for decoding or reconstruction, this step introduces prototype information, making the decoding process no longer unconstrained feature recovery, but guided by the known intrinsic structure of the category. Therefore, the decoded features not only retain the original spectral-spatial information but also demonstrate an alignment trend with the "normal category pattern" at the representation level.

[0026] During the decoding process, known category samples, due to their high consistency with the inherent normal prototype, exhibit decoding results that closely match the prototype structure, thus demonstrating small reconstruction bias and high consistency. Unknown category samples, lacking feature patterns that match the inherent normal prototype, are difficult to constrain effectively during the decoding stage, and their decoding results often deviate more significantly from the prototype in structure or distribution. This difference is further manifested in the feature space, providing favorable conditions for subsequent identification or differentiation of unknown categories.

[0027] Through the prototype-guided decoding and reconstruction mechanism described above, IDTG plays a role in "amplifying differences" at the feature level: on the one hand, it strengthens the consistency and stability of known category samples in the reconstructed feature space; on the other hand, it amplifies the degree of deviation of unknown category samples relative to the inherent normal prototype. Thus, the model maintains the accuracy of known category discrimination while possessing greater sensitivity to unknown categories.

[0028] After decoding and reconstruction based on the intrinsic normal prototype in the IDTG module, the resulting features still retain high-dimensional spatial-spectral information. Although the differences between known and unknown categories have been amplified under the prototype constraint, direct discrimination based on these high-dimensional features may still be affected by redundant information and local noise. To address this, this patent further introduces a Learned Cross-Attention Spectral Token Prototype Generator (LCA-STPG) module after the IDTG module to compress and structurally model the decoded features.

[0029] Specifically, the LCA-STPG module receives the reconstructed features output by the IDTG module as input and introduces a preset number of learnable spectral tokens as aggregation carriers. By constructing a cross-attention mechanism, the learnable spectral tokens actively gather the spectral-spatial information most relevant to classification from the decoded features, compressing the key information originally scattered in high-dimensional features into a set of compact and more discriminative spectral token prototype representations.

[0030] In the e-th episode, the trainable hyperparameters are first temporarily updated once or multiple times based on the support set loss to obtain the adaptive hyperparameters for that episode, which can be expressed as:

[0031]

[0032] in, This represents the initial set of hyperparameters for the SSRN encoder. This represents the set of hyperparameters after the inner loop update. The inner loop learning rate, To support task losses on the set, Indicates by hyperparameters A parameterized or modulated encoder.

[0033] In this process, LCA-STPG can effectively suppress the influence of redundant features and noise components while maintaining the main discriminative information, thereby reducing the feature dimensionality and improving the stability of feature representation. Since the decoded features are already guided and constrained by the inherent normal prototype in IDTG, known class samples are more likely to form a compact and clearly distributed token representation after compression; while unknown class samples, due to their significant deviation during the decoding stage, still maintain a significant difference from the known class token prototype after tokenization and compression.

[0034] Through the compression and aggregation processes described above, the spectral token prototype output by LCA-STPG can be directly used for subsequent classification or unknown category detection, enabling the model to effectively distinguish between known and unknown category samples with low feature complexity. Thus, LCA-STPG completes the transformation from "prototype-guided feature reconstruction" to "discriminative and compact representation" in the overall process, further improving the stability and reliability of hyperspectral open set classification in practical applications.

[0035] Finally, this patent utilizes a feature fusion and discrimination module to jointly model the reconstructed feature representation and the tokenized prototype representation, and introduces multiple objective constraints during training to collaboratively optimize the model. By simultaneously considering optimization strategies that balance representation consistency and inter-class discriminability, the model improves classification performance for known categories while maintaining stable recognition capabilities for unknown category samples. This joint optimization process constrains the model's learning behavior at a holistic level, enabling it to balance classification accuracy and unknown sample rejection in complex hyperspectral scenarios, achieving a complete mapping from intermediate representations to the final classification result. Attached Figure Description

[0036] Figure 1 This is a flowchart of the MGIPNet model in the method of this invention;

[0037] Figure 2 This is a structural diagram of the MGIPNet model in the method of this invention;

[0038] Figure 3 This is a structural diagram of the INP-MHAE module in the method of this invention;

[0039] Figure 4 This is a structural diagram of the IDTG module in the method of this invention;

[0040] Figure 5 This is a structural diagram of the LCA-STPG module in the method of this invention;

[0041] Figure 6 These are the PaviaU dataset ground pseudo-color map and ground truth map used in the method of this invention;

[0042] Figure 7 These are the ground pseudo-color map and ground truth map of the Houston2013 dataset used in the method of this invention;

[0043] Figure 8 These are the ground pseudo-color map and ground truth map of the WHU-Hi-LongKou dataset used in the method of this invention;

[0044] Figure 9 This is a pseudo-color image of the PaviaU dataset classification results of the MGIPNet model and other models of this invention.

[0045] Figure 10 This is a pseudo-color image of the classification results of the MGIPNet model and other models on the Houston 2013 dataset.

[0046] Figure 11 This is a pseudo-color image of the classification results of the MGIPNet model and other models on the WHU-Hi-LongKou dataset. Detailed Implementation

[0047] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0048] The flowchart of the hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction in this specific implementation is as follows: Figure 1 As shown, it includes the following steps:

[0049] Step 1: Perform category-level partitioning on the original hyperspectral dataset to be processed. Configure the multiple categories contained in the dataset into a known category set and an unknown category set, where each unknown category contains at least one class. For samples of the known categories, divide them according to preset rules into a training sample set for model training and a test sample set for performance evaluation. Samples of the unknown categories do not participate in model training; they only serve as input to be judged along with the known category test samples during the testing phase, thus constructing a data configuration environment that meets the requirements of open set classification.

[0050] Step Two: After data partitioning, a spectral-spatial residual network is used as the hyperspectral feature extraction encoder, and a meta-learning mechanism is introduced to optimize its training process. Through task-level training, some previously fixed hyperparameters are transformed into trainable parameters, enabling the encoder to adaptively adjust the feature extraction process based on feedback from different tasks and sample distributions. The resulting feature representations exhibit stronger adaptability and generalization performance when facing complex spectral variations, providing stable foundational features for subsequent open-set modeling.

[0051] Step 3: After obtaining the adaptive feature representation, an intrinsic normal prototype multi-head attention extraction module is introduced to further model the features. This module learns a set of normal prototypes from the spectral-spatial features through learnable tokens and a multi-head attention mechanism. These prototypes can stably represent the intrinsic structure of known categories. As reference patterns for known categories, these prototypes help enhance intra-class compactness, suppress noise interference, and provide structured priors for distinguishing unknown categories.

[0052] Step Four: After obtaining the intrinsic normal prototype, a prototype-guided decoding and reconstruction module is introduced, explicitly embedding the prototype into the feature decoding process. This method ensures that known category samples are constrained by the intrinsic normal prototype during the decoding stage, thus maintaining high consistency; while unknown category samples, lacking a matching prototype structure, exhibit more significant deviations in feature reconstruction. This step amplifies the differences between known and unknown categories at the feature level, providing favorable conditions for subsequent discrimination.

[0053] Step 5: After completing the prototype-guided decoding, the decoded features are further processed by introducing a learned cross-attention spectral token prototype generation module to compress and aggregate the high-dimensional space-spectral features. This module compresses the scattered decoded features into a compact and more discriminative token prototype representation using a small number of learnable spectral tokens, reducing feature redundancy while maintaining key discriminative information, thereby improving the stability and efficiency of the model under open set conditions.

[0054] Step Six: Finally, the reconstructed feature representation is fused with the token prototype representation and input into the classification and discrimination module for processing. During model training, a multi-objective loss function is introduced to jointly optimize the model, achieving a balance between classification accuracy for known categories and detection reliability for unknown categories. Through a global optimization strategy, the model can accurately identify known categories and effectively distinguish or reject unknown category samples in complex hyperspectral scenes, ultimately outputting stable and reliable classification results.

[0055] To verify the effectiveness and applicability of the proposed model in hyperspectral data classification tasks, this method was experimentally validated using several representative internationally accepted hyperspectral remote sensing benchmark datasets, including the Pavia University dataset, the Houston 2013 dataset, and the WHU-Hi-LongKou (LK) dataset. These datasets exhibit significant differences in spatial resolution, number of spectral bands, and scene types, allowing for a comprehensive evaluation of the method's classification performance and open set recognition capabilities from multiple dimensions.

[0056] The Pavia University (PU) dataset, acquired by the ROSIS (Reflective Optics System Imaging Spectrometer) pushbroom imaging spectrometer, covers the urban area of ​​Pavia, Italy. After denoising preprocessing, this dataset retains multiple effective spectral bands, achieving a high spatial resolution of approximately 1.3 meters per pixel. The land cover types are primarily urban elements such as roads, buildings, and vegetation, exhibiting subtle spectral differences and complex spatial structures, making it suitable for validating the model's classification stability in high-resolution urban scenes. The Houston 2013 (HU) dataset, acquired by the ITRES CASI-1500 airborne hyperspectral imaging system, covers the University of Houston campus and its surrounding area, with a spatial resolution of approximately 2.5 meters per pixel. It contains a large number of spectral bands and was one of the standard datasets for the IEEE GRSS 2013 data fusion competition. This dataset includes various artificial land cover and natural vegetation types, with a large number of categories and complex distribution, providing a more challenging experimental environment for open-set hyperspectral classification. The WHU-Hi-LongKou (LK) dataset is a high spatial resolution UAV hyperspectral dataset that primarily covers agricultural planting areas and their surrounding environment, exhibiting higher spatial detail and significant spectral similarity. This dataset contains various crop categories and natural features, and its fine spatial structure and strong spectral aliasing characteristics place higher demands on the model's generalization ability and unknown category detection capabilities.

[0057] The three selected datasets complement each other in terms of spatial resolution, spectral dimension, and application scenarios, covering typical hyperspectral application environments such as urban, mixed-use, and agricultural scenarios, and have been widely adopted in the field of hyperspectral intelligent interpretation. Experimental verification on the aforementioned datasets allows for a comprehensive evaluation of the effectiveness and robustness of the proposed method in complex hyperspectral open-set classification tasks.

[0058] We compared and evaluated our proposed MGIPNet with several representative open-set classification methods. These methods include: MDL4OW (Multitask Deep Learning for the Open World), a multitask deep learning method for open-world scenarios; SSLR (Spectral–Spatial Latent Reconstruction), an open-set classification method based on spectral-spatial latent reconstruction; SSMTR (Spectral–Spatial Masked Transformer), which uses a spectral-spatial masked Transformer architecture for feature modeling; SSMLP-RPL (Spectral–Spatial MLP-Like Network with Reciprocal Points Learning), a method combining a spectral-spatial MLP structure with a reciprocal point learning mechanism; DBCTNet (Double-Branch Convolution-Transformer Network), which uses a convolutional and Transformer dual-branch structure for feature extraction and fusion; and OSC-SCL (Open-Set Classification via Supervised Contrastive Learning), a method that introduces supervised contrastive learning into the hyperspectral open-set classification task.

[0059]

[0060] In terms of overall classification performance, MGIPNet achieved the best results on the PaviaU dataset. Its overall classification accuracy (OA) reached 97.17% ± 0.07, an improvement of approximately 2.18 percentage points compared to the best-performing OSC-SCL (94.99% ± 1.24), and an improvement of approximately 3.77 percentage points compared to DBCTNet (93.40% ± 1.57). In terms of mean accuracy (AA) and F1 score, MGIPNet achieved 97.29% ± 0.15 and 96.75% ± 0.48 respectively, significantly outperforming the other methods, indicating that the model has a more balanced and stable classification ability across categories. Meanwhile, its Kappa coefficient reached 96.29% ± 0.09, an improvement of approximately 2.81 percentage points compared to OSC-SCL, demonstrating significant advantages in overall consistency and classification reliability.

[0061] MGIPNet also demonstrates outstanding performance in terms of category-level results and unknown category detection capabilities. For most known categories, MGIPNet achieves classification accuracy close to or reaching 100%, such as 100.00% for categories 5, 6, and 7, and significantly higher than or no lower than the comparison methods for categories 1, 2, 4, and 8, showcasing excellent intra-class discrimination ability. More importantly, in the unknown category detection metric Novel(%), MGIPNet achieves 85.04% ± 1.55, an improvement of approximately 15.39 percentage points compared to OSC-SCL's 69.65% ± 6.03, and an improvement of approximately 25.35 percentage points compared to SSMLP-RPL. These results indicate that the proposed method significantly enhances the ability to identify unknown samples while maintaining high accuracy in known category classification, demonstrating stronger overall performance advantages in hyperspectral open set classification tasks.

[0062]

[0063] In terms of overall performance metrics, MGIPNet also achieved the best overall classification performance on the Houston2013 dataset. Its overall classification accuracy (OA) reached 94.52%, an improvement of approximately 4.85 percentage points compared to the current best-performing comparison method OSC-SCL (89.67%), and an improvement of approximately 6.26 percentage points compared to SSMTR (88.26%). In terms of mean accuracy (AA) and F1 score, MGIPNet achieved 94.34% and 93.36% respectively, both significantly higher than other methods, indicating that the model can maintain stable and balanced classification performance even on the imbalanced class distribution and complex scenarios of the Houston2013 dataset. Furthermore, its Kappa coefficient reached 94.65%, an improvement of approximately 5.96 percentage points compared to OSC-SCL's 88.69%, demonstrating a significant advantage in overall consistency and discriminative reliability.

[0064] MGIPNet also demonstrates outstanding performance in terms of category-level accuracy and unknown category detection capability. It achieves near-optimal or best-in-class results on multiple known categories, such as 100.00% for categories 3 and 5, and over 96% accuracy for categories 1, 2, 6, 9, and 10, showcasing excellent discrimination ability for multiple land cover types in complex urban scenes. More importantly, in the unknown category detection metric Novel(%), MGIPNet achieves 75.04%, an improvement of approximately 5.96 percentage points compared to OSC-SCL's 69.08%, and an improvement of approximately 13.80 percentage points compared to SSMLP-RPL's 61.24%. These results indicate that the proposed method, while maintaining high accuracy in known category classification, further enhances the ability to identify and distinguish unknown category samples, exhibiting stronger overall performance and robustness in complex urban hyperspectral open set classification tasks.

[0065]

[0066] In terms of overall classification performance, MGIPNet achieved the best overall performance on the WHU-Hi-LongKou (LK) dataset. Its overall classification accuracy (OA) reached 95.34% ± 0.42, an improvement of approximately 2.83 percentage points compared to the best-performing comparative method, SSLR (92.51%), and an improvement of approximately 3.49 percentage points compared to OSC-SCL (91.85%). In terms of mean accuracy (AA) and F1 score, MGIPNet achieved 95.62% and 95.80% respectively, both significantly higher than other methods, indicating that the model has a more balanced and stable recognition ability across categories in agricultural hyperspectral scenes. Meanwhile, its Kappa coefficient reached 94.99%, significantly better than OSC-SCL's 90.29%, demonstrating a clear advantage in overall classification consistency and reliability.

[0067] MGIPNet also demonstrates outstanding performance in terms of category-level accuracy and unknown category detection capability. It achieves near-optimal results on multiple known categories, such as 99.99% ± 0.02 and 99.85% ± 0.13 for categories 6 and 7, respectively, and classification accuracy exceeding 96% for categories 1, 2, 4, and 5, showing good distinguishing ability among spectrally similar crop categories. More importantly, in the unknown category detection metric Novel(%), MGIPNet achieves 73.72%, an improvement of approximately 11.20 percentage points compared to OSC-SCL's 62.52%, and improvements of approximately 22.04 and 28.11 percentage points compared to SSLR and DBCTNet, respectively. These results indicate that the proposed method, in high-resolution agricultural hyperspectral data, not only maintains high accuracy in known category classification but also significantly enhances the ability to identify and reject unknown category samples, demonstrating stronger robustness in open set classification.

[0068] To subjectively evaluate the classification effect, Figure 9 , Figure 10 , Figure 11 The images show ground truth maps and pseudo-color maps of the classification results for the PaviaU, Houston2013, and WHU-Hi-LongKou datasets, respectively. It can be seen that the proposed method more closely approximates the real-world ground cover distribution, significantly reducing the area of ​​misclassification. Experimental results on the PaviaU dataset demonstrate the significant advantages of the proposed model in hyperspectral image open-set classification tasks. Its multi-scale convolution and feature fusion strategy, while extracting features of known categories, greatly improves the ability to identify new categories, providing a new and effective solution for the field of hyperspectral image classification.

Claims

1. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: This invention provides a hyperspectral open set classification method (Meta-Guided Intrinsic Prototype Network, MGIPNet) based on meta-learning and adaptive prototype reconstruction, which includes the following steps: First, the hyperspectral sample data to be processed is classified, with at least one category designated as the unknown category and the remaining n-1 categories as known categories. The known category samples are then divided into training and testing data. Without introducing unknown category samples into the training, a hyperspectral classification model is constructed using the known category training data. During the model testing phase, unknown category samples and known category test data are introduced together as input samples to be judged, achieving classification of known categories and identification of unknown categories. This method first optimizes the SSRN (Spectral-Spatial Residual Network) encoder through a meta-learning-guided trainable hyperparameter mechanism. The episodic training mechanism transforms traditional fixed hyperparameters into dynamically optimizable parameters, enabling the network to adaptively adjust the feature extraction process based on task feedback during training. This overcomes the limitation that traditional SSRN network hyperparameters cannot be adjusted during training, enhancing the model's adaptability and generalization ability to the complex characteristics of spectral space, and laying the foundation for adaptive feature representation in subsequent processing. Subsequently, an Intrinsic Normal Prototype Multi-Head Attention Extractor (INP-MHAE) is employed to learn the core spectral spatial prototypes of known classes from the adaptive features, enhancing feature stability and amplifying the representational differences between known and unknown classes. Further, an Intrinsic Normal Prototype Decoder Token Generation (IDTG) module guides feature reconstruction based on the learned prototypes, enhancing class discriminativeness and highlighting reconstruction errors of unknown samples. Then, a Learned Cross-Attention Spectral Token Prototype Generator (LCA-STPG) is used to compress the reconstructed features into compact spectral token representations, reducing redundancy and improving feature abstraction capabilities. Finally, a feature fusion and classification module concatenates the reconstructed features with the spectral token prototypes and performs joint optimization using a multi-objective loss function. This ensures accurate classification of known classes while effectively improving the rejection capability for unknown class samples, ultimately achieving robust classification and recognition in hyperspectral open-set scenarios.

2. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: First, the original hyperspectral dataset to be processed is introduced, and the samples are reorganized according to the category information it contains. Assuming the dataset contains n different categories, n-1 categories are selected as known categories, and the remaining category is designated as the unknown category. For the known category samples, they are divided into training data and test data according to a pre-defined partitioning rule. The unknown category samples do not participate in model training but are merged with the test data of the known categories during the testing phase to form a test sample set for classification performance evaluation.

3. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: A trainable hyperparameter optimization strategy based on meta-learning mechanism is adopted to train the encoder of hyperspectral images, wherein the encoder is a spectral-spatial residual network for extracting spectral and spatial features of hyperspectral data. During training, a task-level training process that includes inner loop updates and outer loop updates is constructed. At least one hyperparameter that was originally pre-set in the encoder is set as a trainable parameter, and the trainable parameter is adaptively optimized according to the training feedback of different tasks. The inner loop update temporarily adjusts the encoder parameters and corresponding trainable hyperparameters based on the training samples of the current task; the outer loop update globally optimizes the trainable hyperparameters based on the encoder's performance on the validation samples. This enables the trained encoder to adaptively adjust its feature extraction process according to the changes in spectral-spatial distribution in different hyperspectral tasks, thereby outputting feature representations with stronger generalization ability for subsequent hyperspectral open set classification.

4. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: It also includes a prototype extraction step, which generates a set of intrinsic normal prototypes from the spectral-spatial features output by the encoder using the Intrinsic Normal Prototype Multi-Head Attention Extractor (INP-MHAE). Specifically, this includes: Obtain the hyperspectral spectral-spatial feature tensor obtained in step A as the input feature; initialize a preset number of learnable tokens, which serve as query vectors to "probe" and aggregate representative patterns from the input features; input the learnable tokens and the input features into a multi-head attention mechanism to obtain an intrinsic normal prototype set P, where each attention head independently models different spectral-spatial dependencies, and these dependencies are then concatenated and linearly mapped to form the final prototype representation. The calculation of the multi-head attention generation prototype satisfies one of the following forms: in Indicates the number of heads of attention. , , These represent the query, key, and value projection of the h-th header, respectively. The dimension of the key vector. This is an optional mask item. , This is for outputting projection parameters. The INP-MHAE also includes a coherence loss based on consistency constraints, which promotes the representativeness and compactness of prototypes by minimizing the distance between input features and their nearest prototypes. The coherence loss satisfies at least one of the following forms: Where F is the input feature point set, P is the prototype set, B is the batch size, N is the number of feature points, M is the number of prototypes, and λ is the regularization weight. The intrinsic normal prototype set P serves as a reference pattern for known categories and is used in subsequent feature reconstruction or discrimination processes to enhance intra-class consistency of known categories and improve the separability of samples from unknown categories.

5. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: It also includes a decoding and token generation step based on the intrinsic normal prototype. This step, through the intrinsic normal prototype decoding and token generation module IDTG, introduces the intrinsic normal prototype output by INP-MHAE into the feature reconstruction process, specifically including: The feature token output by the encoder is obtained as the query input, and the set of intrinsic normal prototypes output by the Intrinsic Normal Prototype Multi-Head Attention Extractor (INP-MHAE) is obtained. Using the feature token as the query vector and the intrinsic normal prototypes as the key vector and value vector, a prototype-guided decoding process based on the attention mechanism is constructed to generate a prototype-constrained decoded feature representation. Through the prototype-guided decoding, the reconstructed features tend to the intrinsic normal prototypes on known class samples, while generating a larger reconstruction bias on unknown class samples, so as to enhance the stable representation of known classes and amplify the anomalous response of unknown samples. The prototype-guided decoding process satisfies the following form: Where X represents the input feature token; To query the projection; , , where P is the set of intrinsic normal prototypes; LN represents attention operation; LN represents normalization operation; FFN represents feedforward mapping network. The attention weight calculation satisfies: in This represents the distance or similarity constraint between the query and the prototype. The weighting coefficient is used to adjust the guiding strength of the prototype in the decoding process. The IDTG module explicitly introduces the inherent normal prototype into the decoding stage, making the generation process of decoded features constrained by the inherent structure of the known categories, thereby enhancing the intra-class consistency of known categories in the feature space and improving the separability between unknown category samples and known category prototypes.

6. A hyperspectral open set classification method based on meta-learning and adaptive prototype reconstruction, characterized by: It also includes a feature compression and discrimination step after the intrinsic normal prototype-guided decoding. This step uses the Learned Cross-Attention Spectral Token Prototype Generation Module (LCA-STPG) to compress the decoded features and generate a compact representation for classification. Specifically, it includes: The reconstructed features output by the Intrinsic Normal Prototype Decoding Token Generation Module (IDTG) are used as input features. A cross-attention mechanism is constructed using a preset number of learnable spectral tokens, which serve as query vectors to weight and aggregate the high-dimensional spatial—spectral—information in the input features, generating a set of spectral token prototypes. These spectral token prototypes are then used as compact feature representations for subsequent classification or unknown category detection, reducing feature dimensionality and redundancy while preserving key discriminative information. In one embodiment, the spectral token prototype generation process of the LCA-STPG satisfies one of the following forms: in, , For learnable spectral tokens; Features after IDTG decoding; This is the generated set of spectral token prototypes. The LCA-STPG module reduces redundant information and noise interference by compressing high-dimensional space-spectral features into a finite number of spectral token prototypes, thereby improving the stability of classification and discrimination and enhancing the ability to distinguish samples of unknown categories. The Intrinsic Normal Prototype Guided Decoding Module (IDTG) and the Learned Cross-Attention Spectral Token Prototype Generation Module (LCA-STPG) work in a cascaded manner, so that features are first reconstructed under the constraints of known category prototypes, and then tokenized and compressed for discrimination, thereby simultaneously achieving compact clustering of known categories and effective separation of unknown categories within the same feature space.

Images