A hyperspectral open set classification method combining feature reconstruction and prototype constraint
By combining feature reconstruction and prototype constraint methods, the hyperspectral open set classification model is optimized, which solves the problems of insufficient feature extraction and distribution adaptability, and achieves high-precision and robust hyperspectral open set classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2025-02-19
- Publication Date
- 2026-05-15
AI Technical Summary
Existing hyperspectral open set classification methods have shortcomings in feature extraction and distribution adaptability, resulting in insufficient classification accuracy and robustness. In particular, they are prone to identification errors and over-compression of intra-class features in complex scenarios.
This paper proposes a method that combines feature reconstruction and prototype constraints. It extracts feature vectors through an encoder, constructs a prototype library and builds a comparative prototype loss, uses a decoder for feature reconstruction, and combines cross-entropy loss and reconstruction error loss to optimize the classification model. It also uses Weibull and generalized Pareto distributions to evaluate the probability of unknown categories.
It improves the accuracy and efficiency of hyperspectral open set classification, enhances the robustness of the model and its ability to identify unknown categories, reduces the risk of feature overlap, and maintains fast convergence characteristics.
Smart Images
Figure CN119942350B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of this application relate to the field of image classification technology, and in particular to a hyperspectral open set classification method that combines feature reconstruction and prototype constraints. Background Technology
[0002] Hyperspectral imagery, with its rich spatial and spectral information, is widely used in various fields such as environment, agriculture, oceanography, and military. Hyperspectral images typically contain hundreds or even thousands of narrow spectral bands, providing abundant spectral information and enabling the detailed representation of the intrinsic distinguishing attributes of scene targets. By classifying ground objects pixel-by-pixel, each pixel in a hyperspectral image can be assigned an independent label for a ground object, such as grassland, roads, forests, buildings, soil, and rivers. Therefore, hyperspectral image classification has become a powerful tool for scientific exploration of large areas and inaccessible targets.
[0003] In recent years, classification systems based on convolutional neural networks have significantly improved the accuracy of hyperspectral image classification. However, in practical applications, classification systems are often deployed in unstable and open environments where not all categories belong to the label set of a closed system, resulting in test samples with unknown categories. Classification systems not only need to correctly classify samples belonging to the closed set but also need to reject unknown test samples. Relying on a single criterion is insufficient for accurately modeling complex and unknown categories, while generative methods depend on the quality of known categories.
[0004] To address this technical challenge, researchers both domestically and internationally have proposed several hyperspectral open set classification methods based on deep learning. For data from known categories, the system classifies the data according to the characteristics of the known categories it possesses. For data from unknown categories, the system can detect that it does not come from a known category and label it as "unknown category" using a discriminative or generative model. In the future, other learning methods will be used to identify and handle this type of data.
[0005] However, the inventors of this application have discovered the following problems with the currently proposed hyperspectral open set classification methods.
[0006] First, due to the spatial-spectral joint characteristics of hyperspectral images, convolutional neural networks are limited by their receptive field when extracting hyperspectral image classifications, making it difficult to focus on channel information and ensuring effective feature map extraction, which in turn affects the open set classification performance.
[0007] Second, the proposed hyperspectral open set classification methods rely on the distribution of training data for modeling. However, in complex imaging scenarios, the distribution of training data and test data may not be consistent. Identifying unknown categories by outliers can easily lead to identification errors and classification errors.
[0008] Third, in the process of hyperspectral open set classification, due to the task bias of neural networks and the imbalance of training data distribution, the classifier may over-compress intra-class features in pursuit of accuracy, which may cause the classification model to ignore important information that helps to distinguish unknown categories. Summary of the Invention
[0009] In view of this, embodiments of this application propose a hyperspectral open set classification method that combines feature reconstruction and prototype constraints. This method can effectively identify unknown categories while maintaining the accuracy of hyperspectral closed set classification, achieving high-precision and fast hyperspectral open set classification. This has a significant positive impact on promoting the widespread application of hyperspectral images.
[0010] In a first aspect, embodiments of this application propose a hyperspectral open set classification method combining feature reconstruction and prototype constraints. The method includes: dividing the original sample data in the original sample dataset into blocks to obtain a training set composed of several sample data blocks, and using the label of the center pixel at the top layer of each sample data block as the label of the entire sample data block; wherein the original sample data is a hyperspectral image data cube; inputting the sample data blocks into a classification model composed of an encoder, a decoder, a prototype classification module, and an unknown category detection module; using the encoder to extract features from the sample data blocks to obtain corresponding feature vectors; simultaneously calculating the mean of each known category based on the feature vectors; constructing a prototype library; and constructing a comparative prototype loss based on the prototype library; using the decoder to perform feature reconstruction based on the feature vectors to obtain reconstructed sample data blocks; and based on the sample data blocks and the reconstructed sample data blocks... The data blocks are used to construct a reconstruction error loss; a prototype classification module is used to perform closed-set classification based on feature vectors and a prototype library to obtain the closed-set classification results for the sample data blocks, and a cross-entropy loss is constructed based on the closed-set classification results and the labels of the sample data blocks; a total loss function is constructed based on the comparative prototype loss, reconstruction error loss, and cross-entropy loss, and the classification model is iteratively trained until convergence based on the total loss function to obtain the trained classification model; using the unknown category detection module of the trained classification model, open-set identification is performed based on the sample feature distribution of the training set and the reconstruction error loss, and the probability score of the unknown category is evaluated by combining the prototype distance and the reconstruction loss. The probability scores of all sample data blocks in the training set are analyzed, the tail is modeled using the Weibull distribution, and its cumulative distribution function is approximated by the generalized Pareto distribution, and the first 50% of the tail of the entire cumulative distribution function is determined to be the unknown category.
[0011] Optionally, the shape of the original sample data in the original sample dataset is: , This represents the number of rows of pixels in the original sample data. The number of columns of pixels in the original sample data. The number of bands in the original sample data is given. Each pixel in the original sample data is labeled with a category representation.
[0012] The original sample data in the original sample dataset is divided into blocks according to a preset partitioning standard. Several sample data blocks are then extracted from each original sample data block. A training set is formed based on all these sample data blocks, and the label of the center pixel at the top layer of each sample data block is used as the label for the entire sample data block. Each sample data block has the following shape: , , .
[0013] Optionally, the encoder consists of multiple convolutional layers and global average pooling layers;
[0014] The encoder is used to extract features from the sample data block to obtain the corresponding feature vectors, including:
[0015] First, the sample data blocks are concatenated together using two parallel convolutions to obtain the first intermediate feature. First intermediate feature This can be expressed by the formula:
[0016] ;
[0017] in, Represents a sample data block. and This represents two parallel convolutions. Indicates splicing;
[0018] Then, the first intermediate feature... After processing by the residual module with added channel attention mechanism, the second intermediate feature is obtained. Second intermediate feature This can be expressed by the formula:
[0019] ;
[0020] in, and Both represent convolution. This indicates the channel attention mechanism;
[0021] Then the second intermediate feature The third intermediate feature is obtained by weighting the signal with both channel attention and spatial attention. Third intermediate feature This can be expressed by the formula:
[0022] ;
[0023] in, This represents the sigmoid activation function. and This represents a multilayer perceptron. Represents the ReLU activation function. Indicates the second intermediate feature medium pixel The value, Represents convolution. Indicates the total number of channels. Represents the first sample data block One channel, This represents the vector product operation;
[0024] Finally, regarding the third intermediate feature Perform average pooling to obtain the corresponding feature vector. .
[0025] Optionally, the mean of each known category is calculated based on the feature vectors to construct a prototype library, which is achieved through the following formula:
[0026] ;
[0027] ;
[0028] in, This represents the total number of known categories. Indicates the first The prototype corresponding to each known category This represents the prototype library that has been constructed. Indicates belonging to the first The total number of feature vectors for known categories. Indicates the first The one belonging to the first Feature vectors of known categories;
[0029] The mean of feature vectors with confidence levels less than a preset confidence threshold is calculated and used as a placeholder for the unknown category. The placeholder for the unknown category is expressed by the formula:
[0030] ;
[0031] in, This represents the total number of feature vectors with a confidence level less than a preset confidence threshold. Indicates the first A feature vector with a confidence level less than a preset confidence threshold. Placeholder for an unknown category;
[0032] The comparative prototype loss is built based on the prototype library and is achieved through the following formula:
[0033] ;
[0034] ;
[0035]
[0036] ;
[0037] in, express and The Euclidean distance between them This represents the maximum distance between all prototypes in the prototype library. Indicates the first The first of the known categories The distance between each feature vector and each prototype in the prototype library This represents the loss of the constructed comparative prototype.
[0038] Optionally, the decoder employs a deconvolution structure symmetrical to the encoder;
[0039] The reconstruction error loss is constructed based on the sample data block and the reconstructed sample data block, and is achieved through the following formula:
[0040] ;
[0041] in, This indicates the number of sample data blocks input in each iteration. Indicates the number generated by the encoder A shallow semantic representation The decoder reconstructs the first A shallow semantic representation This represents the reconstruction error loss obtained from the construction.
[0042] Optionally, the prototype classification module is used to perform closed-set classification based on feature vectors and a prototype library to obtain closed-set classification results for the sample data blocks, including:
[0043] Calculate the Euclidean distance between the feature vector and each prototype in the prototype library;
[0044] The category corresponding to the prototype with the smallest Euclidean distance is selected as the closed-set classification result for the sample data block.
[0045] The cross-entropy loss is constructed based on the closed-set classification results and the labels of the sample data blocks, and is implemented using the following formula:
[0046] ;
[0047] in, This represents the classification result of the closed set. Indicates a label, This represents the cross-entropy loss of the construct.
[0048] Optionally, a total loss function is constructed based on the contrastive prototype loss, reconstruction error loss, and cross-entropy loss, and is implemented using the following formula:
[0049] ;
[0050] in, , and The preset weight parameters, To construct the total loss function.
[0051] Optionally, the probability score for the unknown category is constructed as follows:
[0052] ;
[0053] in, The preset evaluation adjustment coefficient, This represents the probability that a sample data block belongs to each known category. The probability that a sample data block belongs to an unknown category;
[0054] Analyze the probability scores of all sample data blocks in the training set, model the tail using a Weibull distribution, and approximate its cumulative distribution function using a generalized Pareto distribution, achieved through the following formula:
[0055] ;
[0056] in, Represents the Weibull distribution. This represents the generalized Pareto distribution. and All are learnable parameters. and The value is determined through model training;
[0057] The first 50% of the tail of the cumulative distribution function are identified as the unknown category using the following formula:
[0058] ;
[0059] in, Adjust the preset mapping parameters.
[0060] This application proposes a hyperspectral open-set classification method that combines feature reconstruction and prototype constraints. By choosing to reconstruct shallow semantic representations instead of high-dimensional raw image data, it significantly reduces the computational complexity of the model and allows feature extraction to focus more on key information for class differentiation. The shallow semantic representation carries highly discriminative features of known classes while retaining sufficient discriminative power for the complexity of unknown classes, thus effectively improving the classification efficiency and accuracy of the model in open-set environments. Based on the manifold representation generated by the encoder, efficient differentiation of known classes is achieved by modeling each known class separately. Furthermore, using the encoder reconstruction error as a measure of class affiliation enables effective classification of sample data blocks, ensuring that the classification model performs well in both rejecting unknown classes and recognizing known classes, effectively improving the robustness of classification. By introducing placeholders for unknown classes in contrastive prototype learning, the risk of overlap between known and unknown classes in the feature space is significantly reduced. Moreover, unlike contrastive learning based on reciprocity points, this application allows known classes to be arranged hierarchically around placeholders, optimizing the utilization of the feature space. Even with a large number of categories, it can still maintain fast convergence, effectively improving the overall convergence efficiency and stability of the algorithm.
[0061] In a second aspect, embodiments of this application propose an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a hyperspectral open set classification method combining feature reconstruction and prototype constraints as described in the first aspect above.
[0062] Thirdly, embodiments of this application propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables a hyperspectral open set classification method combining feature reconstruction and prototype constraints as described in the first aspect above.
[0063] It is understood that the beneficial effects of the second and third aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0064] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0065] Figure 1This is a flowchart of a hyperspectral open set classification method combining feature reconstruction and prototype constraints provided in one embodiment of this application;
[0066] Figure 2 This is a schematic diagram of a data preprocessing procedure provided in one embodiment of this application;
[0067] Figure 3 This is a schematic diagram of the structure of a classification model provided in one embodiment of this application;
[0068] Figure 4 This is a schematic diagram of the encoder provided in one embodiment of this application;
[0069] Figure 5 This is a schematic diagram of the decoder provided in one embodiment of this application;
[0070] Figure 6 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation
[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. In the various embodiments of this application, many technical details are presented to enable the reader to better understand this application. However, even without these technical details and various variations and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented. The division of the following embodiments is only for convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0072] In recent years, researchers at home and abroad have proposed some hyperspectral open set classification methods based on deep learning, which can be mainly divided into three types: methods using confidence, methods using prototype distance, and methods using reconstruction loss.
[0073] Liu et al. proposed a hyperspectral open-set classification method based on deep networks. This algorithm assumes that there are relatively consistent activation vectors for most classes. It obtains the average activation vector of each known class sample based on the nearest-neighbor average theory. During testing, the output vector of the penultimate layer of the network is updated based on the distance between the sample activation vector and the average activation vector of each known class, thus adapting to the open-set classification environment. However, this method fails to effectively consider second-order and higher-order statistical features and is not suitable for data conforming to a long-tailed distribution, leading to inaccurate modeling of the Weiber distribution.
[0074] Xie et al. proposed a prototype network based on feature consistency for hyperspectral open set classification. This method designs a three-layer convolutional network to extract discriminative features, introduces a contrastive clustering module to enhance discriminativity, and constructs a prototype set from the extracted features. Finally, a prototype-guided open set module is designed to identify known and unknown samples. This method constructs prototype sets for all categories from the extracted features. However, intra-class features are compressed to a limited number of points during prototype learning, which may cause the model to filter out some necessary information that helps distinguish unknown categories.
[0075] Zhou et al. designed a spectral spatial latent reconstruction method that enhances the learned feature representation by reconstructing the spectral and spatial features of hyperspectral data. This preserves spectral spatial information that helps reject unknown categories and distinguish known categories, and utilizes feature-level reconstruction errors of spectral and spatial features to differentiate unknown categories. However, because the reconstruction method is based solely on known samples, it cannot fully identify the complex and diverse information of unknown categories, and it is highly dependent on the quality of the training set when dealing with open-set problems.
[0076] The inventors of this application have discovered the following problems with currently proposed hyperspectral open set classification methods.
[0077] First, the spatial-spectral joint characteristics of hyperspectral images necessitate simultaneous attention to features in both spatial and channel dimensions during classification tasks. However, traditional convolutional neural networks (CNNs) are limited by a fixed receptive field during feature extraction, making it difficult to effectively capture salient features across dimensions. Furthermore, the feature extraction modules commonly used in traditional CNNs cannot adequately distinguish between relevant and irrelevant features, leading to the generation of numerous redundant feature maps and weakening classification performance.
[0078] Second, in practical applications, the acquisition scenarios for hyperspectral images are complex and diverse, often resulting in inconsistent distributions between training and test data. For example, the spectral characteristics of ground features may differ due to variations in imaging conditions or acquisition by different sensors. This inconsistency makes it difficult for existing open-set identification methods that rely on the distribution of training data to accurately model unknown class samples. This is especially true when using outliers for unknown class identification, where the model may misclassify abnormal samples of known classes as unknown classes or ignore some truly unknown class samples. Furthermore, traditional outlier detection methods typically rely on the feature distribution of known categories in the training data, neglecting the potential complexity of hyperspectral images and the diversity of unknown class samples, leading to high uncertainty in the face of unseen data.
[0079] Third, in the process of hyperspectral classification, the task bias of neural networks and the imbalance of training data distribution often lead to the classifier over-compressing intra-class features while pursuing high accuracy. This phenomenon causes the model to ignore some key information that can distinguish unknown categories, especially when the spectral and spatial features of hyperspectral images interact in a complex manner. Furthermore, the imbalance of data distribution can also cause the network to favor high-frequency features of known categories during training, thereby weakening the model's sensitivity to unknown categories and ultimately affecting the robustness of classification.
[0080] To address the aforementioned technical problems, one embodiment of this application proposes a hyperspectral open set classification method combining feature reconstruction and prototype constraints, applied to electronic devices. The electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the hyperspectral open set classification method combining feature reconstruction and prototype constraints proposed in this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.
[0081] The specific process of the hyperspectral open set classification method combining feature reconstruction and prototype constraints proposed in this embodiment can be described as follows: Figure 1 As shown, it includes:
[0082] Step 101: Divide the original sample data in the original sample dataset into blocks to obtain a training set consisting of several sample data blocks, and use the label of the center pixel of the top layer of the sample data block as the label of the entire sample data block.
[0083] In the specific implementation, before training the model, the server needs to divide the original sample data in the original sample dataset into blocks, thus obtaining a training set composed of several sample data blocks. It's important to note that the original sample data in the original sample dataset is specifically a hyperspectral image data cube. The data block operation is designed based on the spatial-spectral joint characteristics of hyperspectral data, and the resulting sample data blocks exist in the form of three-dimensional data cubes. After completing the data block division, the server also needs to use the label of the center pixel of the top layer of each sample data block as the label of the entire sample data block. This label is used to characterize the category to which the sample data block belongs.
[0084] In one example, the shape of the original sample data is as follows: Figure 2 As shown, the shape of the original sample data in the original sample dataset is... , This represents the number of rows of pixels in the original sample data. The number of columns of pixels in the original sample data. This represents the number of bands in the original sample data. Each pixel in the original sample data is labeled with a category representation. The server divides the data according to a preset classification standard (i.e., ...). and The original sample data in the original sample dataset is divided into blocks, and several sample data blocks are extracted from each original sample data. A training set is formed based on all these sample data blocks. The label of the center pixel of the top layer of each sample data block is used as the label of the entire sample data block to represent its category. It is important to note that each sample data block has the same shape. , , .
[0085] In one example, the server needs to build a test set at the same time as building the training set, in order to perform performance testing on the trained classification model.
[0086] Step 102: Input the sample data block into the classification model consisting of an encoder, a decoder, a prototype classification module and an unknown category detection module. Use the encoder to extract features from the sample data block to obtain the corresponding feature vectors. At the same time, calculate the mean of each known category based on the feature vectors, construct a prototype library, and construct a comparative prototype loss based on the prototype library.
[0087] In the specific implementation, the server needs to pre-build a classification model consisting of an encoder, a decoder, a prototype classification module, and an unknown category detection module. After obtaining the training set, sample data blocks can be input into this training model. The sample data blocks first enter the encoder of the classification model. The encoder extracts features from the sample data blocks to obtain the corresponding feature vectors. At the same time, it calculates the mean of each known category based on the feature vectors, constructs a prototype library, and builds a comparative prototype loss for model training based on the prototype library.
[0088] In one example, the specific structure of the classification model can be as follows: Figure 3 As shown.
[0089] In one example, the specific structure of the encoder can be as follows: Figure 4 As shown, it consists of multiple convolutional layers and global average pooling layers, and its function is to extract spatial and spectral features from hyperspectral images.
[0090] First, the encoder concatenates the sample data blocks together using two parallel convolutions to obtain the first intermediate features. First intermediate feature This can be expressed by the formula:
[0091] ;
[0092] in, Represents a sample data block. and This represents two parallel convolutions. Indicates splicing.
[0093] Then, the encoder processes the first intermediate feature. After processing by the residual module with added channel attention mechanism, the second intermediate feature is obtained. Second intermediate feature This can be expressed by the formula:
[0094] ;
[0095] in, and Both represent convolution. This indicates the channel attention mechanism.
[0096] Then, the encoder will use the second intermediate feature The third intermediate feature is obtained by weighting the signal with both channel attention and spatial attention. Third intermediate feature This can be expressed by the formula:
[0097] ;
[0098] in, This represents the sigmoid activation function. and This represents a multilayer perceptron. Represents the ReLU activation function. Indicates the second intermediate feature medium pixel The value, Represents convolution. Indicates the total number of channels. Represents the first sample data block One channel, This represents the vector product operation.
[0099] Finally, the encoder processes the third intermediate feature. By performing average pooling, the corresponding feature vector can be obtained. .
[0100] In one example, besides feature extraction, the encoder is also responsible for constructing a prototype library. The encoder calculates the mean of each known class based on the feature vectors to construct the prototype library, which can be achieved using the following formula:
[0101] ;
[0102] ;
[0103] ;
[0104] in, This represents the total number of known categories. Indicates the first The prototype corresponding to each known category This represents the prototype library that has been constructed. Indicates belonging to the first The total number of feature vectors for known categories. Indicates the first The one belonging to the first Feature vectors of known categories This indicates the encoder.
[0105] After classification, the encoder selects the feature vectors with confidence levels lower than a preset confidence threshold from the classification results and calculates their mean. This mean is used as a placeholder for the unknown category. The placeholder for the unknown category is expressed by the formula:
[0106] ;
[0107] in, This represents the total number of feature vectors with a confidence level less than a preset confidence threshold. Indicates the first A feature vector with a confidence level less than a preset confidence threshold. Placeholder for an unknown category.
[0108] Every An update is required every iteration, with the update weight set to [value]. , .
[0109] Following this, the encoder constructs a contrastive prototype loss based on the prototype library for model training. The purpose of the contrastive prototype loss is to encourage the classification model to learn semantic information related to known categories, minimizing the overlap of features between classes. The constructed contrastive prototype loss can be expressed by the formula:
[0110] The comparative prototype loss is built based on the prototype library and is achieved through the following formula:
[0111] ;
[0112] ;
[0113]
[0114] ;
[0115] in, express and The Euclidean distance between them This represents the maximum distance between all prototypes in the prototype library. Indicates the first The first of the known categories The distance between each feature vector and each prototype in the prototype library This represents the loss of the constructed comparative prototype.
[0116] By imposing normalization constraints on the feature vectors, the encoder is able to extract more compact and representative feature representations. These features are then passed to the decoder for reconstruction verification, and to the prototype classification module for classification and subsequent open-set recognition.
[0117] Step 103: Use the decoder to perform feature reconstruction based on the feature vector to obtain the reconstructed sample data block, and construct the reconstruction error loss based on the sample data block and the reconstructed sample data block.
[0118] In the specific implementation, the encoder inputs the feature vector into the decoder, the decoder performs feature reconstruction based on the feature vector to obtain the reconstructed sample data block, and constructs the reconstruction error loss based on the sample data block and the reconstructed sample data block.
[0119] In one example, the specific structure of the encoder can be as follows: Figure 5 As shown, the decoder employs a deconvolution structure symmetrical to the encoder. Its main purpose is to verify the quality of features extracted by the encoder by reconstructing the input data, while simultaneously improving the classification model's ability to identify unknown categories. The decoder progressively restores the high-dimensional feature vectors to data with the same shape as the input hyperspectral small cube. The reconstruction error can be calculated by comparing the shallow features of the reconstructed samples output by the decoder with the shallow features of the input samples. The decoder constructs the reconstruction error loss based on the sample data blocks and the reconstructed sample data blocks, which can be achieved using the following formula:
[0120] ;
[0121] in, This indicates the number of sample data blocks input in each iteration. Indicates the number generated by the encoder A shallow semantic representation The decoder reconstructs the first A shallow semantic representation This represents the reconstruction error loss obtained from the construction.
[0122] This backpropagation adjusts the weights of the encoder and decoder, ensuring that the reconstructed sample is as close as possible to the input sample. Since the decoder is trained only on known classes, the model cannot effectively reconstruct samples belonging to unknown classes, resulting in a large reconstruction error. Therefore, the reconstruction error can be used as one of the criteria for detecting unknown classes.
[0123] Step 104: Use the prototype classification module to perform closed-set classification based on feature vectors and prototype library to obtain the closed-set classification results of sample data blocks, and construct cross-entropy loss based on the closed-set classification results and the labels of sample data blocks.
[0124] In the specific implementation, the prototype classification module will perform closed-set classification based on feature vectors and prototype library to obtain closed-set classification results for sample data blocks, and construct cross-entropy loss based on closed-set classification results and labels of sample data blocks.
[0125] In one example, the prototype classification module needs to calculate the Euclidean distance between the feature vector and each prototype in the prototype library, and then select the category corresponding to the prototype with the smallest Euclidean distance as the closed-set classification result for the sample data block.
[0126] In one example, the prototype classification module constructs the cross-entropy loss based on the closed-set classification results and the labels of the sample data blocks, which can be achieved using the following formula:
[0127] ;
[0128] in, This represents the classification result of the closed set. Indicates a label, This represents the cross-entropy loss of the construct.
[0129] Step 105: Construct a total loss function based on the prototype loss, reconstruction error loss, and cross-entropy loss, and iteratively train the classification model based on the total loss function until convergence to obtain the trained classification model.
[0130] In the specific implementation, after obtaining the contrast prototype loss, reconstruction error loss, and cross-entropy loss, the server can construct a total loss function based on the contrast prototype loss, reconstruction error loss, and cross-entropy loss, and then iteratively train the classification model based on the total loss function until convergence, thus obtaining the trained classification model.
[0131] In one example, the server constructs a total loss function based on the contrastive prototype loss, reconstruction error loss, and cross-entropy loss, using the following formula:
[0132] ;
[0133] in, , and The preset weight parameters, To construct the total loss function.
[0134] Step 106: Using the unknown category detection module of the trained classification model, open set identification is performed based on the sample feature distribution and reconstruction error loss of the training set. The probability score of the unknown category is evaluated by combining the prototype distance and reconstruction loss. The probability scores of all sample data blocks in the training set are analyzed. The tail is modeled using the Weibull distribution. The cumulative distribution function is approximated by the generalized Pareto distribution. The first 50% of the tail of the entire cumulative distribution function is determined to be the unknown category.
[0135] In its implementation, the classification model uses prototype distance to classify known categories and combines prototype distance and reconstruction loss to jointly determine unknown categories. The server utilizes the unknown category detection module of the trained classification model. Based on the sample feature distribution of the training set and reconstruction error loss, it performs open-set identification, evaluates the probability score of unknown categories by combining prototype distance and reconstruction loss, analyzes the probability scores of all sample data blocks in the training set, models the tail using a Weibull distribution, approximates its cumulative distribution function using a generalized Pareto distribution, and determines the top 50% of the tail of the entire cumulative distribution function as unknown categories.
[0136] In one example, the probability score for the unknown category is constructed as follows:
[0137] ;
[0138] in, The preset evaluation adjustment coefficient, This represents the probability that a sample data block belongs to each known category. The probability that a sample data block belongs to an unknown category.
[0139] In one example, analyzing the probability scores of all sample data blocks in the training set, modeling the tail using a Weibull distribution, and approximating its cumulative distribution function using a generalized Pareto distribution, can be achieved using the following formula:
[0140] ;
[0141] in, Represents the Weibull distribution. This represents the generalized Pareto distribution. and All are learnable parameters. and The value is determined through model training.
[0142] In one example, determining the top 50% of the tail of the entire cumulative distribution function as the unknown category can be achieved using the following formula:
[0143] ;
[0144] in, Adjust the preset mapping parameters.
[0145] This embodiment proposes a hyperspectral open-set classification method that combines feature reconstruction and prototype constraints. By choosing to reconstruct shallow semantic representations instead of high-dimensional raw image data, it significantly reduces the computational complexity of the model and allows feature extraction to focus more on key information for class differentiation. The shallow semantic representation carries highly discriminative features of known classes while retaining sufficient discriminative power for the complexity of unknown classes, thus effectively improving the classification efficiency and accuracy of the model in open-set environments. Based on the manifold representation generated by the encoder, efficient differentiation of known classes is achieved by modeling each known class separately. Furthermore, using the encoder reconstruction error as a measure of class affiliation enables effective classification of sample data blocks, ensuring that the classification model performs well in both rejecting unknown classes and recognizing known classes, effectively improving the robustness of classification. By introducing placeholders for unknown classes in contrastive prototype learning, the risk of overlap between known and unknown classes in the feature space is significantly reduced. Moreover, unlike contrastive learning based on reciprocity points, this application allows known classes to be arranged hierarchically around placeholders, optimizing the utilization of the feature space. Even with a large number of categories, it can still maintain fast convergence, effectively improving the overall convergence efficiency and stability of the algorithm.
[0146] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.
[0147] Another embodiment of this application proposes an electronic device, the specific structure of which is as follows: Figure 6 As shown, it includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, the instructions being executed by the at least one processor 201 to enable the at least one processor 201 to perform a hyperspectral open set classification method combining feature reconstruction and prototype constraints as described in the above method embodiments.
[0148] The memory and processor can be connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.
[0149] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.
[0150] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, enables a hyperspectral open set classification method combining feature reconstruction and prototype constraints as described in the above method embodiments.
[0151] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (such as a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, or optical disk.
[0152] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.
Claims
1. A hyperspectral open set classification method combining feature reconstruction and prototype constraints, characterized in that, include: The original sample data in the original sample dataset is divided into blocks to obtain a training set consisting of several sample data blocks. The label of the center pixel of the top layer of the sample data block is used as the label of the entire sample data block. The original sample data is a hyperspectral image data cube. The sample data block is input into the classification model consisting of an encoder, a decoder, a prototype classification module, and an unknown category detection module. The encoder extracts features from the sample data block to obtain the corresponding feature vector. At the same time, the mean of each known category is calculated based on the feature vector, a prototype library is constructed, and a comparative prototype loss is built based on the prototype library. The decoder is used to reconstruct features based on feature vectors to obtain reconstructed sample data blocks, and a reconstruction error loss is constructed based on the sample data blocks and the reconstructed sample data blocks. The prototype classification module is used to perform closed-set classification based on feature vectors and prototype library to obtain closed-set classification results for sample data blocks, and cross-entropy loss is constructed based on the closed-set classification results and the labels of sample data blocks. A total loss function is constructed based on the comparative prototype loss, reconstruction error loss, and cross-entropy loss. The classification model is then iteratively trained until convergence based on the total loss function to obtain the trained classification model. Using the unknown category detection module of the trained classification model, open set identification is performed based on the sample feature distribution and reconstruction error loss of the training set. The probability score of the unknown category is evaluated by combining the prototype distance and reconstruction loss. The probability scores of all sample data blocks in the training set are analyzed. The tail is modeled using the Weibull distribution, and its cumulative distribution function is approximated by the generalized Pareto distribution. The first 50% of the tail of the entire cumulative distribution function is determined to be the unknown category.
2. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 1, characterized in that, The shape of the original sample data in the original sample dataset is as follows: , This represents the number of rows of pixels in the original sample data. The number of columns of pixels in the original sample data. The number of bands in the original sample data is given. Each pixel in the original sample data is labeled with a category representation. The original sample data in the original sample dataset is divided into blocks according to a preset partitioning standard. Several sample data blocks are then extracted from each original sample data block. A training set is formed based on all these sample data blocks, and the label of the center pixel at the top layer of each sample data block is used as the label for the entire sample data block. Each sample data block has the following shape: , , .
3. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 1, characterized in that, The encoder consists of multiple convolutional layers and global average pooling layers; The encoder is used to extract features from the sample data block to obtain the corresponding feature vectors, including: First, the sample data blocks are concatenated together using two parallel convolutions to obtain the first intermediate feature. First intermediate feature This can be expressed by the formula: ； in, Represents a sample data block. and This represents two parallel convolutions. Indicates splicing; Then, the residual module with added channel attention mechanism is used to process the first intermediate feature. Processing is performed to obtain the second intermediate feature. Second intermediate feature This can be expressed by the formula: ； in, and Both represent convolution. This indicates the channel attention mechanism; Then, regarding the second intermediate feature By applying a combined weighted sum of channel attention and spatial attention, a third intermediate feature is obtained. Third intermediate feature This can be expressed by the formula: ； in, This represents the sigmoid activation function. and This represents a multilayer perceptron. Represents the ReLU activation function. Indicates the second intermediate feature medium pixel The value, Represents convolution. Indicates the total number of channels. Represents the first sample data block One channel, This represents the vector product operation; Finally, regarding the third intermediate feature Perform average pooling to obtain the corresponding feature vector. .
4. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 3, characterized in that, The mean of each known category is calculated based on the feature vectors, and a prototype library is constructed using the following formula: ； ； in, This represents the total number of known categories. Indicates the first The prototype corresponding to each known category This represents the prototype library that has been constructed. Indicates belonging to the first The total number of feature vectors for known categories. Indicates the first The one belonging to the first Feature vectors of known categories; The mean of feature vectors with confidence levels less than a preset confidence threshold is calculated and used as a placeholder for the unknown category. The placeholder for the unknown category is expressed by the formula: ； in, This represents the total number of feature vectors with a confidence level less than a preset confidence threshold. Indicates the first A feature vector with a confidence level less than a preset confidence threshold. Placeholder for an unknown category; The comparative prototype loss is built based on the prototype library and is achieved through the following formula: ； ； ； in, express and The Euclidean distance between them This represents the maximum distance between all prototypes in the prototype library. Indicates the first The first of the known categories The distance between each feature vector and each prototype in the prototype library This represents the loss of the constructed comparative prototype.
5. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 4, characterized in that, The decoder employs a deconvolution structure symmetrical to the encoder; The reconstruction error loss is constructed based on the sample data block and the reconstructed sample data block, and is achieved through the following formula: ； in, This indicates the number of sample data blocks input in each iteration. Indicates the number generated by the encoder A shallow semantic representation The decoder reconstructs the first A shallow semantic representation This represents the reconstruction error loss obtained from the construction.
6. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 5, characterized in that, The prototype classification module performs closed-set classification based on feature vectors and a prototype library to obtain closed-set classification results for sample data blocks, including: Calculate the Euclidean distance between the feature vector and each prototype in the prototype library; The category corresponding to the prototype with the smallest Euclidean distance is selected as the closed-set classification result for the sample data block. The cross-entropy loss is constructed based on the closed-set classification results and the labels of the sample data blocks, and is implemented using the following formula: ； in, This represents the classification result of the closed set. Indicates a label, This represents the cross-entropy loss of the construct.
7. The hyperspectral open set classification method combining feature reconstruction and prototype constraints according to claim 6, characterized in that, The total loss function is constructed based on the prototype loss, reconstruction error loss, and cross-entropy loss, and is implemented using the following formula: ； in, , and The preset weight parameters, To construct the total loss function.
8. A hyperspectral open set classification method combining feature reconstruction and prototype constraints according to any one of claims 4 to 7, characterized in that, The probability score for the unknown category is constructed as follows: ； in, The preset evaluation adjustment coefficient, This represents the probability that a sample data block belongs to each known category. The probability that a sample data block belongs to an unknown category; Analyze the probability scores of all sample data blocks in the training set, model the tail using a Weibull distribution, and approximate its cumulative distribution function using a generalized Pareto distribution, achieved through the following formula: ； in, Represents the Weibull distribution. This represents the generalized Pareto distribution. and All are learnable parameters. and The value is determined through model training; The first 50% of the tail of the cumulative distribution function are identified as the unknown category using the following formula: ； in, Adjust the preset mapping parameters.
9. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform a hyperspectral open set classification method combining feature reconstruction and prototype constraints as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it can implement a hyperspectral open set classification method that combines feature reconstruction and prototype constraints as described in any one of claims 1 to 8.