Hyperspectral open set recognition system and method based on class semantic reconstruction

Abstract

The invention belongs to the technical field of open set recognition, and discloses a hyperspectral open set recognition system and method based on class semantic reconstruction. The hyperspectral open set recognition system based on class semantic reconstruction comprises a grouping spectral space reservation transformer module which comprises three stages of hierarchical structures, and each stage of hierarchical structure comprises a grouping pixel embedding module and a space enhancement feature transformation module. The spectrum-spatial feature fusion module is used for fusing the stage spectrum-spatial features into discriminative spectrum-spatial features; the class semantic reconstruction module comprises a plurality of class auto-encoders and is used for performing class semantic reconstruction on the discriminative spectral-spatial features to obtain reconstruction errors, constructing and training a classification model through the reconstruction errors and finally realizing open set recognition in combination with a multi-dimensional scoring function; according to the method, inter-class confusion can be reduced, and interference of background noise information is reduced by reconstructing semantic features instead of original pixels.

Description

Technical Field

[0001] The present invention relates to the technical field of open set recognition, and in particular to a hyperspectral open set recognition system and method based on class semantic reconstruction. Background Art

[0002] Hyperspectral imaging continuously collects spectral information in hundreds of narrow bands, which can provide more detailed material representation than traditional three-channel images. It has important value in agricultural monitoring, ecological protection, mineral exploration and other fields.

[0003] However, existing reconstruction-based methods distinguish known from unknown categories by comparing the errors between input data and reconstructed output, but their performance is limited by the inherent characteristics of hyperspectral data and faces significant limitations in practical applications. First, hyperspectral images contain hundreds of bands. Sensor noise, illumination changes, and background interference will directly contaminate the pixel-level reconstruction process, thereby weakening the model's ability to extract discriminative features. Second, existing methods usually use a single autoencoder to uniformly reconstruct all known categories, ignoring the subtle differences in spectral-spatial features of different categories. The model cannot effectively distinguish similar categories, which in turn causes classification confidence ambiguity or even misclassification, resulting in serious inter-class confusion and ambiguity in classification confidence. Summary of the Invention

[0004] The purpose of the present invention is to provide a hyperspectral open set recognition system and method based on class semantic reconstruction to overcome the problems existing in the prior art. The present invention can design an independent autoencoder for each known category, map the features of different categories to separate manifold spaces, thereby reducing inter-class confusion, and at the same time, by reconstructing semantic features rather than original pixels, reducing the interference of background noise information and improving the robustness of unknown category detection; it can also combine grouped pixel embedding and spatial enhancement feature transformation modules to extract discriminative features, and focus on the central pixel through a spatially maintained attention mechanism, thereby suppressing noise while reducing computational complexity.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a hyperspectral open set recognition system based on class semantic reconstruction, comprising:

[0007] The grouped spectral-space preserving transformer module includes a three-stage hierarchical structure. Each stage of the hierarchical structure includes a grouped pixel embedding module and a spatial enhancement feature transformation module, which is used to fuse the stage spectral-spatial features into discriminative spectral-spatial features;

[0008] The class semantic reconstruction module includes several class autoencoders, which are used to perform class semantic reconstruction on the discriminative spectral-spatial features to obtain the reconstruction error. The classification model is constructed and trained based on the reconstruction error, and finally open set recognition is achieved by combining the multi-dimensional scoring function.

[0009] In a second aspect, the present invention provides a hyperspectral open set recognition method based on class semantic reconstruction, based on the above-mentioned hyperspectral open set recognition system based on class semantic reconstruction, comprising the following steps:

[0010] Step 1: construct a hyperspectral image dataset and divide the hyperspectral image dataset into training samples and test samples;

[0011] Step 2: The hyperspectral image dataset is input into the grouped spectral space-preserving transformer module to extract discriminative spectral-spatial features;

[0012] Step 3: The training samples are subjected to category semantic reconstruction of the discriminative spectral-spatial features through the class semantic reconstruction module to obtain the reconstruction error, and the classification model is constructed and trained based on the reconstruction error;

[0013] Step 4: Input the test sample into the trained classification model, classify the test sample into known categories through reconstruction error, and use the multi-dimensional scoring function to identify the unknown category of the test sample after the known category classification, so as to achieve open set recognition;

[0014] Furthermore, the hyperspectral image dataset is input into the grouped spectral space preserving transformer module to extract discriminative spectral-spatial features, specifically including:

[0015] The hyperspectral image dataset is input into the Grouped Spectral Space Preserving Transformer module. The input hyperspectral image dataset passes through the three-stage hierarchical structure of the Grouped Spectral Space Preserving Transformer module in sequence to extract discriminative spectral-spatial features. In each stage of the hierarchical structure, the module passes through the Grouped Pixel Embedding module and the Spatial Enhanced Feature Transformation module in sequence.

[0016] Furthermore, the pixel grouping embedding module and the spatial enhancement feature transformation module specifically include:

[0017] The grouped pixel embedding module divides the high-dimensional spectral dimension of the hyperspectral image into several groups of non-overlapping sub-channels. Each group of non-overlapping sub-channels extracts local spectral features through independent 2D convolution to obtain several local spectral features. These local spectral features are then concatenated to form fusion features. The fusion features are then used to extract global features through a spatially preserved attention mechanism. When in the first and second stage hierarchical structures, stage spectral-spatial features are obtained; when in the third stage hierarchical structure, discriminative spectral-spatial features are obtained.

[0018] Furthermore, the fusion features are extracted into global features through a spatial attention mechanism, specifically including:

[0019] A spatial attenuation matrix is ​​introduced into the spatial preservation attention mechanism. The attention weight of the fused features is dynamically adjusted through the Manhattan distance. The attention of the fused features is decomposed into horizontal attention and vertical attention. The horizontal attention and vertical attention are calculated separately to obtain the horizontal attention calculation results and the vertical attention calculation results are fused to obtain the final attention. The final attention is used to extract global features.

[0020] Furthermore, the spatial attenuation matrix is ​​introduced into the spatial attention mechanism, and the formula specifically includes:

[0021] Q=(XW Q )eα;

[0022]

[0023] V=XW V ;

[0024]

[0025] RetNet(X)=(QK · eM)V;

[0026] Where X represents the input feature; Q represents the query matrix; K represents the key matrix; V represents the value matrix; W Q W represents the learnable parameter matrix used to transform the query matrix; K W represents the learnable parameter matrix used to transform the key matrix; V represents a learnable parameter matrix used to transform the value matrix; α represents a column vector; a i Represents the elements in each column vector; θ represents the learnable column vector; i represents an imaginary number; represents the complex conjugate of α; ⊙ represents the Hadamard product, which is used to multiply the corresponding elements of two vectors or matrices; M ij represents the attenuation matrix; x i Indicates the column index of the pixel; x j Represents the row index of the pixel; Represents a constant ranging from 0 to 1; RetNet represents the RetNet network calculation formula; T represents matrix transpose; M represents the spatial attenuation matrix;

[0027] The Manhattan distance formula specifically includes:

[0028]

[0029] Where x 1 represents a pixel in the input image block; x 2 represents another pixel in the input image patch;

[0030] Furthermore, the training samples are subjected to category semantic reconstruction of the discriminative spectral-spatial features through a class semantic reconstruction module to obtain a reconstruction error, and a classification model is constructed and trained based on the reconstruction error, specifically including:

[0031] The reconstruction error of the discriminative spectral-spatial features in several class autoencoders is calculated for the training samples, and the probability of each class is generated by Softmax. The loss function is constructed based on the negative logarithmic probability of the true label, and the classification model is trained by minimizing the loss function.

[0032] Furthermore, the step of inputting the test sample into the trained classification model and classifying the test sample into known categories by reconstructing the error specifically includes:

[0033] The test sample is input into the trained classification model, and the reconstruction error of the discriminative spectral-spatial features in several category autoencoders is calculated. The probability of each category is generated through Softmax. The larger the error, the lower the probability of the category. The category with the highest probability is used as the classification category to achieve known category classification of the test sample.

[0034] Furthermore, the formula of the multi-dimensional scoring function specifically includes:

[0035]

[0036]

[0037] Where S1 represents the first-order statistical score; F represents the discriminative spectral-spatial features extracted from the test sample; f represents the features extracted by the grouped spectral space preserving transformer module; represents the pixel reconstruction error of the predicted category c; S2 represents the second-order statistical score; T represents the matrix transpose operation; Indicates the strength of category perception features; S3 indicates the third-order statistical score; Sum indicates summation; G c represents the average matrix of predicted categories; Represents element-by-element multiplication of matrices; I p Indicates that the feature F is connected by column and then subjected to exponential operation; p represents a constant;

[0038] Furthermore, the open set recognition of unknown category samples of the test samples by using the multi-dimensional scoring function specifically includes:

[0039] After S1, S2 and S3 are normalized, they are weighted and fused to obtain the final score. If the final score is less than or equal to the pre-set threshold, it is considered an unknown category.

[0040] If the final score is greater than the pre-set threshold, it is a known category.

[0041] The above technical solution has the following advantages or beneficial effects:

[0042] In the first aspect, the present invention provides a hyperspectral open set recognition system based on class semantic reconstruction. Through the grouped spectral space preserving transformer module, the spectral dimension is divided into non-overlapping sub-channels through grouped pixel embedding, and local spectral features are independently extracted. At the same time, combined with the spatial enhancement feature transformation module, a spatial attenuation matrix is ​​introduced to make the model focus on the central pixel; through the class semantic reconstruction module, an independent autoencoder is assigned to each known category, thereby alleviating the problem of inter-class confusion. By reconstructing semantic features instead of original pixels, the interference of background noise information is reduced, and the robustness of unknown category detection is improved.

[0043] In the second aspect, the present invention provides a hyperspectral open set recognition method based on class semantic reconstruction. By utilizing the grouped spectral space-preserving transformer module to extract discriminative spectral-spatial features, it can fully mine the key information in the hyperspectral image, effectively retain the important features of the data, provide high-quality input for subsequent processing, and help improve the recognition accuracy; through the class semantic reconstruction module, the features of the training samples are subjected to category semantic reconstruction and the reconstruction error is obtained, thereby constructing and training a classification model, which can enable the model to better understand the semantic information of different categories, enhance the model's ability to distinguish known categories, and improve classification performance; in open set recognition, known category samples in the test samples can be accurately classified, and at the same time, a multi-dimensional scoring function is used to perform open set recognition on unknown category samples, thereby alleviating the problem of inter-class confusion and reducing the interference of background noise information.

[0044] Furthermore, by extracting discriminative spectral-spatial features, the spectrum is divided into sub-channels using grouped pixel embedding, local spectral features are extracted independently, and the spatial enhancement feature transformation module is used to extract global features and optimize the utilization of the central pixel information of the input hyperspectral image.

[0045] Furthermore, by assigning an independent autoencoder to each known category, the semantic features of different categories are mapped to the separation manifold space, which alleviates the inter-class confusion of the model. By reconstructing semantic features instead of original pixels, the background noise interference of the input data is effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1Schematic diagram of the process of a hyperspectral open set recognition method based on class semantic reconstruction of the present invention;

[0047] Figure 2 This is a module diagram of a hyperspectral open set recognition system based on class semantic reconstruction in the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to specific embodiments, which are intended to explain the present invention rather than to limit it.

[0049] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0051] Example:

[0052] See also Figure 2 The present invention provides a hyperspectral open set recognition system based on class semantic reconstruction, comprising:

[0053] The grouped spectral-space preserving transformer module includes a three-stage hierarchical structure. Each stage of the hierarchical structure includes a grouped pixel embedding module and a spatial enhancement feature transformation module, which is used to fuse the stage spectral-spatial features into discriminative spectral-spatial features, realizing feature fusion from local to global.

[0054] The class semantic reconstruction module includes several class autoencoders, which are used to perform class semantic reconstruction on the discriminative spectral-spatial features to obtain the reconstruction error. The classification model is constructed and trained based on the reconstruction error, and finally open set recognition is achieved by combining the multi-dimensional scoring function.

[0055] See also Figure 1 In one embodiment of the present invention, a hyperspectral open set recognition method based on class semantic reconstruction is provided, comprising the following steps:

[0056] Step 1: construct a hyperspectral image dataset and divide the hyperspectral image dataset into training samples and test samples. The training samples include several known category samples, and the test samples include several known category samples and unknown category samples.

[0057] Step 2: The hyperspectral image dataset is input into the Grouped Spectral Space Preserving Transformer module. The input hyperspectral image dataset passes through the three-stage hierarchical structure of the Grouped Spectral Space Preserving Transformer module in sequence to extract discriminative spectral-spatial features. In each stage of the hierarchical structure, the dataset passes through the Grouped Pixel Embedding Module and the Spatial Enhanced Feature Transformation Module in sequence.

[0058] Specifically, through the grouped pixel embedding module and the spatial enhancement feature transformation module, including: the grouped pixel embedding module divides the high-dimensional characteristic spectral dimension of the hyperspectral image into several groups of non-overlapping sub-channels, and each group of non-overlapping sub-channels extracts local spectral features through independent 2D convolution to obtain several local spectral features, and then splices several local spectral features to form fusion features. Compared with traditional convolution, the grouped pixel embedding module effectively extracts the local detail features of the hyperspectral image through spectral focusing, and extracts global features from the fusion features through the space-preserving attention mechanism. Among them, the spatial attenuation matrix is ​​introduced in the space-preserving attention mechanism. The formula specifically includes:

[0059] Q=(XW Q )eα;

[0060]

[0061] V=XW V ;

[0062]

[0063] RetNet(X)=(QK · e M)V;

[0064] Where X represents the input feature; Q represents the query matrix; K represents the key matrix; V represents the value matrix; W Q W represents the learnable parameter matrix used to transform the query matrix; K W represents the learnable parameter matrix used to transform the key matrix; V represents a learnable parameter matrix used to transform the value matrix; α represents a column vector; a iRepresents the elements in each column vector; θ represents the learnable column vector; i represents an imaginary number; represents the complex conjugate of α; ⊙ represents the Hadamard product, which is used to multiply the corresponding elements of two vectors or matrices; M ij represents the attenuation matrix; x i Indicates the column index of the pixel; x j Represents the row index of the pixel; Represents a constant ranging from 0 to 1; RetNet represents the RetNet network calculation formula; T represents matrix transpose; M represents the spatial attenuation matrix;

[0065] The Manhattan distance is used to dynamically adjust the attention weight of the fusion feature, so that the classification model prioritizes the neighborhood information of the central pixel and reduces the interference of long-distance noise. To further reduce the computational complexity of the attention mechanism, the formula includes:

[0066]

[0067] Where x 1 represents a pixel in the input image block; x 2 represents another pixel in the input image patch;

[0068] The attention of the fusion feature is decomposed into horizontal attention and vertical attention, and the horizontal attention and vertical attention are calculated respectively to obtain the horizontal attention calculation results and the vertical attention calculation results. The horizontal attention calculation results and the vertical attention calculation results are fused to obtain the final attention. The global feature is extracted through the final attention. When it is in the first and second stage hierarchical structures, the stage spectral-spatial features are obtained; when it is in the third stage hierarchical structure, the discriminative spectral-spatial features are obtained;

[0069] Step 3: The training samples are subjected to category semantic reconstruction of the discriminative spectral-spatial features through the class semantic reconstruction module to obtain the reconstruction error, and the classification model is constructed and trained based on the reconstruction error;

[0070] Specifically, the reconstruction error of the discriminative spectral-spatial features in several class autoencoders is calculated for the training samples, and the probability of each class is generated by Softmax. The loss function is constructed based on the negative logarithmic probability of the true label, and the classification model is trained by minimizing the loss function.

[0071] Step 4: Input the test sample into the trained classification model, calculate the reconstruction error of the discriminative spectral-spatial features in several category autoencoders, and generate the probability of each category through Softmax. The larger the error, the lower the probability of the category. The category with the maximum probability is used as the classification category to achieve known category classification of the test sample. The test sample after known category classification is identified by the multidimensional scoring function for unknown category recognition to achieve open set recognition. The formula of the multidimensional scoring function specifically includes:

[0072]

[0073] Where S1 represents the reconstruction statistical score; F represents the discriminative spectral-spatial features extracted from the test sample; f represents the features extracted by the grouped spectral space preserving transformer module; represents the pixel reconstruction error of the predicted category c; S2 represents the first-order statistical score; T represents the matrix transpose operation; Indicates the strength of category perception features; S3 indicates the second-order statistical score; Sum indicates summation; G c represents the average matrix of predicted categories; Represents element-by-element multiplication of matrices; I p Indicates that the feature F is connected by column and then subjected to exponential operation; p represents a constant;

[0074] After S1, S2 and S3 are normalized, they are weighted and fused to obtain the final score. If the final score is less than or equal to the pre-set threshold, it is considered an unknown category.

[0075] If the final score is greater than the pre-set threshold, it is a known category.

[0076] See also Figure 1 In one embodiment of the present invention, a hyperspectral open set recognition method based on class semantic reconstruction is provided, comprising the following steps:

[0077] Step 1: Construct a hyperspectral image dataset. Taking the Pavia University dataset as an example, there are nine known categories in total, and one unknown category is additionally marked. Among the nine known categories, 50 known category samples are selected for each known category, and a total of 450 known category samples are used as training samples. The known category samples that are not selected as training samples and one unknown category are used as test samples (that is, the remaining samples are used as test samples);

[0078] Step 2: The hyperspectral image dataset is input into the Grouped Spectral Space Preserving Transformer module. The input hyperspectral image dataset passes through the three-stage hierarchical structure of the Grouped Spectral Space Preserving Transformer module in sequence to extract discriminative spectral-spatial features. In each stage of the hierarchical structure, the dataset passes through the Grouped Pixel Embedding Module and the Spatial Enhanced Feature Transformation Module in sequence.

[0079] Specifically, through the grouped pixel embedding module and the spatial enhancement feature transformation module, including: the grouped pixel embedding module divides the high-dimensional characteristic spectral dimension of the hyperspectral image into several groups of non-overlapping sub-channels, and each group of non-overlapping sub-channels extracts local spectral features through independent 2D convolution to obtain several local spectral features, and then splices the several local spectral features to form fusion features, and extracts global features from the fusion features through the space-preserving attention mechanism, wherein the spatial attenuation matrix is ​​introduced into the space-preserving attention mechanism. The core inspiration of the space-preserving self-attention mechanism comes from the RetNet architecture. The RetNet architecture is an advanced neural network architecture originally designed for natural language processing tasks. RetNet reduces the retention of early information by introducing the time decay matrix M in the self-attention mechanism, thereby improving computational efficiency; the formula for introducing the spatial attenuation matrix in the space-preserving attention mechanism specifically includes:

[0080] Q=(XW Q )eα;

[0081]

[0082] V=XW V ;

[0083]

[0084] RetNet(X)=(QK · eM)V;

[0085] Where X represents the input feature; Q represents the query matrix; K represents the key matrix; V represents the value matrix; W Q W represents the learnable parameter matrix used to transform the query matrix; K W represents the learnable parameter matrix used to transform the key matrix; V represents a learnable parameter matrix used to transform the value matrix; α represents a column vector; a i Represents the elements in each column vector; θ represents the learnable column vector; i represents an imaginary number; represents the complex conjugate of α; ⊙ represents the Hadamard product, which is used to multiply the corresponding elements of two vectors or matrices; M ij represents the attenuation matrix; x i Indicates the column index of the pixel; xj Represents the row index of the pixel; Represents a constant ranging from 0 to 1; RetNet represents the RetNet network calculation formula; T represents matrix transpose; M represents the spatial attenuation matrix;

[0086] The attention weight of the fusion feature is dynamically adjusted through the Manhattan distance. Specifically, the relative distance between pixels is calculated using the Manhattan distance. The larger the distance, the greater the attenuation weight. The specific formula includes:

[0087]

[0088] Where x 1 represents a pixel in the input image block; x 2 Represents another pixel in the input image patch; in this way, the model is able to better capture the spatial information in the image while paying less attention to distant pixels;

[0089] In order to capture the spatial features in the image more finely, the spatial decomposition technology is introduced. The attention of the fusion feature is decomposed into horizontal attention and vertical attention. The horizontal attention and vertical attention are calculated separately to obtain the horizontal attention calculation results and the vertical attention calculation results. The horizontal attention calculation results and the vertical attention calculation results are fused to obtain the final attention. The final attention is used to extract the global features. The formula for fusing the horizontal attention calculation results and the vertical attention calculation results to obtain the final attention is as follows:

[0090]

[0091] Where H represents the vertical direction. By calculating the attention in the horizontal and vertical directions separately, the model can more effectively capture the spatial structure in the image. Finally, the horizontal and vertical attention results are combined to obtain the complete SRA output.

[0092] When in the first and second stage hierarchical structure, the stage spectrum-space characteristics are obtained; when in the third stage hierarchical structure, the discriminative spectrum-space characteristics are obtained;

[0093] Step 3: The training samples are subjected to category semantic reconstruction of the discriminative spectral-spatial features through the class semantic reconstruction module to obtain the reconstruction error, and the classification model is constructed and trained based on the reconstruction error;

[0094] Specifically, the discriminative spectral-spatial features are encoded and decoded into reconstructed discriminative spectral-spatial features. The semantic reconstruction module adopts a linear encoder and decoder structure and uses the L1-norm to calculate the error between the input discriminative spectral-spatial features (input features) and the reconstructed discriminative spectral-spatial features (reconstructed features). The formula specifically includes:

[0095]

[0096] Where, represents the reconstructed discriminative spectral-spatial features; AE represents autoencoder; Represents the error between the input discriminative spectral-spatial features and the reconstructed discriminative spectral-spatial features;

[0097] In this way, the classification model can quantify the reconstruction error of each category. In order to associate the reconstruction error with the category probability, the model performs softmax normalization on the error and defines the category probability. The formula includes:

[0098]

[0099] Where y represents the category it belongs to; exp represents the exponential function; γ represents the temperature parameter, and C represents the total number of categories of training samples;

[0100] Each pixel of the input image is reconstructed and its category is predicted to make full use of the spectral-spatial semantic features extracted by the grouped spectral space preserving transformer module. The category prediction of the input region is then obtained by averaging the predictions of all pixels. The specific formula includes:

[0101]

[0102] The loss function of the classification model is defined as the negative log probability of the true label category. The formula includes:

[0103] L = -logp(y = c|F);

[0104] The model is trained by minimizing the loss function;

[0105] Step 4: Input the test sample into the trained classification model, calculate the reconstruction error of the discriminative spectral-spatial features in several category autoencoders, and generate the probability of each category through Softmax. The larger the error, the lower the probability of the category. The category with the maximum probability is taken as the classification category to realize the classification of the known category of the test sample. A first-order statistical scoring function S1(F) is formulated based on the reconstruction error. Known category samples usually show a smaller reconstruction error relative to their corresponding labels, while they show a larger reconstruction error for unlabeled categories. On the contrary, samples from unknown categories have a larger reconstruction error for all labeled categories because they cannot be classified into any known category. In order to maximize the use of the semantic features extracted by the grouped spectral space preservation transformer module, we summarize and average the reconstruction errors of each pixel prediction category to obtain the final reconstruction score. Based on this concept, the specific formula of the reconstruction scoring function S1(F) includes:

[0106]

[0107] Where S1 represents the reconstruction statistical score; F represents the discriminative spectral-spatial features extracted from the test sample; f represents the discriminative spectral-spatial features of the input; represents the pixel reconstruction error of the predicted category c;

[0108] For the first-order and second-order statistical scores of the discriminative spectral-spatial features, we first t The semantic features are preprocessed, the absolute values ​​of all elements in the feature set are taken, the first-order statistics of the discriminative spectral-spatial features are calculated, the average activation intensity of each category is calculated, and then these intensities are normalized between different categories. The specific formula includes:

[0109]

[0110] Where, represents the normalized category-aware feature strength; c represents the perceptual feature strength of the c-th category; j represents the perceptual feature strength of the jth category; F t c Represents the feature set of all predicted categories c in the training samples;

[0111] The average activation intensity is used to identify unknown categories. Specifically, the discriminative spectral-spatial features of the same category tend to have more similar activation intensities, while the discriminative spectral-spatial features of unknown categories are significantly different from the average activation intensities of known categories. Therefore, we calculate the category-aware feature intensity The first-order statistical score of the discriminative spectral-spatial features is finally calculated by averaging the weighted extracted test data features on a pixel-by-pixel basis. The formula is as follows:

[0112]

[0113] Where S2 represents the first-order statistical score; T represents the matrix transpose operation;

[0114] For the second-order statistics of the discriminative spectral-spatial features, the Gram matrix is ​​used for calculation. First, F is concatenated by column to form a 2D matrix Where C represents the feature dimension. Then, the Gram matrix is ​​calculated for each feature category and the average matrix G for each category is derived. i , matrix G i It reflects the similarity between features of different categories. The formula includes:

[0115]

[0116] Where, I i Represents the feature set predicted to belong to category i; I p Indicates that the matrix I performs exponential operation; p represents a constant;

[0117] Multiply the Gram matrix corresponding to F with the average matrix Gc of the predicted category and add them pixel by pixel to obtain the second-order statistical score of the feature. The formula is as follows:

[0118]

[0119] Where, S3 represents the second-order statistical score; Sum represents the sum; G c represents the average matrix of predicted categories; Represents element-wise multiplication of matrices;

[0120] S1, S2, and S3 are then standardized and weightedly fused to obtain the final score. If the final score is ≤ a preset threshold, it is an unknown category; if the final score is > a preset threshold, it is a known category.

[0121] For open set recognition in test samples, the present invention adopts a triple scoring mechanism for joint decision-making. By fusing the reconstruction error and the first-order and second-order statistics of the features, it achieves accurate classification of known categories and effective identification of unknown categories. Specifically, for the reconstruction score, a basic discriminant index is constructed based on the normalized reconstruction error. The larger the error, the smaller the category probability. This score reflects the degree of match between the test sample and the known class feature manifold. The higher the value, the stronger the similarity with the known class, while the unknown class has a larger reconstruction error for all known classes. For the first-order statistical score, the feature distribution offset is measured by calculating the class activation strength weight. This score captures the distribution difference between the unknown class and the known class in low-order statistical features. For the second-order statistical score, the Graham matrix is ​​used to measure the correlation of high-order features. This score enhances the sensitivity to the distribution difference of high-order features.

[0122] Hyperspectral imaging technology, with its unique advantages of high spectral resolution and rich ground information, has shown broad application potential in dynamic and open scenes. This technology captures the fine reflective characteristics of the target through continuous narrow bands, providing key data support for multi-dimensional analysis and decision-making in agriculture, geology, environment, medicine and other fields. In the agricultural field, its ability to monitor the growth status of crops in real time can accurately identify the spectral characteristics of pest and disease plants, assist in disease early warning, and significantly improve agricultural production efficiency; in geological exploration, by efficiently distinguishing the spectral differences of mineral components, it can identify complex rock structures, providing high-precision technical means for mineral resource development and geological disaster risk assessment; in environmental monitoring, it dynamically tracks the diffusion path of water pollution, providing a scientific basis for ecological governance and emergency response.

[0123] This paper proposes a hyperspectral open set recognition framework based on semantic reconstruction. By assigning an independent autoencoder to each known category and reconstructing semantic features rather than pixel-level reconstruction, it significantly reduces the background noise interference of the image and the inter-class confusion problem existing in existing methods, and at the same time uses the difference in reconstruction error to identify unknown categories. First, this paper proposes a feature extraction network based on the attention mechanism, combined with the grouped spectral space-preserving transformer, integrating the spatial enhancement feature transformation module and the grouped pixel embedding technology to fully explore the spatial correlation and spectral details of hyperspectral data; secondly, a semantic reconstruction module is proposed to reconstruct the features extracted by the feature extractor according to category, calculate the reconstruction error in the autoencoders of different categories, construct the loss function based on their labels and category reconstruction errors, and use the gradient descent method to update the classification network until the maximum number of iterations is reached; finally, the classification results are obtained based on the reconstruction error of the test samples in the autoencoders of different categories.

[0124] This invention effectively improves the accuracy of open-set recognition of few-sample hyperspectral images and enhances the robustness of the model. It can be widely used in smart agriculture, mineral exploration, environmental testing, medical diagnosis and other fields, providing more accurate and reliable data support and analysis methods for related work in these fields, and promoting further development and innovation of technologies in various fields.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral open set recognition system based on class semantic reconstruction, characterized by: include: The grouped spectral-space preserving transformer module includes a three-stage hierarchical structure. Each stage of the hierarchical structure includes a grouped pixel embedding module and a spatial enhancement feature transformation module, which is used to fuse the stage spectral-spatial features into discriminative spectral-spatial features; The class semantic reconstruction module includes several class autoencoders, which are used to perform class semantic reconstruction on the discriminative spectral-spatial features to obtain the reconstruction error. The classification model is constructed and trained based on the reconstruction error, and finally open set recognition is achieved by combining the multi-dimensional scoring function.

2. A hyperspectral open set recognition method based on class semantic reconstruction, characterized in that: The hyperspectral open set recognition system based on class semantic reconstruction according to claim 1 comprises the following steps: Construct a hyperspectral image dataset and divide the hyperspectral image dataset into training samples and test samples; The hyperspectral image dataset is fed into the Grouped Spectral Space Preserving Transformer module to extract discriminative spectral-spatial features; The training samples are processed through the class semantic reconstruction module to perform category semantic reconstruction on the discriminative spectral-spatial features, and the reconstruction error is obtained. The classification model is then constructed and trained based on the reconstruction error. The test samples are input into the trained classification model, and the test samples are classified into known categories through reconstruction error. The test samples classified into known categories are identified into unknown categories through a multi-dimensional scoring function to achieve open set recognition.

3. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 2 is characterized in that: The hyperspectral image dataset is input into the grouped spectral space preserving transformer module to extract discriminative spectral-spatial features, specifically including: The hyperspectral image dataset is input into the grouped spectral space-preserving transformer module. The input hyperspectral image dataset passes through the three-stage hierarchical structure of the grouped spectral space-preserving transformer module in sequence to extract discriminative spectral-spatial features. Among them, in each stage of the hierarchical structure, it passes through the grouped pixel embedding module and the spatial enhancement feature transformation module in sequence.

4. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 3 is characterized in that: The pixel grouping embedding module and the spatial enhancement feature transformation module specifically include: The grouped pixel embedding module divides the high-dimensional characteristic spectral dimension of the hyperspectral image into several groups of non-overlapping sub-channels. Each group of non-overlapping sub-channels extracts local spectral features through independent 2D convolution to obtain several local spectral features. Then, several local spectral features are spliced ​​to form fusion features. The fusion features are used to extract global features through the space-preserving attention mechanism. When in the first and second stage hierarchical structures, stage spectral-spatial features are obtained; when in the third stage hierarchical structure, discriminative spectral-spatial features are obtained.

5. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 4 is characterized in that: The fusion features are extracted into global features through the space-preserving attention mechanism, specifically including: A spatial attenuation matrix is ​​introduced into the space-preserving attention mechanism. The attention weight of the fused feature is dynamically adjusted through the Manhattan distance. The attention of the fused feature is decomposed into horizontal attention and vertical attention. The horizontal attention and vertical attention are calculated respectively to obtain the horizontal attention calculation results and the vertical attention calculation results. The horizontal attention calculation results and the vertical attention calculation results are fused to obtain the final attention, and the global feature extraction is achieved through the final attention.

6. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 5, characterized in that: The spatial attenuation matrix is ​​introduced into the spatial preservation attention mechanism, and the formula specifically includes: Q=(XW Q )eα; V=XW V ; RetNet(X)=(QK · e M)V; Where X represents the input feature; Q represents the query matrix; K represents the key matrix; V represents the value matrix; W Q W represents the learnable parameter matrix used to transform the query matrix; K W represents the learnable parameter matrix used to transform the key matrix; V represents a learnable parameter matrix used to transform the value matrix; α represents a column vector; a i Represents the elements in each column vector; θ represents the learnable column vector; i represents an imaginary number; represents the complex conjugate of α; ⊙ represents the Hadamard product, which is used to multiply the corresponding elements of two vectors or matrices; M ij represents the attenuation matrix; x i Indicates the column index of the pixel; x j Represents the row index of the pixel; Represents a constant ranging from 0 to 1; RetNet represents the RetNet network calculation formula; T represents matrix transpose; M represents the spatial attenuation matrix; The Manhattan distance formula specifically includes: Where x 1 represents a pixel in the input image block; x 2 Represents another pixel in the input image patch.

7. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 2, characterized in that: The method of performing category semantic reconstruction on the discriminative spectral-spatial features of the training samples through a category semantic reconstruction module to obtain a reconstruction error, and constructing and training a classification model based on the reconstruction error specifically includes: The reconstruction error of the discriminative spectral-spatial features in several category autoencoders is calculated for the training samples, and the probability of each category is generated by Softmax. The loss function is constructed according to the negative logarithmic probability of the true label, and the classification model is trained by minimizing the loss function.

8. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 2, characterized in that: The test sample is input into the trained classification model, and the test sample is classified into known categories by reconstruction error, specifically including: The test sample is input into the trained classification model, and the reconstruction error of the discriminative spectral-spatial features in several category autoencoders is calculated. The probability of each category is generated by Softmax. The larger the error, the lower the probability of the category. The category with the maximum probability is used as the classification category to achieve known category classification of the test sample.

9. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 2, characterized in that: The formula of the multi-dimensional scoring function specifically includes: Where S1 represents the first-order statistical score; F represents the discriminative spectral-spatial features extracted from the test sample; f represents the features extracted by the grouped spectral space preserving transformer module; represents the pixel reconstruction error of the predicted category c; S2 represents the second-order statistical score; T represents the matrix transpose operation; Indicates the strength of category perception features; S3 indicates the third-order statistical score; Sum indicates summation; G c represents the average matrix of predicted categories; Represents element-by-element multiplication of matrices; I p It means that the feature F is connected by columns and then the exponential operation is performed; p is a constant.

10. The hyperspectral open set recognition method based on class semantic reconstruction according to claim 9, characterized in that: The open set recognition of unknown category samples of the test samples by using the multi-dimensional scoring function specifically includes: After S1, S2 and S3 are normalized, they are weighted and fused to obtain the final score. If the final score is less than or equal to the pre-set threshold, it is considered an unknown category. If the final score is greater than the pre-set threshold, it is a known category.

Images