Few-shot HSI classification method based on prototype mask comparison and cross-stage feature refinement

Abstract

The present invention discloses a small-sample HSI classification method based on prototype mask comparison and cross-stage feature refinement. The method comprises: performing meta-task training in a cross-source domain and a target domain, unifying the spectral dimensions of samples of a support set and a query set for each meta-task, and then performing multi-dimensional feature fusion that is resistant to neighboring pixel dependence to obtain support features and query features; constructing positive and negative sample pairs representing prototypes under each meta-task and performing prototype mask comparison through graph convolution; performing feature refinement to eliminate redundancy based on emphasized mapping of a cross-domain shared multi-layer perceptron, coordinating cross-stage feature refinement of support features and query features with the multi-layer perceptron and directional calibration to obtain refined support features and refined query features, and optimizing different losses in the two training stages; and using a small sample from the target domain in the testing stage, sequentially performing dimension unification mapping, a feature extraction module, and features extracted after the multi-layer perceptron to train a K-nearest neighbor classifier for classification prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement. Background Art

[0002] Hyperspectral imaging (HSI) provides rich spatial spectral information by recording spectral data across hundreds of continuous narrowband spectral ranges. This capability enables HSI classification technology to accurately identify different ground features and has been widely used in a variety of fields, including environmental monitoring, mineral exploration, and precision agriculture.

[0003] Traditional methods based on statistics and machine learning have been extensively studied in the field of HSI classification. However, these methods struggle to fit nonlinear features and may fail to capture complex spatial and spectral relationships. In recent years, deep learning-based methods have demonstrated remarkable performance in HSI classification. However, deep network learning generally requires a large number of labeled samples. Due to the difficulty in acquiring HSI samples, the high level of expertise required, and the time-consuming nature of the training process, labeled samples are extremely limited in practice, hindering the adequate training of deep networks. Researchers have introduced the FSL technique, a meta-learning technique, to HSI classification. Meta-learning focuses on "learning how to learn." By simulating multiple small-sample tasks in a source domain with abundant labeled samples, it can rapidly adapt to new tasks in a target domain with few labeled samples. However, the source and target domains do not come from the same domain with the same data distribution, and this difference in feature distribution introduces domain shift. Researchers are currently using various techniques to find a mapping space that aligns the features of the source and target domains as closely as possible to mitigate the effects of domain shift. However, existing techniques still require optimization and improvement to improve classification performance.

[0004] At present, the research on HSI classification in small sample scenarios mainly starts from the meta-learning paradigm, and tries to learn feature mappings that are aligned at the feature distribution level under the training mode of cross-domain meta-tasks to alleviate the domain shift problem, thereby effectively utilizing the knowledge transferred from the source domain to promote effective classification in the target domain even with extremely small training samples.

[0005] Z. Ye, J. Wang, T. Sun, J. Zhang, and W. Li, "Cross-Domain Few-Shot Learning Based on Graph Convolution Contrast for Hyperspectral Image Classification," in IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-14, 2024, Art no. 5504614, doi: 10.1109 / TGRS.2024.3352093, proposed a cross-domain Few-Shot Learning (FSL) framework for HSI classification based on graph convolution contrast, referred to as GCC-FSL. This method incorporates supervised contrastive learning and graph convolution on top of conventional convolution operations. By effectively utilizing label information, supervised contrastive learning can construct more task-oriented positive and negative sample pairs, enabling better mining of valuable information from prototype features themselves. Local features extracted by traditional convolution operators often overlook the capture of global information, resulting in insufficient perceptual information when aligning cross-domain data distributions. However, graph convolution can construct a graph for all positive and negative pairs to achieve global perception. The secondary extraction of high-order features further helps to explore a mapping space suitable for contrastive learning. GCC-FSL constructs positive and negative sample pairs using a cross-domain prototype matching rule based on Euclidean distance to implement supervised contrastive learning. After constructing a graph structure for the cross-domain prototype matching pairs, it uses the inter-layer propagation mechanism of graph convolution to aggregate neighbor information and extract high-order features of nodes from global relationships to construct a contrastive loss, thereby aligning the data distribution of the source and target domains.

[0006] Conventional convolution operations fail to fully account for the rich independent information and interactions in HSI's spatial and spectral dimensions. Existing work has introduced cross-domain FSL techniques to HSI classification, aiming to transfer source domain knowledge by learning a cross-domain meta-task. However, the bias introduced by incompletely eliminating domain shift still makes feature distribution alignment suboptimal, and adaptation to the target domain meta-task struggles to generalize effectively to unlabeled samples. Current mainstream prototype-based methods for small-sample HSI classification fail to consider similar redundant components across prototypes, leading to intra-domain reliance on redundancy and domain shift between the source and target domains due to unrecognized latent feature distributions. GCC-FSL combines supervised contrastive learning with graph convolution to address this problem from the prototype level. However, due to the uncertainty of cross-domain data distribution differences, the construction of positive and negative pairs for target domain prototypes may result in some source domain prototypes being identified as positive matches multiple times. The alignment of target domain prototypes cannot guarantee independence, and the assumption of sufficient similarity between positive pairs may not hold. Summary of the Invention

[0007] The purpose of the present invention is to overcome the problem of redundant components between prototypes and propose a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, comprising the following steps:

[0010] Cross-generate support and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset;

[0011] After unifying the spectral dimensions of the samples in the support set and query set, the feature extraction module performs multi-dimensional feature fusion that is resistant to neighboring pixel dependence to obtain support features and query features.

[0012] Based on a parameter-free adaptive mask generator, we construct positive and negative sample pairs representing the prototypes for each meta-task and perform prototype mask comparison through graph convolution.

[0013] Combined with the prototype pairs representing each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of a multi-layer perceptron shared across domains. A two-stage training mode is set depending on whether emphasis mapping is performed. Each stage refines features at different levels based on directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on support features and query features, obtaining refined support features and refined query features. Different losses are optimized in the two training stages.

[0014] The first stage of the two-stage training model uses support features and query features to perform few-shot learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform few-shot learning;

[0015] In the testing phase, a small sample of the target domain is used to train a K-nearest neighbor classifier for classification prediction using the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

[0016] Furthermore, the multi-dimensional feature fusion that resists neighboring pixel dependence is specifically as follows: under the condition that neighboring pixels of the input HSI cube are randomly removed, features are extracted from the spatial, spectral, and spatial-spectral levels in a parallel structure and fused to obtain fused features;

[0017] The fusion features are:

[0018]

[0019] in, To fusion features, For the feature extraction process, is a random mask of neighboring pixels, is the maximum ratio of random mask, For the mapping layer, is the input HSI cube, represents the number of spectral channels, 9×9 represents the spatial dimension, and the output after dimension unification is , is the unified dimension of the source domain and the target domain, As the output dimension, the fusion features will be extracted All operations are recorded as .

[0020] Furthermore, the feature extraction module includes a neighboring pixel removal mechanism and three parallel branches. The outputs of the three parallel branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then the features of different levels are fused in sequence through three-dimensional convolution, adaptive three-dimensional average pooling, and fully connected operations. The three parallel branches include a spatial branch, a spectral branch, and a spatial-spectral branch.

[0021] The spatial branch first adopts a size of 3×3 and the number of groups is A two-dimensional convolution kernel of 1×1×3 is used to extract spatial information, and then a three-dimensional convolution kernel of 1×1×3 is used to mine high-order spectral features based on spatial aggregation;

[0022] The spectral branch first uses a 3D convolution kernel of size 1×1×3 to extract spectral features, and then uses a 3×3 convolution kernel with a grouping number of The two-dimensional convolution kernel mines high-order spatial information based on spectral aggregation;

[0023] The spatial-spectral branch uses two consecutive two-dimensional convolutional layers, and the convolution kernel size of each convolution layer is 3×3. It mines features from both spatial and spectral levels and adopts a residual structure.

[0024] Furthermore, the parameter-free adaptive mask generator constructs positive and negative sample pairs representing the prototype under each meta-task and performs prototype mask comparison through graph convolution. Specifically, the parameter-free adaptive mask generator generates an emphasis mask group and a hidden mask group complementary to the emphasis mask group according to the conditional distribution of the representative prototype group obtained in each meta-task. According to the emphasis mask group and the hidden mask group, each prototype is adaptively screened for components related to strong categories to construct positive pairs representing the prototype, and redundant components are left to construct negative pairs representing the prototype. Positive and negative sample pairs representing the prototype under each meta-task are obtained. After the comparison group is constructed into a prototype comparison graph using the positive and negative sample pairs representing the prototype under each meta-task, high-order dependencies between nodes are extracted through graph convolution for supervised comparative learning.

[0025] Furthermore, the emphasis mask group is:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] Among them, the subscript t represents the target domain, To emphasize the mask group, represents the vector concatenation operator in the column direction, To emphasize the mask group Column vector, corresponding to the The emphasis mask of feature channels, represents the number of prototype categories, which is the support set and queryset The number of categories, For the The threshold interval of feature channels, For the The average value of feature channels, For the The standard deviation of the feature channels, Indicates the Class represents the prototype vector No. feature components, To support the set Middle The sample set of the class, the support set Each category has samples, query set Each class has M samples, is the feature extraction function, which calculates the first Class represents the prototype vector All by = get;

[0033] The hidden mask group is:

[0034]

[0035]

[0036] in, To hide the mask group, The first Column vector, corresponding to the Hidden masks of feature channels;

[0037] Perform scale calibration for all prototype masks, and the scale correction for the i-th prototype mask in the mask group is:

[0038]

[0039] in, and are the i-th row vectors of the emphasis mask group and the hidden mask group, corresponding to the i-th emphasis mask and hidden mask representing the prototype, respectively. represents the scale correction operator in the row direction, parameter It indicates the proportion of feature components that are masked out by the mask group operation for each representative prototype at most, and it also indicates the proportion of feature components that are retained by the hidden mask group operation for each representative prototype at most;

[0040] The positive and negative sample pairs representing the prototypes under each meta-task are obtained as follows:

[0041]

[0042] in, is a collection representing a prototype group Each element of is taken as a row in the matrix, express The matrix form of the corresponding positive sample group is, express The matrix form of the corresponding negative sample group, Represents the dot product operation of matrices of the same size, and Each row is an element, and the resulting set and are the positive sample group and the negative sample group, Each element of Positive samples of corresponding position elements, Each element of Negative samples of elements at corresponding positions;

[0043] The new features used in supervised contrastive learning for extracting high-order dependencies between nodes through graph convolution are:

[0044]

[0045]

[0046] in, Prototype comparison group The adjacency matrix No. Rank Column elements, is a parameter that controls the width of the radial basis function, is the 2-norm of the vector, The elements in are nodes, and It is nodes Hedi nodes The corresponding eigenvector, The matrix form of is the input of the first layer of graph convolution, that is , , is the identity matrix, yes The degree matrix of , for activation function, It is the first cross-domain sharing in graph convolution The weight matrix of the layer, For graph convolution The output of the layer is also composed of The new features obtained ,Will Each row of is taken as an element to get , The elements in are the new features for contrastive learning.

[0047] Furthermore, the first stage of the two-stage training mode and the second stage of the two-stage training mode are performed in a cycle, the loss of the directional calibration constrains the different representative prototypes of the first stage of the two-stage training mode and the second stage of the two-stage training mode from the unification of direction and distance, the first stage of the two-stage training mode learns reliable prototypes and refines the features by directional calibration, and the second stage of the two-stage training mode emphasizes the refinement of the mapping based on the cross-domain shared multi-layer perceptron, and performs secondary refinement of the directional calibration on the mapping features.

[0048] Furthermore, the total loss of the first stage of the cross-stage feature refinement is:

[0049]

[0050] in, is the total loss of the first stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, calculated of In both stages , calculate the first stage and Used for Similarly, the first stage of cross-training also optimizes the total loss of the source domain ;

[0051] The total loss of the second stage of cross-stage feature refinement is:

[0052]

[0053] in, is the total loss of the second stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of orientation calibration, is the mean square error loss, Only optimize in the second stage, calculate the second stage and Used for Similarly, the second stage of cross-training also optimizes the total loss of the source domain ;

[0054] The mean square error loss is:

[0055]

[0056] in, is the mean square error loss, The first step to emphasize the mapping characteristics of the multilayer perceptron feature components, Indicates the positive comparison group The class represents the first pair of prototype vectors feature components;

[0057] The dynamic triplet loss is:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] in, It is a dynamic triplet loss. The constraints of the dynamic triplet loss are all the new features of the representative prototypes and their positive and negative pairs that have undergone graph convolution. Indicates taking the maximum value of the input. The length is Vector and The Euclidean distance of is the facing distance, is the negative distance, is the set of key negative samples where the negative pair distance must be considered. Indicates that in the candidate negative pair Only The nearest negative sample pair distance participates in the loss calculation, The negative distance is at least greater than the positive distance. and For dynamic adjustment Hyperparameters of

[0064] The FSL loss is:

[0065]

[0066]

[0067]

[0068] in, is the FSL loss, It is a sample The corresponding category label, For query set Each sample The class distribution of For query sets The Prototype vector for class sample classification;

[0069] The overall loss of the orientation calibration is:

[0070]

[0071]

[0072]

[0073] in, is the overall loss of orientation calibration, is the inter-class distance loss for directional calibration, is the intra-class distance loss for orientation calibration, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, is the 2-norm of the vector, Indicates that only The farthest intra-class distance is used to participate in the loss calculation.

[0074] In a second aspect, the present invention provides a small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement, comprising:

[0075] The dataset partitioning module is used to generate support sets and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset;

[0076] The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is anti-neighboring pixel dependent through the feature extraction module to obtain support features and query features;

[0077] The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution;

[0078] The cross-stage feature refinement module is used to combine the positive pairs representing the prototypes under each meta-task and refine the features to eliminate redundancy based on the emphasis mapping of the cross-domain shared multi-layer perceptron. The module is set to a two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training mode uses the support features and query features to perform small-sample learning, and the second stage of the two-stage training mode uses the refined support features and refined query features to perform small-sample learning.

[0079] The classification module uses a small sample of the target domain in the testing phase. It sequentially passes through the dimension unified mapping, feature extraction module, and multi-layer perceptron to extract features and train the K-nearest neighbor classifier for classification prediction.

[0080] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement is implemented.

[0081] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, provides the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement.

[0082] Compared with the prior art, the present invention has the following beneficial technical effects:

[0083] The present invention proposes a small-sample HSI classification method based on prototype mask contrast and cross-stage feature refinement. The small-sample HSI classification framework based on prototype mask contrast and cross-stage feature refinement is abbreviated as PMC-CFR. It takes the prototype mask contrast (PMC) module as the core, and further implements cross-stage feature refinement (CFR) on this basis. It is extremely critical to find a suitable angle to construct representative positive and negative sample pairs. The present invention aims to use the prototype as the starting point and effectively remove similar redundant components between prototypes by constructing positive and negative sample pairs. It uses graph convolution to learn the spatial mapping of the application contrast loss from the global perspective of the contrast group. Under the promotion of supervised contrastive learning, the contrast samples and additional calibration measures are further used in the training process to extract more category-representative refined features.

[0084] Furthermore, the multi-dimensional feature extraction and fusion module designed in the present invention not only weakens the dependence of the pixels to be classified on neighboring objects, but also comprehensively extracts and fuses features from three levels: spatial, spectral, and spatial-spectral. It extracts features from different angles, taking into account both the independent information of each dimension and their interactions. More comprehensive mining and fusion of features can provide a deeper understanding of the essential information of HSI objects.

[0085] Furthermore, the prototype mask comparison module proposed in the present invention can adaptively generate an emphasis mask group and a complementary hidden mask group according to the representative prototype group of each meta-task through a parameter-free adaptive mask generator. The mask group is applied to the prototype group to obtain a positive prototype group that filters out strong category-related feature components, and a negative prototype group that retains redundant components. Each positive and negative sample pair constructs a prototype comparison graph. Through graph convolution, the high-order dependency relationship between the prototype and its variants can be extracted to realize contrastive learning, ensuring that the redundant components in the feature vector are easier to separate.

[0086] Furthermore, the present invention utilizes the generated set of positive samples after removing redundant components to design a two-stage training model, and on this basis, realizes cross-stage feature refinement. The training is set as two stages. The first stage extracts reliable prototypes and performs directional calibration to achieve feature refinement. The second stage implements emphasized mapping based on a cross-domain shared multi-layer perceptron and performs directional calibration on the mapped de-redundant features. The emphasized mapping and directional calibration in the second stage achieve a dual feature refinement effect. The coordinated implementation of directional calibration and shared MLP can effectively alleviate the domain shift problem, promote the alignment of feature distributions in the source and target domains, and thus improve classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only schematic and are used to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings:

[0088] Figure 1 This is a flowchart of the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement of the present invention.

[0089] Figure 2 This is a structural diagram of the small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement in the present invention.

[0090] Figure 3 This is an electronic device diagram of the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement of the present invention.

[0091] Figure 4This is the overall framework diagram of the PMC-CFR of the present invention.

[0092] Figure 5 This is the overall structure of the multi-dimensional feature fusion module that is resistant to neighboring pixel dependence of the present invention.

[0093] Figure 6 The overall process of constructing positive and negative samples for the parameter-free adaptive mask generator of the present invention is described.

[0094] Figure 7 This is the gradual improvement effect after the orientation calibration of the present invention.

[0095] Figure 8 : The pseudo-color image and real object image of the IP dataset in the embodiment of the present invention.

[0096] Figure 9 : The pseudo-color image and real object image of the UP dataset in the embodiment of the present invention.

[0097] Figure 10 This is a classification diagram of various methods used in the comparative experiment of the embodiment of the present invention on the IP dataset.

[0098] Figure 11 This is a classification diagram of various methods used in the comparative experiment of the embodiment of the present invention on the UP dataset.

[0099] Figure 12 This is the two-dimensional feature visualization of DMCM, GCC-FSL, and PMC-CFR on the IP dataset and UP dataset in the embodiments of the present invention. DETAILED DESCRIPTION

[0100] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0101] Example 1

[0102] See also Figure 1 This embodiment proposes a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, including the following steps:

[0103] Cross-generate support and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset;

[0104] After unifying the spectral dimensions of the samples in the support set and query set, the feature extraction module performs multi-dimensional feature fusion that is resistant to neighboring pixel dependence to obtain support features and query features.

[0105] Based on a parameter-free adaptive mask generator, we construct positive and negative sample pairs representing the prototypes for each meta-task and perform prototype mask comparison through graph convolution.

[0106] Combined with the prototype pairs representing each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of a multi-layer perceptron shared across domains. A two-stage training mode is set depending on whether emphasis mapping is performed. Each stage refines features at different levels based on directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on support features and query features, obtaining refined support features and refined query features. Different losses are optimized in the two training stages.

[0107] The first stage of the two-stage training model uses support features and query features to perform few-shot learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform few-shot learning;

[0108] In the testing phase, a small sample of the target domain is used to train a K-nearest neighbor classifier for classification prediction using the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

[0109] See also Figure 4 A small-sample HSI classification framework based on prototype mask contrast and cross-stage feature refinement is proposed, with the Prototype Mask Contrast module as the core. On this basis, cross-stage feature refinement is further implemented. The proposed scheme is abbreviated as PMC-CFR.

[0110] Figure 4 The overall PMC-CFR framework is shown, consisting of three key components: feature extraction, prototype mask comparison, and cross-stage feature refinement. In the figure, stage 1 represents the first stage of training, and stage 2 represents the second stage. Different losses are optimized at different stages.

[0111] First, the feature extraction part is implemented based on a multi-dimensional feature fusion module that is resistant to neighboring pixel dependence. Under the condition that neighboring pixels of the input HSI cube are randomly removed, features are extracted and fused from the spatial, spectral, and spatial-spectral levels in a parallel structure, which fully considers the independent information and interaction of the two dimensions of space and spectrum.

[0112] Secondly, the prototype mask contrast module uses a parameter-free adaptive mask generator to generate an emphasis mask group and a complementary hidden mask group based on the conditional distribution of the prototype group obtained from each meta-task. This adaptively filters components with strong category correlation for each prototype to form positive pairs, while retaining redundant components to form negative pairs. After constructing the contrast group as a prototype contrast graph, graph convolution is used to extract high-order dependencies between nodes to achieve supervised contrastive learning, making redundant components of features easier to separate. It is worth noting that graph convolution only maps feature representations to a new representation space using a contrastive loss; these components are discarded during testing.

[0113] Finally, cross-stage feature refinement is performed using a cross-domain shared emphasis mapping implemented by a multi-layer perceptron (MLP). Meanwhile, an orientation calibration loss constrains the different representative prototypes in the two stages by unifying their orientation and distance. Training is divided into two stages. In the first stage, reliable prototypes are learned to ensure feature discriminability while performing orientation calibration for feature refinement. In the second stage, the MLP is used to implement cross-domain shared emphasis mapping and orientation calibration of the secondary mapping features. This combination of emphasis mapping and orientation calibration in the second stage achieves a dual feature refinement effect. The coordinated implementation of orientation calibration and shared MLP for cross-stage feature refinement effectively mitigates domain shift. Cross-domain FSL is achieved using different prototypes in the two stages during training. In the testing stage, the feature extraction phase followed by the MLP is used as the feature extractor. A K-nearest neighbor classifier is trained using a small number of labeled samples from the target domain for prediction. The core implementation process and principles of this embodiment will be detailed in these three parts, further illustrated by simulation experiments and experimental results.

[0114] (1) Feature extraction: The training process is performed cross-training in the source domain and the target domain. However, the spectral dimensions of cross-domain data are often different. Using a mapping layer to unify the dimensions is a widely adopted measure.

[0115] Specifically, it is implemented through two cross-domain non-shared mapping layers, each of which is implemented by a layer of two-dimensional convolution, including A convolution kernel of size 1×1, To unify the number of output channels, it is followed by a two-dimensional batch normalization layer, and the mapping layer can automatically integrate the information of the spectral dimension.

[0116] The input HSI cube is denoted as , represents the number of spectral channels, 9×9 represents the spatial dimension, and the mapping layer is recorded as , then the processed output is , is the unified dimension of the source domain and the target domain. Then Input into the multi-dimensional feature fusion module that is resistant to neighboring pixel dependence. The structure of this module is as follows Figure 5 This module aims to mine the independent information and interactions of spatial and spectral dimensions from three perspectives under the condition of random removal of adjacent pixels.

[0117] First of all, Random mask drops up to The neighboring pixels of the ratio are used to weaken the neighborhood dependence when extracting discriminative features. This operation is recorded as Next, a parallel three-branch structure is used to extract features from the spatial, spectral, and spatial-spectral levels.

[0118] The spatial branch first adopts a size of 3×3 and the number of groups is The two-dimensional convolution kernel extracts spatial information without involving the interaction of spectral dimensions, and then uses a three-dimensional convolution kernel of size 1×1×3 to mine high-order spectral features on the basis of spatial aggregation. The convolution order of the spectral branch is opposite to that of the spatial branch. It first extracts spectral information without involving the interaction of spatial dimensions, and then mines high-order spatial information on the basis of spectral aggregation. The spatial spectral branch uses two two-dimensional convolution layers in succession. The convolution kernel size of each convolution layer is 3×3. It mines features from both spatial and spectral levels and adopts a residual structure. The outputs of the three branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then pass through three-dimensional convolution (convolution kernel size is 3×3×3), adaptive three-dimensional average pooling, full connection and other operations in sequence to fully integrate features at different levels. The process of extracting features in this module is recorded as , we can get the formula (1) as the final feature representation.

[0119] (1)

[0120] Unify the number of dimensions Set to 96, and each layer parameter data is cleverly set to maintain the input dimension and output dimension This helps avoid information loss and preserve subtle differences. For the convenience of representation, all operations in the feature extraction part are simply recorded as The types of each layer of the multi-dimensional feature fusion module that is resistant to neighboring pixel dependence and their corresponding related parameters are shown in Table 1.

[0121] Table 1 Detailed configuration of each layer of the multi-dimensional feature fusion module based on proximity dependency

[0122]

[0123] (2) Prototype mask comparison: First, a parameter-free adaptive mask generator is used to construct positive and negative pairs representing the prototypes for each meta-task.

[0124] Each meta-task In the training of , taking the target domain as an example, its meta-task , including the support set and queryset , support set Include Class, each class has samples, then the meta-task It is called C-way K-shot, and the corresponding query set Each category has samples. Since each meta-task is constructed The training samples are randomly selected, so the support set is considered and queryset The feature representation of all samples in the target domain is used to reduce sample fluctuations, and the mean of each type of feature is used as the representative prototype. The representative prototype of the class It can be calculated by formula (2).

[0125] (2)

[0126] in, and Indicates the first The sample set of the class, express Belong to the Collection of classes Each sample in , Represents the feature extraction function, which is different in the two stages of training. The prototype mask comparison process uses and its variants have passed the first stage accomplish.

[0127] for Each class represents a prototype, which is called a prototype group and is recorded as Some channels in the high-dimensional feature space may contribute to or even be crucial for the discrimination of certain classes, but these channels may be seen as commonalities in other classes, not separable, or even redundant. The parameter-free adaptive mask generator generates the mask from each feature channel. To evaluate the distribution, use formula (3) and formula (4) to calculate the The average value of feature channels and standard deviation , the channel threshold interval is generated by formula (5) .

[0128] (3)

[0129] (4)

[0130] (5)

[0131] in, Indicates the target domain The class represents the prototype vector feature components.

[0132] For the first The feature components within the interval are hidden as common features of some classes that do not have the ability to distinguish, and the feature components outside the interval are emphasized as significant features of some classes. Thus, the emphasis mask of the channel is generated as shown in formulas (6) and (7): and hidden mask (Upper right corner Indicates that the mask is the first column vector).

[0133] (6)

[0134] (7)

[0135] in, The column vector A portion.

[0136] Similarly, we can get the mask of all channels representing each prototype feature, and take each channel mask as a column in turn, and splice it into a matrix by formula (8) and formula (9) and record it as the emphasis mask group and hide mask groups .

[0137] (8)

[0138] (9)

[0139] in, Represents the vector concatenation operator in the column direction.

[0140] The distribution of some channels fluctuates greatly, which may lead to a high proportion of some prototypes being masked by the emphasis mask or a high proportion being left by the concealment mask. Therefore, for each prototype mask of the mask group, Row represents the The mask corresponding to the prototype is used for scale correction. The mask corresponding to the prototype is recorded as and , upper right corner mark Indicates that the mask is the first Row vector. Statistics 0 and Is the ratio of 1 within the appropriate value? If it is greater, 0 and 1 are randomly inverted to ensure that the ratio of feature components masked by positive samples or retained by negative samples is maintained at an appropriate value. The ratio correction process is implemented by formula (10).

[0141] (10)

[0142] in, represents the scale correction operator in the row direction, whose parameters It indicates the proportion of feature components that are masked out by the emphasized mask group operation for each representative prototype, and it also indicates the proportion that is retained by the hidden mask group operation for each representative prototype.

[0143] Random proportion correction is dominated by screening features to retain the representativeness of variant properties, while random placement introduces uncertain interference and increases the difference Finally, Each element of is represented as a row in sequence, and its matrix form is obtained. , and the mask group are calculated by formula (11) and .

[0144] (11)

[0145] in, Represents the dot product (Hadamard product) operation on matrices of the same size.

[0146] Will and Each row of is taken as an element, and we can get Positive control group and negative control group For each , The overall process of obtaining positive and negative samples corresponding to each prototype is as follows: Figure 6 As shown in Figure 2, the color of the squares that make up the mask is white, indicating that the bit is 1, and black, indicating that the bit is 0. The scale correction operation in the row direction sets the parameter β to 0.6 for illustration.

[0147] Will It is recorded as the prototype comparison group, each element of which can be regarded as the feature vector corresponding to a node. The eigenvector corresponding to the node is , the adjacency matrix can be constructed from the relationship between nodes , calculated in turn by formula (12) No. Row, No. Column elements , after adding the self-loop and normalizing, we get , and thus construct a prototype comparison chart. Each element of is represented as a row in turn, and its matrix form is obtained. As the first layer input of graph convolution, The graph convolution operation is performed by formula (13), and the features of the prototype contrast group are mapped to the new feature space with contrast loss using the global structural information of the prototype contrast graph. The obtained graph convolution The output of the layer is the new feature representation .

[0148] (12)

[0149] (13)

[0150] in, is a parameter that controls the width of the radial basis function, and is a node and The corresponding eigenvector, is the 2-norm of the vector, for activation function, It is the first cross-domain sharing in graph convolution The weight matrix of the layer.

[0151] Each row of is used as a new feature representation of the corresponding prototype, and its collection form is obtained. , and decomposed into 、 and .Will Each element in , as the first The class represents the prototype's baseline, only The corresponding For its right side, The corresponding and All the other ones except themselves are considered as necessary negative pairs, which are simply referred to as sets , including the negative pairs generated by itself and other initial prototypes. These negative pairs are the most valuable and must be paid attention to. In addition to the above-mentioned prototype variants, all other candidate negative pairs are referred to as the set , the number of candidate negative pairs is large and the range is large, so only The nearest pairs of samples are involved in the contrastive loss calculation, which can also prevent the negative effects of close distances from being mitigated by long distances. Dynamic triplet loss is used to perform contrastive learning, as shown in Equation (14). This loss aims to make the positive sample pairs closer and the negative sample pairs farther apart, making it easier to separate redundant components of the input features.

[0152] (14)

[0153] (15)

[0154] (16)

[0155] (17)

[0156] (18)

[0157] in, Indicates taking the maximum value of the input, and formula (15) is the length of Vector and The Euclidean distance of , formula (16) and formula (17) are the positive distance and negative distance respectively, is the set of key negative samples where the negative pair distance must be considered. Indicates Only The nearest negative sample pair distance participates in the loss calculation, and the formula (18) The negative distance is at least greater than the positive distance. and It is a hyperparameter, which is adaptively adjusted according to the positive and negative distances .

[0158] (3) Cross-stage feature refinement:

[0159] 1. Emphasis Mapping Based on Multi-layer Perceptron: The positive samples representing each prototype obtained in the prototype mask comparison module filter out the feature components that are strongly related to the class, while removing the redundant components that are common in the prototype group. However, this relies on the conditional distribution statistics based on the prototype group. Such positive samples have refined features based on the prototype and have more refined discriminability. Therefore, it is very important to achieve feature refinement on the test samples from this perspective. It should be noted that The emphasis mask group represented by Row elements, corresponding to the The emphasis mask of the class feature means that even if the mask group is calculated and retained from the training sample in the test phase, the test sample cannot determine its corresponding mask position because it has no label. Therefore, MLP is introduced here to realize the emphasis mapping of the feature, which is recorded as , used to fit the emphasis mask to refine the features. The feature representation in the second feature mapping is then performed through MLP, and the output obtained is the same as the positive comparison group The mean squared error (MSE) of the features of the positive samples of the corresponding category is optimized, as shown in formula (19).

[0160] (19)

[0161] According to each The emphasized mask group of all label sample statistics under , effectively utilizes all label sample information, reflects the category prototype representation trend of a meta-task, and at the same time, each time The training samples are all randomly drawn, and the differences in meta-tasks further promote the exploration of commonalities from the representation trends of various category prototypes. Therefore, this kind of feature component screening with strong category correlation has better generalization, and by exploring common mappings from different representation trends through MLP, it can effectively remove redundancy from test samples without relying on labels. Under the cross-domain learning of meta-tasks, the sharing of MLP can explore de-redundant mappings that are more conducive to feature distribution alignment, and the domain offset problem can be effectively alleviated. The implementation of MLP is lightweight, consisting of only two fully connected layers with a Activation function, detailed parameters are shown in Table 2.

[0162] Table 2 Detailed configuration of shared MLP

[0163]

[0164] 2. Orientation calibration: In order to make full use of and The label samples in the , make the intra-class distance closer and the inter-class distance farther, and design the loss of directional calibration. Figure 7 This is the gradual improvement effect after directional calibration. The figure is illustrated by two-dimensional visualization. The prototype in the yellow circle indicates a stable distribution under multi-dimensional tasks, the prototype near the green circle indicates a large fluctuation in the distribution under multi-dimensional tasks, the black circle indicates the position when the inter-class distance is pulled apart, and the dark area indicates the opposite direction range of the illustrated situation. According to formula (2), the representative prototype of each class is obtained for inter-class directional calibration. If the prototypes of the two classes are directly pulled apart by distance, the problems existing when only prototyping is performed based on distance are as follows: Figure 7 As shown in (a), compared with the prototype of a certain type in different The distribution is stable (floating in the yellow circle), when the same category is in different The prototype distribution under fluctuates greatly (fluctuates near the two green circles). It can be seen that the effect of farther distance can be produced in multiple directions (near the black circle), but they are actually farther away. The calibration effect of the multi-dimensional task is not only not cumulative, but has a negative effect of confrontation. The Euclidean distance metric ignores the directionality, that is, the distance contribution of different feature components may be different, but the Euclidean distance focuses on the sum of the contributions, which is in line with expectations. The loss of directional calibration is shown in formula (20). The cosine similarity is introduced as a directional guide in the numerator of the Euclidean distance term. After the introduction of cosine similarity, the special case of prototype calibration with directional guidance is as follows Figure 7 As shown in (b), taking into account the balance between distance and direction, the different classes are kept apart. Prototypes of the same class tend to be closer in the high-dimensional space, that is, in the opposite direction (the dark area). However, cosine similarity will weaken the influence of the distance metric, especially for prototypes that are already in the opposite direction. Their calibration loss will depend entirely on the direction. Therefore, the 2-norm of the prototype is added as a constraint in Equation (20). Introducing the 2-norm of the prototype as a constraint solves the problem that special samples cannot be calibrated. Figure 7 As shown in (c), the directions are already opposite, thus ignoring the prototype of the distance metric. By constraining the modulus to be sufficiently large, the inter-class distance is ensured to be sufficiently large. In this case, even a small negative number in the Euclidean distance term can be considered beneficial. Conversely, if the directions are similar, the constraint does not guarantee a sufficiently large inter-class distance, and it is necessary to separate the inter-class prototypes from the unity of direction and distance. The 2-norm constraint ensures that the Euclidean distance between classes is always large enough under various directions. Directional calibration aims to explore a range of opposite directions in high-dimensional space that keeps classes apart, helping to ensure that the features of samples of the same class are distributed as close as possible within the range guided by the direction.

[0165] After obtaining the distribution of similar features in the directional range, the intra-class distance is directly narrowed by formula (21), and the unified setting is to only take The farthest participating loss calculation, Setting it to a smaller value can focus more on optimizing the compactness within the class from extreme samples and avoid the risk of overfitting caused by over-smoothing the global distribution.

[0166] (20)

[0167] (twenty one)

[0168] (twenty two)

[0169] in, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, Indicates that only The farthest intra-class distances are used to calculate the loss, and Equation (22) is the overall loss of Directional Calibration (DC).

[0170] 3. Two-stage training mode: The proposed PMC-CFR sets a two-stage training mode to fully refine the features, and the two stages of training are performed cyclically. The interval between the two stages is , each training Next, This is recorded as stage one, and the This is recorded as stage 2, and repeated.

[0171] Each stage of training includes FSL loss. Taking the target domain as an example, the first Prototype vector of the class pass From formula (23), we get Each sample Class distribution Calculate according to formula (24) and obtain the FSL loss by formula (25).

[0172] (twenty three)

[0173] (twenty four)

[0174] (25)

[0175] in, yes The corresponding category label.

[0176] The two stages of training have different feature extraction functions , the first stage does not contain shared MLP, then , and the second stage contains a shared MLP, then From formula (23), we can get The prototype of the classification is subjected to FSL using Equation (25), and the representative prototype is obtained by Equation (2). The directional calibration loss is optimized using Equation (22). Both stages optimize the contrast loss, and only the second stage optimizes the MSE using Equation (19). Therefore, the total loss that needs to be optimized in the first stage is:

[0177] (26)

[0178] The total loss that needs to be optimized in the second stage is:

[0179] (27)

[0180] Meta-task training is performed crosswise between the source domain and the target domain. Similarly, the two stages of training are in the source domain. and In the testing phase, by Features are extracted and K-nearest neighbor classifier is used for prediction.

[0181] (IV) Simulation conditions and experimental setup: All experiments are implemented based on the Pytorch framework on a platform with an i9-12900HX CPU, an NVIDIA GeForce RTX 4060 16GB GPU, and 16GB of memory.

[0182] As a cross-domain FSL method, the Chikusei dataset is selected as the source domain dataset, from which 200 labeled samples are selected for training per class, while only 5 labeled samples are selected for each class in the target domain dataset, and the dataset is expanded to 200 for training by adding random Gaussian noise and random flipping data augmentation operations. Each C-way K-shot meta-task is considered as an iteration. is set to the number of categories of the target dataset, and Set to 1 to query the number of samples of each class in the set Set to 19, that is, 20 samples of each class will be randomly selected from 200 labeled samples for training in each meta-task, and the source domain and target domain will be cross-trained for meta-task training. The spatial size of the input HSI cube is 9×9, and the number of graph convolution layers is Set to 3, two-stage interval Set to 500, the number of nearest neighbors used by the K-nearest neighbor classifier for classification is set to 1. The number of samples used to filter the loss calculation in the contrast loss and directional calibration loss It is uniformly set to 3 and used for dynamic adjustment in contrast loss. of and Set to 8 and 0.5, as well as the directional calibration loss used to adjust the magnitude of the distance between or within classes. The number of training iterations was set to 10,000, and the Adam optimizer was used to optimize the model parameters. The learning rate was set to 0.0005, and the learning rate was decayed by 20% every 4,000 iterations.

[0183] (V) Performance Verification: The overall accuracy (OA), average accuracy (AA) and Kappa coefficient are used to comprehensively evaluate the classification performance of the proposed scheme. Each experiment is randomly and independently run ten times, and the average results and standard deviation are reported.

[0184] In order to evaluate the effectiveness of the proposed PMC-CFR in the small sample scenario of HSI, nine related schemes were selected for comparison in the experiment, including support vector machine (SVM) based on radial basis function kernel, 3DCNN, DCFSL, Gia-CFSL, CMFSL, ADAFSL, DMCM, HyMuT and GCC-FSL. Among them, SVM belongs to the traditional machine learning method, 3DCNN is a conventional deep learning method, which does not involve cross-domain and FSL, and the other seven methods are advanced HSI classification methods in cross-domain small sample scenarios. All methods select 5 labeled samples per category from the target domain dataset in the same random manner for training, and the other samples are used for testing. For cross-domain classification methods for small sample scenarios of HSI, the Chikusei dataset with rich category information is used as the source domain dataset for learning transfer knowledge. The target domain datasets select the widely used Indian Pines (IP) dataset and University of Pavia (UP) for performance evaluation. Table 3 shows the detailed information of the 16 categories of the IP dataset, and Figure 8 (a) shows a pseudo-color image, Figure 8 (b) shows the real ground feature map. The sample details of the 9 categories of the UP dataset are shown in Table 4, and the pseudo-color images are shown in Figure 9 (a) and the real feature map are shown in Figure 9 (b).

[0185] 1. Performance on the IP Dataset: Table 5 shows the classification results of various methods on the IP dataset. Due to the lack of labeled samples, SVM and 3DCNN were unable to extract highly discriminative features, resulting in lower classification results. However, the 3DCNN's deep learning-based feature extraction still outperformed the traditional SVM method. Methods based on cross-domain small-shot learning all achieved significant improvements in classification performance. Overall, the proposed PMC-CFR achieved the best values ​​for OA, AA, and Kappa, at 79.79%, 87.58%, and 77.13%, respectively. Compared to the second-best performing DMCM, OA, AA, and Kappa improved by 5.38%, 4.67%, and 6.07%, respectively. Classification accuracy across all categories was generally superior to that of other methods. Notably, the classification accuracy for Corn-notill (2 categories), Hay-windowed (8 categories), and Soybean-clean (12 categories) significantly outperformed other methods. Figure 10 The classification diagrams corresponding to the experimental results are shown. Figure 10 (a) in the figure is the real feature map; Figure 10 (b) in the figure is 3DCNN; Figure 10 (c) in the figure is DCFSL; Figure 10(d) in the figure is Gia-CFSL; Figure 10 (e) in the table is CMFSL; Figure 10 (f) in the table is ADAFSL; Figure 10 (g) in the equation is DMCM; Figure 10 (h) in the figure is HyMuT; Figure 10 (i) in the figure is GCC-FSL; Figure 10 (j) in the table is PMC-CFR;

[0186] The classification effect of PMC-CFR is the smoothest, and the classified areas are more complete.

[0187] Table 3 Names and sample numbers of each category in the IP dataset

[0188]

[0189] Table 4 Name and number of samples of each category in the UP dataset

[0190]

[0191] 2. Performance on the UP dataset: Table 6 shows the experimental results on the UP dataset. It can be seen that SVM and 3DCNN perform similarly, and the small-sample learning paradigm still effectively improves classification performance. Compared to the second-best DMCM, PMC-CFR achieved results of 90.48%, 91.28%, and 87.53% on OA, AA, and Kappa, representing improvements of 4.05%, 3.59%, and 5.34%, respectively. PMC-CFR achieved the highest classification accuracy across all eight categories, achieving leading performance in the Bare Soil (6 categories) and Self-Blocking Bricks (8 categories) categories despite the high number of misclassifications by other methods. Figure 11 The classification diagram corresponding to the experimental results is shown. Figure 11 (a) in the figure is the real feature map; Figure 11 (b) in the figure is 3DCNN; Figure 11 (c) in the figure is DCFSL; Figure 11 (d) in the figure is Gia-CFSL; Figure 11 (e) in the table is CMFSL; Figure 11 (f) in the table is ADAFSL; Figure 11 (g) in the equation is DMCM; Figure 11 (h) in the figure is HyMuT; Figure 11 (i) in the example is GCC-FSL; Figure 11(j) in the figure is PMC-CFR. It can be seen that the proposed method has a good classification effect in the edge areas of Gravel (3 categories) and Bitumen (7 categories), which reduces the terrain dependence of land feature classification to a certain extent.

[0192] Table 5 Classification results of each method on the IP dataset (five labeled samples per class)

[0193]

[0194] Table 6 Classification results of each method on the UP dataset (five labeled samples per class)

[0195]

[0196] (6) Feature visualization: In order to further evaluate the predictive effect of features extracted by different methods from an intuitive visual perspective, two-dimensional visualization of high-dimensional features is achieved based on t-SNE. Experiments will be conducted on DMCM, GCC-FSL, and PMC-CFR on the IP dataset and UP dataset. Figure 12 The t-SNE visualization effects of each method on two datasets are shown. Figure 12 (a) in the figure is the IP feature of DMCM; Figure 12 (b) in the figure is the IP feature of GCC-FSL; Figure 12 (c) in the figure is the IP feature of PMC-CFR; Figure 12 (d) in the figure is the UP feature of DMCM; Figure 12 (e) in the figure is the UP feature of GCC-FSL; Figure 12 (f) in the figure represents the UP feature of PMC-CFR. The figure clearly shows that the proposed PMC-CFR demonstrates superior intra-class clustering and inter-class feature classification across all datasets. For the IP dataset, the clustering of the comparison methods across categories was relatively loose and the intra-class tightness was poor, while PMC-CFR brought samples of the same category closer together and sharpened the separation boundaries between samples of different categories. Specifically, for the Soybean-mintill class (11 categories) in the IP dataset, the comparison methods exhibited excessive dispersion, making them easily confused with other categories. However, PMC-CFR achieved good feature clustering for this class and strong separation from features of other categories. For Asphalt (1 category) in the UP dataset, the comparison methods exhibited either overlapping or scattered clustering, but the proposed method avoided these issues. In summary, PMC-CFR effectively enhances the discriminability of features across different categories in the feature space.

[0197] This embodiment aims to fully train deep networks for HSI classification using only a small sample size. However, to address the domain shift problem inherent in cross-domain FSL, current state-of-the-art prototype-based methods generally fail to account for the redundancy of similar feature components across prototypes. Fitting these redundant components across domains makes it difficult to optimally align the feature distributions of the source and target domains, effectively failing to address the resulting domain shift. This paper proposes a small-sample HSI classification scheme (PMC-CFR) based on prototype mask comparison and cross-stage feature refinement to address potential prototype redundancy and domain shift in prototype-based classification models. This embodiment employs a feature extraction module with multiple convolutional modes to extract and fuse features from spatial, spectral, and spatial-spectral levels. The additional neighboring pixel removal mechanism ensures weak neighborhood dependency among samples. Next, a parameter-free adaptive mask generator adaptively generates an emphasis mask set and a complementary hidden mask set based on the representative prototype set for each meta-task. Applying these mask sets to the prototype sets yields a positive prototype set that filters out strongly class-correlated feature components, and a negative prototype set that retains redundant components. By implementing graph convolution based on the graph structure of the comparison group, high-order dependencies between the prototype and its variants can be extracted to achieve contrastive learning. The process of prototype mask comparison ensures that the redundant components in the feature vector are easier to separate. Finally, the training is set to two stages. The first stage extracts reliable prototypes and performs directional calibration to achieve feature refinement. The second stage implements cross-domain shared emphasis mapping based on MLP, and performs directional calibration on the mapping results. The emphasis mapping and directional calibration in the second stage achieve a dual feature refinement effect. The cross-stage feature refinement achieved by coordinating shared MLP and directional calibration can effectively alleviate the domain offset problem. Therefore, this embodiment can show good classification performance using a very small number of training samples.

[0198] HSI classification shows great promise in multiple application areas. In agriculture, fine-grained crop and soil classification based on HSI can help farmers monitor crop health, assess soil quality, and optimize crop management, thereby increasing yields and resource efficiency. In environmental monitoring, HSI classification can effectively identify different vegetation types, detect water pollution, and assess land use changes, providing important data support for ecological protection and sustainable development. Furthermore, in urban planning, this technology can be used for land cover classification, urban heat island effect monitoring, and infrastructure assessment, helping city managers make more informed decisions. It also has extensive applications in mining and resource exploration. Overall, HSI classification technology is becoming an increasingly important tool for research and application in various fields, driving innovation and development in related industries. Meanwhile, with advances in hyperspectral imaging technology, the acquisition of hyperspectral data has become more common, but the time and labor costs required to label HSI remain high. How to use extremely limited labeled samples to train models and improve classification accuracy is a crucial issue. FSL can enable models to quickly adapt to new environmental changes when labeled data is scarce, enabling more flexible applications.

[0199] Therefore, achieving efficient HSI classification based on FSL is an application trend in this field. The problem solved by this embodiment is to quickly achieve effective HSI classification through small samples in various environments, thereby reducing the application cost of implementing this technology in various scenarios.

[0200] Example 2

[0201] See also Figure 2 , a small-shot HSI classification system based on prototype mask comparison and cross-stage feature refinement, including:

[0202] The dataset partitioning module is used to generate support sets and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset;

[0203] The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is anti-neighboring pixel dependent through the feature extraction module to obtain support features and query features;

[0204] The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution;

[0205] The cross-stage feature refinement module is used to combine the positive pairs representing the prototypes under each meta-task and refine the features to eliminate redundancy based on the emphasis mapping of the cross-domain shared multi-layer perceptron. The module is set to a two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training mode uses the support features and query features to perform small-sample learning, and the second stage of the two-stage training mode uses the refined support features and refined query features to perform small-sample learning.

[0206] The classification module uses a small sample of the target domain in the testing phase. It sequentially passes through the dimension unified mapping, feature extraction module, and multi-layer perceptron to extract features and train the K-nearest neighbor classifier for classification prediction.

[0207] Example 3

[0208] See also Figure 3 , an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement is implemented.

[0209] Example 4

[0210] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement is implemented.

[0211] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, read-only optical disks, optical storage devices, etc.) containing computer-usable program code.

Claims

1. A small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, characterized by: The following steps are involved: Cross-generate support and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset; After unifying the spectral dimensions of the samples in the support set and query set, the feature extraction module performs multi-dimensional feature fusion that is resistant to neighboring pixel dependence to obtain support features and query features. Based on a parameter-free adaptive mask generator, positive and negative sample pairs representing the prototype under each meta-task are constructed and prototype mask comparison is performed through graph convolution. Specifically, based on the conditional distribution of the representative prototype group obtained in each meta-task, the parameter-free adaptive mask generator generates an emphasis mask group and a hidden mask group complementary to the emphasis mask group. According to the emphasis mask group and the hidden mask group, each prototype is adaptively filtered for components with strong category correlation to construct a positive pair representing the prototype, and redundant components are retained to construct a negative pair representing the prototype. The positive and negative sample pairs representing the prototype under each meta-task are obtained. After the comparison group is constructed into a prototype comparison graph using the positive and negative sample pairs representing the prototype under each meta-task, the high-order dependencies between nodes are extracted through graph convolution for supervised contrastive learning. Combined with the prototype pairs representing each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of a multi-layer perceptron shared across domains. A two-stage training mode is set depending on whether emphasis mapping is performed. Each stage refines features at different levels based on directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on support features and query features, obtaining refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training model uses support features and query features to perform few-shot learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform few-shot learning; In the testing phase, a small sample of the target domain is used to train a K-nearest neighbor classifier for classification prediction using the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

2. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1 is characterized in that: Specifically, the multi-dimensional feature fusion method that resists neighboring pixel dependence is to extract features from the spatial, spectral, and spatial-spectral levels in a parallel structure and fuse them to obtain fused features under the condition that neighboring pixels of the input HSI cube are randomly removed; The fusion features are: in, To fusion features, For the feature extraction process, is a random mask of neighboring pixels, is the maximum ratio of random mask, For the mapping layer, is the input HSI cube, represents the number of spectral channels, 9×9 represents the spatial dimension, and the output after dimension unification is , is the unified dimension of the source domain and the target domain, As the output dimension, the fusion features will be extracted All operations are recorded as .

3. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 2 is characterized in that: The feature extraction module includes a neighboring pixel removal mechanism and three parallel branches. The outputs of the three parallel branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then the features of different levels are fused through three-dimensional convolution, adaptive three-dimensional average pooling, and fully connected operations. The three parallel branches include a spatial branch, a spectral branch, and a spatial-spectral branch. The spatial branch first adopts a size of 3×3 and the number of groups is A two-dimensional convolution kernel of 1×1×3 is used to extract spatial information, and then a three-dimensional convolution kernel of 1×1×3 is used to mine high-order spectral features based on spatial aggregation; The spectral branch first uses a 3D convolution kernel of size 1×1×3 to extract spectral features, and then uses a 3×3 convolution kernel with a grouping number of The two-dimensional convolution kernel mines high-order spatial information based on spectral aggregation; The spatial-spectral branch uses two consecutive two-dimensional convolutional layers, and the convolution kernel size of each convolution layer is 3×3. It mines features from both spatial and spectral levels and adopts a residual structure.

4. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1 is characterized in that The emphasis mask group is: Among them, the subscript t represents the target domain, To emphasize the mask group, represents the vector concatenation operator in the column direction, To emphasize the mask group Column vector, corresponding to the The emphasis mask of feature channels, represents the number of prototype categories, which is the support set and queryset The number of categories, For the The threshold interval of feature channels, For the The average value of feature channels, For the The standard deviation of the feature channels, Indicates the Class represents the prototype vector No. feature components, To support the set Middle The sample set of the class, the support set Each category has samples, query set Each class has M samples, is the feature extraction function, which calculates the first Class represents the prototype vector All by = get; The hidden mask group is: in, To hide the mask group, The first Column vector, corresponding to the Hidden masks of feature channels; Perform scale calibration for all prototype masks, and the scale correction for the i-th prototype mask in the mask group is: in, and are the i-th row vectors of the emphasis mask group and the hidden mask group, corresponding to the i-th emphasis mask and hidden mask representing the prototype, respectively. represents the scale correction operator in the row direction, parameter It indicates the proportion of feature components that are masked out by the mask group operation for each representative prototype at most, and it also indicates the proportion of feature components that are retained by the hidden mask group operation for each representative prototype at most; The positive and negative sample pairs representing the prototypes under each meta-task are obtained as follows: in, is a collection representing a prototype group Each element of is taken as a row in the matrix, express The matrix form of the corresponding positive sample group is, express The matrix form of the corresponding negative sample group, Represents the dot product operation of matrices of the same size, and Each row is an element, and the resulting set and are the positive sample group and the negative sample group, Each element of Positive samples of corresponding position elements, Each element of Negative samples of elements at corresponding positions; The new features used in supervised contrastive learning for extracting high-order dependencies between nodes through graph convolution are: in, Prototype comparison group The adjacency matrix No. Rank Column elements, is a parameter that controls the width of the radial basis function, is the 2-norm of the vector, The elements in are nodes, and It is nodes Hedi nodes The corresponding eigenvector, The matrix form of is the input of the first layer of graph convolution, that is , , is the identity matrix, yes The degree matrix of , for activation function, It is the first cross-domain sharing in graph convolution The weight matrix of the layer, For graph convolution The output of the layer is also composed of The new features obtained ,Will Each row of is taken as an element to get , The elements in are the new features for contrastive learning.

5. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1 is characterized in that The first stage of the two-stage training model and the second stage of the two-stage training model are performed cyclically, the loss of directional calibration constrains the different representative prototypes of the first stage of the two-stage training model and the second stage of the two-stage training model from the unification of direction and distance, the first stage of the two-stage training model learns reliable prototypes and refines the features by directional calibration, the second stage of the two-stage training model emphasizes the refinement of the mapping based on a cross-domain shared multi-layer perceptron, and performs secondary refinement of the directional calibration on the mapping features.

6. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1, characterized in that The total loss of the first stage of cross-stage feature refinement is: in, is the total loss of the first stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, calculated of In both stages , calculate the first stage and Used for Similarly, the first stage of cross-training also optimizes the total loss of the source domain ; The total loss of the second stage of cross-stage feature refinement is: in, is the total loss of the second stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of orientation calibration, is the mean square error loss, Only optimize in the second stage, calculate the second stage and Used for Similarly, the second stage of cross-training also optimizes the total loss of the source domain ; The mean square error loss is: in, is the mean square error loss, The first step to emphasize the mapping characteristics of the multilayer perceptron feature components, Indicates the positive comparison group The class represents the first pair of prototype vectors feature components; The dynamic triplet loss is: in, It is a dynamic triplet loss. The constraints of the dynamic triplet loss are all the new features of the representative prototypes and their positive and negative pairs that have undergone graph convolution. Indicates taking the maximum value of the input. The length is Vector and The Euclidean distance of is the facing distance, is the negative distance, is the set of key negative samples where the negative pair distance must be considered. Indicates that in the candidate negative pair Only The nearest negative sample pair distance participates in the loss calculation, The negative distance is at least greater than the positive distance. and For dynamic adjustment Hyperparameters of The FSL loss is: in, is the FSL loss, It is a sample The corresponding category label, For query set Each sample The class distribution of For query sets The Prototype vector for class sample classification; The overall loss of the orientation calibration is: in, is the overall loss of orientation calibration, is the inter-class distance loss for directional calibration, is the intra-class distance loss for orientation calibration, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, is the 2-norm of the vector, Indicates that only The farthest intra-class distance is used to participate in the loss calculation.

7. A small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement, characterized by: include: The dataset partitioning module is used to generate support sets and query sets from the original hyperspectral images of the source domain dataset or the target domain dataset; The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is anti-neighboring pixel dependent through the feature extraction module to obtain support features and query features; The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution. Specifically, based on the conditional distribution of the representative prototype group obtained in each meta-task, the parameter-free adaptive mask generator generates an emphasis mask group and a hidden mask group complementary to the emphasis mask group. According to the emphasis mask group and the hidden mask group, each prototype is adaptively filtered for components with strong category correlation to construct a positive pair representing the prototype, and redundant components are retained to construct a negative pair representing the prototype. The positive and negative sample pairs representing the prototype under each meta-task are obtained. After the comparison group is constructed into a prototype comparison graph using the positive and negative sample pairs representing the prototype under each meta-task, the high-order dependencies between nodes are extracted through graph convolution for supervised contrastive learning. The cross-stage feature refinement module is used to combine the positive pairs representing the prototypes under each meta-task and refine the features to eliminate redundancy based on the emphasis mapping of the cross-domain shared multi-layer perceptron. The module is set to a two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training mode uses the support features and query features to perform small-sample learning, and the second stage of the two-stage training mode uses the refined support features and refined query features to perform small-sample learning. The classification module uses a small sample of the target domain in the testing phase. It sequentially passes through the dimension unified mapping, feature extraction module, and multi-layer perceptron to extract features and train the K-nearest neighbor classifier for classification prediction.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the computer program implements the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement described in any one of claims 1 to 6.

Images