Small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement

Abstract

The invention discloses a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, and the method comprises the steps: carrying out the meta-task training in a source domain and a target domain in a crossing manner, carrying out the spectral dimension unification of samples of a support set and a query set in each meta-task, carrying out the multi-dimensional feature fusion resistant to the dependence of adjacent pixels, and carrying out the classification of the multi-dimensional features. Obtaining support features and query features; constructing positive and negative sample pairs representing a prototype under each meta-task, and performing prototype mask comparison through graph convolution; according to the method, feature redundancy elimination refining is carried out based on emphasis mapping of a cross-domain shared multi-layer perceptron, cross-stage feature refining is carried out on support features and query features through the multi-layer perceptron and directional calibration coordination, refined support features and refined query features are obtained, and different losses are optimized in two training stages; in the test stage, small samples of a target domain are used, and a K-nearest neighbor classifier is trained for classification prediction through features extracted after dimensionality unified mapping, a feature extraction module and a multi-layer perceptron in sequence.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement. Background Art

[0002] HSI (Hyperspectral Imaging) provides rich spatial spectral information by recording spectral data in hundreds of continuous narrow-band spectral ranges. Thanks to this feature, HSI classification technology can accurately identify different ground materials and has been widely used in many fields such as environmental monitoring, mineral exploration, and precision agriculture.

[0003] Traditional methods based on statistics and machine learning have been widely studied in the field of HSI classification, but traditional methods are difficult to fit nonlinear features and may not be able to capture complex spatial and spectral relationships. In recent years, methods based on deep learning have performed well in the field of HSI classification. However, the learning of deep networks generally requires a large number of labeled samples. Due to the difficulty of collecting, high professional knowledge requirements, and time-consuming costs, HSI has extremely limited labeled samples in actual training, resulting in the inability to fully train deep networks. FSL technology, which belongs to meta-learning, has been introduced by researchers into the HSI classification task. Meta-learning focuses on "learning how to learn" by simulating multiple small sample tasks in the source domain with sufficient labeled samples, so that it can quickly adapt to new tasks in the target domain with a small number of labeled samples. However, the source domain and the target domain are not from the same domain with the same data distribution, so the difference in feature distribution also introduces the influence of domain shift. At present, relevant researchers are using various technical means to find a mapping space that can align the features of the source domain and the target domain as much as possible to alleviate the influence of domain shift. Existing technical means still need to be optimized and improved to improve classification performance.

[0004] At present, the research on HSI classification in small sample scenarios mainly starts from the meta-learning paradigm, and tries to learn feature mappings that are aligned at the feature distribution level under the training mode of cross-domain meta-tasks to alleviate the domain shift problem, thereby effectively utilizing the knowledge transferred from the source domain to promote effective classification in the target domain even with extremely small training samples.

[0005] Z. Ye, J. Wang, T. Sun, J. Zhang and W. Li, "Cross-Domain Few-Shot Learning Based on Graph Convolution Contrast for Hyperspectral Image Classification," in IEEE Transactions on Geoscience and Remote Sensing, vol.62, pp. 1-14, 2024, Art no. 5504614, doi: 10.1109 / TGRS.2024.3352093 proposed a cross-domain FSL (Few-shot Learning) HSI classification framework based on graph convolution contrast, referred to as GCC-FSL. This method introduces supervised contrastive learning and graph convolution on conventional convolution operations. Supervised contrastive learning can construct more task-oriented positive and negative sample pairs by effectively utilizing label information, and can better mine more valuable information from the prototype features themselves. The local features extracted by traditional convolution operators often ignore the capture of global information, resulting in the limitation of insufficient perceptual information when aligning cross-domain data distribution. However, graph convolution can construct a graph for all positive and negative pairs to achieve global perception, and the high-order features extracted secondary are more helpful in exploring the mapping space suitable for contrastive learning. GCC-FSL constructs positive and negative sample pairs through cross-domain prototype matching rules based on Euclidean distance to achieve supervised contrastive learning. After constructing a graph structure for cross-domain prototype matching pairs, it uses the inter-layer propagation mechanism of graph convolution to aggregate neighbor information, extract high-order features of nodes from global relationships to construct contrastive loss, and thus align the data distribution of the source domain and the target domain.

[0006] Conventional convolution operations cannot fully consider the independent information and interactions of HSI's rich spatial and spectral dimensions. Existing work introduces cross-domain FSL technology into HSI classification tasks, aiming to achieve the transfer of source domain knowledge by learning cross-domain meta-tasks, but the bias introduced by the incomplete elimination of domain shift still makes it difficult to optimize the feature distribution alignment, and the adaptation on the target domain meta-task is difficult to effectively generalize to unlabeled samples. At present, various prototype-based mainstream methods for small sample HSI classification do not consider the similar redundant components between various prototypes, which easily leads to redundancy dependence within the domain and domain shift between the source and target domains from the potential feature distribution that is not understood. GCC-FSL combines supervised contrastive learning and graph convolution to start from the prototype level, but due to the uncertainty of cross-domain data distribution differences, the measures to construct positive and negative pairs for the target domain prototype may cause some categories of prototypes in the source domain to be used as positive matches multiple times, and the alignment of category prototypes in the target domain cannot guarantee mutual independence. At the same time, the assumption that the similarity of positive pairs is high enough may not hold. Summary of the invention

[0007] The purpose of the present invention is to overcome the problem of redundant components between prototypes, and propose a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement.

[0008] In order to achieve the above object, the present invention adopts the following technical scheme: In a first aspect, the present invention provides a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, comprising the following steps: The support set and query set are generated by cross-generating the original hyperspectral images of the source domain dataset or the target domain dataset; After the spectral dimensions of the samples in the support set and the query set are unified, the multi-dimensional features that are resistant to neighboring pixel dependence are fused through the feature extraction module to obtain the support features and query features. Based on the parameter-free adaptive mask generator, we construct positive and negative sample pairs representing the prototypes in each meta-task and perform prototype mask comparison through graph convolution. Combined with the positive pairs representing the prototypes under each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains. The two-stage training mode is set according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain the refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training model uses support features and query features to perform small sample learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform small sample learning; In the test phase, a small sample of the target domain is used to train the K nearest neighbor classifier for classification prediction through the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

[0009] Furthermore, the multi-dimensional feature fusion with resistance to neighboring pixel dependence is specifically, under the condition that neighboring pixels of the input HSI cube are randomly removed, features are extracted from three levels of space, spectrum, and space-spectrum respectively in a parallel structure and fused to obtain fused features; The fusion features are:

[0010] in, To fusion features, For the feature extraction process, is a random mask of neighboring pixels, is the maximum ratio of random mask, is the mapping layer, is the input HSI cube, represents the number of spectral channels, 9×9 represents the spatial dimension, and the output after dimension unification is , is the unified dimension of the source domain and the target domain, is the output dimension, and the fusion features will be extracted All operations are recorded as .

[0011] Furthermore, the feature extraction module includes a neighboring pixel removal mechanism and three parallel branches. The outputs of the three parallel branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then the features of different levels are fused in sequence through three-dimensional convolution, adaptive three-dimensional average pooling, and full connection operation. The three parallel branches include a spatial branch, a spectral branch, and a spatial-spectral branch. The spatial branch first adopts a size of 3×3 and the number of groups is A two-dimensional convolution kernel of 1×1×3 is used to extract spatial information, and a three-dimensional convolution kernel of 1×1×3 is used to mine high-order spectral features based on spatial aggregation. The spectral branch first uses a 3D convolution kernel of size 1×1×3 to extract spectral features, and then uses a 3×3 convolution kernel with a grouping number of The two-dimensional convolution kernel mines high-order spatial information based on spectral aggregation; The spatial-spectral branch uses two two-dimensional convolutional layers in succession, and the convolution kernel size of each convolutional layer is 3×3. It mines features from both spatial and spectral levels and adopts a residual structure.

[0012] Furthermore, the parameter-free adaptive mask generator constructs positive and negative sample pairs representing the prototype under each meta-task and performs prototype mask comparison through graph convolution. Specifically, the parameter-free adaptive mask generator generates an emphasis mask group and a hidden mask group complementary to the emphasis mask group according to the conditional distribution of the representative prototype group obtained in each meta-task, and adaptively filters the components related to the strong category of each prototype according to the emphasis mask group and the hidden mask group to construct a positive pair representing the prototype, and leaves the redundant components to construct a negative pair representing the prototype, so as to obtain the positive and negative sample pairs representing the prototype under each meta-task, and after constructing the comparison group into a prototype comparison graph using the positive and negative sample pairs representing the prototype under each meta-task, the high-order dependencies between nodes are extracted through graph convolution for supervised contrast learning.

[0013] Furthermore, the emphasis mask group is:

[0014]

[0015]

[0016]

[0017]

[0018]

[0019] Among them, the subscript t represents the target domain, To emphasize the mask group, represents the vector concatenation operator in the column direction, To emphasize the mask group Column vector, corresponding to The emphasis mask of feature channels, represents the number of prototype categories, which is the support set and queryset The number of categories, For the The threshold interval of feature channels, For the The average value of feature channels, For the The standard deviation of the feature channels, Indicates Class represents the prototype vector No. feature components, To support the set Middle The sample set of the class, the support set Each category has samples, query set Each class has M samples. is the feature extraction function, which calculates the first Class represents the prototype vector All by = get; The hidden mask group is:

[0020]

[0021] in, To hide the mask group, To hide the mask group Column vector, corresponding to Hidden masks of feature channels; Perform scale calibration for all prototype masks, and the scale correction for the i-th prototype mask of the mask group is:

[0022] in, and are the i-th row vectors of the emphasis mask group and the hidden mask group, corresponding to the i-th emphasis mask and hidden mask representing the prototype, respectively. represents the scale correction operator in the row direction, with parameters It indicates the proportion of feature components that are masked out by the emphasized mask group operation for each representative prototype, and it also indicates the proportion of feature components that are retained by the hidden mask group operation for each representative prototype; The positive and negative sample pairs representing the prototypes under each meta-task are obtained as follows:

[0023] in, is a collection representing a prototype group Each element of is taken as a row in the matrix, express The corresponding matrix form of the positive sample group is, express The corresponding matrix form of the negative sample group is, Represents the dot product operation of matrices of the same size, and Each row of is taken as an element, and the resulting set and are the positive sample group and the negative sample group, Each element of The positive sample of the corresponding position element, Each element of Negative samples of elements at corresponding positions; The new features used for supervised contrastive learning by extracting high-order dependencies between nodes through graph convolution are:

[0024]

[0025] in, The prototype comparison group The adjacency matrix of No. Line List elements, is a parameter that controls the width of the radial basis function, is the 2-norm of the vector, The elements in are nodes, and It is Nodes and Nodes The corresponding eigenvector, The matrix form of is the input of the first layer of graph convolution, that is , , is the identity matrix, yes The degree matrix of , for Activation function, It is the first cross-domain sharing in graph convolution. The weight matrix of the layer, For graph convolution The output of the layer is also composed of The new features obtained ,Will Each row of is taken as an element to get , The elements in are the new features for contrastive learning.

[0026] Furthermore, the first stage of the two-stage training mode and the second stage of the two-stage training mode are performed in a cycle, the loss of directional calibration constrains different representative prototypes of the first stage of the two-stage training mode and the second stage of the two-stage training mode from the unification of direction and distance, the first stage of the two-stage training mode learns reliable prototypes and refines the features by directional calibration, the second stage of the two-stage training mode emphasizes the refinement of mapping based on a cross-domain shared multi-layer perceptron, and performs secondary refinement of directional calibration on the mapping features.

[0027] Furthermore, the total loss of the first stage of the cross-stage feature refinement is:

[0028] in, is the total loss of the first stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, calculated of In both stages , calculate the first stage and Used for Similarly, the first stage of cross-training also optimizes the total loss of the source domain ; The total loss of the second stage of cross-stage feature refinement is:

[0029] in, is the total loss of the second stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, is the mean square error loss, Only optimize in the second stage, calculate the second stage and Used for Similarly, the second stage of cross-training also optimizes the total loss of the source domain ; The mean square error loss is:

[0030] in, is the mean square error loss, The first step of emphasizing the mapping features for the multilayer perceptron feature components, Indicates the positive comparison group The class represents the first pair of prototype vectors. feature components; The dynamic triplet loss is:

[0031]

[0032]

[0033]

[0034]

[0035] in, It is a dynamic triplet loss. The constraints of the dynamic triplet loss are all new features of the representative prototypes and their positive and negative pairs that have undergone graph convolution. It means taking the maximum value of the input. The length is Vector and The Euclidean distance of is the facing distance, is the negative distance, is the set of key negative samples where the negative pair distance must be considered. Indicates that in the candidate negative pair Only The nearest negative sample pair distance participates in the loss calculation, is a value where the negative distance is at least greater than the positive distance, and For dynamic adjustment Hyperparameters of The FSL loss is:

[0036]

[0037]

[0038] in, is the FSL loss, It is a sample The corresponding category label, For query set Each sample The class distribution of For query sets The Prototype vector for class sample classification; The overall loss of the orientation calibration is:

[0039]

[0040]

[0041] in, is the overall loss of directional calibration, is the inter-class distance loss for directional calibration, is the intra-class distance loss for orientation calibration, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, is the 2-norm of the vector, Indicates that only The farthest intra-class distance is used to calculate the loss.

[0042] In a second aspect, the present invention provides a small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement, comprising: A dataset partitioning module is used to generate a support set and a query set cross-wise from the original hyperspectral images of the source domain dataset or the target domain dataset; The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is resistant to neighboring pixel dependence through the feature extraction module to obtain support features and query features; The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution; The cross-stage feature refinement module is used to refine the features to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains in combination with the positive pairs representing the prototypes under each meta-task, and set the two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and the query features to obtain the refined support features and the refined query features. The two training stages optimize different losses. The first stage of the two-stage training mode uses the support features and the query features to perform small sample learning, and the second stage of the two-stage training mode uses the refined support features and the refined query features to perform small sample learning. The classification module uses a small sample of the target domain in the test phase, and trains the K nearest neighbor classifier for classification prediction through the features extracted by the dimension unified mapping, feature extraction module, and multi-layer perceptron.

[0043] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement when executing the computer program.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, provides the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement.

[0045] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention proposes a small sample HSI classification method based on prototype mask contrast and cross-stage feature refinement. The small sample HSI classification framework based on prototype mask contrast and cross-stage feature refinement is abbreviated as PMC-CFR. The prototype mask contrast (PMC) module is used as the core, and cross-stage feature refinement (CFR) is further realized on this basis. It is extremely critical to find a suitable angle to construct representative positive and negative sample pairs. The present invention aims to take the prototype as the starting point, effectively remove similar redundant components between prototypes by constructing positive and negative sample pairs, and use graph convolution to learn the spatial mapping of application contrast loss from the global perspective of the contrast group. Under the promotion of supervised contrast learning, the contrast samples and additional calibration measures are further used in the training process to extract more category-representative refined features.

[0046] Furthermore, the multi-dimensional feature extraction and fusion module designed in the present invention not only weakens the dependence of the pixels to be classified on the neighboring objects, but also comprehensively extracts and fuses features from three levels: space, spectrum, and space-spectrum, extracts features from different angles, and considers both the independent information of each dimension and their interaction. More comprehensive mining and fusion of features can enable a deeper understanding of the essential information of HSI objects.

[0047] Furthermore, the prototype mask comparison module proposed in the present invention can adaptively generate an emphasis mask group and a complementary hidden mask group according to the representative prototype group of each meta-task through a parameter-free adaptive mask generator. The mask group is applied to the prototype group to obtain a positive prototype group that filters out strong category-related feature components, and a negative prototype group that retains redundant components. A prototype comparison graph is constructed for each positive and negative sample pair. Through graph convolution, high-order dependencies between the prototype and its variants can be extracted to realize contrast learning, ensuring that the redundant components in the feature vector are easier to separate.

[0048] Furthermore, the present invention uses the generated positive sample group after removing redundant components to design a two-stage training mode, and on this basis realizes cross-stage feature refinement, and sets the training into two stages. The first stage extracts reliable prototypes and performs directional calibration to realize feature refinement, and the second stage realizes emphasized mapping based on cross-domain shared multi-layer perceptron, and performs directional calibration on the de-redundant features obtained by mapping, and the emphasized mapping and directional calibration in the second stage realize a double feature refinement effect. The coordinated implementation of directional calibration and shared MLP can effectively alleviate the domain shift problem, so as to promote the alignment of feature distributions of the source domain and the target domain, thereby improving classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings: Figure 1 This is a flow chart of the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement of the present invention.

[0050] Figure 2 This is a structural diagram of the small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement of the present invention.

[0051] Figure 3 This is an electronic device diagram of the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement of the present invention.

[0052] Figure 4 It is the overall framework diagram of the PMC-CFR of the present invention.

[0053] Figure 5 This is the overall structure of the multi-dimensional feature fusion module that is resistant to neighboring pixel dependence of the present invention.

[0054] Figure 6 The overall process of constructing positive and negative samples for the parameter-free adaptive mask generator of the present invention.

[0055] Figure 7 This is the gradual improvement effect after the directional calibration of the present invention.

[0056] Figure 8 2. False color image and real object image of the IP data set in the embodiment of the present invention.

[0057] Fig. 9 1 is a pseudo-color image and a real object image of the UP data set in an embodiment of the present invention.

[0058] Fig.10 This is a classification diagram of various methods in the comparative experiment of the embodiment of the present invention on the IP data set.

[0059] Fig.11 This is a classification diagram of various methods in the comparative experiment of the embodiment of the present invention on the UP dataset.

[0060] Fig.12 This is the two-dimensional feature visualization of DMCM, GCC-FSL, and PMC-CFR on the IP dataset and UP dataset in the embodiments of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0062] Embodiment 1 See also Figure 1 , this embodiment proposes a small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, including the following steps: The support set and query set are generated by cross-generating the original hyperspectral images of the source domain dataset or the target domain dataset; After the spectral dimensions of the samples in the support set and the query set are unified, the multi-dimensional features that are resistant to neighboring pixel dependence are fused through the feature extraction module to obtain the support features and query features. Based on the parameter-free adaptive mask generator, we construct positive and negative sample pairs representing the prototypes in each meta-task and perform prototype mask comparison through graph convolution. Combined with the positive pairs representing the prototypes under each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains. The two-stage training mode is set according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain the refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training model uses support features and query features to perform small sample learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform small sample learning; In the test phase, a small sample of the target domain is used to train the K nearest neighbor classifier for classification prediction through the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

[0063] See also Figure 4 A small sample HSI classification framework based on prototype mask contrast and cross-stage feature refinement is proposed. The Prototype Mask Contrast module is used as the core, and cross-stage Feature Refinement is further implemented on this basis. The proposed scheme is abbreviated as PMC-CFR.

[0064] Figure 4The overall framework of PMC-CFR is shown, which mainly includes three key parts: feature extraction, prototype mask comparison and cross-stage feature refinement. In the figure, stage 1 represents the first stage of training, and stage 2 represents the second stage of training. The optimized losses at different stages are different.

[0065] First, the feature extraction part is implemented based on a multi-dimensional feature fusion module that is resistant to neighboring pixel dependence. Under the condition that neighboring pixels of the input HSI cube are randomly removed, features are extracted and fused from three levels: spatial, spectral, and spatial-spectral. This fully considers the independent information and interaction of the two dimensions of space and spectrum.

[0066] Secondly, the prototype mask contrast module generates an emphasis mask group and a complementary hidden mask group according to the conditional distribution of the prototype group obtained by each meta-task through a parameter-free adaptive mask generator, thereby adaptively screening the components with strong category correlation for each prototype to construct positive pairs and leaving the redundant components to construct negative pairs. After constructing the contrast group as a prototype contrast graph, the high-order dependencies between nodes are extracted through graph convolution to achieve supervised contrastive learning, making the redundant components of the features easier to separate. It is worth noting that graph convolution only maps the feature representation to a new representation space with contrast loss applied, which is discarded during testing.

[0067] Finally, cross-stage feature refinement is performed based on the cross-domain shared emphasis mapping implemented by the multi-layer perceptron (MLP), and the loss of directional calibration constrains the different representative prototypes of the two stages from the unification of direction and distance. The training is divided into two stages. In the first stage, a reliable prototype is learned to ensure the discriminability of the feature, and directional calibration is performed to achieve feature refinement. In the second stage, cross-domain shared emphasis mapping is implemented based on MLP, and directional calibration is performed on the secondary mapping features. The emphasized mapping and directional calibration in the second stage achieve a dual feature refinement effect. Cross-stage feature refinement achieved by directional calibration and shared MLP can effectively alleviate the domain shift problem. The training process implements cross-domain FSL through different prototypes in the two stages, and the test stage uses the feature extraction part followed by the entire MLP as the feature extractor, and uses a small number of labeled samples in the target domain to train the K nearest neighbor classifier for prediction. Next, the core implementation process and principle of this embodiment will be elaborated in detail from these three parts, and the embodiment will be further explained through simulation experiments and experimental results.

[0068] (i) Feature extraction: The training process is performed by cross-training in the source domain and the target domain. However, the spectral dimensions of cross-domain data are often different. Using a mapping layer to unify the dimensions is a widely adopted measure.

[0069] Specifically, it is implemented through two cross-domain non-shared mapping layers, each of which is implemented by a layer of two-dimensional convolution, including A convolution kernel of size 1×1, To unify the number of output channels, it is followed by a 2D batch normalization layer, and the mapping layer can automatically integrate the information of the spectral dimension.

[0070] The input HSI cube is denoted as , represents the number of spectral channels, 9×9 represents the spatial dimension, and the mapping layer is recorded as , then the processed output is , is the unified dimension of the source domain and the target domain. Then Input into the multi-dimensional feature fusion module that resists neighboring pixel dependence. The structure of this module is as follows Figure 5 This module aims to mine the independent information and interactions of the spatial and spectral dimensions from three perspectives under the condition of random removal of adjacent pixels.

[0071] First of all Random mask drops up to The neighboring pixels of the ratio are reduced to weaken the neighborhood dependence when extracting discriminative features. This operation is recorded as Next, a parallel three-branch structure is used to extract features from the spatial, spectral, and spatial-spectral levels.

[0072] The spatial branch first adopts a size of 3×3 and the number of groups is A two-dimensional convolution kernel is used to extract spatial information without involving the interaction of spectral dimensions, and then a three-dimensional convolution kernel of size 1×1×3 is used to mine high-order spectral features based on spatial aggregation. The convolution order of the spectral branch is opposite to that of the spatial branch. Spectral information is first extracted without involving the interaction of spatial dimensions, and then high-order spatial information is mined based on spectral aggregation. The spatial-spectral branch uses two two-dimensional convolution layers in succession. The convolution kernel size of each convolution layer is 3×3. Features are mined from both spatial and spectral levels, and a residual structure is adopted. The outputs of the three branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then pass through three-dimensional convolution (convolution kernel size is 3×3×3), adaptive three-dimensional average pooling, full connection and other operations in sequence to fully integrate features at different levels. The process of extracting features in this module is recorded as , we can get the formula (1) as the final feature representation.

[0073] (1) Unify the number of dimensions Set to 96, and each layer parameter data is cleverly set to maintain the input dimension With output dimension This helps avoid information loss and preserve subtle differences. For the convenience of representation, all operations in the feature extraction part are simply recorded as The types of each layer of the multi-dimensional feature fusion module with anti-neighboring pixel dependence and its corresponding related parameters are shown in Table 1.

[0074] Table 1 Detailed configuration of each layer of the multi-dimensional feature fusion module based on neighbor dependency

[0075] (ii) Prototype mask comparison: First, a parameter-free adaptive mask generator is used to construct positive and negative pairs representing the prototypes for each meta-task.

[0076] Each meta-task In the training of , including the support set and queryset , support set Include Class, each class has samples, then the meta-task It is called C-way K-shot, and the corresponding query set Each category has samples. Since each meta-task is constructed The training samples are randomly selected, so the support set is considered and queryset The feature representation of all samples in the target domain is used to reduce sample fluctuations, and the mean of each type of feature is used as the representative prototype. The representative prototype of the class It can be calculated by formula (2).

[0077] (2) in, and Indicates the first The sample set of the class, express Belong to the Collection of classes Each sample in , Represents the feature extraction function, which is different in the two stages of training and used in the prototype mask comparison process and its variants have passed the first stage accomplish.

[0078] for The class represents a prototype, which is called a prototype group and is recorded as Some channels in the high-dimensional feature space may contribute to or even be critical to the discrimination of some classes, but these channels may be seen as commonalities in other classes, not separable, or even redundant. The parameter-free adaptive mask generator performs masking on each feature channel. To evaluate the distribution, the formula (3) and formula (4) are used to calculate the The average value of feature channels and standard deviation , the channel threshold interval is generated by formula (5) .

[0079] (3) (4) (5) in, Indicates the target domain The class represents the prototype vector feature components.

[0080] For The feature components within the interval are hidden as common features of some classes that have no distinguishing ability, and the feature components outside the interval are emphasized as significant features of some classes. Thus, the emphasis mask of the channel is generated as shown in equations (6) and (7): and hide mask (Upper right corner Indicates that the mask is the first column vector).

[0081] (6) (7) in, The column vector A quantity.

[0082] Similarly, we can get the mask of each channel representing the prototype feature. We take the mask of each channel as a column and concatenate them into a matrix by equation (8) and equation (9) and record it as the emphasis mask group: and hide mask group .

[0083] (8) (9) in, Represents the vector concatenation operator in the column direction.

[0084] The distribution fluctuation of some channels is large, which may cause some prototypes to be masked out too much by the emphasis mask or left too much by the concealment mask. Therefore, for each prototype mask of the mask group, Row indicates the The mask corresponding to the prototype is used for scale correction. The mask corresponding to the prototype is recorded as and , upper right corner mark Indicates that the mask is the first Row vector. Statistics 0 and Whether the proportion of 1 is within the appropriate value, if it is greater, 0 and 1 are randomly inverted to ensure that the proportion of feature components masked by positive samples or retained by negative samples is maintained at an appropriate value. The proportion correction process is implemented by formula (10).

[0085] (10) in, represents the scale correction operator in the row direction, whose parameters It indicates the proportion of feature components that are masked out by the emphasized mask group operation for each representative prototype, and it also indicates the proportion of feature components that are retained by the hidden mask group operation for each representative prototype.

[0086] The random proportion correction is dominated by the screening characteristics, which retains the representativeness of the variant properties, while the random number introduces uncertain interference and increases the difference Finally, Each element of is represented as a row in sequence, and its matrix form is obtained. , and the mask group are calculated by formula (11) and .

[0087] (11) in, Represents a dot product (Hadamard product) operation on matrices of the same size.

[0088] Will and Each row of is taken as an element, and we can get The positive control group and negative control group For each , is its only positive sample pair, and all other samples are regarded as negative sample pairs. The overall process of obtaining positive and negative samples corresponding to each prototype is as follows Figure 6As shown in FIG. 1 , the color of the squares constituting the mask, white indicates that the bit is 1, black indicates that the bit is 0, and the scale correction operation in the row direction sets the parameter β to 0.6 for illustration.

[0089] Will Denoted as the prototype comparison group, each element in it can be regarded as a feature vector corresponding to a node, and the The feature vector corresponding to the node is , the adjacency matrix can be constructed from the relationship between nodes , calculated by formula (12) No. Row, No. Column elements , after adding the self-loop and normalizing, we get , and thus construct a prototype comparison chart. Each element of is represented as a row in turn, and its matrix form is obtained. As the first layer input of graph convolution, The graph convolution operation is performed by formula (13), and the features of the prototype comparison group are mapped to the new feature space with contrast loss using the global structural information of the prototype comparison graph. The output of the layer is the new feature representation .

[0090] (12) (13) in, is a parameter that controls the width of the radial basis function, and Is a node and The corresponding eigenvector, is the 2-norm of the vector, for Activation function, It is the first cross-domain sharing in graph convolution. The weight matrix of the layer.

[0091] Each row of is used as a new feature representation of the corresponding prototype, and its collection form is obtained , and decomposed into , and .Will Each element in , as the first The class represents the prototype's base, only The corresponding For its right side, The corresponding and All except itself are considered as necessary negative pairs, which are simply referred to as sets , including the negative pairs generated by itself and other initial prototypes. These negative pairs are the most valuable and must be paid attention to. In addition to the above-mentioned prototype variants, all other candidate negative pairs are abbreviated as a set , the number of candidate negative pairs is large and the range is large, so only The nearest pair is involved in the contrast loss calculation, which can also prevent the negative impact of close distance from being mitigated by long distance. Dynamic triplet loss is used to perform contrastive learning, as shown in formula (14). This loss aims to make the positive sample pairs closer and the negative sample pairs farther, making it easier to separate redundant components of the input features.

[0092] (14) (15) (16) (17) (18) in, Indicates taking the maximum value of the input. Formula (15) is a length Vector and The Euclidean distance of, formula (16) and formula (17) are the positive distance and negative distance respectively. is the set of key negative samples where the negative pair distance must be considered. Indicated in Only The nearest negative sample pair distance participates in the loss calculation, and the formula (18) is a value where the negative distance is at least greater than the positive distance, and is a hyperparameter, which is adaptively adjusted according to the positive and negative distances .

[0093] 3. Cross-stage feature refinement: 1. Emphasis mapping based on multi-layer perceptron: The positive samples representing each type of prototype obtained in the prototype mask comparison module screen out the feature components that are strongly related to this type, and remove the redundant components that are common in the prototype group, but this depends on the conditional distribution statistics based on the prototype group. Such positive samples have refined features based on the prototype and have more refined discriminability. Therefore, it is critical to achieve feature refinement on the test sample from this perspective. It should be noted that The emphasis mask group represented by The row element corresponds to This means that even if the mask group is calculated and retained from the training samples in the test phase, the test samples cannot determine the corresponding mask position because they have no labels. Therefore, MLP is introduced here to realize the emphasized mapping of features, which is recorded as , which is used to fit the feature refinement of the emphasized mask. The feature representation in is then subjected to secondary feature mapping through MLP, and the obtained output is consistent with the positive comparison group The mean square error (MSE) of the features of the positive samples of the corresponding category is optimized, as shown in formula (19).

[0094] (19) According to each The emphasized mask group of all label sample statistics under , effectively utilizes all label sample information, reflects the category prototype representation trend of a meta-task, and at the same time, each time The training samples are randomly selected, and the differences in meta-tasks further promote the exploration of commonalities from the representation trends of various category prototypes. Therefore, this kind of feature component screening with strong category correlation has better generalization, and by exploring common mappings from different representation trends through MLP, it can effectively remove redundancy from test samples without relying on labels. Under the cross-domain learning of meta-tasks, the sharing of MLP can explore de-redundant mappings that are more conducive to feature distribution alignment, and the domain offset problem can be effectively alleviated. The implementation of MLP is lightweight, consisting of only two fully connected layers with a Activation function, detailed parameters are shown in Table 2.

[0095] Table 2 Detailed configuration of shared MLP

[0096] 2. Directional calibration: In order to further make full use of and The labeled samples in make the intra-class distance closer and the inter-class distance farther, and design the loss of directional calibration. Figure 7 This is the gradual improvement effect after directional calibration. The figure is illustrated by two-dimensional visualization. The prototype in the yellow circle indicates stable distribution under multi-dimensional tasks, the prototype near the green circle indicates large distribution fluctuations under multi-dimensional tasks, the black circle indicates the position when the class is pulled apart, and the dark area indicates the opposite direction of the illustrated situation. According to formula (2), the representative prototype of each class is obtained for inter-class directional calibration. If the prototypes of two classes are directly pulled apart by distance, the problems existing when only prototyping is performed based on distance are as follows: Figure 7 As shown in (a), compared with the prototype of a certain class in different The distribution is stable (floating in the yellow circle), when the same category is in different The prototype distribution under σ2 fluctuates greatly (fluctuates near the two green circles). It can be seen that multiple directions can produce a farther distance effect (near the black circle), but they are farther away from each other. The calibration effect of multiple-dimensional tasks is not only not cumulative, but has a negative effect of confrontation. The Euclidean distance measurement ignores directionality, that is, the distance contribution of different feature components may be different, but the Euclidean distance focuses on the sum of contributions, which is in line with expectations. The loss of directional calibration is shown in formula (20). Cosine similarity is introduced in the numerator of the Euclidean distance term as a directional guide. After the introduction of cosine similarity, there are special cases in prototype calibration with directional guidance, such as Figure 7 As shown in (b) of the figure, the mutual balance between distance and direction is taken into account, thereby making different classes farther apart. The prototypes of the same category tend to be closer in the high-dimensional space, that is, in the opposite direction (the dark area). However, cosine similarity will weaken the influence of distance measurement, especially for prototypes that are already in the opposite direction, their calibration loss will depend entirely on the direction. Therefore, the 2-norm of the prototype is added as a constraint in equation (20). Introducing the 2-norm of the prototype as a constraint solves the problem that special samples cannot be calibrated. Figure 7 As shown in (c), the prototype of the distance metric is ignored because the directions have tended to be opposite. The constraint term limits its own modulus to be large enough, thereby ensuring that the distance between classes is large enough. At this time, even if the Euclidean distance term is a very small negative number, it can be considered beneficial. On the contrary, if the directions tend to be close, the constraint term does not guarantee that the distance between classes is large enough, and it is necessary to separate the prototype between classes from the unity of direction and distance. The 2-norm constraint ensures that the Euclidean distance between classes is always large enough under various directions. Directional calibration aims to explore a reverse direction range in high-dimensional space to ensure that the classes are far away, which helps the features of the same class of samples to be distributed as close as possible to the direction.

[0097] After obtaining the distribution of similar features in the directional range, the intra-class distance is directly shortened by formula (21), and the uniform setting is to take only The farthest participant loss calculation, Setting it to a smaller value can focus more on optimizing the compactness within the class from extreme samples and avoid the risk of overfitting caused by over-smoothing the global distribution.

[0098] (20) (twenty one) (twenty two) in, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, Indicates that only The farthest intra-class distance is used to calculate the loss, and formula (22) is the overall loss of directional calibration (DC).

[0099] 3. Two-stage training mode: The proposed PMC-CFR sets a two-stage training mode to fully refine the features, and the two stages of training are performed cyclically. The interval between the two stages is denoted as , each training Next, This is recorded as stage one, and then This is recorded as stage 2, and repeated.

[0100] Each stage of training includes FSL loss. Taking the target domain as an example, the first Prototype vector of the class pass From formula (23), we get Each sample Class distribution Calculate according to formula (24) and get the FSL loss by formula (25).

[0101] (twenty three) (twenty four) (25) in, yes The corresponding category label.

[0102] The two stages of training have different feature extraction functions , the first stage does not contain a shared MLP, then , and the second stage contains a shared MLP, then From formula (23), we can get The prototype of the classification is subjected to FSL by equation (25), and the representative prototype is obtained by equation (2), and the directional calibration loss is optimized by equation (22). Both stages optimize the contrast loss, and only the second stage optimizes the MSE by equation (19). Therefore, the total loss that needs to be optimized in the first stage is: (26) The total loss that needs to be optimized in the second stage is: (27) Meta-task training is performed crosswise between the source domain and the target domain. Similarly, the two stages of training are in the source domain and In the testing phase, by Features are extracted and K nearest neighbor classifier is used for prediction.

[0103] (IV) Simulation conditions and experimental settings: All experiments are implemented based on the Pytorch framework on a platform with an i9-12900HX CPU, NVIDIA GeForce RTX 4060 16GB GPU, and 16GB of memory.

[0104] As a cross-domain FSL method, the Chikusei dataset is selected as the source domain dataset, from which 200 labeled samples are selected for training in each category, while only 5 labeled samples are selected for each category in the target domain dataset, and the dataset is expanded to 200 for training by adding random Gaussian noise and random flipping data augmentation operations. Each C-way K-shot meta-task is used as an iteration. is set to the number of categories of the target dataset, and Set to 1 to query the number of samples of each class in the set Set to 19, that is, 20 labeled samples will be randomly selected from 200 for each class in each meta-task for training, and the source domain and target domain will be cross-trained for meta-task training. The spatial size of the input HSI cube is 9×9, and the number of graph convolution layers is Set to 3, two-stage interval Set to 500, the number of nearest neighbors used by the K-nearest neighbor classifier for classification is set to 1. The number of samples used to filter the loss calculation in the contrast loss and directional calibration loss It is uniformly set to 3 and used for dynamic adjustment in contrast loss of and Set to 8 and 0.5, as well as the directional calibration loss used to adjust the magnitude of the distance between or within classes. Set to 0.5. The number of training iterations is set to 10000, and the Adam optimizer is used to optimize the model parameters. The learning rate is set to 0.0005, and the learning rate is decayed by 20% every 4000 iterations.

[0105] (V) Performance Verification: The overall accuracy (OA), average accuracy (AA) and Kappa coefficient are used to comprehensively evaluate the classification performance of the proposed scheme, and each experiment is randomly and independently run ten times to report the average results and standard deviation.

[0106] In order to evaluate the effectiveness of the proposed PMC-CFR in the small sample scenario of HSI, nine related schemes were selected for comparison in the experiment, including support vector machine (SVM) based on radial basis function kernel, 3DCNN, DCFSL, Gia-CFSL, CMFSL, ADAFSL, DMCM, HyMuT and GCC-FSL. Among them, SVM belongs to the traditional machine learning method, 3DCNN is a conventional deep learning method, which does not involve cross-domain and FSL, and the other seven methods are advanced HSI classification methods in cross-domain small sample scenarios. All methods select 5 labeled samples per category from the target domain dataset in the same random manner for training, and the other samples are used for testing. For the cross-domain classification methods for small sample scenarios of HSI, the Chikusei dataset with rich category information is used as the source domain dataset for learning transfer knowledge. The target domain datasets select the widely used Indian Pines (IP) dataset and University of Pavia (UP) for performance evaluation. Table 3 shows the detailed information of the 16 categories of the IP dataset, and Figure 8 (a) shows a pseudo-color image. Figure 8 (b) shows the real ground map. The sample details of the 9 categories of the UP dataset are shown in Table 4, and the pseudo-color images are shown in Fig. 9 (a) and the real terrain map Fig. 9 (b).

[0107] 1. Performance on IP dataset: Table 5 shows the classification results of various methods on IP dataset. Due to the lack of labeled samples, SVM and 3DCNN cannot extract classification features with high discrimination, and the classification results are relatively low. However, for 3DCNN, the feature extraction based on deep learning still makes the classification results better than the traditional method SVM. The classification performance of the methods based on cross-domain small sample learning has been greatly improved. Overall, the proposed PMC-CFR achieved the best values ​​in OA, AA and Kappa, which are 79.79%, 87.58% and 77.13% respectively. Compared with the second best DMCM, OA, AA and Kappa have increased by 5.38%, 4.67% and 6.07% respectively. The classification accuracy of each category is basically better than other methods. It is worth noting that the classification accuracy of Corn-notill class (2 categories), Hay-windowed class (8 categories) and Soybean-clean class (12 categories) is significantly ahead of other methods. Fig.10 The classification diagrams corresponding to the experimental results are shown. Fig.10 (a) in the figure is the real feature map; Fig.10 (b) in the figure is 3DCNN; Fig.10 (c) in it is DCFSL; Fig.10(d) in the figure is Gia-CFSL; Fig.10 (e) in it is CMFSL; Fig.10 (f) in it is ADAFSL; Fig.10 (g) in the figure is DMCM; Fig.10 (h) in the figure is HyMuT; Fig.10 (i) in the figure is GCC-FSL; Fig.10 (j) in the figure is PMC-CFR; The classification effect of PMC-CFR is the smoothest, and the classified areas have better integrity.

[0108] Table 3 Names and sample numbers of each category in the IP dataset

[0109] Table 4 Name and number of samples of each category in the UP dataset

[0110] 2. Performance on UP dataset: Table 6 shows the experimental results on UP dataset. It can be seen that SVM and 3DCNN perform similarly, and the small sample learning paradigm still effectively improves the classification performance. Compared with the second-best DMCM, PMC-CFR achieved 90.48%, 91.28% and 87.53% results on OA, AA and Kappa, which are 4.05%, 3.59% and 5.34% higher respectively. PMC-CFR has the highest classification accuracy on 8 categories. Bare Soil (6 categories) and Self-Blocking Bricks (8 categories) can still achieve leading performance despite the high misclassification of other methods. Fig.11 The classification diagram corresponding to the experimental results is shown. Fig.11 (a) in the figure is the real feature map; Fig.11 (b) in the figure is 3DCNN; Fig.11 (c) in it is DCFSL; Fig.11 (d) in the figure is Gia-CFSL; Fig.11 (e) in it is CMFSL; Fig.11 (f) in it is ADAFSL; Fig.11 (g) in the figure is DMCM; Fig.11 (h) in the figure is HyMuT; Fig.11 (i) is GCC-FSL; Fig.11 (j) in the figure is PMC-CFR. It can be seen that the proposed method has a good classification effect in the edge areas of Gravel (3 categories) and Bitumen (7 categories), which reduces the terrain dependence of ground feature classification to a certain extent.

[0111] Table 5 Classification results of each method on the IP dataset (five labeled samples per class)

[0112] Table 6 Classification results of each method on the UP dataset (five labeled samples per class)

[0113] (vi) Feature visualization: In order to further evaluate the predictive effect of features extracted by different methods from an intuitive visual perspective, two-dimensional visualization of high-dimensional features is achieved based on t-SNE. Here, experiments are conducted on DMCM, GCC-FSL, and PMC-CFR on the IP dataset and UP dataset. Fig.12 The t-SNE visualization effects of each method on two datasets are shown. Fig.12 (a) in the figure is the IP feature of DMCM; Fig.12 (b) in the figure is the IP feature of GCC-FSL; Fig.12 (c) in the figure is the IP feature of PMC-CFR; Fig.12 (d) in the figure is the UP feature of DMCM; Fig.12 (e) in the figure is the UP feature of GCC-FSL; Fig.12 (f) in the figure is the UP feature of PMC-CFR. It can be seen intuitively from the figure that the proposed PMC-CFR shows better intra-class clustering effect and inter-class feature classification on each data set. For the IP data set, the clustering effect of each category of the comparison method is relatively loose and the intra-class tightness is poor, while PMC-CFR makes the distance between samples of the same category closer and makes the separation boundary between samples of different categories clearer. Specifically for the Soybean-mintill class (11 categories) of the IP data set, the performance of the comparison method is too dispersed and easily confused with other categories, while PMC-CFR has a good degree of feature clustering in this category and a strong degree of separation from other category features. For Asphalt (1 category) of the UP data set, the comparison methods all show overlapping or clustered dispersion, but the performance of the proposed method avoids these problems. Based on the above analysis, PMC-CFR effectively enhances the distinguishability of different category features in the feature space.

[0114] This embodiment aims to fully train the deep network for HSI classification using only small samples. In the face of the domain shift problem existing in the current cross-domain FSL, the current advanced prototype-based methods basically do not consider the redundancy of similar feature components between various prototypes. The redundant components fitted to each domain make it difficult to align the feature distribution of the source domain and the target domain, and the resulting domain shift problem has not been effectively solved. This paper proposes a small sample HSI classification scheme (PMC-CFR) based on prototype mask comparison and cross-stage feature refinement to solve the potential prototype redundancy and domain shift problems under cross-domain learning under the prototype-based classification mode. The feature extraction module with multiple convolution modes used in this embodiment extracts and fuses features from three levels: space, spectrum, and space-spectrum. The additional adjacent pixel removal mechanism introduced here also ensures the weak neighborhood dependency of the sample. Next, the parameter-free adaptive mask generator adaptively generates an emphasized mask group and a complementary hidden mask group according to the representative prototype group of each meta-task. The mask group is applied to the prototype group to obtain a positive prototype group that filters out strong category-related feature components, and a negative prototype group that leaves redundant components. By implementing graph convolution based on the graph structure of the comparison group, high-order dependencies between the prototype and its variants can be extracted to achieve contrastive learning. The process of prototype mask comparison ensures that redundant components in the feature vector are easier to separate. Finally, the training is set to two stages. The first stage extracts reliable prototypes and performs directional calibration to achieve feature refinement. The second stage implements cross-domain shared emphasis mapping based on MLP, and performs directional calibration on the mapping results. The emphasis mapping and directional calibration in the second stage achieve a double feature refinement effect. The cross-stage feature refinement achieved by the coordination of shared MLP and directional calibration can effectively alleviate the problem of domain offset. Therefore, this embodiment can show good classification performance with very small training samples.

[0115] HSI classification has shown broad prospects in multiple application fields. In the agricultural field, fine classification of crops and soil based on HSI can help farmers monitor crop health, evaluate soil quality and optimize crop management, thereby improving yield and resource utilization efficiency. In environmental monitoring, HSI classification can effectively identify different types of vegetation, detect water pollution and evaluate land use changes, providing important data support for ecological protection and sustainable development. In addition, in urban planning, this technology can be used for land cover classification, urban heat island effect monitoring and infrastructure assessment, helping urban managers make more scientific decisions. It is also widely used in mining and resource exploration. Overall, HSI classification technology is increasingly becoming an important tool for research and application in various fields, which can promote innovation and development in related industries. At the same time, with the advancement of hyperspectral imaging technology, obtaining hyperspectral data has become more common, but the time and labor cost required to label HSI are still very high. How to use extremely limited labeled samples for model training and improve classification accuracy is an extremely critical issue. FSL can enable the model to quickly adapt to new environmental changes and achieve more flexible applications when label data is scarce.

[0116] Therefore, realizing efficient HSI classification based on FSL is an application trend in this field. The problem solved by this embodiment is to quickly realize effective HSI classification through small samples in various environments, thereby reducing the application cost of implementing this technology in various scenarios.

[0117] Embodiment 2 See also Figure 2 , a small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement, including: A dataset partitioning module is used to generate a support set and a query set cross-wise from the original hyperspectral images of the source domain dataset or the target domain dataset; The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is resistant to neighboring pixel dependence through the feature extraction module to obtain support features and query features; The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution; The cross-stage feature refinement module is used to refine the features to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains in combination with the positive pairs representing the prototypes under each meta-task, and set the two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and the query features to obtain the refined support features and the refined query features. The two training stages optimize different losses. The first stage of the two-stage training mode uses the support features and the query features to perform small sample learning, and the second stage of the two-stage training mode uses the refined support features and the refined query features to perform small sample learning. The classification module uses a small sample of the target domain in the test phase, and trains the K nearest neighbor classifier for classification prediction through the features extracted by the dimension unified mapping, feature extraction module, and multi-layer perceptron.

[0118] Embodiment 3 See also Figure 3 , an electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement is implemented.

[0119] Embodiment 4 A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement is implemented.

[0120] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, read-only optical disk, optical storage, etc.) containing computer-usable program code.

Claims

1. A small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement, characterized in that: The following steps are involved: The support set and query set are generated by cross-generating the original hyperspectral images of the source domain dataset or the target domain dataset; After the spectral dimensions of the samples in the support set and the query set are unified, the multi-dimensional features that are resistant to neighboring pixel dependence are fused through the feature extraction module to obtain the support features and query features. Based on the parameter-free adaptive mask generator, we construct positive and negative sample pairs representing the prototypes in each meta-task and perform prototype mask comparison through graph convolution. Combined with the positive pairs representing the prototypes under each meta-task, the features are refined to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains. The two-stage training mode is set according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and query features to obtain the refined support features and refined query features. Different losses are optimized in the two training stages. The first stage of the two-stage training model uses support features and query features to perform small sample learning, and the second stage of the two-stage training model uses refined support features and refined query features to perform small sample learning; In the test phase, a small sample of the target domain is used to train the K nearest neighbor classifier for classification prediction through the features extracted through dimensional unified mapping, feature extraction module, and multi-layer perceptron.

2. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1, characterized in that: The multi-dimensional feature fusion that resists neighboring pixel dependence is specifically, under the condition that neighboring pixels of the input HSI cube are randomly removed, features are extracted from three levels of space, spectrum, and space-spectrum respectively in a parallel structure and fused to obtain fused features; The fusion features are: in, To fusion features, For the feature extraction process, is a random mask of neighboring pixels, is the maximum ratio of random mask, is the mapping layer, is the input HSI cube, represents the number of spectral channels, 9×9 represents the spatial dimension, and the output after dimension unification is , is the unified dimension of the source domain and the target domain, is the output dimension, and the fusion features will be extracted All operations are recorded as .

3. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 2 is characterized in that: The feature extraction module includes a neighboring pixel removal mechanism and three parallel branches. The outputs of the three parallel branches are spliced ​​in the channel dimension after three-dimensional average pooling, and then three-dimensional convolution, adaptive three-dimensional average pooling, and full connection operations are used to fuse features at different levels. The three parallel branches include a spatial branch, a spectral branch, and a spatial-spectral branch. The spatial branch first adopts a size of 3×3 and the number of groups is A two-dimensional convolution kernel of 1×1×3 is used to extract spatial information, and a three-dimensional convolution kernel of 1×1×3 is used to mine high-order spectral features based on spatial aggregation. The spectral branch first uses a 3D convolution kernel of size 1×1×3 to extract spectral features, and then uses a 3×3 convolution kernel with a grouping number of The two-dimensional convolution kernel mines high-order spatial information based on spectral aggregation; The spatial-spectral branch uses two two-dimensional convolutional layers in succession, and the convolution kernel size of each convolutional layer is 3×3. It mines features from both spatial and spectral levels and adopts a residual structure.

4. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1, characterized in that: The method is based on a parameter-free adaptive mask generator to construct positive and negative sample pairs representing the prototype under each meta-task and perform prototype mask comparison through graph convolution. Specifically, based on the parameter-free adaptive mask generator, an emphasis mask group and a hidden mask group complementary to the emphasis mask group are generated according to the conditional distribution of the representative prototype group obtained in each meta-task. According to the emphasis mask group and the hidden mask group, components related to strong categories are adaptively screened for each prototype to construct positive pairs representing the prototype, and redundant components are left to construct negative pairs representing the prototype, so as to obtain positive and negative sample pairs representing the prototype under each meta-task. After the comparison group is constructed as a prototype comparison graph using the positive and negative sample pairs representing the prototype under each meta-task, high-order dependencies between nodes are extracted through graph convolution for supervised comparative learning.

5. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 4 is characterized in that: The emphasis mask set is: Among them, the subscript t represents the target domain, To emphasize the mask group, represents the vector concatenation operator in the column direction, To emphasize the mask group Column vector, corresponding to The emphasis mask of feature channels, represents the number of prototype categories, which is the support set and queryset The number of categories, For the The threshold interval of feature channels, For the The average value of feature channels, For the The standard deviation of the feature channels, Indicates Class represents the prototype vector No. feature components, To support the set Middle The sample set of the class, the support set Each category has samples, query set Each class has M samples. is the feature extraction function, which calculates the first Class represents the prototype vector All by = get; The hidden mask group is: in, To hide the mask group, To hide the mask group Column vector, corresponding to Hidden masks of feature channels; Perform scale calibration for all prototype masks, and the scale correction for the i-th prototype mask of the mask group is: in, and are the i-th row vectors of the emphasis mask group and the hidden mask group, corresponding to the i-th emphasis mask and hidden mask representing the prototype, respectively. represents the scale correction operator in the row direction, with parameters It indicates the proportion of feature components that are masked out by the emphasized mask group operation for each representative prototype, and it also indicates the proportion of feature components that are retained by the hidden mask group operation for each representative prototype; The positive and negative sample pairs representing the prototypes under each meta-task are obtained as follows: in, is a collection representing a prototype group Each element of is taken as a row in the matrix, express The corresponding matrix form of the positive sample group is, express The corresponding matrix form of the negative sample group is, Represents the dot product operation of matrices of the same size, and Each row of is taken as an element, and the resulting set and are the positive sample group and the negative sample group, Each element of The positive sample of the corresponding position element, Each element of Negative samples of elements at corresponding positions; The new features used for supervised contrastive learning by extracting high-order dependencies between nodes through graph convolution are: in, The prototype comparison group The adjacency matrix of No. Line List elements, is a parameter that controls the width of the radial basis function, is the 2-norm of the vector, The elements in are nodes, and It is Nodes and Nodes The corresponding eigenvector, The matrix form of is the input of the first layer of graph convolution, that is , , is the identity matrix, yes The degree matrix of , for Activation function, It is the first cross-domain sharing in graph convolution. The weight matrix of the layer, For graph convolution The output of the layer is also composed of The new features obtained ,Will Each row of is taken as an element to get , The elements in are the new features for contrastive learning.

6. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1, characterized in that: The first stage of the two-stage training mode and the second stage of the two-stage training mode are performed cyclically, the loss of directional calibration constrains different representative prototypes of the first stage of the two-stage training mode and the second stage of the two-stage training mode from the unification of direction and distance, the first stage of the two-stage training mode learns reliable prototypes and refines the features by directional calibration, the second stage of the two-stage training mode emphasizes the refinement of mapping based on a cross-domain shared multi-layer perceptron, and performs secondary refinement of directional calibration on the mapping features.

7. The small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement according to claim 1, characterized in that: The total loss of the first stage of cross-stage feature refinement is: in, is the total loss of the first stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, calculated of In both stages , calculate the first stage and Used for Similarly, the first stage of cross-training also optimizes the total loss of the source domain ; The total loss of the second stage of cross-stage feature refinement is: in, is the total loss of the second stage of cross-stage feature refinement in the target domain, is the FSL loss, is the dynamic triplet loss, is the overall loss of directional calibration, is the mean square error loss, Only optimize in the second stage, calculate the second stage and Used for Similarly, the second stage of cross-training also optimizes the total loss of the source domain ; The mean square error loss is: in, is the mean square error loss, The first step of emphasizing the mapping features for the multilayer perceptron feature components, Indicates the positive comparison group The class represents the first pair of prototype vectors feature components; The dynamic triplet loss is: in, It is a dynamic triplet loss. The constraints of the dynamic triplet loss are all new features of the representative prototypes and their positive and negative pairs that have undergone graph convolution. It means taking the maximum value of the input. The length is Vector and The Euclidean distance of is the facing distance, is the negative distance, is the set of key negative samples where the negative pair distance must be considered. Indicates that in the candidate negative pair Only The nearest negative sample pair distance participates in the loss calculation, is a value where the negative distance is at least greater than the positive distance, and For dynamic adjustment Hyperparameters of The FSL loss is: in, is the FSL loss, It is a sample The corresponding category label, For query set Each sample The class distribution of For query sets The Prototype vector for class sample classification; The overall loss of the orientation calibration is: in, is the overall loss of directional calibration, is the inter-class distance loss for directional calibration, is the intra-class distance loss for orientation calibration, Used to adjust the magnitude of the distance between or within classes. Used to control the influence of the 2-norm constraint term, is the 2-norm of the vector, Indicates that only The farthest intra-class distance is used to calculate the loss.

8. A small sample HSI classification system based on prototype mask comparison and cross-stage feature refinement, characterized by: include: A dataset partitioning module is used to generate a support set and a query set cross-wise from the original hyperspectral images of the source domain dataset or the target domain dataset; The feature extraction module is used to unify the spectral dimensions of the samples of the support set and the query set, and then perform multi-dimensional feature fusion that is resistant to neighboring pixel dependence through the feature extraction module to obtain support features and query features; The prototype mask comparison module is used to construct positive and negative sample pairs representing the prototype under each meta-task based on a parameter-free adaptive mask generator and perform prototype mask comparison through graph convolution; The cross-stage feature refinement module is used to refine the features to eliminate redundancy based on the emphasis mapping of the multi-layer perceptron shared across domains in combination with the positive pairs representing the prototypes under each meta-task, and set the two-stage training mode according to whether the emphasis mapping is performed. Each stage refines the features at different levels based on the directional calibration. The multi-layer perceptron and the directional calibration coordinate to perform cross-stage feature refinement on the support features and the query features to obtain the refined support features and the refined query features. The two training stages optimize different losses. The first stage of the two-stage training mode uses the support features and the query features to perform small sample learning, and the second stage of the two-stage training mode uses the refined support features and the refined query features to perform small sample learning. The classification module uses a small sample of the target domain in the test phase, and trains the K nearest neighbor classifier for classification prediction through the features extracted by the dimension unified mapping, feature extraction module, and multi-layer perceptron.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the small sample HSI classification method based on prototype mask comparison and cross-stage feature refinement as described in any one of claims 1 to 7.

Images