Hyperspectral remote sensing image new class discovery method based on dual relation distillation constraint

Abstract

The invention discloses a hyperspectral remote sensing image new class discovery method based on dual relation distillation constraint, comprising the following steps: acquiring a hyperspectral remote sensing image data set, and randomly dividing and acquiring a data training set, a data verification set and a data test set according to the hyperspectral remote sensing image data set; carrying out training data selection on the data training set to obtain a supervised training sample set and a new class discovery training sample set; constructing a new class discovery network model of hyperspectral remote sensing image new class discovery based on the dual relation distillation constraint, performing model training on the new class discovery network model according to the partial supervision training sample set and the new class discovery training sample set, and performing model verification on the trained new class discovery network model through the data verification set to obtain a new class discovery network model of hyperspectral remote sensing image new class discovery. Obtaining a new class discovery network optimization model; and realizing new class prediction of the hyperspectral remote sensing images in the data test set according to the new class discovery network optimization model. The problem that a traditional classification mode gradually shows limitation due to the fact that the traditional classification mode depends on sufficient annotation data and is difficult to adapt to emergencies of new categories is solved. Existing classification technologies mostly depend on limited labeled samples, so that the new class recognition capability is insufficient, and the adaptability is poor.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints. Background Art

[0002] Faced with the growing hyperspectral remote sensing big data and the ever-changing environment, the traditional classification model has gradually revealed its limitations because it relies on sufficient annotated data and is difficult to adapt to the sudden emergence of new categories. That is, most of the existing classification technologies rely on limited annotated samples, resulting in insufficient recognition of new categories and poor adaptability. Therefore, it is particularly important to explore methods for discovering new categories in the hyperspectral field. Summary of the invention

[0003] The present invention provides a new class discovery method for hyperspectral remote sensing images based on dual relationship distillation constraints to overcome the above technical problems.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] A new class discovery method for hyperspectral remote sensing images based on dual relationship distillation constraints includes the following steps:

[0006] S1: Acquire a hyperspectral remote sensing image dataset, wherein the hyperspectral remote sensing image dataset includes spectral characteristics and metric space characteristics of target objects and corresponding target object categories;

[0007] Randomly divide and obtain a data training set, a data validation set, and a data test set according to the hyperspectral remote sensing image data set;

[0008] And select training data from the data training set to obtain a supervised training sample set and a new class discovery training sample set;

[0009] S2: Construct a new class discovery network model for new class discovery of hyperspectral remote sensing images based on dual relationship distillation constraints, and the new class discovery network model includes a supervised training network unit for extracting spectral features and metric space features in a supervised training sample set, a new class discovery network unit for extracting spectral features and metric space features in a new class discovery training sample set, and a category relationship distribution distillation module for capturing the category relationship between known categories and new categories of target objects in the spectral space dimension according to the outputs of the supervised training network unit and the new class discovery network unit;

[0010] S3: According to the supervised training sample set and the new class discovery training sample set, the new class discovery network model is trained, and the trained new class discovery network model is verified through the data verification set to obtain the new class discovery network optimization model;

[0011] S4: New class prediction of the hyperspectral remote sensing images in the data test set is achieved based on the new class discovery network optimization model.

[0012] Furthermore, the method for selecting training data from the data training set described in S1 is:

[0013] Divide the data training set into a supervised training sample set and a new class discovery training sample set according to a preset ratio;

[0014] The supervised training sample set is a set of hyperspectral remote sensing image data that contains the spectral features and metric spatial features of the target objects, and the target object categories corresponding to the target objects as known categories of the target objects;

[0015] The new class discovery training sample set is a set of hyperspectral remote sensing image data that contains target object categories corresponding to the spectral / metric spatial characteristics of the target objects as new categories of the target objects.

[0016] Further, the supervised training network unit in S2 includes a first feature extractor and a first classifier connected in sequence;

[0017] The new class discovery network unit includes a second feature extractor, a new class relationship prediction distillation module and a second classifier connected in sequence;

[0018] The first feature extractor and the second feature extractor are used to extract spectral features and metric space features from the supervised training sample set and the new class discovery training sample set, respectively;

[0019] At the same time, the output feature of the second feature extractor is the instance feature;

[0020] The first classifier is used to classify the target object category according to the output of the first feature extractor;

[0021] The new class relationship prediction distillation module is used to obtain a cluster heap about feature distribution according to the output of the second feature extractor based on the GMM model, and define the cluster heap as a feature prototype group;

[0022] The new class relationship prediction distillation module is also used to obtain the spectral angle cosine similarity between the instance feature and the feature prototype group, and obtain the allocation probability of the instance feature and the feature prototype based on the spectral angle cosine similarity;

[0023] The second classifier is used to predict the distribution probability output by the distillation module according to the new class relationship to obtain the predicted pseudo label of the new class;

[0024] The category relationship distribution distillation module is used to use the output of the first classifier to provide distribution-level distillation constraints for the output of the second classifier, so as to capture the category relationship between the known categories and new categories of the target objects in the spectral-spatial dimension, and realize the prediction of new categories of hyperspectral remote sensing images.

[0025] Furthermore, the first feature extractor has the same structure as the second feature extractor, both of which include a first two-dimensional convolutional layer, a plurality of feature network layers connected in sequence, an average pooling layer, a CBAM attention layer, and a fully connected layer;

[0026] The feature network layer includes a second two-dimensional convolutional layer, a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer, a second batch normalization layer and a second activation layer connected in sequence;

[0027] The first two-dimensional convolutional layer is used to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map;

[0028] The target features are the spectral features and metric space features of the target object;

[0029] The second two-dimensional convolutional layer is used to extract target features from the first feature map to obtain a second feature map;

[0030] The first batch normalization layer is used to perform a normalization operation on the target features in the second feature map to obtain a third feature map;

[0031] The first activation layer is used to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map;

[0032] The third two-dimensional convolutional layer is used to extract target features from the fourth feature map to obtain a fifth feature map;

[0033] The second batch normalization layer is used to perform a normalization operation on the target features in the fifth feature map to obtain a sixth feature map;

[0034] The second activation layer is used to perform nonlinear feature transformation on the fused feature map to obtain a seventh feature map; and the fused feature map is a feature map obtained by performing a pixel-by-pixel addition operation on the outputs of the second two-dimensional convolution layer and the second batch normalization layer;

[0035] The average pooling layer is used to perform a global dimensionality reduction operation on the seventh feature map to obtain an eighth feature map;

[0036] The CBAM attention layer is used to perform an enhanced feature expression operation on the eighth feature map to obtain a ninth feature map;

[0037] The fully connected layer is used to perform high-dimensional mapping of the ninth feature map to obtain the final feature extraction map.

[0038] Furthermore, the S3 specifically includes the following steps:

[0039] S31: using the supervised training sample set as input data of the supervised training network unit, using the new class discovery training sample set as input data of the new class discovery network unit, and using the target object category as output data of the new class discovery network model;

[0040] Performing model training on the new class discovery network model to obtain the trained new class discovery network model;

[0041] S32: Performing model verification on the trained new class discovery network model through a data verification set, that is, determining whether the output of the trained new class discovery network model converges;

[0042] If it is confirmed that the output of the trained new class discovery network model converges, then the trained new class discovery network model is used as the new class discovery network optimization model;

[0043] Otherwise, based on the back propagation method, the weight parameters of the trained new class discovery network model are adaptively adjusted, and S31 is repeatedly executed.

[0044] Furthermore, the loss function L for judging whether the output of the trained new class discovery network model converges in S32 is expressed as:

[0045] L=α*L CLS +β*L KD_distribution +γ*L KD_prototype

[0046]

[0047]

[0048]

[0049] Where: α represents the hyperparameter of the constrained cross entropy loss; β represents the hyperparameter of the auxiliary prediction distribution-level loss; γ represents the hyperparameter of the prototype group-level clustering loss; L S represents the constrained cross entropy loss function; L KD_distribution represents the knowledge distillation loss function used to assist in predicting the distribution loss of the prototype group; L KD_prototype represents the distillation loss function of the prototype group clustering loss; j represents the total number of generated prototype groups; G i Represents the prototype group distribution generated by the original data; G′ i represents the prototype group distribution generated by setting replica data; N Q It represents the total number of samples in the new class discovery training sample set; Q represents the number of categories; Represents the predicted distribution of supervised training network units for new class features; The predicted distribution of new class features by the new class discovery network unit; g(x it ) represents the prediction of the new class discovery network model for the known categories; y it Represents the true label of the known category; N represents the total number of samples in the supervised training sample set used to train the new class discovery network; S represents the number of sample categories in the supervised training sample set used to train the new class discovery network; i represents the sample or prototype or prototype group currently indexed in sequence.

[0050] Beneficial effect: The present invention provides a new category discovery method for hyperspectral remote sensing images based on dual relationship distillation constraints. In the new category discovery stage, the category relationship distribution distillation module is used to set distillation constraints using the relationship information between new categories and known categories to eliminate the recognition gap between the model for different categories, and provide effective prior knowledge for new hyperspectral unlabeled categories. Through the new category relationship prediction distillation module in the new category discovery network unit, the spectral angle cosine similarity between new category samples is gradually integrated, and the inter-class discriminability and intra-class closeness of the data are further improved from class prototype to class prototype group, which effectively improves the prediction of pseudo-labels for new categories while enhancing the new category label prediction ability under noise interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 It is a flow chart of the new class discovery method of hyperspectral remote sensing images based on dual relationship distillation constraints of the present invention;

[0053] Figure 2 Schematic diagram of the structure of a new class discovery network model for new class discovery of hyperspectral remote sensing images based on dual relationship distillation constraints in this embodiment;

[0054] Figure 3 It is a pseudo-color image of the test data set in this embodiment;

[0055] Figure 4 This is a graph showing the new classification results of the hyperspectral remote sensing image of the test data set in this embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] This embodiment provides a new class discovery method for hyperspectral remote sensing images based on dual relationship distillation constraints. Figure 1 As shown, the following steps are included:

[0058] S1: Acquire a hyperspectral remote sensing image dataset, wherein the hyperspectral remote sensing image dataset includes spectral characteristics and metric space characteristics of target objects and corresponding target object categories;

[0059] According to the hyperspectral remote sensing image data set, the data training set, the data verification set and the data test set are randomly divided and obtained; and the training data of the data training set is selected to obtain the supervised training sample set and the new class discovery training sample set;

[0060] Specifically, the method for selecting training data for the data training set is:

[0061] The data training set is divided into a supervised training sample set and a new class discovery training sample set according to a preset ratio; and the supervised training sample set is a set of hyperspectral remote sensing image data that concentrates the hyperspectral remote sensing image data and the target object category corresponding to the spectral characteristics and metric space characteristics of the target object as a set of hyperspectral remote sensing image data of known categories of the target object; the new class discovery training sample set is a set of hyperspectral remote sensing image data that concentrates the hyperspectral remote sensing image data and the target object category corresponding to the spectral / metric space characteristics of the target object as a set of hyperspectral remote sensing image data of new categories of the target object.

[0062] The hyperspectral image Salinas in this embodiment is captured by the AVIRIS sensor over the Salinas Valley in California, USA, covering 16 ground object categories, including 54129 pixels, composed of 512×217 pixels, and the number of bands is 224, as shown in Table 1:

[0063] Table 1. Number of samples of target object categories in the Salinas dataset:

[0064]

[0065] Among them, eight categories, Fallow, Fallow-smooth, Stubble, Celery, Lettuce-romaine-4wk, Lettuce-romaine-6wk, Lettuce-romaine-7wk, and Vinyard-untrained, were set as unlabeled categories to be discovered in the new category discovery phase, and the remaining eight categories were used as known categories to provide inter-category auxiliary information in the initial training phase;

[0066] S2: Construct a new class discovery network model for new class discovery of hyperspectral remote sensing images based on dual relationship distillation constraints, and the new class discovery network model includes a supervised training network unit for extracting spectral features and metric space features in a supervised training sample set, a new class discovery network unit for extracting spectral features and metric space features in a new class discovery training sample set, and a category relationship distribution distillation module for capturing the category relationship between known categories and new categories of target objects in the spectral space dimension according to the outputs of the supervised training network unit and the new class discovery network unit;

[0067] Specifically, Figure 2 As shown, the supervised training network unit includes a first feature extractor and a first classifier connected in sequence; the new class discovery network unit includes a second feature extractor, a new class relationship prediction distillation module and a second classifier connected in sequence; the first feature extractor and the second feature extractor are used to extract spectral features and metric space features from the supervised training sample set and the new class discovery training sample set, respectively, and the output features of the second feature extractor are instance features;

[0068] In a specific embodiment, the first feature extractor has the same structure as the second feature extractor, and both include a first two-dimensional convolutional layer (9×9×64), a plurality of feature network layers connected in sequence, an average pooling layer, a CBAM attention layer (1×1×256), and a fully connected layer;

[0069] The feature network layer includes a second two-dimensional convolutional layer (9×9×64), a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer (5×5×128), a second batch normalization layer and a second activation layer connected in sequence;

[0070] The first two-dimensional convolutional layer is used to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map; and the target features are the spectral features and metric space features of the target object;

[0071] The second two-dimensional convolution layer is used to extract target features from the first feature map to obtain a second feature map; the first batch normalization layer is used to normalize the target features in the second feature map to obtain a third feature map;

[0072] The first activation layer is used to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map; the third two-dimensional convolution layer is used to extract target features from the fourth feature map to obtain a fifth feature map; the second batch normalization layer is used to perform normalization operation on the target features in the fifth feature map to obtain a sixth feature map;

[0073] The second activation layer is used to perform nonlinear feature transformation on the fused feature map to obtain a seventh feature map; and the fused feature map is a feature map obtained by performing a pixel-by-pixel addition operation on the outputs of the second two-dimensional convolution layer and the second batch normalization layer;

[0074] The average pooling layer is used to perform a global dimensionality reduction operation on the seventh feature map to obtain an eighth feature map; the CBAM attention layer is used to perform an enhanced feature expression operation on the eighth feature map to obtain a ninth feature map;

[0075] The fully connected layer is used to perform high-dimensional mapping of the ninth feature map to obtain the final feature extraction map;

[0076] The first classifier is used to classify the target object category according to the output of the first feature extractor;

[0077] The new class relationship prediction distillation module is used to obtain a cluster heap about feature distribution according to the output of the second feature extractor based on the GMM model, and define the cluster heap as a feature prototype group;

[0078] The new class relationship prediction distillation module is also used to obtain the spectral angle cosine similarity between the instance feature and the feature prototype group, and obtain the allocation probability of the instance feature and the feature prototype based on the spectral angle cosine similarity;

[0079] Specifically, this embodiment adopts a class prototype to class prototype group level clustering mode to achieve clustering of unlabeled new categories and assign pseudo labels, and the prototype level clustering method enhances the robustness and correctness of clustering;

[0080] The calculation formula for obtaining the spectral angle cosine similarity between the instance feature and the feature prototype group is:

[0081]

[0082] In the formula: feature represents the instance feature extracted by the feature extractor; C i represents the i-th feature prototype group; SAM(feature,C i ) represents the spectral angle cosine similarity between the instance feature and the feature prototype group;

[0083] Based on SAM(feature,C i) value to obtain instance features feature and prototype C i The probability of allocation is

[0084]

[0085] The second classifier is used to predict the distribution probability output by the distillation module according to the new class relationship to obtain the predicted pseudo label of the new class;

[0086] The category relationship distribution distillation module is used to use the output of the first classifier to provide distribution-level distillation constraints for the output of the second classifier, so as to capture the category relationship between the known categories and new categories of the target objects in the spectral-spatial dimension, and realize the prediction of new categories of hyperspectral remote sensing images.

[0087] S3: According to the supervised training sample set and the new class discovery training sample set, the new class discovery network model is trained, and the trained new class discovery network model is verified through the data verification set to obtain the new class discovery network optimization model;

[0088] The specific steps include:

[0089] S31: using the supervised training sample set as input data of the supervised training network unit, using the new class discovery training sample set as input data of the new class discovery network unit, and using the target object category as output data of the new class discovery network model;

[0090] Performing model training on the new class discovery network model to obtain the trained new class discovery network model;

[0091] S32: Performing model verification on the trained new class discovery network model through a data verification set, that is, determining whether the output of the trained new class discovery network model converges;

[0092] Among them, the loss function L that determines whether the output of the trained new class discovery network model converges is expressed as

[0093] L=α*L' CLS +β*L KD_distribution +γ*L KD_prototype

[0094]

[0095]

[0096]

[0097] Where: α represents the hyperparameter of the constrained cross entropy loss; β represents the hyperparameter of the auxiliary prediction distribution level loss; γ represents the hyperparameter of the prototype group level clustering loss; L' CLSrepresents the constrained cross entropy loss function; L KD_distribution represents the knowledge distillation loss function used to assist in predicting the distribution loss of the prototype group. It is used in the new class discovery phase to use the model in the supervised training phase to impose distribution-level distillation constraints on unlabeled new categories, effectively capturing the class relationship information between new categories and known categories in the spectral space dimension; L KD_prototype represents the distillation loss function of the prototype group clustering loss; j represents the total number of generated prototype groups; G i Represents the prototype group distribution generated by the original data; G′ i represents the prototype group distribution generated by setting replica data; N Q It represents the total number of samples in the new class discovery training sample set; Q represents the number of categories; Represents the predicted distribution of supervised training network units for new class features; The new class discovery network unit predicts the distribution of new class features, and the predicted distribution refers to the probability of each data in each category obtained by letting the current new class discovery model predict the current data. The new class discovery network model is the final model of this embodiment, and the prediction distribution of the new class discovery model is obtained in order to obtain the KL divergence constraint of the prediction distribution of the supervised training model. This process forces the new class discovery model to have a stronger classification ability, and its expression is p I =softmax(M(x k ) / T), where M(x k ) indicates that the new category discovery network model extracts features from new categories; x k Indicates new type of data; T indicates temperature coefficient; g(x it ) represents the prediction of the new class discovery network model for the known categories; y it Represents the true label of the known category; N represents the total number of samples in the supervised training sample set used to train the new class discovery network; S represents the number of sample categories in the supervised training sample set used to train the new class discovery network; i represents the sample or prototype or prototype group currently indexed in sequence;

[0098] If it is confirmed that the output of the trained new class discovery network model converges, then the trained new class discovery network model is used as the new class discovery network optimization model;

[0099] Otherwise, based on the back propagation method, the weight parameters of the trained new class discovery network model are adaptively adjusted, and S31 is repeatedly executed.

[0100] S4: New class prediction of the hyperspectral remote sensing images in the data test set is achieved based on the new class discovery network optimization model.

[0101] This embodiment proposes a network model DRDC with dual relation distillation constraints to solve the problem of new class discovery in the hyperspectral field. The model mainly consists of two modules: the category relation distribution distillation module (CRDD) and the new class relation prediction distillation module (NCRPD); the CRDD module uses the relationship information between new categories and known categories to set distillation constraints in the new class discovery stage to eliminate the recognition gap between different categories of the model. It provides effective prior knowledge for new unlabeled categories of hyperspectral images while maintaining the memory of old knowledge, which is particularly important for the differences in spectral and spatial characteristics of different materials in hyperspectral images; the NCRPD module further improves the inter-class discrimination and intra-class compactness of data from class prototype to class prototype group by gradually integrating the SAM distance relationship between new class samples, realizing the pseudo-label prediction of new categories while designing prototype-level distillation to enhance the new class label prediction ability under noise interference.

[0102] In this embodiment, an experiment was conducted on the Salinas dataset using a new hyperspectral image class discovery method based on dual relationship distillation constraints of the present invention. The results of the experiment are shown in Table 2:

[0103] Table 2. Salinas classification accuracy (%)

[0104]

[0105]

[0106] OA (Overall Accuracy) represents the overall classification accuracy, AA (Average accuracy) represents the average classification accuracy, and Kappa represents the Kappa coefficient. The Kappa coefficient refers to a multivariate discrete method for evaluating the classification accuracy and error matrix of remote sensing images. It also considers various missed and misclassified pixels outside the diagonal line, and can punish the bias of the model, thereby more comprehensively evaluating the classification effect. Figure 3 to Figure 4 The figure shows the pseudo-color image and classification result image of the test dataset.

[0107] In order to more objectively evaluate the role of each step in the model of a new hyperspectral image class discovery method based on dual relationship distillation constraints in this embodiment, an existing ablation experiment is added for illustration. A single module or a combination of different modules is added to the ordinary prototype network to compare the experimental results. The specific experimental results are shown in Table 3:

[0108] Table 3 Classification accuracy of different modules (%)

[0109]

[0110] The following conclusions can be drawn from the above experiments:

[0111] (1) The experimental results in Table 2 show that the proposed method for new class discovery of hyperspectral images based on dual relationship distillation constraints has good results, proving that this method has shown superior performance in the new class discovery problem.

[0112] (2) The ablation experiment data in Table 3 show that the three modules, namely the attention module (CBAM), the category relation distribution distillation module (CRDD) and the new category relation prediction distillation module (NCRPD), have played a role in improving the performance to a certain extent and have a certain effect on the overall classification of the model.

[0113] (3) The ablation experiment data in Table 3 show that the combined effect of the attention module and the category relationship distribution distillation module is better than that of a single module, and is also better than the combination of any other two modules. This shows that the combination of our modules improves the performance of the model.

[0114] (4) The ablation experimental data in Table 3 show that the effects of different module combinations are different. The experimental results in the table show that there are complex interactions between the modules, and all three modules need to be integrated to give full play to their advantages and achieve significant performance improvements.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A new class discovery method for hyperspectral remote sensing images based on dual relation distillation constraints, characterized in that: The following steps are involved: S1: Acquire a hyperspectral remote sensing image dataset, wherein the hyperspectral remote sensing image dataset includes spectral characteristics and metric space characteristics of target objects and corresponding target object categories; Randomly divide and obtain a data training set, a data validation set, and a data test set according to the hyperspectral remote sensing image data set; And select training data from the data training set to obtain a supervised training sample set and a new class discovery training sample set; S2: Construct a new class discovery network model for new class discovery of hyperspectral remote sensing images based on dual relationship distillation constraints, and the new class discovery network model includes a supervised training network unit for extracting spectral features and metric space features in a supervised training sample set, a new class discovery network unit for extracting spectral features and metric space features in a new class discovery training sample set, and a category relationship distribution distillation module for capturing the category relationship between known categories and new categories of target objects in the spectral space dimension according to the outputs of the supervised training network unit and the new class discovery network unit; S3: According to the supervised training sample set and the new class discovery training sample set, the new class discovery network model is trained, and the trained new class discovery network model is verified through the data verification set to obtain the new class discovery network optimization model; S4: New class prediction of the hyperspectral remote sensing images in the data test set is achieved based on the new class discovery network optimization model.

2. The method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints according to claim 1, characterized in that: The method for selecting training data for the data training set described in S1 is: Divide the data training set into a supervised training sample set and a new class discovery training sample set according to a preset ratio; The supervised training sample set is a set of hyperspectral remote sensing image data that contains the spectral features and metric spatial features of the target objects, and the target object categories corresponding to the target objects as known categories of the target objects; The new class discovery training sample set is a set of hyperspectral remote sensing image data that contains target object categories corresponding to the spectral / metric spatial characteristics of the target objects as new categories of the target objects.

3. The method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints according to claim 2, characterized in that: The supervised training network unit in S2 includes a first feature extractor and a first classifier connected in sequence; The new class discovery network unit includes a second feature extractor, a new class relationship prediction distillation module and a second classifier connected in sequence; The first feature extractor and the second feature extractor are used to extract spectral features and metric space features from the supervised training sample set and the new class discovery training sample set, respectively; At the same time, the output feature of the second feature extractor is the instance feature; The first classifier is used to classify the target object category according to the output of the first feature extractor; The new class relationship prediction distillation module is used to obtain a cluster heap about feature distribution according to the output of the second feature extractor based on the GMM model, and define the cluster heap as a feature prototype group; The new class relationship prediction distillation module is also used to obtain the spectral angle cosine similarity between the instance feature and the feature prototype group, and obtain the allocation probability of the instance feature and the feature prototype based on the spectral angle cosine similarity; The second classifier is used to predict the distribution probability output by the distillation module according to the new class relationship to obtain the predicted pseudo label of the new class; The category relationship distribution distillation module is used to use the output of the first classifier to provide distribution-level distillation constraints for the output of the second classifier, so as to capture the category relationship between the known categories and new categories of the target objects in the spectral-spatial dimension, and realize the prediction of new categories of hyperspectral remote sensing images.

4. The method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints according to claim 3, characterized in that: The first feature extractor has the same structure as the second feature extractor, and both include a first two-dimensional convolutional layer, a plurality of feature network layers connected in sequence, an average pooling layer, a CBAM attention layer, and a fully connected layer; The feature network layer includes a second two-dimensional convolutional layer, a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer, a second batch normalization layer and a second activation layer connected in sequence; The first two-dimensional convolutional layer is used to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map; The target features are the spectral features and metric space features of the target object; The second two-dimensional convolutional layer is used to extract target features from the first feature map to obtain a second feature map; The first batch normalization layer is used to perform a normalization operation on the target features in the second feature map to obtain a third feature map; The first activation layer is used to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map; The third two-dimensional convolutional layer is used to extract target features from the fourth feature map to obtain a fifth feature map; The second batch normalization layer is used to perform a normalization operation on the target features in the fifth feature map to obtain a sixth feature map; The second activation layer is used to perform nonlinear feature transformation on the fused feature map to obtain a seventh feature map; and the fused feature map is a feature map obtained by performing a pixel-by-pixel addition operation on the outputs of the second two-dimensional convolution layer and the second batch normalization layer; The average pooling layer is used to perform a global dimensionality reduction operation on the seventh feature map to obtain an eighth feature map; The CBAM attention layer is used to perform an enhanced feature expression operation on the eighth feature map to obtain a ninth feature map; The fully connected layer is used to perform high-dimensional mapping of the ninth feature map to obtain the final feature extraction map.

5. The method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints according to claim 4, characterized in that: The S3 specifically includes the following steps: S31: using the supervised training sample set as input data of the supervised training network unit, using the new class discovery training sample set as input data of the new class discovery network unit, and using the target object category as output data of the new class discovery network model; Performing model training on the new class discovery network model to obtain the trained new class discovery network model; S32: Performing model verification on the trained new class discovery network model through a data verification set, that is, determining whether the output of the trained new class discovery network model converges; If it is confirmed that the output of the trained new class discovery network model converges, then the trained new class discovery network model is used as the new class discovery network optimization model; Otherwise, based on the back propagation method, the weight parameters of the trained new class discovery network model are adaptively adjusted, and S31 is repeatedly executed.

6. The method for discovering new classes of hyperspectral remote sensing images based on dual relationship distillation constraints according to claim 5, characterized in that: The loss function L used in S32 to determine whether the output of the trained new class discovery network model converges is expressed as: L=α*L' CLS +β*L KD_distribution +γ*L KD_prototype Where: α represents the hyperparameter of the constrained cross entropy loss; β represents the hyperparameter of the auxiliary prediction distribution level loss; γ represents the hyperparameter of the prototype group level clustering loss; L' CLS represents the constrained cross entropy loss function; L KD_distribution represents the knowledge distillation loss function used to assist in predicting the distribution loss of the prototype group; L KD_prototype represents the distillation loss function of the prototype group clustering loss; j represents the total number of generated prototype groups; G i Represents the prototype group distribution generated by the original data; G′ i represents the prototype group distribution generated by setting replica data; N Q It represents the total number of samples in the new class discovery training sample set; Q represents the number of categories; Represents the predicted distribution of supervised training network units for new class features; The predicted distribution of new class features by the new class discovery network unit; g(x it ) represents the prediction of the new class discovery network model for the known categories; y it Represents the true label of the known category; N represents the total number of samples in the supervised training sample set used to train the new class discovery network; S represents the number of sample categories in the supervised training sample set used to train the new class discovery network; i represents the sample or prototype or prototype group currently indexed in sequence.

Images