Hyperspectral remote sensing image novelty discovery method based on dual relationship distillation constraint

Abstract

The application discloses a hyperspectral remote sensing image new class discovery method based on double relationship distillation constraint, including obtaining a hyperspectral remote sensing image dataset, randomly dividing the hyperspectral remote sensing image dataset and obtaining a data training set, a data verification set and a data test set; and performing training data selection on the data training set to obtain a supervised training sample set and a new class discovery training sample set; constructing a new class discovery network model for the hyperspectral remote sensing image new class discovery based on double relationship distillation constraint, performing model training on the new class discovery network model according to the partial supervised training sample set and the new class discovery training sample set, and performing model verification on the trained new class discovery network model through the data verification set to obtain a new class discovery network optimization model; and realizing new class prediction of the hyperspectral remote sensing image in the data test set according to the new class discovery network optimization model. The application solves the limitation of a traditional classification mode which gradually reveals limitations due to the difficulty in adapting to the sudden emergence of new classes because of its dependence on sufficient labeled data. That is, the existing classification technology mostly depends on limited labeled samples, resulting in insufficient new class recognition ability and poor adaptability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral image classification, and particularly relates to a hyperspectral remote sensing image new class discovery method based on double relationship distillation constraint. BACKGROUND

[0002] In the face of growing hyperspectral remote sensing big data and changing environment, the traditional classification mode gradually reveals limitations because it is difficult to adapt to the sudden emergence of new classes due to its dependence on sufficient labeled data. That is, existing classification techniques mostly rely on limited labeled samples, resulting in insufficient new class recognition ability and poor adaptability. Therefore, it is particularly important to explore methods for new class discovery in the field of hyperspectral. SUMMARY

[0003] The present application provides a hyperspectral remote sensing image new class discovery method based on double relationship distillation constraint to overcome the above technical problems.

[0004] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0005] A hyperspectral remote sensing image new class discovery method based on double relationship distillation constraint, comprising the following steps:

[0006] S1: obtaining a hyperspectral remote sensing image dataset, wherein the hyperspectral remote sensing image dataset comprises spectral features and metric space features of target ground objects and corresponding target ground object classes;

[0007] According to the hyperspectral remote sensing image dataset, a data training set, a data validation set and a data test set are randomly divided and obtained;

[0008] The data training set is subjected to training data selection to obtain a supervised training sample set and a new class discovery training sample set;

[0009] S2: constructing a new class discovery network model for hyperspectral remote sensing image new class discovery based on double relationship distillation constraint, wherein the new class discovery network model comprises a supervised training network unit for extracting spectral features and metric space features in the supervised training sample set, a new class discovery network unit for extracting spectral features and metric space features in the new class discovery training sample set, and a class relationship distribution distillation module for capturing class relationship between known classes and new classes of target ground objects in spectral space dimensions according to outputs of the supervised training network unit and the new class discovery network unit;

[0010] S3: model training of the new class discovery network model according to the supervised training sample set and the new class discovery training sample set, and model validation of the trained new class discovery network model through the data validation set to obtain a new class discovery network optimization model;

[0011] S4: Based on the new class discovery network optimization model, new class prediction is achieved for hyperspectral remote sensing images in the data test set.

[0012] Furthermore, the method for selecting training data for the training set described in S1 is as follows:

[0013] The training data set is divided into a supervised training sample set and a new class discovery training sample set according to a preset ratio;

[0014] Furthermore, the supervised training sample set is a collection of hyperspectral remote sensing image data containing the spectral features and metric spatial features of the target land cover in the hyperspectral remote sensing image dataset, which is a collection of hyperspectral remote sensing image data with known target land cover categories.

[0015] The training sample set for new class discovery is a collection of hyperspectral remote sensing image data that takes the target land cover category corresponding to the spectral / metric spatial characteristics of the target land cover from the hyperspectral remote sensing image dataset as the new category of the target land cover.

[0016] Furthermore, the supervised training network unit described in S2 includes a first feature extractor and a first classifier connected in sequence;

[0017] The novel class discovery network unit includes a second feature extractor, a novel class relationship prediction distillation module, and a second classifier connected in sequence.

[0018] The first feature extractor and the second feature extractor are used to extract spectral features and metric space features from the supervised training sample set and the new class discovery training sample set, respectively.

[0019] Meanwhile, the output features of the second feature extractor are the instance features;

[0020] The first classifier is used to classify the target land cover category based on the output of the first feature extractor;

[0021] The novel class relationship prediction distillation module is used to obtain clusters about feature distribution based on the output of the second feature extractor according to the GMM model, and to define the clusters as feature prototype groups;

[0022] The novel relation prediction distillation module is also used to obtain the spectral angle cosine similarity between instance features and feature prototype groups, and to obtain the allocation probability of instance features and feature prototypes based on the spectral angle cosine similarity.

[0023] The second classifier is used to predict the allocation probability output by the distillation module based on the new class relationship, and to obtain the predicted pseudo-label of the new class.

[0024] The category relationship distribution distillation module is used to utilize the output of the first classifier to provide a distribution level distillation constraint for the output of the second classifier, so as to capture the category relationship between the known categories and the new categories of the target ground object in the spectral space dimension, and realize the prediction of the new category of the hyperspectral remote sensing image.

[0025] Further, the first feature extractor and the second feature extractor are structurally identical, and each includes a first two-dimensional convolutional layer, a plurality of feature network layers connected in sequence, an average pooling layer, a CBAM attention layer, and a full connection layer.

[0026] The feature network layer includes a second two-dimensional convolutional layer, a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer, a second batch normalization layer, and a second activation layer connected in sequence.

[0027] The first two-dimensional convolutional layer is used to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map.

[0028] The target features are the spectral features and the metric space features of the target ground object.

[0029] The second two-dimensional convolutional layer is used to extract target features from the first feature map to obtain a second feature map.

[0030] The first batch normalization layer is used to normalize the target features in the second feature map to obtain a third feature map.

[0031] The first activation layer is used to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map.

[0032] The third two-dimensional convolutional layer is used to extract target features from the fourth feature map to obtain a fifth feature map.

[0033] The second batch normalization layer is used to normalize the target features in the fifth feature map to obtain a sixth feature map.

[0034] The second activation layer is used to perform nonlinear feature transformation on the fusion feature map to obtain a seventh feature map, and the fusion feature map is a feature map obtained by performing pixel-by-pixel addition operation on the outputs of the second two-dimensional convolutional layer and the second batch normalization layer.

[0035] The average pooling layer is used to perform global dimension reduction on the seventh feature map to obtain an eighth feature map.

[0036] The CBAM attention layer is used to perform enhanced feature expression operation on the eighth feature map to obtain a ninth feature map.

[0037] The full connection layer is used to perform high-dimensional mapping of the features of the ninth feature map to obtain a final feature extraction map.

[0038] Further, the S3 specifically comprises the following steps:

[0039] S31: taking the supervised training sample set as the input data of the supervised training network unit, and taking the new class discovery training sample set as the input data of the new class discovery network unit, and taking the target ground object class as the output data of the new class discovery network model;

[0040] Model training is performed on the new class discovery network model to obtain a trained new class discovery network model;

[0041] S32: model verification is performed on the trained new class discovery network model through the data verification set, i.e., whether the output of the trained new class discovery network model converges is determined;

[0042] If it is confirmed that the output of the trained new class discovery network model converges, the trained new class discovery network model is taken as the new class discovery network optimization model at this time;

[0043] Otherwise, the weight parameters of the trained new class discovery network model are adaptively adjusted based on the back propagation method, and S31 is repeatedly executed.

[0044] Further, the loss function L for determining whether the output of the trained new class discovery network model converges in the S32 has an expression as follows:

[0045] L = a * L CLS + β * L KD_distribution + γ * L KD_prototype

[0046]

[0047]

[0048]

[0049] In the formula, a represents a hyperparameter of the constraint cross-entropy loss; β represents a hyperparameter of the auxiliary prediction distribution level loss; γ represents a hyperparameter of the prototype group level clustering loss; L S represents the constraint cross-entropy loss function; L KD_distribution represents the knowledge distillation loss function for the auxiliary prediction prototype group distribution loss; L KD_prototype represents the distillation loss function of the prototype group clustering loss; j represents the total number of generated prototype groups; G i represents the prototype group distribution generated by the original data; G′ i represents the prototype group distribution generated by the set copy data; N Q represents the total number of samples in the new class discovery training sample set; Q represents the number of categories. represents the prediction distribution of the new class feature by the supervised training network unit; represents the prediction distribution of the new class feature by the new class discovery network unit; it represents the prediction of the known class by the new class discovery network model; it represents the real label of the known class; N represents the total number of samples used to train the new class discovery network in the supervised training sample set; S represents the number of sample classes used to train the new class discovery network in the supervised training sample set; i represents the sample or prototype or prototype group currently indexed in sequence.

[0050] Beneficial effects: The application provides a hyperspectral remote sensing image new class discovery method based on double relationship distillation constraints, through a class relationship distribution distillation module in the new class discovery stage, relationship information between new classes and known classes is used to set distillation constraints, so as to eliminate the recognition gap of different classes by the model, and provide effective prior knowledge for the hyperspectral unlabeled new class; through a new class relationship prediction distillation module in the new class discovery network unit, the spectral angle cosine similarity between new class samples is gradually integrated, the data inter-class discriminability and intra-class tightness are further improved from the class prototype to the class prototype group, and the pseudo-label prediction of the new class is effectively improved, and the new class label prediction ability under noise interference is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0052] Figure 1 The flowchart of the hyperspectral remote sensing image new class discovery method based on double relationship distillation constraints of the application;

[0053] Figure 2 The structural diagram of the new class discovery network model of the hyperspectral remote sensing image new class discovery based on double relationship distillation constraints in the embodiment;

[0054] Figure 3 The pseudo-color map of the test data set in the embodiment;

[0055] Figure 4 The hyperspectral remote sensing image new class classification result map of the test data set in the embodiment. DETAILED DESCRIPTION

[0056] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0057] The embodiment provides a hyperspectral remote sensing image new class discovery method based on double relationship distillation constraints, as shown in the formula: Figure 1

[0058] S1: Obtain a hyperspectral remote sensing image dataset, and the hyperspectral remote sensing image dataset comprises spectral features and metric space features of target ground objects and corresponding target ground object categories;

[0059] According to the hyperspectral remote sensing image dataset, a data training set, a data verification set and a data test set are randomly divided and obtained; and training data selection is performed on the data training set to obtain a supervised training sample set and a new class discovery training sample set;

[0060] Specifically, the method for performing training data selection on the data training set is that,

[0061] The data training set is divided into the supervised training sample set and the new class discovery training sample set according to a preset proportion; the supervised training sample set is a set of hyperspectral remote sensing image data in which the target ground object categories corresponding to the spectral features and the metric space features of the target ground objects in the hyperspectral remote sensing image dataset are taken as known categories of the target ground objects; and the new class discovery training sample set is a set of hyperspectral remote sensing image data in which the target ground object categories corresponding to the spectral features and the metric space features of the target ground objects in the hyperspectral remote sensing image dataset are taken as new categories of the target ground objects.

[0062] The hyperspectral image Salinas in the embodiment is captured by an AVIRIS sensor above the Salinas Valley in California, USA, covers 16 ground object categories, includes 54129 pixels, is composed of 512*217 pixels, and has 224 bands, as shown in Table 1.

[0063] Table 1. Sample number of target ground object categories in the Salinas dataset

[0064]

[0065] ​Among them, the eight categories Fallow, Fallow-smooth, Stubble, Celery, Lettuce-romaine-4wk, Lettuce-romaine-6wk, Lettuce-romaine-7wk, and Vinyard-untrained were set as unlabeled categories and discovered in the new class discovery phase, while the remaining eight categories were used as known categories to provide inter-class auxiliary information in the initial training phase;

[0066] S2: Construct a new class discovery network model for new class discovery in hyperspectral remote sensing images based on dual relation distillation constraints. The new class discovery network model includes a supervised training network unit for extracting spectral features and metric space features from the supervised training sample set, a new class discovery network unit for extracting spectral features and metric space features from the new class discovery training sample set, and a class relation distribution distillation module for capturing the class relationship between known and new classes of target land cover in the spectral space dimension based on the outputs of the supervised training network unit and the new class discovery network unit.

[0067] Specifically, such as Figure 2 As shown, the supervised training network unit includes a first feature extractor and a first classifier connected in sequence; the new class discovery network unit includes a second feature extractor, a new class relationship prediction distillation module, and a second classifier connected in sequence; the first feature extractor and the second feature extractor are used to extract spectral features and metric space features from the supervised training sample set and the new class discovery training sample set, respectively, and the output feature of the second feature extractor is the instance feature;

[0068] In a specific embodiment, the first feature extractor and the second feature extractor have the same structure, both including a first two-dimensional convolutional layer (9×9×64), several feature network layers connected in sequence, an average pooling layer, a CBAM attention layer (1×1×256), and a fully connected layer.

[0069] The feature network layer includes a second two-dimensional convolutional layer (9×9×64), a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer (5×5×128), a second batch normalization layer, and a second activation layer connected in sequence.

[0070] The first two-dimensional convolutional layer is used to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map; and the target features are the spectral features and metric space features of the target land cover.

[0071] The second two-dimensional convolutional layer is used to extract target features from the first feature map to obtain the second feature map; the first batch normalization layer is used to normalize the target features in the second feature map to obtain the third feature map.

[0072] The first activation layer is configured to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map; the third two-dimensional convolution layer is configured to perform target feature extraction on the fourth feature map to obtain a fifth feature map; and the second batch normalization layer is configured to perform normalization operation on the target feature in the fifth feature map to obtain a sixth feature map.

[0073] The second activation layer is configured to perform nonlinear feature transformation on the fusion feature map to obtain a seventh feature map; and the fusion feature map is a feature map obtained by performing pixel-by-pixel addition operation on the output of the second two-dimensional convolution layer and the second batch normalization layer.

[0074] The average pooling layer is configured to perform global dimension reduction operation on the seventh feature map to obtain an eighth feature map; and the CBAM attention layer is configured to perform enhanced feature expression operation on the eighth feature map to obtain a ninth feature map.

[0075] The full connection layer is configured to perform high-dimensional mapping of features on the ninth feature map to obtain a final feature extraction map.

[0076] The first classifier is configured to perform classification of target ground object categories according to the output of the first feature extractor.

[0077] The new class relationship prediction distillation module is configured to obtain a clustering heap about feature distribution based on the GMM model according to the output of the second feature extractor, and define the clustering heap as a feature prototype group.

[0078] The new class relationship prediction distillation module is further configured to obtain a spectral angle cosine similarity between an instance feature and the feature prototype group, and obtain an assignment probability of the instance feature and the feature prototype based on the spectral angle cosine similarity.

[0079] Specifically, the embodiment adopts a class prototype to class prototype group level clustering mode to realize clustering and pseudo-labeling of unlabeled new classes, and the prototype level clustering mode enhances the robustness and correctness of clustering.

[0080] The calculation formula of the spectral angle cosine similarity between the instance feature and the feature prototype group is

[0081]

[0082] In the formula, feature represents an instance feature extracted by a feature extractor from a sample; C i represents an i-th feature prototype group; SAM(feature, C i ) represents a spectral angle cosine similarity between an instance feature and a feature prototype group.

[0083] Based on SAM(feature, C i) value acquisition instance feature and prototype C i The distribution probability of the prototype C

[0084]

[0085] The second classifier is used to predict the distribution probability output by the distillation module according to the new class relationship, and obtain the predicted pseudo label of the new class;

[0086] The class relationship distribution distillation module is used to utilize the output of the first classifier to provide distribution level distillation constraint for the output of the second classifier, so as to capture the class relationship between the known classes and the new classes of the target ground objects in the spectral space dimension, and realize the prediction of the new class of the hyperspectral remote sensing image.

[0087] S3: Model training is performed on the new class discovery network model according to the supervised training sample set and the new class discovery training sample set, and model verification is performed on the trained new class discovery network model through the data verification set, so as to obtain a new class discovery network optimization model;

[0088] Specifically, the following steps are included:

[0089] S31: The supervised training sample set is used as the input data of the supervised training network unit, the new class discovery training sample set is used as the input data of the new class discovery network unit, and the target ground object class is used as the output data of the new class discovery network model;

[0090] The model training is performed on the new class discovery network model, and a trained new class discovery network model is obtained;

[0091] S32: The model verification is performed on the trained new class discovery network model through the data verification set, that is, whether the output of the trained new class discovery network model converges is determined;

[0092] Wherein, the loss function L for determining whether the output of the trained new class discovery network model converges is expressed as

[0093] L=α*L' CLS +β*L KD_distribution +γ*L KD_prototype

[0094]

[0095]

[0096]

[0097] In the formula, α represents a hyperparameter of constraint cross-entropy loss; β represents a hyperparameter of auxiliary prediction distribution level loss; γ represents a hyperparameter of prototype group level clustering loss; L' CLSrepresents a constrained cross-entropy loss function; L KD_distribution represents a knowledge distillation loss function for assisting in predicting the prototype group distribution loss, which is used to impose distribution-level distillation constraints on the unlabeled new classes by the model of the supervised training phase in the new class discovery phase, effectively capturing the class relationship information of the new classes and known classes in the spectral dimension; L KD_prototype represents a distillation loss function of the prototype group clustering loss; j represents the total number of generated prototype groups; G i represents the prototype group distribution generated by the original data; G' i represents the prototype group distribution generated by the set of duplicate data; N Q represents the total number of samples in the new class discovery training sample set; Q represents the number of classes; represents the prediction distribution of the new class feature by the supervised training network unit; represents the prediction distribution of the new class feature by the new class discovery network unit, and the prediction distribution refers to the probability of each data under each class predicted by the current new class discovery model for the current data. The new class discovery network model is the final model of the embodiment, and the prediction distribution of the new class discovery model is obtained to constrain the prediction distribution of the supervised training model. This process forces the classification ability of the new class discovery model to be stronger, and its expression is p I = softmax(M(x k ) / T), where M(x k ) represents feature extraction of the new class by the new class discovery network model; x k represents new class data; T represents a temperature coefficient; g(x it ) represents the prediction of the known class by the new class discovery network model; y it represents the true label of the known class; N represents the total number of samples in the supervised training sample set used to train the new class discovery network; S represents the number of sample classes in the supervised training sample set used to train the new class discovery network; i represents the sample or prototype or prototype group indexed in order;

[0098] If it is confirmed that the output of the trained new class discovery network model converges, the trained new class discovery network model at this time is used as a new class discovery network optimization model;

[0099] Otherwise, based on the back propagation method, the weight parameters of the trained new class discovery network model are adaptively adjusted, and S31 is repeatedly executed.

[0100] S4: Implement new class prediction for hyperspectral remote sensing images in the data test set according to the new class discovery network optimization model.

[0101] The embodiment proposes a network model DRDC of double relationship distillation constraint to solve the new class discovery problem in the field of hyperspectral. The model mainly consists of two modules of class relationship distribution distillation module (CRDD) and new class relationship prediction distillation module (NCRPD). The CRDD module sets the distillation constraint by using the relationship information between the new class and the known class in the new class discovery stage, so as to eliminate the identification difference of the model to different classes. It provides effective prior knowledge for the unlabeled new class of hyperspectral, and at the same time, it retains the memory of old knowledge, which is particularly important for the difference of different materials in spectral space in hyperspectral image. The NCRPD module further improves the inter-class discriminability and intra-class compactness of data from class prototype to class prototype group by gradually integrating the SAM distance relationship between new class samples, realizes the pseudo-label prediction of new class, and designs the prototype-level distillation to enhance the new class label prediction ability under noise interference.

[0102] In the embodiment, for the Salinas data set, a new class discovery method of hyperspectral image based on double relationship distillation constraint is used for experiment, and the experimental results are shown in Table 2:

[0103] Table 2. Classification accuracy of Salinas (%)

[0104]

[0105]

[0106] Among them, OA (Overall Accuracy) represents the overall classification accuracy, AA (Average accuracy) represents the average classification accuracy, and Kappa represents the Kappa coefficient. The Kappa coefficient is a multivariate discrete method for evaluating the classification accuracy and error matrix of remote sensing image, which considers various missed and wrong pixels except the diagonal line, can punish the bias of the model, and can more comprehensively evaluate the effect of classification. As Figures 3-4 The pseudo-color map and the classification result map of the test data set are shown as shown in the following table.

[0107] In order to more objectively evaluate the role of each step in the model of the new class discovery method of hyperspectral image based on double relationship distillation constraint in the embodiment, the existing ablation experiment is added to illustrate. On the basis of the ordinary prototype network, a single module or a combination of different modules is added to compare the experimental results, and the specific experimental results are shown in Table 3:

[0108] Table 3. Classification accuracy of different modules (%)

[0109]

[0110] Through the above experiment, the following conclusions can be obtained:

[0111] (1) Through the experimental results in Table 2, it is shown that the proposed new class discovery method for hyperspectral images based on double relationship distillation constraint has good effect, which proves that the method has superior performance in new class discovery problem.

[0112] (2) The ablation experiment data in Table 3 show that the attention module (CBAM), the class relationship distribution distillation module (CRDD) and the new class relationship prediction distillation module (NCRPD) all play a role in improving the performance to a certain extent, and have a certain effect on the overall classification of the model.

[0113] (3) The ablation experiment data in Table 3 show that the effect of the combination of the attention module and the class relationship distribution distillation module is better than that of the single module, and is also better than that of any two module combinations, which shows that the combination of our modules improves the performance of the model.

[0114] (4) The ablation experiment data in Table 3 show that the effects of different module combinations are different, and the experimental results in the table show that there is a complex interaction relationship between the modules, and all three modules need to be integrated to play their advantages and obtain significant performance improvement.

[0115] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for discovering new classes of hyperspectral remote sensing images based on double relationship distillation constraints, characterized in that, The method comprises the following steps: S1: obtaining a hyperspectral remote sensing image dataset, wherein the hyperspectral remote sensing image dataset comprises spectral features and metric space features of a target ground object and a corresponding target ground object category; According to the hyperspectral remote sensing image dataset, a data training set, a data verification set and a data test set are randomly divided and obtained; And the training data selection is performed on the data training set to obtain a supervised training sample set and a new category discovery training sample set; S2: constructing a new category discovery network model for new category discovery of the hyperspectral remote sensing image based on a double relationship distillation constraint, wherein the new category discovery network model comprises a supervised training network unit for extracting spectral features and metric space features in the supervised training sample set, a new category discovery network unit for extracting spectral features and metric space features in the new category discovery training sample set, and a category relationship distribution distillation module for capturing a category relationship between known categories and new categories of the target ground object in a spectral space dimension according to outputs of the supervised training network unit and the new category discovery network unit; The supervised training network unit comprises a first feature extractor and a first classifier connected in sequence; The new category discovery network unit comprises a second feature extractor, a new category relationship prediction distillation module and a second classifier connected in sequence; The first feature extractor and the second feature extractor are respectively used for extracting spectral features and metric space features in the supervised training sample set and the new category discovery training sample set; Meanwhile, the output feature of the second feature extractor is an instance feature; The first classifier is used for classifying the target ground object category according to the output of the first feature extractor; The new category relationship prediction distillation module is used for obtaining a clustering heap about feature distribution based on the GMM model according to the output of the second feature extractor, and defining the clustering heap as a feature prototype group; The new category relationship prediction distillation module is also used for obtaining a spectral angle cosine similarity between the instance feature and the feature prototype group, and obtaining an assignment probability of the instance feature and the feature prototype based on the spectral angle cosine similarity; The second classifier is used for obtaining a predicted pseudo label of the new category according to the assignment probability output by the new category relationship prediction distillation module; The category relationship distribution distillation module is used for providing a distribution level distillation constraint for the output of the second classifier by using the output of the first classifier, so as to capture the category relationship between the known categories and the new categories of the target ground object in the spectral space dimension, and realize the prediction of the new category of the hyperspectral remote sensing image; S3: model training is performed on the new category discovery network model according to the supervised training sample set and the new category discovery training sample set, and model verification is performed on the trained new category discovery network model through the data verification set, so as to obtain a new category discovery network optimization model; S4: the new category discovery network optimization model is used to realize the new category prediction of the hyperspectral remote sensing image in the data test set. 2.The hyperspectral remote sensing image new class discovery method based on dual relationship distillation constraint of claim 1, wherein, The method for performing training data selection on the data training set in S1 is as follows: The data training set is divided into the supervised training sample set and the new category discovery training sample set according to a preset ratio; The supervised training sample set is a set of hyperspectral remote sensing image data whose target feature corresponding to the target object is the target object category in the hyperspectral remote sensing image data, and the target object category is the known category of the hyperspectral remote sensing image data; The new class discovery training sample set is a set of hyperspectral remote sensing image data whose target feature corresponding to the target object is the target object category in the hyperspectral remote sensing image data, and the target object category is the new category of the hyperspectral remote sensing image data.

3. The hyperspectral remote sensing image new class discovery method based on dual relationship distillation constraint according to claim 2, characterized in that, The first feature extractor and the second feature extractor are structurally identical, and each includes a first two-dimensional convolutional layer, a plurality of feature network layers connected in sequence, an average pooling layer, a CBAM attention layer, and a full connection layer; The feature network layer includes a second two-dimensional convolutional layer, a first batch normalization layer, a first activation layer, a third two-dimensional convolutional layer, a second batch normalization layer, and a second activation layer connected in sequence; The first two-dimensional convolutional layer is configured to extract target features from the input hyperspectral remote sensing image data to obtain a first feature map; The target feature is the spectral feature and the metric space feature of the target object; The second two-dimensional convolutional layer is configured to extract target features from the first feature map to obtain a second feature map; The first batch normalization layer is configured to normalize the target features in the second feature map to obtain a third feature map; The first activation layer is configured to perform nonlinear feature transformation on the third feature map to obtain a fourth feature map; The third two-dimensional convolutional layer is configured to extract target features from the fourth feature map to obtain a fifth feature map; The second batch normalization layer is configured to normalize the target features in the fifth feature map to obtain a sixth feature map; The second activation layer is configured to perform nonlinear feature transformation on the fusion feature map to obtain a seventh feature map; and the fusion feature map is a feature map obtained by performing pixel-by-pixel addition operation on the output of the second two-dimensional convolutional layer and the second batch normalization layer; The average pooling layer is configured to perform global dimension reduction on the seventh feature map to obtain an eighth feature map; The CBAM attention layer is configured to perform enhanced feature expression operation on the eighth feature map to obtain a ninth feature map; The full connection layer is configured to perform high-dimensional mapping of the ninth feature map to obtain a final feature extraction map.

4. The hyperspectral remote sensing image new class discovery method based on dual relationship distillation constraint of claim 3, characterized in that, The S3 specifically includes the following steps: S31: taking the supervised training sample set as the input data of the supervised training network unit, taking the new class discovery training sample set as the input data of the new class discovery network unit, and taking the target object category as the output data of the new class discovery network model; S32: performing model verification on the trained new class discovery network model through the data verification set, i.e., judging whether the output of the trained new class discovery network model converges or not; If it is confirmed that the output of the trained new class discovery network model converges, the trained new class discovery network model is taken as the new class discovery network optimization model at this time; Otherwise, based on the back propagation method, the weight parameters of the trained new class discovery network model are adaptively adjusted, and S31 is repeatedly executed. ​ 5. The hyperspectral remote sensing image new class discovery method based on dual relationship distillation constraint according to claim 4, characterized in that, The loss function used in the S32 to determine whether the output of the trained new class discovery network model converges whose expression is In the formula: represents a hyperparameter of the constrained cross-entropy loss; represents a hyperparameter of the auxiliary prediction distribution level loss; represents a hyperparameter of the prototype group level clustering loss; represents a constrained cross-entropy loss function; represents a knowledge distillation loss function for the auxiliary prediction prototype group distribution loss; represents a distillation loss function of the prototype group clustering loss; represents the total number of generated prototype groups; represents the prototype group distribution generated by the original data; represents the prototype group distribution generated by the set copy data; represents the total number of samples in the new class discovery training sample set; represents the number of categories; represents the prediction distribution of the new class feature by the supervised training network unit; represents the prediction distribution of the new class feature by the new class discovery network unit; represents the prediction of the known categories by the new class discovery network model; represents the real label of the known categories; N denotes the total number of samples in the supervised training sample set that are used to train the new class discovery network; S denotes the number of sample classes in the supervised training sample set that are used to train the new class discovery network; i denotes the current sample or prototype or prototype group that is indexed in order.

Images