Class-incremental learning method and system based on mixed sample selection and multi-granularity distillation

Abstract

The application relates to the technical field of artificial intelligence, and discloses a class incremental learning method and system based on mixed sample selection and multi-granularity distillation. The method comprises the following steps: acquiring a hyperspectral remote sensing image and preprocessing; constructing a teacher model and a student model, and completing initial task training; after completing each incremental task, mixed sample selection is performed through a two-stage strategy, a compact quota is adaptively calculated for each class, compact and diverse samples are screened, role marks are assigned, and the samples are stored in a buffer area; in the training of a new task, constraints are applied to an output layer, an intermediate layer and a feature space through logical value relationship alignment, intermediate layer pooling distillation and role distinguishing prototype distillation respectively; a classification loss and three distillation losses are weighted and summed to update the student model, a student model that has completed the training of the current incremental task is obtained, and a ground object classification result is output. Through sample screening and multi-level distillation, the application effectively alleviates the catastrophic forgetting in hyperspectral incremental learning.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and specifically relates to a class incremental learning method and system based on mixed sample selection and multi-granularity distillation. Background Technology

[0002] Incremental hyperspectral image classification is of great significance for supporting applications such as dynamic updating of land cover types, continuous monitoring of land use, and detection of disasters and changes. By enabling the classification model to continuously absorb newly emerging land cover categories without relying on retraining of all historical data, the system can dynamically expand the land cover category library with low storage and computational costs.

[0003] Incremental class learning is a key technology supporting the above capabilities. Its task is to enable a deep model consisting of a feature extraction network and a classification head to learn several batches of new classes sequentially according to a task sequence, and to maintain a high discriminative ability for all previously learned classes after training for each task. Traditional incremental class learning methods are mainly based on a framework that combines sample replay and knowledge distillation. That is, under a limited storage budget, old class samples are retained, and the old model frozen before training is used as a teacher to impose distillation constraints on the current model.

[0004] However, traditional methods face many challenges in incremental classification scenarios for hyperspectral images: hyperspectral data has a high-dimensional joint spectral-space distribution, and existing sampling strategies are unable to balance representativeness and diversity, resulting in insufficient representativeness of the sample buffer; existing distillation methods only apply constraints at a single representation level, making it difficult to fully protect multi-granularity knowledge such as logistic relationship structure, intermediate representation structure, and feature prototype structure; existing feature-level distillation methods do not distinguish between buffer samples, causing the category prototype estimation to be contaminated by boundary samples, which limits further improvement in recognition accuracy.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address or at least alleviate one or more of the above problems, a class incremental learning method and system based on hybrid sample selection and multi-granularity distillation is provided, which can significantly improve the old class preservation ability and overall recognition accuracy of hyperspectral image class incremental classification.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a class incremental learning method based on mixed sample selection and multi-granularity distillation is provided, comprising the following steps: Acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; After each incremental task training is completed, a two-stage selection strategy is used to select mixed samples. The compact quota for each category is adaptively calculated. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples, and they are stored in the sample buffer. During the training process of the new task, knowledge distillation is performed. Logical value relationship alignment constraints are adopted in the output layer to require the student model to maintain the same relative relationship between samples as the teacher model. At the same time, intermediate layer output is constrained by intermediate layer pooling distillation. The frozen intermediate layer representation of the teacher model is used as the anchoring target to constrain the intermediate layer representation of the student model to align with the teacher model. In the feature space, role-distinguishing prototype distillation is performed. Based on the determined role labels of compact and diverse samples, the feature prototypes of each category are calculated using compact samples as anchor targets, and the category prototypes generated by the student model are constrained to align with the anchor targets. The classification loss is weighted and summed with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively to construct the total loss function to update the student model parameters until convergence, thus obtaining the student model that has completed the training of the current incremental task. This student model is then used as the initial model for the next incremental task. The student model is then used to classify hyperspectral remote sensing images and output the land cover classification results.

[0008] Furthermore, a two-stage selection strategy is employed for mixed sample selection, adaptively calculating the compact quota for each category, including: The joint feature-spectral distance metric is defined by weighted summation of the cosine distance between the depth features of the samples and the spectral angle of the original spectral vector. Calculate and normalize the comprehensive intra-class dispersion score of each category in the deep feature space and the original spectral space; determine the compact quota of each category based on the normalized dispersion score. The higher the normalized dispersion score, the smaller the compact quota of the category, and the lower the normalized dispersion score, the larger the compact quota of the category; wherein, the compact quota of each category is between a preset upper bound and a lower bound.

[0009] Furthermore, the first stage involves screening for compact samples, and the second stage involves selecting diverse samples, including: The joint feature-spectral distance metric is defined by weighted summation of the cosine distance between the depth features of the samples and the spectral angle of the original spectral vector. The first stage of compact sample selection: For each category, using the category prototype as the anchor point, an iterative group selection strategy is adopted to successively select samples from the training sample pool that minimize the Euclidean distance between the mean feature of the selected samples and the category prototype, until the compact quota of each category is reached. The selected samples are then marked as compact samples. The second stage is the selection of diverse samples: from the residual pool after removing compact samples from each category, the compactness objective is to minimize the average distance from the selected subset to the category prototype, and the diversity objective is to maximize the average joint distance between samples within the selected subset. The remaining storage slots are selected and marked as diverse samples.

[0010] Furthermore, different role labels are assigned to the selected compact and diverse samples, and these labels are stored in the sample buffer, including: The role flag of the compact sample selected in the first stage is set to true, and the role flag of the diverse sample selected in the second stage is set to false. The role flag, along with the sample data and category label, is stored in a buffer to determine whether the sample participates in the category prototype calculation in the subsequent prototype distillation.

[0011] Furthermore, a logical value alignment constraint is adopted at the output layer, requiring the student model to maintain the same relative relationships between samples as the teacher model, including: Obtain the output matrix of the samples belonging to the old category in the current training batch on the Logit output layer of the student model and teacher model; For each old category, calculate the pairwise difference matrix between the logistic values ​​of each sample in the student model and the teacher model in that category; The element-wise subtraction of the pairwise difference matrices of the student model and the teacher model in the same category is used as the square of the Frobenius norm, which is then used as the relation alignment loss for that category. Summing and averaging the relation alignment losses for all old categories yields the logical relation alignment loss. The logistic value alignment loss constrains the logistic value layer output of the student model through gradient backpropagation, aligning the relative response relationship between samples in the old category of the student model with that of the teacher model, so as to maintain the relative structural stability of the discrimination boundary between the old categories.

[0012] Furthermore, by constraining the intermediate layer output through intermediate layer pooling distillation, the frozen intermediate layer representation of the teacher model is used as an anchoring target to constrain the intermediate layer representation of the student model to align with the teacher model, including: The teacher model and the student model use the same network architecture, both consisting of multiple stacked Transformer modules; Obtain the feature maps output by each Transformer module of the student model; The feature maps output by each Transformer module are sequentially normalized and spatial dimension aggregated and compressed to extract compact description vectors for each Transformer module. Obtain the compact description vectors of the student model and the frozen teacher model on each Transformer module, respectively, and calculate the mean square error between them as the distillation loss of each Transformer module. The distillation loss of all Transformer modules is weighted and summed to obtain the intermediate layer pooling distillation loss, which constrains the output of each Transformer module of the student model to align with the corresponding frozen output of the teacher model.

[0013] Furthermore, role-distinguishing prototype distillation is performed in the feature space. Based on the determined role labels of compact and diverse samples, the feature prototypes of each category are calculated using compact samples as anchor targets. The category prototypes generated by the student model are constrained to align with the anchor targets, including: Deep features of compact samples with true role markers in the buffer are extracted using the frozen teacher model. The feature vectors are summed by category and then normalized using the L2 norm to obtain the normalized prototypes for each category. In the current batch, extract the normalized features of compact samples that belong to the old category and whose role flag is true under the current student model, as well as the normalized features of the sample under the frozen teacher model. Calculate the cosine similarity between student model features and corresponding class prototypes, and the cosine similarity between teacher model features and corresponding class prototypes, respectively. The square of the difference between the two is used as the prototype distillation loss for the sample. The prototype distillation loss for role differentiation is obtained by summing and averaging the prototype distillation loss of all compact samples that belong to the old category and whose role label is true. The role-distinguishing prototype distillation loss constraint ensures that the feature orientation of the student model on compact samples remains consistent with that of the teacher model, in order to maintain the stability of the prototype center of the old category.

[0014] Furthermore, the total loss function is expressed as:

[0015] in, For cross-entropy loss, For logical value alignment loss, This represents the loss from intermediate layer pooling distillation. To differentiate the prototype distillation loss for the character, , , These are the weighting coefficients.

[0016] To achieve the above objectives, according to a second aspect of the present invention, a class incremental learning system based on mixed sample selection and multi-granularity distillation is provided, comprising: The acquisition and construction module is used to acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; The mixed sample selection and adaptive quota allocation module is used to select mixed samples through a two-stage selection strategy after each incremental task training is completed. It adaptively calculates the compact quota for each category. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples and stored in the sample buffer. The knowledge distillation module from the relation layer to the intermediate layer is used to perform knowledge distillation during the training process of new tasks. In the output layer, a logical value relation alignment constraint is adopted, requiring the student model to maintain the same relative relationship between samples as the teacher model. At the same time, the intermediate layer output is constrained by the intermediate layer pooling distillation, and the frozen intermediate layer representation of the teacher model is used as the anchor target to constrain the intermediate layer representation of the student model to align with the teacher model. The role differentiation prototype distillation module is used to perform role differentiation prototype distillation in the feature space. Based on the determined role labels of compact and diverse samples, the module calculates feature prototypes of each category using compact samples as anchor targets, and constrains the category prototypes generated by the student model to align with the anchor targets. The distillation optimization and output module is used to weight and sum the classification loss with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively, to construct the total loss function to update the student model parameters until convergence, obtain the student model that has completed the training of the current incremental task, and use it as the initial model for the next incremental task. The student model is then used to classify hyperspectral remote sensing images and output the land cover classification results.

[0017] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: This invention significantly improves the old class retention capability and overall recognition accuracy of incremental classification of hyperspectral images through innovative technologies such as hybrid sample selection and adaptive quota allocation, multi-granularity knowledge distillation, and role-distinguishing prototype distillation. The organic combination of these technologies not only alleviates the catastrophic forgetting problem under the high-dimensional spectral-space distribution of hyperspectral data, but also provides an efficient solution for continuous learning under limited storage and computing resources, offering reliable technical support for the continuous intelligent upgrading of hyperspectral remote sensing ground cover classification systems.

[0018] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0020] In the attached diagram: Figure 1 This is a flowchart illustrating the incremental learning method based on mixed sample selection and multi-granularity distillation in this specific embodiment. Figure 2 This is a schematic diagram of data processing for the incremental learning method based on mixed sample selection and multi-granularity distillation in this specific embodiment. Figure 3 This is a schematic diagram of the incremental learning system based on mixed sample selection and multi-granularity distillation in this specific embodiment. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0022] In the following description, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] Example 1: Please see Figure 1 and Figure 2 This invention provides a class incremental learning method based on mixed sample selection and multi-granularity distillation, comprising the following steps: Acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; After each incremental task training is completed, a two-stage selection strategy is used to select mixed samples. The compact quota for each category is adaptively calculated. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples, and they are stored in the sample buffer. During the training process of the new task, knowledge distillation is performed. Logical value relationship alignment constraints are adopted in the output layer to require the student model to maintain the same relative relationship between samples as the teacher model. At the same time, intermediate layer output is constrained by intermediate layer pooling distillation. The frozen intermediate layer representation of the teacher model is used as the anchoring target to constrain the intermediate layer representation of the student model to align with the teacher model. In the feature space, role-distinguishing prototype distillation is performed. Based on the determined role labels of compact and diverse samples, the feature prototypes of each category are calculated using compact samples as anchor targets, and the category prototypes generated by the student model are constrained to align with the anchor targets. The classification loss is weighted and summed with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively to construct the total loss function to update the student model parameters until convergence, thus obtaining the student model that has completed the training of the current incremental task, and serving as the initial model for the next incremental task.

[0025] In some embodiments, a two-stage selection strategy is used to select mixed samples, adaptively calculating a compact quota for each category, including: Calculate and normalize the comprehensive intra-class dispersion score of each category in the deep feature space and the original spectral space; determine the compact quota of each category based on the normalized dispersion score. The higher the normalized dispersion score, the smaller the compact quota of the category, and the lower the normalized dispersion score, the larger the compact quota of the category; wherein, the compact quota of each category is between a preset upper bound and a lower bound.

[0026] 1. Mixed sample selection and adaptive quota allocation: After training the model for each incremental task, a limited number of representative samples for each category must be selected from the training samples involved in that task and stored in a sample buffer for replay training and distillation constraints in subsequent incremental tasks. Traditional sample selection methods, such as random sampling or simple population selection, struggle to simultaneously consider both the representativeness of the sample set to the category center and the coverage of category distribution diversity. To address this, this invention designs a hybrid sample selection and adaptive quota allocation module, employing a two-stage selection strategy to solve the aforementioned problems. The first stage of this strategy performs compact sample screening based on the feature space, ensuring that the mean of the selected samples quickly approximates the desired category prototype. The second stage uses global optimization among the remaining samples to simultaneously consider representativeness and diversity, covering the intra-class distribution span. Both stages of sample evaluation require unified quantitative indicators as the screening basis. Therefore, this invention first defines the following joint feature-spectral distance metric as a unified similarity evaluation benchmark throughout the two-stage selection process.

[0027] 1.1 Joint Feature-Spectral Distance Metric: To enable the sample selection process to consider both spatial and spectral information, this invention defines a unified joint distance metric. Let the sample... of L2 The normalized eigenvector is The spectral vector obtained by averaging and normalizing the pixel values ​​of each band in its spatial neighborhood is: s i Then the sample With sample joint distance between D ij Defined as: ; in, cos(·) Represents cosine similarity. arccos(·) This represents the spectral angle obtained by the inverse cosine function. ∈[0,1] is a weighting coefficient used to balance the contributions of the deep feature space and the original spectral space to the distance metric. This joint distance takes into account both the semantic discriminative information reflected by the deep features and the original physical properties preserved by the spectral vector, making the subsequently selected samples representative in both the feature space and the spectral space.

[0028] 1.2 Adaptive Compact Quota Calculation: Different land cover categories exhibit significant differences in dispersion within the combined spectral and spatial dimensions. For instance, vegetation categories show greater spectral variability while artificial land cover categories exhibit stronger spectral consistency. Therefore, this invention adaptively calculates a compact quota for each category. .

[0029] First, calculate the category. c Dispersion score : ; in, For category c The average spectral vector, Indicates category c The average score is calculated from all samples in the training pool. This score combines the intra-class discrepancies from both the deep feature space and the original spectral space. The larger the value, the more dispersed the samples of that category are in the spectral-space joint space.

[0030] To eliminate the impact of differences in units and value ranges between different categories, the dispersion scores of all categories are... Normalize: ; in, For all training samples greater than N The set of categories, For the categories in this set j The dispersion score. After normalization ∈[0,1], the larger the value, the higher the relative dispersion of the category.

[0031] Subsequently, the compact quota for each category is calculated based on the normalized dispersion score: ; in, and These are the preset upper and lower bounds for the compact quota, with values ​​of [value]. = N This means that compact samples can occupy up to all available storage slots. Take 0 or a smaller positive integer; in this embodiment, take 1 to ensure that each category has at least one clean prototype for prototype distillation. ∈[0,1] is the adaptive intensity coefficient, which controls the degree of influence of dispersion on quota adjustment. In this embodiment, it is taken as... =0.8, retaining a certain adjustment range but avoiding extreme allocation; round (·) This is the floor function.

[0032] This adaptive quota calculation method ensures that categories with higher dispersion receive smaller compact quotas and correspondingly larger diverse quotas, thus reserving more quotas to cover the boundaries and variation regions of that category. Conversely, categories with lower dispersion receive larger compact quotas to more accurately approximate their category centers. Through this adaptive allocation, the sample selection strategy of this invention can flexibly adjust the allocation of storage resources according to the inherent distribution characteristics of each category, thereby achieving a more reasonable sample configuration within a limited buffer budget.

[0033] In some embodiments, the first stage involves screening compact samples, and the second stage involves selecting diverse samples, including: The first stage of compact sample selection: For each category, using the category prototype as the anchor point, an iterative group herding strategy is adopted to successively select samples from the training sample pool that minimize the Euclidean distance between the mean feature of the selected samples and the category prototype, until the compact quota of each category is reached. The selected samples are then marked as compact samples. The second stage is the selection of diverse samples: from the residual pool after removing compact samples from each category, the compactness objective is to minimize the average distance from the selected subset to the category prototype, and the diversity objective is to maximize the average joint distance between samples within the selected subset. The remaining storage slots are selected and marked as diverse samples.

[0034] 1.3 Compact Sample Selection (Phase 1) In incremental learning frameworks, the number of storage slots for each class in the sample buffer is typically limited by the storage budget, denoted as . Based on this, a compact sample quota is further set for each category. This is used to control the number of compact samples in this category, with the remainder... The remaining slots will be used for subsequent selection of diverse samples.

[0035] The first phase involves selecting compact samples for each category. c Let its training sample pool be The category prototype is This is the mean vector of L2-normalized deep features for all training samples of this category. The first stage employs an iterative population herding strategy from... Selected A compact sample set. Let the selected sample set be... , No. Step from the remaining candidate pool Select a sample This ensures that after adding the sample to the selected set, all The feature mean and category prototype of each selected sample The Euclidean distance is the smallest, that is: ; in, Indicates that they have been selected The samples in For those already selected The feature vector of the sample in the sample. min Indicates all candidates Choose the one that minimizes the distance; the result is... Repeat the above process. Step by step, we obtain These samples are distributed closely around the class center in the feature space, and can accurately approximate the class prototype with a smaller number of samples. They are labeled as "compact" samples.

[0036] 1.4 Diverse Sample Selection and Global Optimization (Second Stage) Select After [number] compact samples, the remaining storage slots for this category in the buffer are [number]. In the second stage, after removing compact samples from the training sample pool for that category, diverse samples are selected from the remaining pool. The residual pool is defined as follows: ; in, For residual error, This is the set of all selected compact samples.

[0037] For residual pool any candidate subset Its compactness The metric is defined as the feature of all samples within a subset. To category prototype Mean cosine distance: ; in, Representing a subset The number of samples in the sample, Representing a subset The Middle i one sample L2 Normalized deep feature vectors, This represents the cosine similarity between the feature and the class prototype. The smaller the value, the closer the samples in the subset are to the category center.

[0038] This subset diversity The metric is defined as the average of the pairwise joint distances between all samples within a subset: ; in, The joint feature-spectral distance defined in Section 1.1, The larger the value, the more dispersed the sample distribution within the subset is, and the wider the coverage of the distribution range.

[0039] The goal of the second phase is to extract from the residual pool Select a subset This achieves an optimal balance between compactness and diversity. This optimal subset is obtained by minimizing the following objective function: ; in, This indicates that the candidate subset must be selected from the residual pool. This indicates that the size of the subset must be equal to the remaining storage space. ∈[0,1] is a tradeoff coefficient used to adjust the relative importance of compactness and diversity in the objective function.

[0040] This optimization problem is solved using a greedy search strategy. Starting from an empty set, at each step, a sample that maximizes the decrease in the current objective function value is selected from the remaining samples in the residual pool and added to the subset. This operation is repeated until the subset is full. One sample.

[0041] Although diverse samples may not strictly revolve around the class center in the feature space, they can cover a wider range of spectral-space variation patterns in the class distribution, providing the model with richer information on the old class distribution during subsequent replay training. These samples are labeled as "diverse" samples.

[0042] In some embodiments, different role labels are assigned to selected compact samples and diverse samples, and these are stored in a sample buffer, including: Set the role flag of the compact sample selected in the first stage to true, and set the role flag of the diverse sample selected in the second stage to false; store the role flag, sample data and category label together in the buffer, which will be used to determine whether the sample participates in the category prototype calculation in the subsequent prototype distillation.

[0043] 1.5 Character ID Storage: After completing the selection, a total of [number] items were selected for each category. N Each sample is stored in a buffer. The samples selected in the first stage... Each compact sample is assigned a boolean role flag, with a value of True; the second phase of selection Each diverse sample is assigned a role label with a value of False. This role label, along with the sample data and category labels, is stored in a buffer. M In subsequent training, it is determined whether the sample should participate in the prototype calculation.

[0044] In some embodiments, a logical value alignment constraint is employed at the output layer, requiring the student model to maintain the same relative sample relationships as the teacher model, including: Obtain the output matrix of the old category samples in the current training batch on the logical value output layer of the student model and the teacher model; For each old category, calculate the pairwise difference matrix between the logistic values ​​of each sample in the student model and the teacher model in that category; The element-wise subtraction of the pairwise difference matrices of the student model and the teacher model in the same category is used as the square of the Frobenius norm, which is then used as the relation alignment loss for that category. Summing and averaging the relation alignment losses for all old categories yields the logical relation alignment loss. Logistic value alignment loss aligns the student model's logistic value layer output with gradient backpropagation, thereby aligning the student model's relative response relationships between samples in the old categories with those of the teacher model, thus maintaining the relative structural stability of the discrimination boundaries between the old categories.

[0045] 2. Knowledge distillation from the relational layer to the intermediate layer: The hybrid sample selection strategy selects compact and diverse samples for each category and stores them in a buffer for later replay. However, relying solely on sample replay is insufficient to prevent the forgetting of old knowledge. When the model learns a new category, its parameter updates inevitably disturb the learned old category discrimination boundaries and feature representations. To address this, this invention introduces a knowledge distillation mechanism during the training process of the new task. The old model, frozen after the previous training stage, is used as a teacher to transfer the structured knowledge of the old categories to the new model currently being trained.

[0046] The knowledge for classifying hyperspectral images exists in different forms at different representation levels. At the logistic layer, the knowledge is manifested as the relative relationships between the output values ​​of each class, which encodes the semantic similarity structure between classes. At the intermediate Transformer block layer, the knowledge is manifested as the structured distribution pattern of spectral-space joint features. Therefore, this invention designs a relational layer-intermediate layer knowledge distillation, applying distillation constraints simultaneously at both levels.

[0047] 2.1 Logical value alignment loss Traditional logit distillation typically uses KL divergence to directly match the output probability distributions of teachers and students in each category. This "point-by-point matching" approach focuses on the absolute output value of a single sample in each category, ignoring the relative relationships between the outputs of different samples within the same batch. In hyperspectral classification, spectrally similar land cover categories are easily confused. If only absolute output values ​​are matched, the model may disrupt this inter-class relationship structure when learning new tasks, leading to distortion of the discrimination boundaries between similar categories.

[0048] To this end, the present invention uses Logit Relational Alignment (LRA), which does not require the student model to have completely consistent output values ​​with the teacher model in every class, but rather requires the student model to maintain the same relative relationship between samples as the teacher model.

[0049] Let the number of samples belonging to the old category in the current training batch be . The total number of old categories is Let the logit output matrix of the student model on these old categories be denoted as . ∈ The corresponding matrix for the teacher model is ∈ For the first c Old categories, ∈ Indicates all in the batch The logistic value vector of each sample in that category. To measure the logistic values ​​of these samples in the [missing context] category... c The pairwise response differences across classes are calculated, and the pairwise difference matrix of the vector is plotted. The pairwise difference matrices calculated by the student and teacher are then subtracted element-wise, and the square of the Frobenius norm is taken as the relation alignment loss for that class. The summation and averaging over all old classes yields the LRA loss. ; During training, the loss constrains the logistic layer output of the student model through gradient backpropagation, making its relative response relationship between samples in the old category as close as possible to that of the teacher model, thereby protecting the discriminative structure between the old categories from being covered by subsequent tasks.

[0050] In some embodiments, by constraining the intermediate layer output through intermediate layer pooling distillation, the frozen intermediate layer representation of the teacher model is used as an anchoring target to constrain the intermediate layer representation of the student model to align with the teacher model, including: The teacher model and the student model use the same network architecture, both consisting of multiple stacked Transformer modules; Obtain the feature maps output by each Transformer module of the student model; The feature maps output by each Transformer module are sequentially normalized and spatial dimension aggregated and compressed to extract compact description vectors for each Transformer module. Obtain the compact description vectors of the student model and the frozen teacher model on each Transformer module, respectively, and calculate the mean square error between them as the distillation loss of each Transformer module. The distillation loss of all Transformer modules is weighted and summed to obtain the intermediate layer pooling distillation loss, which constrains the output of each Transformer module of the student model to align with the corresponding frozen output of the teacher model.

[0051] 2.2 Intermediate layer pooling distillation loss While LRA preserves the semantic relationship structure between categories at the output layer, the discriminative information for hyperspectral image classification is not only present in the final logistic output. The intermediate modules of the Transformer encoder also encode important spectral-space joint feature patterns. These intermediate representations contain multi-level information from local spectral features to global spatial context, forming a crucial foundation for the final classification decision. If constraints are only imposed on the logistic layer, the representation structure of the intermediate layers may drift during incremental learning, thereby weakening the distillation effect of subsequent logistic layers.

[0052] To address this, this invention proposes Intermediate Pooled Distillation (IPD), which applies distillation constraints to the outputs of each intermediate module of the Transformer encoder. Let the Transformer encoder have a total of... L The first module. For the first... l Each module outputs: ∈ ,in For batch size, The number of spatial patches, The feature dimension for each patch. To reduce computational overhead and extract a compact representation, for The following processes are performed sequentially: First, layer normalization is performed to unify the dimensions; then, spatial average pooling is performed to... Each patch is averaged along its spatial dimension and aggregated into a single feature vector; finally, it is flattened, and the pooled result is... Flattening the vector into a one-dimensional vector yields the compact descriptor of the module. ∈ The descriptors for teachers and students are denoted as follows: and The former, as a fixed objective, does not participate in gradient updates. The IPD loss is the weighted sum of the mean squared errors between the teacher and student description vectors across all modules: ; in, For each module, a fixed weight is assigned to satisfy... =1, used to adjust the relative importance of different levels of representation in distillation. By minimizing this loss, when the current model updates its parameters, the outputs of its intermediate modules are constrained to the neighborhood of the old model's output, thus protecting the multi-level spectral-space feature structure from being covered.

[0053] LRA and IPD together constitute the knowledge distillation module from the relation layer to the intermediate layer. LRA protects the inter-category relations from the output layer, while IPD protects the spectral-space feature structure from the intermediate layer. The two protect old knowledge at different granularities, enabling the model to retain the learned knowledge more comprehensively when learning new categories.

[0054] In some embodiments, role-distinguishing prototype distillation is performed in the feature space. Based on the determined role labels of compact and diverse samples, feature prototypes of each category are calculated using compact samples as anchor targets, constraining the category prototypes generated by the student model to align with the anchor targets, including: Deep features of compact samples with true role markers in the buffer are extracted using the frozen teacher model. The feature vectors are summed by category and then normalized using the L2 norm to obtain the normalized prototypes for each category. In the current batch, extract the normalized features of compact samples that belong to the old category and whose role flag is true under the current student model, as well as the normalized features of the sample under the frozen teacher model. Calculate the cosine similarity between student model features and corresponding class prototypes, and the cosine similarity between teacher model features and corresponding class prototypes, respectively. The square of the difference between the two is used as the prototype distillation loss for the sample. The prototype distillation loss for role differentiation is obtained by summing and averaging the prototype distillation loss of all compact samples that belong to the old category and whose role label is true. The role-distinguishing prototype distillation loss constraint ensures that the feature orientation of the student model on compact samples remains consistent with that of the teacher model, in order to maintain the stability of the prototype center of the old class.

[0055] 3. Role Differentiation Prototype Distillation Module: Knowledge distillation from the relational layer to the intermediate layer imposes distillation constraints on the logical value layer and the intermediate layer, but neither of these layers directly involves the class prototype structure in the feature space. In incremental learning, the feature prototype of each old class is a key anchor point for maintaining the class discrimination boundary. If the prototype drifts, the classification decision boundary will shift accordingly, causing old class samples to be misclassified as new classes or neighboring old classes.

[0056] Existing prototype distillation methods typically utilize all samples stored in a buffer to calculate class prototypes. However, in the sample selection strategy of this invention, the buffer simultaneously contains two types of samples with different functions: compact samples selected in Section 1.3 are located in the class center region and serve as stable prototype estimation anchors; while diverse samples selected in Section 1.4 are distributed across class boundaries and variation regions, aiming to expand the replay coverage. If both types of samples are mixed for prototype calculation, boundary samples will "skew" the prototype estimation, causing the calculated prototype to deviate from the true class center and reducing the effectiveness of distillation. To address this, this invention proposes Role-Differentiated Prototype Distillation (RDPD), which ensures that the calculation of class prototypes is based solely on compact samples, i.e., samples with the role flag set to True, while diverse samples do not participate in prototype calculation. This avoids imposing excessive constraints on diverse samples distributed in the boundary region, thus limiting the model's plasticity.

[0057] 3.1 Category Prototype Calculation After training for each incremental task, the frozen old model is used. Only compact samples with true role flags are selected from the buffer to calculate the L2 normalized prototypes for each category: ; in, For category c The set of all compact samples For the old model to the samples The extracted deep feature vectors are summed in the numerator over the feature vectors of all compact samples, and the denominator is normalized using the L2 norm of the sum to ensure that the prototype vectors are unit vectors, resulting in a set of normalized prototypes. , representing the center position of the category.

[0058] 3.2 Prototype Distillation Losses When training on a new task, the current model's feature extractor The data will be updated with gradients to adapt to the new class, which may cause a shift in the response of features from the old class in the prototype direction. To prevent this shift, RDPD imposes explicit constraints on compact samples during training. For compact samples in the current batch that belong to the old class and have a role flag of True... Extract the L2-normalized feature vectors of the current model and the old model respectively. and Calculate the relationship between these two features and the category prototype. The cosine similarity is calculated, and the square of the difference between the old and new models on this similarity is used as the distillation loss: ; in, This is a compact set of samples in the current batch that belong to the old category and have the role flag set to True. This represents the cosine similarity operation.

[0059] This loss requires that the features of the same compact sample on both the old and new models maintain directional consistency relative to the same class prototype. By minimizing this loss, when the current model updates its parameters, the features of its compact samples are constrained to the neighborhood of the old model's feature orientation, thus protecting the stability of the old class's central structure. This role-discriminating design ensures that the samples used for prototype distillation are reliable "clean" anchors, avoiding contamination of prototype estimation by boundary samples while preserving the contribution of diverse samples to distribution coverage during replay training.

[0060] 4. Overall Loss Function The deep model architecture used in this invention is the Lightweight Self-Gaussian Attention Visual Transformer (LSGA-ViT), which consists of a feature extractor. and classification head It consists of two parts. Each incremental task... t During training, the model's total loss function consists of four terms: ; Among them, the first item The first term is cross-entropy loss, applied to new class training samples for the current task and old class samples replayed from the buffer. This loss ensures that the model can correctly distinguish all currently seen classes. The latter three terms are the three core distillation losses of this invention. Protect the relationship structure between categories at the logical value level. Protect the spectral-empty feature structure at the intermediate module level. Protect the stability of category prototypes at the feature level. Weighting coefficients. , , The relative intensity of these three distillation losses was controlled separately.

[0061] Example 2: Step 1: Data Acquisition and Preprocessing 1.1 Data Acquisition: The purpose of data acquisition is to obtain hyperspectral remote sensing images covering different land cover categories, providing a data foundation for subsequent incremental class learning. This embodiment uses four publicly available hyperspectral remote sensing benchmark datasets as examples, but new hyperspectral remote sensing data can also be acquired in practical applications.

[0062] The Pavia University dataset was acquired by the ROSIS-3 sensor. The imaging area is the University of Pavia, Italy. The image size is 610×340 pixels, the spatial resolution is 1.3 meters / pixel, the original spectral bands are 115, and 103 bands are retained after noise removal. The wavelength range is 0.43-0.86μm, and it includes 9 land cover categories: asphalt, grass, gravel, trees, metal sheets, bare soil, asphalt roofs, bricks, and shadows.

[0063] Indian Pines dataset: acquired by AVIRIS sensors, the imaging area is farmland in Indiana, USA, the image size is 145×145 pixels, the spatial resolution is 20 meters / pixel, the original spectral bands are 220, after removing water absorption bands, 200 are retained, covering the wavelength range of 0.4-2.5μm, including 16 land cover categories, covering crops (corn, soybean, wheat, etc.) and vegetation types, the category distribution is highly unbalanced.

[0064] Salinas dataset: acquired by AVIRIS sensors, imaging area is Salinas Valley, California, USA, image size 512×217 pixels, spatial resolution 3.7 m / pixel, 224 original spectral bands, after removing water absorption bands, 204 are retained, including 16 agricultural categories (broccoli, celery, grapes, etc.).

[0065] The Houston 2013 dataset was acquired by the ITRES CASI-1500 sensor. The imaging area is the University of Houston campus and surrounding area. The image size is 349×1905 pixels, the spatial resolution is 2.5 meters / pixel, and the original spectral bands are 144, covering the wavelength range of 0.38-1.05μm. It includes 15 urban land cover categories: grassland, trees, soil, water bodies, residential areas, commercial areas, roads, highways, railways, parking lots, tennis courts, running tracks, etc.

[0066] 1.2 Data Preprocessing: The purpose of data preprocessing is to convert the raw hyperspectral images into a standard format suitable for model input, remove noise and redundant information, and improve training efficiency.

[0067] Principal Component Analysis Dimensionality Reduction: To reduce computational complexity and remove redundant information between adjacent bands, principal component analysis (PCA) is performed on the original hyperspectral image for dimensionality reduction. The entire hyperspectral image is expanded into a two-dimensional matrix (number of pixels × number of bands) pixel by pixel. The covariance matrix is ​​calculated and eigenvalue decomposition is performed. The top 36 principal components with a cumulative contribution rate of over 99% are selected, compressing the spectral dimension of each pixel from the original number of bands to 36 dimensions. Each pixel after dimensionality reduction is represented as a 36-dimensional feature vector.

[0068] Spatial Neighborhood Extraction: Spatial context information of hyperspectral images is crucial for classification; therefore, a fixed-size spatial neighborhood window is extracted centered on each labeled pixel. A 7×7 pixel spatial window is used to construct a 36×7×7 three-dimensional tensor as the model input for this sample, where 36 represents the spectral dimension and 7×7 represents the spatial dimension. For edge pixels in the image, a reflection-filling strategy is used to complete the neighborhood range.

[0069] Normalization: All extracted samples are normalized by minimum-maximum value for each band, mapping the pixel values ​​of each band to the [0,1] interval, thus eliminating the impact of dimensional differences between different bands on model training. The normalization formula is: ; in, This represents the normalized result of the pixel value vector for the b-th band. This is the vector of all pixel values ​​in the b-th band. and These are the minimum and maximum values ​​for that band, respectively.

[0070] Incremental Task Construction: After dividing each dataset into training and testing sets, the four datasets are arranged sequentially into four incremental learning tasks. Task 1 is the University of Pavia dataset (9 classes), Task 2 is the Indian Pine Forest dataset (16 classes), Task 3 is the Salinas dataset (16 classes), and Task 4 is the Houston 2013 dataset (15 classes), for a total of 56 classes. Global class labels are uniformly assigned according to task order as 0-8 (Task 1), 9-24 (Task 2), 25-40 (Task 3), and 41-55 (Task 4), with no overlap between tasks. The new class set included in Task 1 is denoted as . , No. n The set of new categories contained in each task is denoted as . During training for each task, the current training data includes samples of the new category and selected samples of the old category; after training for each task, the model needs to classify and identify all seen categories, and no task identifiers are available during inference.

[0071] Step 2: Teacher and Student Model Construction: The purpose of this model construction is to establish a unified model architecture that plays the roles of teacher and student at different stages. This invention employs a Light Self-Gaussian-Attention Vision Transformer (LSGA-ViT) as the unified backbone network. The model consists of two stacked Transformer modules, each containing a multi-head self-attention layer, a feedforward network, layer normalization, and residual connections. The self-attention mechanism uses a self-Gaussian attention approximation to reduce computational complexity.

[0072] All model parameters were initialized using Xavier uniform initialization, and the classifier head weights were initialized using a normal distribution. Model training used the Adam optimizer with an initial learning rate of 5 × 10⁻⁶. -4 The training time is reduced by 0.9 every 50 rounds. Each task is trained for 150 rounds with a batch size of 128.

[0073] In incremental learning, after each task is completed, the parameters of the current model are completely copied and frozen, serving as the teacher model for the next task. The student model is the new model currently being trained, and its structure is exactly the same as the teacher model.

[0074] Step 3: Initial Task Training The purpose of initial task training is to allow the model to converge on the data of the first task (Pavia) and establish basic feature representation capabilities.

[0075] 3.1 Training Data Preparation: The 3,921 training samples from Task 1 were randomly shuffled and divided into batches of size 128. Each sample was a 36×7×7 three-dimensional tensor with a class index of 0-8.

[0076] 3.2 Forward Propagation and Loss Calculation: Each batch of samples is input into the LSGA-ViT model, passing sequentially through an embedding layer, a Transformer encoder, and a classification head, resulting in a 9-dimensional logistic output vector. Training is performed using only the cross-entropy loss function. ; in, =128 is the batch size. =9 represents the number of categories for Task 1. For the first Each sample in the true category In this invention, log represents the logarithm with the natural constant e as the base.

[0077] 3.3 Backpropagation and Parameter Update: The Adam optimizer is used to backpropagate the cross-entropy loss, calculating the gradients of all trainable parameters (feature extractor parameters). A gradient clipping threshold of 1.0 is set to prevent gradient explosion. A gradient descent step is performed to update the parameters. Every 10 training epochs, the overall classification accuracy of the current model is evaluated on the test set, and the checkpoint with the highest accuracy is saved. Training terminates after 150 epochs. If the validation accuracy does not improve within 20 consecutive epochs, an early stopping strategy is used to terminate training prematurely. After training, the final model parameters are saved as the initial teacher model for subsequent incremental learning.

[0078] Step 4: Selecting and storing mixed samples in the buffer: After the initial task training is completed, samples for each category need to be selected and stored in a buffer for subsequent incremental task replay training and distillation constraints.

[0079] 4.1 Calculate the joint feature-spectral distance: Let the sample and samples The L2 normalized depth features are respectively and The L2 normalized spectral vectors are respectively and Then the joint distance between the two for: ; in, Represents cosine similarity. =0.6 is the weighting coefficient for balancing depth features and spectral information. This distance matrix is ​​used for subsequent selection of diverse samples.

[0080] 4.2 Adaptive Quota Calculation: Let the maximum number of spaces stored in each type of buffer be... N =5. First, calculate the dispersion score for each category. It combines the intra-class discreteness of the deep feature space and the spectral space; and includes all training samples with a number greater than 100. N The dispersion scores of the categories are min-max normalized to obtain Then calculate the compact quota for each category using the following formula: ; in, =5, =1, =0.8 is the adaptive intensity coefficient. The higher the dispersion of a category, the smaller the compact quota it receives, reserving more slots for diverse samples.

[0081] 4.3 Compact Sample Selection: Selected using an iterative group selection strategy A compact sample. Category prototype. This represents the mean of the L2-normalized deep features for all samples in this category. The next step is to select the sample from the remaining candidate pool that minimizes the Euclidean distance between the "feature mean after adding candidate samples" and the class prototype. ; in, For training the sample pool, The selected sample set; Iteration Step 1: Select all compact samples in sequence and mark them as True.

[0082] 4.4 Selection of diverse samples: Compact samples are removed from the training pool to obtain the residual pool. Select from the residual pool A variety of samples. For the candidate subset Its compactness measure and diversity measurement These are defined as the average distance from a sample within a subset to the prototype and the average distance between any two samples within a subset, respectively. The optimal diversity subset is obtained by minimizing the following objective function. : ; in, =0.5 is the tradeoff coefficient. Starting from an empty set, each step selects the candidate sample that maximizes the decrease in the objective function and adds it, repeating until the quota is full. These samples are marked as False.

[0083] Each category N =5 samples were stored in the buffer M Each time a new task is added, the buffer expands in the same way.

[0084] Step 5: Subsequent incremental task training: 5.1 Loading the model: The model parameters from the previous task's training are completely copied and frozen, serving as the teacher model for the current task. The teacher model does not participate in gradient updates during training; it only provides supervision signals to a fixed objective. The student model parameters can be updated during training.

[0085] In each training epoch, the training batch consists of a mixture of two types of samples: one part comes from the new category training data of the current task, and the other part is sampled and replayed from the buffer. The batch size remains at 128.

[0086] 5.2 Forward Propagation and Calculation of Various Losses: The samples are input into the teacher model and the student model respectively. The student model concatenates all the classification header logical values ​​into a complete vector in category order, with a length of [length missing]. The teacher model only outputs the old category logical values, with a length of [length missing]. Calculate the following four losses: Cross-entropy loss : Applied to all samples in the batch, calculate the negative log-likelihood between the true label and the model's predicted probability: ; in, The set of all categories currently seen. Indicates the first in the current batch One sample, For the model to the first Each sample in its true category The logical output value, For the model to the first Each sample in category The output value is the logical value. This loss ensures that the model maintains good discriminative ability for all seen categories.

[0087] Logical value alignment loss This calculation is performed only on samples belonging to the old category within the batch, to preserve the relative discriminative structure between categories. By constraining the student model to maintain the same relative response relationship between samples as the teacher model on the old category, it prevents the discriminative boundary between spectrally similar categories from being distorted during incremental learning, as expressed in: .

[0088] Intermediate layer pooling distillation loss This loss is calculated only for samples of the old class in the batch, and is used to protect the spectral-space feature structure of the intermediate layer from being covered. This loss constrains the intermediate layer representation of the student model to remain within the neighborhood of the teacher model's output during incremental updates, thereby preserving multi-level information in the hyperspectral image from local spectral features to global spatial context, as expressed in: .

[0089] Character differentiation prototype distillation loss This loss is calculated only for compact samples with the role flag set to True in the batch; diverse samples are not included. This loss ensures that the feature responses of the old and new models remain consistent in the direction of the "clean" class prototype, avoids contamination of prototype estimation by diverse samples, and stabilizes the class center structure in the feature space, expressed as: .

[0090] Total loss : ; The four losses are weighted and summed, with the weighting coefficients... = =0.5, =0.1. This total loss simultaneously optimizes the model's classification ability, the relationship structure between old categories, the spectral-empty feature representation of intermediate layers, and the consistency of feature responses of category prototypes. The four loss terms protect old knowledge at different granularities and work together to achieve a balance between stability and plasticity in the incremental learning process.

[0091] Step 6: Model Evaluation After training for each incremental task is completed, a comprehensive evaluation is performed on the test set of all currently seen categories. The main evaluation metrics are overall accuracy (OA), which is the percentage of correctly classified test samples out of the total number of test samples; average class accuracy (AA), which reports the arithmetic mean of the accuracy of each category; and the Kappa coefficient, a statistical indicator that measures the consistency between the classification results and the true labels, used to evaluate how well the model's predictions match the actual situation.

[0092] Example 3: To verify the effectiveness of the class incremental learning method based on mixed sample selection and multi-granularity distillation proposed in this invention, this embodiment conducted a systematic comparative experiment on four publicly available hyperspectral remote sensing benchmark datasets, and performed quantitative comparison and in-depth analysis with eight mainstream class incremental learning methods.

[0093] 1. Comparison Method This invention selects the following eight representative incremental learning methods as comparative baselines, covering three major paradigms: regularization methods, replay methods, and knowledge distillation methods: (1) Finetuning: The simplest baseline method, which directly fine-tunes the model on new task data without using any old class sample replay or distillation constraints, and is used to verify the severity of catastrophic forgetting.

[0094] (2) EWC (Elastic Weight Consolidation): A regularization method that estimates the importance of each parameter to the old task through the Fisher information matrix and imposes penalty constraints on the update of important parameters.

[0095] (3) LwF (Learning without Forgetting): A knowledge distillation method that uses the softened logits of the old model output as the distillation target when training on a new task, constraining the output distribution of the current model on the old category.

[0096] (4) DER (Dynamically Expandable Representation): A dynamic architecture method that learns to dynamically adjust the capacity of the feature space through scalable representations.

[0097] (5) iCaRL (Incremental Classifier and Representation Learning): The replay method selects representative samples of each class and stores them in a buffer through the herding strategy, and combines them with the nearest mean classifier for classification.

[0098] (6) BiC (Bias Correction): The playback combined with distillation method introduces a learnable bias correction layer on the basis of iCaRL to alleviate the bias of the new category on the classification of the old category.

[0099] (7) LLL-AFCA (Adaptive Feature Consolidation): The replay combined with distillation method adopts a dual-branch architecture to maintain the consistency of new and old features, and performs well on multiple benchmarks.

[0100] (8) HyperKD (Lifetime Hyperspectral Image Classification Based on Cross-Spectral-Spatial Knowledge Distillation): A hyperspectral class incremental learning method based on cross-network knowledge distillation and cross-selection modules, which is one of the most advanced methods in the field of hyperspectral CIL.

[0101] Among them, method (1) is the baseline without taking any forgetting mitigation measures, method (2) is a regularization method, method (3) is a distillation method, and methods (4)-(8) are all hybrid methods of playback and distillation.

[0102] 2. Experimental Data Results and Analysis 2.1 Description of Evaluation Indicators To comprehensively evaluate the performance of various incremental learning methods, this experiment uses the following evaluation metrics: Overall Accuracy (OA): The percentage of correctly classified test samples out of all test samples. It is the most intuitive metric for classification performance. A higher OA indicates a stronger overall classification ability of the model across all known categories.

[0103] Task Average Accuracy (TAC): At a given incremental stage, the TAC is the arithmetic mean of the model's overall accuracy (OA) across all learned datasets. For example, the TAC for Task 2 is the average of the OA from the University of Pavia dataset and the Indian Pine Forest dataset; the TAC for Task 4 is the average of the OA from all four datasets. TAC comprehensively reflects the model's overall ability to retain all learned tasks at the current stage and is a core metric for measuring incremental learning performance.

[0104] Avg(4): The arithmetic mean of the final TAC of the four incremental tasks. This metric comprehensively evaluates the average performance of the model throughout the entire incremental learning sequence, taking into account both the learning effect of the initial task and the retention and adaptation ability of subsequent tasks.

[0105] Avg(10): The combined average of OA across all tasks for all datasets, including 1 OA for Task 1, 2 OAs for Task 2, 3 OAs for Task 3, and 4 OAs for Task 4, for a total of 10 data points. This metric more comprehensively reflects the overall performance of the model at different learning stages and on different datasets.

[0106] Table 1 shows the comparison between the proposed method and other methods on four datasets. All values ​​are presented as "mean ± standard deviation" (e.g., "86.64 ± 4.66"). The best results for each task are highlighted in bold. The mean is the average of five independent experiments, and the standard deviation reflects the stability of the experimental results. The smaller the standard deviation, the smaller the performance fluctuation of the method under different random initializations, and the stronger its robustness.

[0107] 2.2 Analysis of Results for Each Task (1) Task 1 Task 1 involves only one dataset, the University of Pavia dataset, on which all methods were trained from scratch, avoiding the problem of forgetting old classes. Experimental results show that the operational accuracy (OA) of each method is concentrated between 85.48% and 86.87%, with the OA of the method in this invention being 86.64% ± 4.66%, which is basically on par with the best method, BiC, at 86.87% ± 4.59%. The standard deviation of all methods is between 4% and 5%, indicating a similar level. This result demonstrates that with consistent initial experimental conditions, the methods are comparable under the same backbone network, laying the foundation for fair comparisons in subsequent incremental tasks.

[0108] (2) Task 2 After entering the incremental learning phase, the performance of various methods diverged significantly. The method of this invention achieved an OA of 83.91% ± 3.17% on the old task, the University of Pavia dataset, significantly outperforming the second-best method, HyperKD, which achieved 77.97% ± 1.81%, representing an improvement of 5.94 percentage points. This indicates that the hybrid sample selection and multi-granularity distillation mechanism proposed in this invention effectively mitigates catastrophic forgetting.

[0109] On the new task, the Indian Pine Forest dataset, the OA of the proposed method is 91.05% ± 1.52%, which is slightly lower than iCaRL's 93.60% ± 1.16%, but still ranks among the top methods. iCaRL performs best on new classes, mainly because its nearest-nearest-means classifier adapts to the feature space of new classes quickly, but this advantage comes at the cost of sacrificing the ability to preserve old classes.

[0110] From the perspective of TAC (Tracking Accuracy), the method of this invention achieves 87.48% ± 2.31%, the highest among all methods, and is 2.30 percentage points higher than the second-best method, HyperKD, which achieves 85.18% ± 0.90%. This indicates that the method of this invention achieves a better balance between "learning new classes" and "maintaining old classes".

[0111] (3) Task 3 As tasks accumulate, the forgetting of old classes intensifies, further widening the performance gap between different methods. The OA of the method in this invention on the three datasets are 79.04%±0.83 (University of Pavia dataset), 87.78%±0.93 (Indian Pine Forest dataset), and 96.79%±0.28 (Salinas dataset), all of which are at the best or second-best level.

[0112] Of particular note is that on the old task, the University of Pavia dataset, the advantage of the method of this invention further expands with the accumulation of tasks, exceeding the second-best method HyperKD by 5.06 percentage points. At the same time, the TAC reaches 87.87%±0.56%, which is the highest among all methods, and the standard deviation is the smallest among all methods, indicating that the method of this invention has stable performance and strong robustness under different random seeds.

[0113] (4) Task 4 In the final task, the model needs to identify all 56 land cover categories simultaneously. The OA of the method of this invention on the four datasets are 75.11%±3.91 (University of Pavia dataset), 84.53%±1.63 (Indian Pine Forest dataset), 94.81%±0.65 (Salinas dataset), and 83.76%±1.82 (Houston 2013 dataset).

[0114] On the Pavia dataset, our method ranked first with 75.11%, a 1.44 percentage point improvement over HyperKD's 73.67%. On the Salinas dataset, our method ranked first with 94.81%, a 0.25 percentage point improvement over HyperKD's 94.56%. On the Indian Pine Forest dataset, our method ranked second with 84.53%, slightly lower than EWC's 84.83%, but higher than HyperKD's 83.42%.

[0115] On the Houston 2013 dataset, LLL-AFCA achieved the best OA of 88.71%, but the TAC of the method in this invention still ranked first with 84.55%±1.44, indicating that it is still the best in overall multi-task balance.

[0116] (5) Global average index In terms of global average performance, the Avg(10) of the method of this invention is 86.34%±1.00 and the Avg(4) is 86.64%±1.64, which are the highest among all methods, and are 1.10 and 1.09 percentage points higher than the second-best method HyperKD, respectively.

[0117] Table 1: Comparison of overall accuracy (%) of different methods on four hyperspectral classification tasks

[0118] Example 4: Based on the same inventive concept, please see Figure 3 The present invention also provides a class of incremental learning systems based on hybrid sample selection and multi-granularity distillation, comprising: The acquisition and construction module is used to acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; The mixed sample selection and adaptive quota allocation module is used to select mixed samples through a two-stage selection strategy after each incremental task training is completed. It adaptively calculates the compact quota for each category. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples and stored in the sample buffer. The knowledge distillation module from the relation layer to the intermediate layer is used to perform knowledge distillation during the training process of new tasks. In the output layer, a logical value relation alignment constraint is adopted, requiring the student model to maintain the same relative relationship between samples as the teacher model. At the same time, the intermediate layer output is constrained by the intermediate layer pooling distillation, and the frozen intermediate layer representation of the teacher model is used as the anchor target to constrain the intermediate layer representation of the student model to align with the teacher model. The role differentiation prototype distillation module is used to perform role differentiation prototype distillation in the feature space. Based on the determined role labels of compact and diverse samples, the module calculates feature prototypes of each category using compact samples as anchor targets, and constrains the category prototypes generated by the student model to align with the anchor targets. The distillation optimization and output module is used to weight and sum the classification loss with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively, to construct the total loss function to update the student model parameters until convergence, obtain the student model that has completed the training of the current incremental task, and use it as the initial model for the next incremental task. The student model is then used to classify hyperspectral remote sensing images and output the land cover classification results.

[0119] Unless otherwise defined, the technical or scientific terms used in this invention shall have the general meaning understood by one of ordinary skill in the art to which this invention pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this invention do not indicate quantitative limitations and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this invention are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “A plurality” used in this invention refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. In this invention, the terms "first," "second," "third," etc., are used only to distinguish similar objects and do not represent a specific order of objects.

[0120] Obviously, the accompanying drawings are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, certain design, manufacturing, or production modifications made according to the technical content disclosed in the present invention are merely conventional technical means and should not be considered as insufficient disclosure of the present invention.

[0121] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A class incremental learning method based on mixed sample selection and multi-granularity distillation, characterized in that, Includes the following steps: Acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; After each incremental task training is completed, a two-stage selection strategy is used to select mixed samples. The compact quota for each category is adaptively calculated. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples, and they are stored in the sample buffer. During the training process of the new task, knowledge distillation is performed. Logical value relationship alignment constraints are adopted in the output layer to require the student model to maintain the same relative relationship between samples as the teacher model. At the same time, intermediate layer output is constrained by intermediate layer pooling distillation. The frozen intermediate layer representation of the teacher model is used as the anchoring target to constrain the intermediate layer representation of the student model to align with the teacher model. In the feature space, role-distinguishing prototype distillation is performed. Based on the determined role labels of compact and diverse samples, the feature prototypes of each category are calculated using compact samples as anchor targets, and the category prototypes generated by the student model are constrained to align with the anchor targets. The classification loss is weighted and summed with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively to construct the total loss function to update the student model parameters until convergence, thus obtaining the student model that has completed the training of the current incremental task. This student model is then used as the initial model for the next incremental task. The student model is then used to classify hyperspectral remote sensing images and output the land cover classification results.

2. The method according to claim 1, characterized in that, A two-stage selection strategy is used for mixed sample selection, adaptively calculating the compact quota for each category, including: The joint feature-spectral distance metric is defined by weighted summation of the cosine distance between the depth features of the samples and the spectral angle of the original spectral vector. Calculate and normalize the comprehensive intra-class dispersion score of each category in the deep feature space and the original spectral space; determine the compact quota of each category based on the normalized dispersion score. The higher the normalized dispersion score, the smaller the compact quota of the category, and the lower the normalized dispersion score, the larger the compact quota of the category; wherein, the compact quota of each category is between a preset upper bound and a lower bound.

3. The method according to claim 1, characterized in that, The first stage involves screening compact samples, and the second stage involves selecting diverse samples, including: The joint feature-spectral distance metric is defined by weighted summation of the cosine distance between the depth features of the samples and the spectral angle of the original spectral vector. The first stage of compact sample selection: For each category, using the category prototype as the anchor point, an iterative group selection strategy is adopted to successively select samples from the training sample pool that minimize the Euclidean distance between the mean feature of the selected samples and the category prototype, until the compact quota of each category is reached. The selected samples are then marked as compact samples. The second stage is the selection of diverse samples: from the residual pool after removing compact samples from each category, the compactness objective is to minimize the average distance from the selected subset to the category prototype, and the diversity objective is to maximize the average joint distance between samples within the selected subset. The remaining storage slots are selected and marked as diverse samples.

4. The method according to claim 3, characterized in that, Different role labels are assigned to the selected compact and diverse samples, and these are stored in the sample buffer, including: The role flag of the compact sample selected in the first stage is set to true, and the role flag of the diverse sample selected in the second stage is set to false. The role flag, along with the sample data and category label, is stored in a buffer to determine whether the sample participates in the category prototype calculation in the subsequent prototype distillation.

5. The method according to claim 1, characterized in that, The output layer employs logical value alignment constraints, requiring the student model to maintain the same relative relationships between samples as the teacher model, including: Obtain the output matrix of the old category samples in the current training batch on the logical value output layer of the student model and the teacher model; For each old category, calculate the pairwise difference matrix between the logistic values ​​of each sample in the student model and the teacher model in that category; The element-wise subtraction of the pairwise difference matrices of the student model and the teacher model in the same category is used as the square of the Frobenius norm, which is then used as the relation alignment loss for that category. Summing and averaging the relation alignment losses for all old categories yields the logical relation alignment loss. The logistic value alignment loss constrains the logistic value layer output of the student model through gradient backpropagation, aligning the relative response relationship between samples in the old category of the student model with that of the teacher model, so as to maintain the relative structural stability of the discrimination boundary between the old categories.

6. The method according to claim 1, characterized in that, By constraining the intermediate layer output through intermediate layer pooling distillation, and using the frozen intermediate layer representation of the teacher model as the anchoring target, the intermediate layer representation of the student model is constrained to align with the teacher model, including: The teacher model and the student model use the same network architecture, both consisting of multiple stacked Transformer modules; Obtain the feature maps output by each Transformer module of the student model; The feature maps output by each Transformer module are sequentially normalized and spatial dimension aggregated and compressed to extract compact description vectors for each Transformer module. Obtain the compact description vectors of the student model and the frozen teacher model on each Transformer module, respectively, and calculate the mean square error between them as the distillation loss of each Transformer module. The distillation loss of all Transformer modules is weighted and summed to obtain the intermediate layer pooling distillation loss, which constrains the output of each Transformer module of the student model to align with the corresponding frozen output of the teacher model.

7. The method according to claim 3, characterized in that, In the feature space, role-distinguishing prototype distillation is performed. Based on the determined role markers of compact and diverse samples, the feature prototypes of each category are calculated using compact samples as anchor targets. The category prototypes generated by the student model are constrained to align with the anchor targets, including: Deep features of compact samples with true role markers in the buffer are extracted using the frozen teacher model. The feature vectors are summed by category and then normalized using the L2 norm to obtain the normalized prototypes for each category. In the current batch, extract the normalized features of compact samples that belong to the old category and whose role flag is true under the current student model, as well as the normalized features of the sample under the frozen teacher model. Calculate the cosine similarity between student model features and corresponding class prototypes, and the cosine similarity between teacher model features and corresponding class prototypes, respectively. The square of the difference between the two is used as the prototype distillation loss for the sample. The prototype distillation loss for role differentiation is obtained by summing and averaging the prototype distillation loss of all compact samples that belong to the old category and whose role label is true. The role-distinguishing prototype distillation loss constraint ensures that the feature orientation of the student model on compact samples remains consistent with that of the teacher model, in order to maintain the stability of the prototype center of the old category.

8. The method according to claim 1, characterized in that, The total loss function is expressed as: ； in, For cross-entropy loss, For logical value alignment loss, This represents the loss from intermediate layer pooling distillation. To differentiate the prototype distillation loss for the character, , , These are the weighting coefficients.

9. A class-based incremental learning system based on hybrid sample selection and multi-granularity distillation, characterized in that, include: The acquisition and construction module is used to acquire and preprocess hyperspectral remote sensing images covering different land cover categories; Construct teacher and student models and train them on initial tasks to obtain the initial teacher model for subsequent incremental learning; The mixed sample selection and adaptive quota allocation module is used to select mixed samples through a two-stage selection strategy after each incremental task training is completed. It adaptively calculates the compact quota for each category. The first stage is to screen compact samples, and the second stage is to select diverse samples. Different role labels are assigned to the selected compact samples and diverse samples and stored in the sample buffer. The knowledge distillation module from the relation layer to the intermediate layer is used to perform knowledge distillation during the training process of new tasks. In the output layer, a logical value relation alignment constraint is adopted, requiring the student model to maintain the same relative relationship between samples as the teacher model. At the same time, the intermediate layer output is constrained by pooling distillation in the intermediate layer, and the frozen intermediate layer representation of the teacher model is used as the anchor target to constrain the intermediate layer representation of the student model to align with the teacher model. The role differentiation prototype distillation module is used to perform role differentiation prototype distillation in the feature space. Based on the determined role labels of compact and diverse samples, the module calculates feature prototypes of each category using compact samples as anchor targets, and constrains the category prototypes generated by the student model to align with the anchor targets. The distillation optimization and output module is used to weight and sum the classification loss with the three distillation losses calculated in the output layer, intermediate layer and feature space respectively, to construct the total loss function to update the student model parameters until convergence, obtain the student model that has completed the training of the current incremental task, and use it as the initial model for the next incremental task. The student model is then used to classify hyperspectral remote sensing images and output the land cover classification results.

Images