Image classification method and system based on attribute-guided concept bottleneck continual learning

Abstract

This invention discloses an image classification method and system based on attribute-guided concept bottleneck continuous learning. The method first generates attribute text related to image categories to construct an attribute concept candidate pool, builds and updates the attribute concept bottleneck matrix. Next, it generates visual features corresponding to the image and applies mask weights to these features, modulating the concept score to calculate the attribute response result. Then, it inputs the old category name and the prototype of the new category for the current task into a semantic-guided prototype enhancement module to generate pseudo-features for the old category. Finally, it constructs a knowledge distillation architecture to constrain the concept scores, and inputs the distilled and constrained output with the current task image into the classification layer. Combined with the attribute response result and the pseudo-features of the old category, it performs joint optimization to achieve image classification prediction output. This invention exhibits good generalization ability, stability, and class increment adaptation capability, providing an effective and reliable technical means for the application of image classification in continuous learning scenarios.

Description

Technical Field

[0001] This invention belongs to the field of continuous learning image classification, specifically involving an image classification method and system based on attribute-guided concept bottleneck continuous learning. It can improve the classification performance of the model and its ability to retain old knowledge in scenarios where categories are continuously expanded. Background Technology

[0002] In recent years, with the development of artificial intelligence and deep learning technologies, image classification, as one of the core tasks in the field of computer vision, has been widely applied in medicine, industrial inspection, intelligent monitoring, and autonomous driving. In these applications, image classification models typically need to perform semantic understanding on input images and accurately classify them into a predefined set of categories to support downstream decision-making and system operation.

[0003] Traditional image classification methods are mostly based on fixed sets of categories and static datasets, acquiring the ability to recognize predefined categories through one-time training. However, in real-world applications, categories and data often increase continuously over time. For example, medical imaging systems need to continuously identify new disease types, industrial inspection systems need to continuously adapt to new defect categories, and intelligent monitoring systems need to continuously learn new target objects. Therefore, image classification models not only need to have the ability to recognize the current category, but also need to be able to continuously learn new categories in subsequent stages.

[0004] To address the aforementioned issues, continuous learning has been introduced into image classification tasks, enabling models to progressively learn new categories in sequential tasks while maintaining performance on older categories to mitigate catastrophic forgetting. Existing methods primarily achieve a balance between new and old knowledge through strategies such as parameter constraints, sample replay, and network expansion. However, in complex image classification scenarios, these methods still suffer from insufficient model stability and weak interpretability. The Concept Bottleneck Model (CBM) provides a modeling framework with good interpretability for image classification. This method introduces an explicit concept layer between image features and category prediction, representing the classification process as a two-stage reasoning process of "image-concept-category." Such methods typically construct a concept space using attributes, linguistic descriptions, or concept sets, and then make category predictions based on concept scores, thus enabling the model to provide more explicit discrimination criteria. Therefore, combining the Concept Bottleneck Model with continuous learning helps improve both the model's interpretability and its adaptability to new categories.

[0005] However, existing image classification methods based on continuous learning from concept bottlenecks still have certain shortcomings. First, in the concept construction stage, they typically rely on large language models or text descriptions to generate concept sets, which are prone to containing abstract semantics or redundant attributes that are weakly related to the visual features of the image, leading to noise in concept representation and thus affecting classification performance and the retention of old knowledge. Second, in terms of concept space modeling, most methods adopt a globally shared concept space, with different categories sharing the same set of concept representations. While this approach improves the efficiency of concept reuse, it is also prone to introducing attribute dependencies unrelated to the target category, resulting in spurious inferences. As new categories are continuously added, the expansion of the shared concept space may also cause changes in the concept representations of existing categories, triggering semantic drift and further reducing the stability of the model during continuous learning.

[0006] Therefore, in continuous learning image classification tasks, how to construct a set of attribute concepts that are highly consistent with visual features, design a concept subspace that is closely related to category discrimination, and maintain the stability of concept semantics during the continuous expansion of categories has become a key problem that urgently needs to be solved. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes a continuous learning method and system for image classification based on attribute-guided concept bottlenecks. First, for each category in the current task, a large language model is used to generate category-related attribute description text. The generated results are then filtered based on the visual semantic similarity between image samples and attribute text, constructing a candidate pool of attribute concepts with visual correspondences. Subsequently, an attribute concept selection network is introduced. Under the combined effect of classification supervision and semantic distribution constraints, representative and discriminative attribute concepts are further selected from the candidate pool. Furthermore, based on the semantic relevance between category names and attribute concepts, a class-related attribute mask corresponding to the category is generated. This attribute mask is then used to modulate the concept response results of the input image dimensionally, reducing the spurious correlation caused by the activation of irrelevant concepts. Then, using the semantic similarity between old category names and the prototype of the new category in the current task, a pseudo-feature representation of the old category is generated. This pseudo-feature representation is then used together with real samples from the current task for classification training, enhancing the model's ability to retain historical category knowledge. Finally, a concept score distillation mechanism is introduced, which uses the concept bottleneck layer frozen and saved in the previous stage as the teacher end and the concept bottleneck layer in the current stage as the student end. Under the same input conditions, the concept scores on the old concept dimension are aligned and constrained, thereby maintaining the stability of the old concept output, mitigating concept drift and catastrophic forgetting, and realizing continuous learning of image classification for scenarios with continuous category expansion.

[0008] In one aspect, this invention provides a continuous learning method for image classification based on attribute-guided conceptual bottlenecks, comprising the following steps:

[0009] Step 1: For each image category in the current task, use a large language model to generate attribute text related to the category, and combine the visual semantic similarity between the image samples and the attribute text to filter the generated results to obtain a candidate pool of attribute concepts with visual correspondence.

[0010] Step 2: Input the attribute concept candidate pool obtained in Step 1 into the attribute concept selection network. Under the combined effect of classification supervision and semantic distribution constraints, select representative and discriminative attribute concepts from the candidate pool and construct the attribute concept bottleneck matrix for the current task. Provide a structured concept representation for each category. At the same time, cumulatively update the matrix with the concept bottlenecks of historical tasks to maintain the concept knowledge in continuous tasks.

[0011] Step 3: Based on the attribute concept bottleneck matrix output in Step 2, calculate the semantic correlation between category names and attribute concepts, generate visual features corresponding to the image, apply mask weights to the visual features, and calculate the attribute response results by modulating the concept scores dimension by dimension.

[0012] Step 4: Based on the visual features obtained in Step 3, input the old category name and the prototype of the new category for the current task into the semantic guidance prototype enhancement module, calculate the semantic similarity between the old category and the new category, generate the pseudo-feature representation of the old category, and combine it with the real samples of the current task for classification training to enhance the model's ability to retain historical category knowledge.

[0013] Step 5: Build a knowledge distillation architecture before the image classifier. Use the concept bottleneck layer frozen and saved in the previous stage as the teacher end and the concept bottleneck layer in the current stage as the student end. Under the same input conditions, perform element-wise alignment constraints on the concept scores in the old concept dimension to mitigate concept drift and catastrophic forgetting.

[0014] Step Six: Input the concept bottleneck output after distillation constraints obtained in Step Five along with the current task image samples into the classification layer. Combine the attribute response results obtained in Step Three and the old category pseudo-features generated by semantic guidance obtained in Step Four for joint optimization. Calculate the category prediction loss, alignment loss, regularization constraint and concept score distillation loss. Update the model parameters through backpropagation to achieve the final image classification prediction output in the scenario of continuous category expansion.

[0015] In another aspect, the present invention also provides an image classification system based on attribute-guided conceptual bottleneck continuous learning, for implementing the aforementioned image classification method, comprising the following modules:

[0016] The attribute concept candidate pool module is used to generate attribute text related to each image category in the current task using a large language model, and then filter and construct the attribute concept candidate pool.

[0017] The attribute concept bottleneck matrix module is used to input the attribute concept candidate pool into the attribute concept selection network, select representative and discriminative attribute concepts from the candidate pool, construct the attribute concept bottleneck matrix, and update it.

[0018] The attribute response module generates visual features corresponding to the image based on the attribute concept bottleneck matrix, applies mask weights to the visual features, and modulates the concept score to calculate the attribute response result.

[0019] The old category pseudo-feature module is used to generate old category pseudo-features by inputting the old category name and the prototype of the new category for the current task into the semantic guidance prototype enhancement module based on visual features.

[0020] The knowledge distillation module is used to build a knowledge distillation architecture before the image classifier, and to perform element-wise alignment constraints on the concept scores in the old concept dimension under the same input conditions.

[0021] The image classification output module is used to input the output after knowledge distillation constraints and the current task image samples into the classification layer. It combines the attribute response results and old category pseudo features for joint optimization to achieve the final image classification prediction output in the scenario of continuous category expansion.

[0022] The beneficial effects of this invention are as follows:

[0023] This invention proposes a continuous learning method for image classification based on attribute-guided concept bottlenecks. This method focuses on optimizing the concept set construction, class-related attribute selection, and old concept retention processes in continuous learning scenarios. By introducing an attribute concept candidate pool and a concept bottleneck selection mechanism, the model can filter attribute concepts with visual correspondence and discriminative ability from a large number of attribute descriptions, providing a stable and reliable semantic foundation for subsequent concept expression. Furthermore, by combining the semantic correlation between categories and attributes, class-related modulation is applied to the concept response results, enabling the model to selectively focus on key attributes for different categories and reduce the impact of spurious correlations caused by shared concept spaces. Simultaneously, through a semantically guided prototype enhancement mechanism and a concept fraction distillation mechanism, the model can maintain the stability of old category feature representations and old concept semantic boundaries during continuous learning, effectively mitigating concept drift and catastrophic forgetting problems. The classification results based on attribute concepts not only clearly reflect the key attributes on which the model's predictions depend but also balance new knowledge learning and old knowledge retention even with the continuous addition of new categories, giving the model high classification performance, interpretability, and continuous learning capabilities. Overall, this method has good generalization ability, stability and class incremental adaptation ability, providing an effective and reliable technical means for the application of image classification models in continuous learning scenarios. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below:

[0025] Figure 1 This is a diagram illustrating the overall architecture of the classification method of this invention.

[0026] Figure 2 This is an architecture diagram of the attribute concept candidate pool construction part of the present invention;

[0027] Figure 3 This is a schematic diagram of the conceptual fractional distillation part of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This embodiment provides a continuous learning method for image classification based on attribute-guided conceptual bottlenecks, such as... Figures 1 to 3 As shown, it includes the following steps:

[0030] Step 1: Starting with category descriptions, generate category-related attributes using a large language model, clean and deduplicate attribute phrases, then calculate visual-text similarity using the CLIP model and filter by threshold to obtain a candidate set. Finally, summarize these to form the attribute concept candidate pool for this task, such as... Figure 2 As shown.

[0031] Step one specifically includes:

[0032] First, we will build a set of attributes that are shared across categories. , Given the number of attributes, a large language model is used to generate corresponding attribute sentences for each image category in each attribute dimension. The category set is... For any category With attributes Language model generates attribute sentences Get the attribute text Then, each attribute sentence is subjected to phrase extraction and normalization cleaning. A phrase extraction operator is introduced. The candidate concept phrase set is obtained as shown in the following formula:

[0033]

[0034] in Long sentences are truncated into phrase-level descriptions, and duplicates are removed and formatted uniformly to make the conceptual units more similar to sortable visual attribute fragments. It is a set of conceptual phrases.

[0035] Next, categories The image sample set is , For category The number of images for any candidate phrase Using a frozen CLIP image encoder With text encoder Calculate the cosine similarity between the phrase and each image within the category, let... The top ratio is used, and then the one with the highest similarity is selected. The average score of each sample is calculated to obtain the aggregate score of the candidate phrase in that category. Finally, only phrases with scores greater than or equal to the filtering threshold are retained. The phrases are used as valid attributes to form a set of attribute phrases after category constraints. The filtering results for all categories in the current task are then aggregated into the final attribute pool. As shown in the following formula:

[0036]

[0037]

[0038]

[0039]

[0040] in For cosine similarity, For category The Zhang Image and Candidate Phrases similarity, For the top proportion, Candidate phrases In category Aggregate matching scores above, The filtering threshold is... For category The set of attribute phrases that are retained. For the first A set of task categories, For the first The attribute pool is obtained by summing all categories of each task.

[0041] Subsequently, the frozen CLIP text encoder was used. right Each attribute concept is encoded, and the resulting embedding vector is processed. Normalization yields the candidate attribute concept embedding matrix for the current task, as shown in the following formula:

[0042]

[0043] in This represents the total number of candidate attribute concepts for the current task. No. The concept of candidate attributes for The corresponding text embedding vector, For vectors Norm, The feature dimension is denoted by . The attribute concept embedding matrices from multiple tasks are accumulated to obtain the attribute concept candidate pool.

[0044] Step 2: Input the attribute concept candidate pool obtained in Step 1 into the attribute concept selection network, perform semantic distribution modeling on the candidate concepts, introduce Mahalanobis distance regularization term to train the attribute selection network to generate concept centers, and then filter the attributes according to the center distance to finally form the concept bottleneck matrix.

[0045] Step two specifically includes:

[0046] To select a representative subset of attribute concepts from the attribute concept candidate pool, an attribute concept selection network is trained using the image of the current task as input and category labels as supervision. The task includes A new category, for a batch For each training sample, the classification loss formula is as follows:

[0047]

[0048]

[0049] in and These are the weights and biases of the first-level linear mapping, respectively. For batch normalization operations, and These are the weights and biases of the second-layer linear mapping, respectively. For the sample The category prediction probability vector, The number of training samples, For the first The number of categories for each task For the sample truth class, Selecting network output samples for attribute concepts Category The predicted probability.

[0050] To further ensure consistency between the learning center and the candidate attribute concept embedding distribution, a Mahalanobis distance regularization term is introduced during training, and the candidate attribute concept embedding matrix obtained in step one for the current task is statistically analyzed. The overall distribution characteristics in the semantic space are calculated by plotting the mean vector and the inverse covariance matrix, which are then used to measure the deviation of the learned center from the distribution center of the candidate attribute concept, as shown in the following formula:

[0051]

[0052]

[0053]

[0054] in This represents the central position of the candidate attribute concept for the current task in the semantic space. Describe its distribution shape, This indicates a covariance operation. It is a constant. It is the identity matrix. For the first A central vector.

[0055] The total loss function of the attribute concept selection network, which combines classification supervision and distribution constraints, is expressed as follows:

[0056]

[0057] in The weighting coefficients are used to balance the classification loss and the Mahalanobis distance regularization term.

[0058] Under classification supervision, the weights of the attribute concept selection network gradually cluster towards representative discriminative semantics. Simultaneously, constrained by the Mahalanobis distance regularization term, their positions are limited to high-density regions within the current task's candidate attribute concept embedding distribution. After training, the weight matrix of the first-layer linear mapping of the attribute concept network is used as the center vector. You can get The central vectors are given by the following formula:

[0059]

[0060] in This is the set of semantic centers learned for the current task.

[0061] Subsequently, attribute concept selection is performed based on the distance between these center vectors and the embeddings of candidate attribute concepts. Specifically, the center vectors are first evaluated based on their distances to the embeddings of candidate attribute concepts. Normalize the vector, then calculate the Euclidean distance between it and all candidate attribute concept embeddings, and select the attribute concept that is closest to it and has not been selected before as the candidate attribute concept index that best matches the center vector. As shown in the following formula:

[0062]

[0063] in The squared Euclidean distance is calculated by first determining the difference between the two vectors in each dimension, and then... Norm and square, For vectors Norm, In order to be with the first The index of candidate attribute concepts that best match the center vector.

[0064] Finally, based on the candidate attribute concept index The bottleneck matrix of attribute concepts obtained by filtering Each row corresponds to the optimal attribute selected for the center vector, as shown in the following formula:

[0065]

[0066] Finally, the newly selected attribute concept bottleneck matrix for the current task is concatenated row-by-row with the attribute concept bottleneck matrices retained from historical tasks to obtain the result up to the [number missing]. The cumulative attribute concept bottleneck matrix of each task As shown in the following formula:

[0067]

[0068] in This is a line-by-line concatenation operation. This represents the total number of all retained attribute concepts up to the current task. Each row in the vector corresponds to a final retained attribute concept vector, which contains both important attribute semantic information newly introduced in the current task and stable concept knowledge learned in historical tasks.

[0069] Step 3: Using the attribute concept bottleneck matrix output in Step 2, calculate the semantic relevance between category names and attribute concepts, generate class-related attribute masks corresponding to the categories, and apply the masks to the concept response results of the input image, modulating the concept scores dimension by dimension to obtain the attribute response results;

[0070] Step three specifically includes:

[0071] The text encoder is denoted as The category set is The attribute text set is The candidate attribute concept embedding matrix was calculated in step one. First, the category name and attribute text of each category are input into a text encoder, and the output vector is processed. Normalization yields category embeddings, which are then formed by the candidate attribute concept embedding matrix. The attribute embedding is obtained as shown in the following formula:

[0072]

[0073] in For category The text encoding vector, For vectors, the second normal form, This is the normalized class embedding vector.

[0074] Subsequently, the category embedding and candidate attribute concept embedding matrix are calculated. The semantic cosine similarity matrix between As shown in the following formula:

[0075]

[0076] in For cosine similarity, for and The semantic similarity is calculated. This similarity matrix reflects the correlation between each category and each attribute; the larger the value, the closer the category and attribute are in the semantic space.

[0077] Furthermore, regarding the similarity matrix The Middle OK Column elements Perform normalization mapping to generate categories. For attributes The mask weights are given by the following formula:

[0078]

[0079] in For the sigmoid function, For the threshold, The smoothing coefficient controls the degree of smoothing. For category For attributes The mask weights. In this way, the contribution of each attribute is not simply 0 or 1, but rather the model can gradually learn the importance of the attributes during training.

[0080] After obtaining the mask weights, they are applied to the original response results of the input samples in the attribute space. For the input image... First, through the visual encoder Extracting visual features Then, the bottleneck matrix of the current task's attribute concept is used. With visual features The product of these features and the matching scores between the image features and the concept vectors of each attribute is used to obtain the original attribute score vector. As shown in the following formula:

[0081]

[0082]

[0083] Then, order Attribute score vector Will be with attribute mask vector Multiplying by dimension, we obtain the attribute response result. As shown in the following formula:

[0084]

[0085] in This represents element-wise multiplication. This is the attribute response result after being modulated by a category-specific attribute mask.

[0086] Step 4: Input the old category name and the prototype of the new category for the current task into the semantic guidance prototype enhancement module, calculate the semantic similarity between the old category and the new category, and generate a pseudo-feature representation of the old category;

[0087] Step four specifically includes:

[0088] Let the first The set of categories currently being learned in each task is denoted as . The set of historically learned categories is denoted as The text encoder is denoted as... The visual encoder is Old category The category name is denoted as New category in the current task The prototype is denoted as This prototype is based on the visual features of the samples of this category obtained in step three. The average was calculated. For any old category... First, calculate the cosine similarity between the text embedding of its category name and the prototype of the new category for the current task, and select the new category that is semantically closest as the reference category, as shown in the following formula:

[0089]

[0090]

[0091] in For category Number of images, Indicates a difference from the old category The new category that is semantically closest provides a basis for subsequent prototype compensation.

[0092] Further utilization of categories The image feature distribution offset information in the current task is used to compensate for the old category prototype. Let the category be... Its prototype in the current task is denoted as Old category The prototype is denoted as Then the old category The pseudo-features are represented by the following formula:

[0093]

[0094] in For category Image features in the current task, Indicates reference category The offset of the current image features relative to its prototype. This represents the generated old class. In this way, the variation pattern of the reference class in the current feature space is transferred to the old class prototype, thus enabling the generated old class representation to better adapt to the feature distribution of the current stage.

[0095] Step 5: Using the frozen concept bottleneck layer from the previous stage as the teacher's end and the concept bottleneck layer that can be updated in the current stage as the student's end, obtain the concept score matrix on the same input sample, and extract the part corresponding to the old concept from the student's output and align it with the teacher's output to finally form the concept score distillation term.

[0096] Step five specifically includes:

[0097] In image classifiers Previously introduced conceptual bottleneck layer After completing the first After training on each task, the conceptual bottleneck layer parameters from the previous stage are frozen as teacher-side conceptual mappings; in the second... During training for the first task, the updatable concept bottleneck layer parameters of the current stage are used as the concept mapping for the student side. For the first task... Mini-batch sample set for each task First, image features are extracted using the frozen CLIP image encoder. Then, the corresponding student concept score matrices are obtained by mapping through the concept bottleneck layers on both the teacher's and student's ends. The formula for calculating the student's concept score matrix at the current stage is as follows:

[0098]

[0099] in For the first Number of samples for each task For frozen CLIP image encoder, For the first A small batch of samples for each task For the first The weight matrix of the student-side conceptual bottleneck layer in each task training phase. As of the date Total number of concepts accumulated for each task Output the cumulative concept score matrix to the student.

[0100] The formula for calculating the teacher-side concept score matrix frozen and saved in the previous stage is as follows:

[0101]

[0102] in For the first The teacher-side concept bottleneck layer weight matrix is ​​frozen and saved after each task is completed. As of the date The total number of old concepts accumulated for each task The old concept score matrix output for the teacher.

[0103] Because the student-side output coverage extends to the [number]th [year]. The cumulative concept space of each task, while the teacher's side only covers the old concept space, therefore, the previous concept score is extracted from the student's concept score matrix. The column yields the old concept score matrix corresponding to the teacher's side, calculated using the following formula:

[0104]

[0105] in This is the concept score matrix for students on the old concept dimension.

[0106] To mitigate potential scaling differences between teachers and students at different training stages and to prevent teachers from dominating gradient updates in the early stages of training due to excessively large output values, intra-batch standardization is performed on the teacher-side concept score matrix to obtain a standardized concept score matrix. The calculation formula is as follows:

[0107]

[0108]

[0109]

[0110] in This is the mean vector of the scores for old concepts taught by teachers in the current small batch. This is the standard deviation vector of the old concept scores from the current small batch of teachers. To prevent constant terms with a denominator of zero, This is the standardized old concept score matrix for teachers.

[0111] Finally, the old concept score matrix for students. Compared with the standardized teacher-side old concept score matrix After applying element-wise mean square error constraints, the concept of fractional distillation is defined, and the calculation formula is as follows:

[0112]

[0113] in The square of the vector difference Norm, For students The concept score vector of each sample on the old concept dimension For teachers The old concept score vector after standardization of each sample.

[0114] Step 6: Input the student-side concept score matrix after distillation constraints obtained in Step 5, along with the current task image sample features, the class-related attribute mask generated in Step 3, and the old category pseudo-features generated in Step 4, into the final classification layer. Calculate the category prediction loss, alignment loss, regularization constraint, and concept score distillation loss, and perform multi-task joint optimization.

[0115] Step six specifically includes:

[0116] First, the category prediction loss is used to supervise the final classification result of the model. Then, the old category pseudo-features are completed in the semantically guided prototype enhancement module after step four. After generation, the real image features of each new category in the current task are combined with the pseudo features of the old categories to form a training feature set. The data is then input into the classification layer for prediction, and the calculation formula is as follows:

[0117]

[0118]

[0119] in The feature set used during the training phase, which includes the current task category. Real image features and historical categories Enhanced pseudo-features, This is the weight matrix for the conceptual bottleneck layer. This is the classification layer weight matrix. This represents the model's prediction results in the cumulative category space.

[0120] Secondly, the alignment loss is used to ensure consistency between the concept response output of the concept bottleneck layer and the CLIP concept activation matrix. This applies to the attribute response results calculated in step three after category-specific attribute mask modulation. The corresponding CLIP concept activation matrix is ​​obtained, and the calculation formula is as follows:

[0121]

[0122] To enhance the discriminative power of the concept score distribution, we first... and The formula for calculating the element-wise cubic transformation is as follows:

[0123]

[0124] in, For the corresponding CLIP concept activation matrix, the exponentiation operation is performed element-wise. and These are the concept score matrix and CLIP concept activation matrix after cubic transformation, respectively.

[0125] Based on this, the alignment loss is defined as follows:

[0126]

[0127] in, and Let represent the concept score vectors after power transformation. For cosine similarity, As of the date The total number of cumulative concepts for each task.

[0128] Furthermore, the final classification layer serves as the mapping from the concept space to the category space, and its weights can be viewed as an explicit representation of the contribution of each concept to category discrimination. To improve the interpretability of the classifier and the transparency of the decision-making process, a regularization constraint in the form of an elastic network is introduced onto the weight matrix of the final classification layer. The calculation formula is shown below:

[0129]

[0130] in This is the final classification layer weight matrix. To balance the parameters, It is the sum of the absolute values ​​of the matrix elements. It is the Frobenius norm. Regularization terms help promote weight sparsity, allowing the model to classify based on only a few key concepts, thus... Setting it to 0.99 makes the model more inclined to obtain sparse concept-to-category connections, thus improving the interpretability of concepts.

[0131] The model employs a multi-task joint optimization strategy during training. The optimization objective of the model is composed of the category prediction loss, alignment loss, regularization constraints, and the concept score distillation loss obtained in step five, as shown in the following formula:

[0132]

[0133] in , and These are the weight coefficients corresponding to the concept alignment loss, sparse regularization loss, and concept fractional distillation loss, respectively.

[0134] In another aspect, the present invention also provides an image classification system based on attribute-guided concept bottleneck continuous learning, for implementing the aforementioned image classification method, comprising the following modules: an attribute concept candidate pool module, an attribute concept bottleneck matrix module, an attribute response module, an old category pseudo-feature module, a knowledge distillation module, and an image classification output module:

[0135] The attribute concept candidate pool module is used to generate attribute text related to each image category in the current task using a large language model, and then filter and construct the attribute concept candidate pool.

[0136] The attribute concept bottleneck matrix module is used to input the attribute concept candidate pool into the attribute concept selection network, select representative and discriminative attribute concepts from the candidate pool, construct the attribute concept bottleneck matrix, and update it.

[0137] The attribute response module generates visual features corresponding to the image based on the attribute concept bottleneck matrix, applies mask weights to the visual features, and modulates the concept score to calculate the attribute response result.

[0138] The old category pseudo-feature module is used to generate old category pseudo-features by inputting the old category name and the prototype of the new category of the current task into the semantic guidance prototype enhancement module based on visual features.

[0139] The knowledge distillation module is used to build a knowledge distillation architecture before the image classifier, and to perform element-wise alignment constraints on the concept scores in the old concept dimension under the same input conditions.

[0140] The image classification output module is used to input the output after knowledge distillation constraints and the current task image sample into the classification layer, and perform joint optimization by combining the attribute response results and old category pseudo features to achieve the final image classification prediction output in the scenario of continuous category expansion.

[0141] Experimental implementation details:

[0142] This method was experimentally implemented in a Python 3.8.10 (Ubuntu 18.04) and CUDA 11.3 environment. The hardware environment consisted of an RTX 4080 Super and an 11th Gen Intel(R) Core(TM) i7-11700. The Adam optimizer was used, with a batch size of 64, a learning rate of 0.001, and a total of 60 training epochs. (Phrase score filtering threshold) Set to 0.2, top filter ratio Set to 0.2, mask threshold Set to 0.15, the trade-off parameter for the classification layer. Set to 0.99, the similarity loss tradeoff coefficient. Set to 1, the weight of the sparse regularization term. Set to 0.001, the weighting factor for distillation loss. The value was set to 0.3. This experiment used ViT-B / 16 as the backbone network. The dataset was split using a no-sample increment setting (Bm Inc-n). The initial number of task categories m and the incremental number of task categories n were variable to evaluate the model's continuous learning ability under different increment conditions. The average incremental accuracy and the final average accuracy were used as performance evaluation metrics.

[0143] Datasets: To comprehensively evaluate the model's performance in different visual scenarios, the experiments used six publicly available image datasets, covering both coarse-grained and fine-grained datasets, providing a good experimental foundation for verifying the model's generalization performance in language-guided concept modeling and interpretable continuous learning scenarios.

[0144] Three representative visual datasets were selected from the coarse-grained datasets: CIFAR-100, Tiny-ImageNet, and ImageNet-subset. The CIFAR-100 dataset contains 60,000 images across 100 object categories. This dataset covers common concepts such as animals, vehicles, and everyday objects, and different categories share a large number of high-level semantic attributes, making it suitable for constructing concept-level intermediate representations. The Tiny-ImageNet dataset covers approximately 120,000 images across 200 categories. This dataset is a streamlined subset of the ImageNet dataset, designed to retain high visual diversity while reducing computational costs. The ImageNet-subset dataset is built upon the large-scale ImageNet visual database, typically formed by selecting 100 categories from the original ImageNet-1K dataset, resulting in a total of over 130,000 images.

[0145] Three fine-grained visual datasets were selected: CUB-200, Oxford Flower, and Food-101. The CUB-200 dataset covers 200 bird species, with approximately 11,788 images. These images are mostly taken from real-world scenes, and each category includes several variations in pose, angle, and lighting. The Oxford Flower dataset covers 102 flower categories, with approximately 8,189 images. The images were taken from various shooting environments and backgrounds, showing significant differences in flower morphology and color, and exhibiting a degree of similarity and challenge between categories. The Food-101 dataset covers 101 food categories, with approximately 101,000 images. These images were taken from real-life scenes under diverse conditions, with significant background interference and lighting variations.

[0146] Table 1. Comparative experimental results (%) on the coarse-grained dataset CIFAR-100

[0147]

[0148] Table 2. Comparative experimental results (%) on the coarse-grained dataset Tiny-ImageNet

[0149]

[0150] Table 3. Comparative experimental results (%) on the coarse-grained dataset ImageNet-subset

[0151]

[0152] Tables 1 to 3 present the comparative experimental results on coarse-grained datasets. This invention achieved optimal results under different incremental settings for CIFAR-100, Tiny-ImageNet, and ImageNet-subset, demonstrating that the proposed method exhibits good stability and generalization ability in coarse-grained continuous learning scenarios. Specifically, under the B-10 Inc-10 setting of CIFAR-100, the CLG-CBM model... and The CLG-CBM model outperforms the EASE model by 0.58% and 0.49% respectively, indicating that while it incorporates concept bottleneck modeling, the distinction between categories is relatively clear in coarse-grained tasks. The additional concept modeling process may introduce semantic redundancy, resulting in slightly lower performance compared to EASE, which directly expands the subspace. This invention outperforms the CLG-CBM model by 1.47% and 2.46%, and also outperforms the EASE model by 0.89% and 1.97%, demonstrating that it retains the advantages of concept modeling while further improving the overall performance and task retention capabilities of the model. Under the B-50Inc-5 setting of CIFAR-100, this invention also achieves optimal results of 79.48% and 77.41%, further illustrating its superior knowledge retention as the initial category size increases and the incremental process lengthens. Similar conclusions are drawn from the other two coarse-grained datasets, where the advantages of this invention in stability and final performance become increasingly apparent as the incremental process progresses.

[0153] Table 4. Comparative experimental results (%) of average incremental accuracy on fine-grained datasets.

[0154]

[0155] Table 4 presents the comparative experimental results on fine-grained datasets. This invention maintains a stable advantage on the CUB-200, Flowers, and Food-101 datasets, especially on the CUB-200 dataset, where it achieves an improvement of approximately 1.8% compared to the CLG-CBM model in both partitioning settings. In bird recognition tasks where visual differences between categories are small, the discriminative ability of attribute representation is more demanding. This method better preserves the semantic structure within categories and performs more stably in multi-stage incremental learning. Compared to the CPP and EASE models, this invention's advantage on fine-grained datasets is more pronounced, indicating that relying solely on cue optimization or subspace expansion is insufficient to fully characterize the subtle differences between fine-grained categories. This invention more effectively maintains semantic consistency between new and old tasks.

[0156] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. An image classification method based on attribute-guided concept bottleneck continual learning, characterized in that, Includes the following steps: Step 1: For each image category in the current task, use a large language model to generate category-related attribute text, and then filter and construct an attribute concept candidate pool; Step 2: Input the attribute concept candidate pool into the attribute concept selection network, select representative and discriminative attribute concepts from the candidate pool, construct the attribute concept bottleneck matrix and update it; Step 3: Based on the attribute concept bottleneck matrix, generate the visual features corresponding to the image, apply the mask weights to the visual features, and modulate the concept score to calculate the attribute response results; Step 4: Based on the visual features, input the old category name and the prototype of the new category for the current task into the semantic guidance prototype enhancement module to generate pseudo-features for the old category; Step 5: Build a knowledge distillation architecture before the image classifier, and perform element-wise alignment constraints on the concept scores in the old concept dimension under the same input conditions; Step 6: Input the output constrained by knowledge distillation together with the current task image samples into the classification layer, and perform joint optimization by combining the attribute response results and old category pseudo features to achieve the final image classification prediction output in the scenario of continuous category expansion.

2. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 1, characterized in that, Step one specifically includes the following process: Construct a set of attributes shared across categories. , Given the number of attributes, a large language model is used to generate corresponding attribute sentences for each image category in each attribute dimension. The category set is... For any category With attributes Language model generates attribute sentences Get the attribute text Subsequently, each attribute sentence is subjected to phrase extraction and normalization cleaning; a phrase extraction operator is introduced. A set of candidate concept phrases is obtained. ; category The image sample set is , For category The number of images for any candidate phrase Using a frozen CLIP image encoder With text encoder Calculate the cosine similarity between the phrase and each image within the category, let... The top ratio is used, and then the one with the highest similarity is selected. The average score of each sample is calculated to obtain the aggregate score of the candidate phrase in that category. Finally, only phrases with scores greater than or equal to the filtering threshold are retained. The phrases are used as valid attributes to form a set of attribute phrases after category constraints. The filtering results for all categories in the current task are then aggregated into the final attribute pool. ; Subsequently, the frozen CLIP text encoder was used. right Each attribute concept is encoded, and the resulting embedding vector is processed. Normalization yields the candidate attribute concept embedding matrix for the current task. The attribute concept embedding matrix of multiple tasks is accumulated to obtain the attribute concept candidate pool.

3. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 2, characterized in that, Step two specifically includes the following process: Using the image of the current task as input and category labels as supervision, train an attribute concept selection network. The task includes A new category, for a batch Calculate the classification cross-entropy loss using training samples. ; During training, a Mahalanobis distance regularization term is introduced, and the embedding matrix of candidate attributes for the current task is statistically analyzed. In the semantic space, the overall distribution characteristics are calculated, and the mean vector and covariance inverse matrix are plotted. These are then used to measure the deviation of the learned center from the distribution center of the candidate attribute concept. ; The total loss function of a network that combines classification supervision and distributed constraints with attribute concept selection. Depend on and We get the result by weighted summation; After training, the weight matrix of the first-layer linear mapping of the attribute concept network is used as the center vector. ,get A set of central vectors ; Subsequently, attribute concept selection is performed based on the distance between the center vector and the embeddings of candidate attribute concepts: first, the center vector is... Normalize, then calculate the Euclidean distance between the embeddings and all candidate attribute concepts, and select the attribute concept that is closest to the embeddings and has not been selected before as the index of the candidate attribute concept that best matches the center vector. ; Based on candidate attribute concept index The bottleneck matrix of attribute concepts obtained by filtering Each row corresponds to the optimal attribute selected by the center vector; Finally, the newly selected attribute concept bottleneck matrix for the current task is concatenated row-by-row with the attribute concept bottleneck matrices retained from historical tasks to obtain the result up to the [number missing]. The cumulative attribute concept bottleneck matrix of each task , Each row in the vector corresponds to a final retained attribute concept vector, which contains both important attribute semantic information newly introduced in the current task and stable concept knowledge learned in historical tasks.

4. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 3, characterized in that, Step three specifically includes the following process: The text encoder is denoted as The category set is The attribute text set is First, input each category name into the text encoder and process the output vector. Normalization yields category embeddings. ; Subsequently, the category embedding and candidate attribute concept embedding matrix are calculated. The semantic cosine similarity matrix between For the similarity matrix The Middle OK Column elements Perform normalization mapping to generate categories. For attributes Mask weights ; For the input image First, through the visual encoder Extracting visual features Then, the bottleneck matrix of the current task's attribute concept is used. With visual features The product of these features and the matching scores between the image features and the concept vectors of each attribute is used to obtain the original attribute score vector. ; make Attribute score vector With attribute mask vector Multiplying dimension by dimension yields the attribute response results in the class-related attribute subspace. .

5. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 4, characterized in that, Step four specifically includes the following process: Let the first The set of categories currently being learned in each task is denoted as . The set of historically learned categories is denoted as The text encoder is denoted as The visual encoder is Old category The category name is denoted as New category in the current task The prototype is denoted as This prototype is based on the visual features of the samples of this category obtained in step three. The average is calculated; for any old category First, calculate the cosine similarity between the text embedding of its category name and the prototype of the new category for the current task, and then select the new category that is semantically closest as the reference category. ; Use of categories The image feature distribution offset information in the current task is used to compensate for the old category prototype: Let the category be... Its prototype in the current task is denoted as Old category The prototype is denoted as Then the old category pseudo-features Depend on and reference categories The current image features are summed relative to their original offsets to obtain the reference category. The offset of the current image feature relative to its prototype is .

6. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 5, characterized in that, Step five specifically includes the following process: In image classifiers Previously introduced conceptual bottleneck layer After completing the first After training on each task, the conceptual bottleneck layer parameters from the previous stage are frozen as teacher-side conceptual mappings; in the second... During training for the first task, the updatable concept bottleneck layer parameters of the current stage are used as the concept mapping for the student side; for the second task... Mini-batch sample set for each task First, image features are extracted using a frozen CLIP image encoder. Then, the corresponding student concept score matrices are obtained by mapping through the concept bottleneck layers on both the teacher's and student's ends. ; The teacher-side concept score matrix that was frozen and saved in the previous stage Extracting the top scores from the student-side concept score matrix The column yields the old concept score matrix corresponding to the teacher's side. ; In-batch standardization of the teacher-side concept score matrix yields the standardized concept score matrix. ; Finally, the old concept score matrix for students. Compared with the standardized teacher-side old concept score matrix Perform element-wise mean square error constraints .

7. The image classification method based on attribute-guided concept bottleneck continuous learning according to claim 6, characterized in that, Step six specifically includes the following process: First, the category prediction loss is used to supervise the final classification result of the model; after completing the old category pseudo-features... After generation, the real image features of each new category in the current task are combined with the pseudo features of the old categories to form a training feature set. The data is then input into the classification layer for prediction, yielding the classification prediction value. And calculate the cross-entropy classification loss. ; Secondly, the cumulative attribute concept bottleneck matrix for the current stage is calculated. Calculate the corresponding attribute response results ; right and Perform an element-wise cubic transformation, then calculate the alignment loss. ; A regularization constraint in the form of an elastic network is introduced onto the weight matrix of the final classification layer. ; A multi-task joint optimization strategy is employed during training to optimize the objective. Loss Prediction by Category Alignment loss Regularization constraints and concept fractional distillation loss Together they constitute.

8. An image classification system based on attribute-guided conceptual bottleneck continuous learning, used to implement the image classification method according to any one of claims 1 to 7, characterized in that, Includes the following modules: The attribute concept candidate pool module is used to generate attribute text related to each category of the image in the current task using a large language model, and then filter and construct the attribute concept candidate pool. The attribute concept bottleneck matrix module is used to input the attribute concept candidate pool into the attribute concept selection network, select representative and discriminative attribute concepts from the candidate pool, construct the attribute concept bottleneck matrix, and update it. The attribute response module generates visual features corresponding to the image based on the attribute concept bottleneck matrix, applies mask weights to the visual features, modulates the concept score, and calculates the attribute response result. The old category pseudo-feature module is used to generate old category pseudo-features by inputting the old category name and the prototype of the new category of the current task into the semantic guidance prototype enhancement module based on visual features; The knowledge distillation module is used to build a knowledge distillation architecture before the image classifier, and to perform element-wise alignment constraints on the concept scores in the old concept dimension under the same input conditions. The image classification output module is used to input the output after knowledge distillation constraints and the current task image samples into the classification layer. It combines the attribute response results and old category pseudo features for joint optimization to achieve the final image classification prediction output in the scenario of continuous category expansion.

Images