Continuous generalized zero sample learning method and system based on generation method
By introducing a diversified attribute enhancement module and a cross-task feature distillation module, the problems of insufficient diversity and catastrophic forgetting in the generation of unseen category features are solved, and the model's generalization and knowledge retention capabilities in dynamic learning scenarios are improved.
Patent Information
- Application Number
- CN202510808980.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-26
AI Technical Summary
Existing continuous generalized zero-shot learning methods suffer from insufficient diversity and catastrophic forgetting in generating features of unseen categories, which affects the model's generalization ability and long-term knowledge transfer capability in dynamic learning scenarios.
A collaborative approach of diversified attribute enhancement modules and cross-task feature distillation modules is adopted to generate diversified semantic descriptions and cross-task feature alignment through a large language model, thereby enhancing the expressive power of unseen category features and alleviating catastrophic forgetting.
It improves the diversity and stability of the model's feature generation in unseen categories, and enhances the model's generalization ability and long-term knowledge retention ability in dynamic learning environments.
Smart Images
Figure CN120706501A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning technology, and in particular to a continuous generalized zero-shot learning method and system based on a generative approach. Background Art
[0002] Deep learning, trained on large-scale labeled datasets, has demonstrated remarkable performance in multiple domains. However, in real-world scenarios, access to labeled data is often extremely limited due to the high cost and time required for manual labeling. This lack of labeled data, coupled with data imbalance, severely limits the model's ability to effectively recognize unseen or unlabeled data. To address these challenges, researchers have proposed Continuous Generalized Zero-Shot Learning (CGZSL). CGZSL aims to balance performance on seen and unseen classes in multi-task learning while retaining effective memory of previous tasks. This enables the model to smoothly adapt to new tasks during incremental learning and maintain robustness and generalization capabilities across changing data distributions. As the model continuously adapts to new classes in a dynamic learning environment, previously learned knowledge may be gradually lost over time, a phenomenon known as catastrophic forgetting. The introduction of new tasks often disrupts the retention of previous knowledge, making it difficult for the model to effectively retain previously learned knowledge. This limitation not only affects the model's stability when adapting to dynamic tasks but also weakens its ability to transfer knowledge across tasks over the long term. In recent years, the research focus on CGZSL has shifted to designing stronger knowledge retention mechanisms to mitigate the impact of catastrophic forgetting, thereby enhancing the model's long-term adaptability and cross-task generalization capabilities.
[0003] Wei et al. first attempted to address the CGZSL problem, but their approach relied on task-level supervision and could not handle test samples without task IDs, limiting its applicability in more practical and dynamic scenarios. Skorokhodov et al. proposed a category normalization technique for zero-shot learning and extended it to continuous zero-shot learning. Gautam et al. proposed a method combining experience replay and generative models to address the problem of incremental category learning. While CGZSL has made some progress, our research offers a novel perspective based on its generative framework. On the one hand, the lack of diversity and authenticity of generated features for unseen categories is a key issue that hinders the generalization ability of models in dynamic and incremental learning scenarios. Generative frameworks such as GANs and VAEs attempt to synthesize visual features of unseen categories to address data gaps, but the generated features often fail to fully represent the true distribution within and between classes. This lack of diversity leads to biased feature representation, which weakens the generalization ability of the generated features. Single-attribute-driven generation methods struggle to capture the true distribution of unseen categories, limiting the model's ability to accurately represent and classify them. On the other hand, although generating replays is one of the mainstream methods to alleviate catastrophic forgetting, the quality of the replay data it generates will gradually decline as the number of tasks experienced by the model increases. This degradation not only affects the model's ability to retain knowledge from earlier tasks, but also reduces the effectiveness of the replay mechanism in long-term continuous learning. To address these challenges, this paper proposes a generation framework called AugGen, which includes diversified attribute enhancement (DAE) and cross-task feature distillation (TFD) branches to address the problems of insufficient diversity and catastrophic forgetting in the generation of unseen category features, thereby effectively addressing the CGZSL challenge. Summary of the Invention
[0004] To solve the above technical problems, the present invention proposes a continuous generalized zero-shot learning method and system based on a generative approach, which can solve the problems of insufficient diversity and catastrophic forgetting in generating features of unseen categories.
[0005] The present invention provides a continuous generalized zero-shot learning method based on a generative approach, comprising:
[0006] Get category attributes and Gaussian noise;
[0007] Based on the category attributes and Gaussian noise, a generator and a discriminator are combined to obtain pseudo features and identifier projections;
[0008] Inputting the pseudo features and identifier projections into a zero-shot learning model to obtain a learning result, wherein the zero-shot learning model includes: a diversified attribute enhancement module and a cross-task feature distillation module;
[0009] The diversified attribute enhancement module is used to enhance the expressive power of unseen category features by introducing diversified semantic descriptions generated by a large language model;
[0010] The cross-task feature distillation module is used to align the feature spaces of the old task and the new task, so that the model can retain existing knowledge while adapting to the new task.
[0011] Optionally, the diversified attribute enhancement module includes: a prompt attribute generation unit and an adaptive feature fusion unit;
[0012] The prompt attribute generation unit is used to obtain prompt attributes using a large language model;
[0013] The adaptive feature fusion unit is used to optimize the output features according to the original attributes, the hint attributes, the pseudo features and the identifier projection.
[0014] Optionally, based on the hint attributes, pseudo features, and identifier projection, output features are obtained including:
[0015] Input the prompt attribute and the original unseen category attribute into the generator for fusion to obtain fusion features;
[0016] Obtaining a normalized loss based on the fused features and the identified pseudo-projection;
[0017] The output features are optimized according to the normalized loss.
[0018] Optionally, the fusion feature is:
[0019] G(a p ,a)=g nor [1] * G(a p ,z)+g nor [2] * G(a,z)
[0020] Among them, G(a p ,a) is the fusion feature after fusion, g nor [] is the adaptive weight parameter, G(a p ,z) is the generated prompt feature, and G(a,z) is the generated original feature.
[0021] Optionally, the normalized loss includes:
[0022]
[0023] Among them, y i Indicates label, A t Represents the set of all attributes, is the high-dimensional mapping of the discriminator to the attribute, ce is the cross entropy function, is the fusion feature, a j is a known unseen category attribute For the prompt attribute, is the unseen category attribute training set, i is the index of all attributes, and j is the index of the unseen category attribute training set.
[0024] Optional, used to align the feature spaces of the old and new tasks so that the model can adapt to the new task while retaining the existing knowledge, including:
[0025] Get the discriminator of the current task and the frozen discriminator of the previous task;
[0026] The attribute space of the frozen discriminator is taken as a shared input, and the intermediate features of the discriminator and the frozen discriminator are aligned.
[0027] Optionally, a feature distillation loss function is constructed when aligning the discriminator and the frozen discriminator intermediate features, wherein the feature distillation loss function is:
[0028]
[0029] Among them, N is the feature dimension of the middle layer of the discriminator, h t (a i ) is the intermediate feature output of the current discriminator, h t-1 (a i ) is the intermediate feature of the frozen discriminator, || || 2 is the square of the l2 norm.
[0030] The present invention also provides a continuous generalized zero-shot learning system based on a generation method, comprising: a learning information acquisition module, an information processing module, and a learning result analysis module;
[0031] The learning information acquisition module is used to obtain category attributes and Gaussian noise;
[0032] The information processing module is used to obtain pseudo features and identifier projections based on the category attributes and Gaussian noise in combination with the generator and the discriminator;
[0033] The learning result analysis module is used to analyze the pseudo features and identifier projections using the diversified attribute enhancement module and the cross-task feature distillation module to obtain the learning results.
[0034] Compared with the prior art, the present invention has the following advantages and technical effects:
[0035] This paper addresses the issues of insufficient feature diversity and catastrophic forgetting in unseen categories by combining Diversified Attribute Enhancement (DAE) with Cross-Task Feature Distillation (TFD). The framework consists of two branches: the DAE branch utilizes hint attributes generated by LLMs and combines them with the original attributes via an adaptive feature fusion network to enhance feature diversity in unseen categories; the TFD branch utilizes an alignment mechanism between discriminators to preserve feature representations of old tasks during task transfer, mitigating catastrophic forgetting. This collaborative approach enables the model to generate more expressive features in incremental learning while maintaining cross-task stability and generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:
[0037] Figure 1 Flowchart of a continuous generalized zero-shot learning method based on a generative method according to an embodiment of the present invention;
[0038] Figure 2 This is a flow chart of a diversified attribute enhancement module according to an embodiment of the present invention;
[0039] Figure 3 4 is a flow chart of a cross-task feature distillation module according to an embodiment of the present invention. DETAILED DESCRIPTION
[0040] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0042] This embodiment proposes a continuous generalized zero-shot learning method based on a generative method, such as Figure 1 As shown, the specific steps include:
[0043] Get category attributes and Gaussian noise;
[0044] Based on the category attributes and Gaussian noise, a generator and a discriminator are combined to obtain pseudo features and identifier projections;
[0045] Inputting the pseudo features and identifier projections into a zero-shot learning model to obtain a learning result, wherein the zero-shot learning model includes: a diversified attribute enhancement module and a cross-task feature distillation module;
[0046] The diversified attribute enhancement module is used to enhance the expressive power of unseen category features by introducing diversified semantic descriptions generated by a large language model;
[0047] The cross-task feature distillation module is used to align the feature spaces of the old task and the new task, so that the model can retain existing knowledge while adapting to the new task.
[0048] Furthermore, the diversified attribute enhancement module includes: a prompt attribute generation unit and an adaptive feature fusion unit;
[0049] The prompt attribute generation unit is used to obtain prompt attributes using a large language model;
[0050] The adaptive feature fusion unit is used to optimize the output features according to the original attributes, the hint attributes, the pseudo features and the identifier projection.
[0051] Furthermore, according to the prompt attributes, pseudo features and identifier projection, the output features are obtained including:
[0052] Input the prompt attribute and the original unseen category attribute into the generator for fusion to obtain fusion features;
[0053] Obtaining a normalized loss based on the fused features and the identified pseudo-projection;
[0054] The output features are optimized according to the normalized loss.
[0055] Specifically, such as Figure 2 As shown, the Diversified Attribute Enhancement (DAE) branch contains the hint attribute generation and the adaptive feature fusion network.
[0056] Prompt attribute generation: In order to enrich the semantic information of unseen categories, this embodiment introduces large language models (LLMs) to generate a new set of attributes for unseen categories. Specifically, this embodiment uses Llama 3.0 to generate a new set of attributes for each unseen category, called prompt attributes, denoted as a p In this embodiment, each label attribute a of the unseen category;
[0057] Construct an LLM hint template: "This is the attribute of a class in a data set, please generate a similar attribute based on these attributes." To ensure that the hint attributes generated by LLMs maintain relevance to the original annotated attributes while maintaining a certain degree of difference and avoiding excessive deviation, this embodiment introduces Euclidean distance as an evaluation criterion. This embodiment uses the following formula to constrain the original attributes and hint attributes:
[0058]
[0059] Among them, a represents the original attribute, a p represents the generated hint attribute, and n is the dimension of the attribute. This example verifies the effectiveness of attribute diversity by performing controlled perturbations on the original attribute vector. When the attribute changes moderately (distance of approximately 0.4), the model performance is comparable to the original attribute, indicating that this degree of difference can introduce beneficial diversity while maintaining semantic relevance. Therefore, this example uses 0.4 as the reference threshold for generating new attributes to balance diversity and semantic consistency.
[0060] Adaptive feature fusion network: To enable the generator to generate more diverse features for unseen categories, this embodiment introduces an adaptive feature fusion network. This network can dynamically integrate features from multiple sources, improve the diversity of generated features, and ensure that these features maximize diversity and representativeness while maintaining semantic consistency during the fusion process. Specifically, this embodiment will prompt attribute a p Together with the original attribute a, it is fed into the generator G as input to generate features and introduce an adaptive weight parameter g nor , with dynamic balance to a p The final feature output is calculated as follows:
[0061] G(a p ,a)=g nor [1] * G(a p ,z)+g nor [2] * G(a,z)
[0062] Among them, G(a p ,a) is the fusion feature after fusion, g nor [] is the adaptive weight parameter, G(a p ,z) is the generated prompt feature, and G(a,z) is the generated original feature.
[0063] In order to achieve adaptive adjustment of weights, this embodiment uses the ReLU activation function and the normalization processing parameter gamma to obtain the normalized weight g nor . g nor The calculation of , where ∈ is a small constant, is as follows:
[0064]
[0065] In order to optimize the generated features and align them with the attribute projection results of the real category, this embodiment constructs a hybrid seen category normalization loss L msnl ,as follows:
[0066]
[0067] Among them, y i Indicates label, A t Represents the set of all attributes, is the high-dimensional mapping of the discriminator to the attribute, c e is the cross entropy function, is the fusion feature, a j is a known unseen category attribute For the prompt attribute, is the training set of unseen category attributes, i is the index of all attributes (unseen category attributes and seen category attribute sets), and j is the index of the training set of unseen category attributes. The known unseen category attribute a j With the prompt attribute Combine to generate features.
[0068] Furthermore, the feature spaces of the old and new tasks are aligned to allow the model to adapt to the new task while retaining the existing knowledge, including:
[0069] Get the discriminator of the current task and the frozen discriminator of the previous task;
[0070] The attribute space of the frozen discriminator is taken as a shared input, and the intermediate features of the discriminator and the frozen discriminator are aligned.
[0071] Furthermore, a feature distillation loss function is constructed when aligning the discriminator and the frozen discriminator intermediate features, wherein the feature distillation loss function is:
[0072]
[0073] Among them, N is the feature dimension of the middle layer of the discriminator, h t (a i ) is the intermediate feature output of the current discriminator, h t-1 (ai ) is the intermediate feature of the frozen discriminator, || || 2 is the square of the l2 norm.
[0074] Specifically, such as Figure 3 As shown, the cross-task feature distillation (TFD) branch: To alleviate the problem of catastrophic forgetting, this embodiment proposes a hybrid method that combines cross-task feature distillation and generative replay. Both techniques aim to help the model adapt to new tasks while retaining the learned knowledge. Specifically, let the current task task t The discriminator is D t , the previous task t-1 The frozen discriminator is D t-1 , this embodiment uses the attribute space of the old discriminator as a shared input and aligns the intermediate features of the two discriminators. Assume that the input is the attribute vector a i In this embodiment, the current discriminator D t The intermediate feature output h t (a i ) and the previous discriminator D t-1 The intermediate feature h t-1 (a i ) alignment. This alignment is crucial because it ensures that the model retains the feature representations learned by the previous task and prevents them from being overwritten by the new task. To maintain knowledge consistency between the old and new tasks, this example constructs a feature distillation loss function:
[0075]
[0076] Where N represents the feature dimension of the middle layer of the discriminator.
[0077] This embodiment also provides a continuous generalized zero-shot learning system based on a generative method, comprising: a learning information acquisition module, an information processing module, and a learning result analysis module;
[0078] The learning information acquisition module is used to obtain category attributes and Gaussian noise;
[0079] The information processing module is used to obtain pseudo features and identifier projections based on the category attributes and Gaussian noise in combination with the generator and the discriminator;
[0080] The learning result analysis module is used to analyze the pseudo features and identifier projections using the diversified attribute enhancement module and the cross-task feature distillation module to obtain the learning results.
[0081] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.
Claims
1. A continuous generalized zero-shot learning method based on a generative approach, characterized by: include: Get category attributes and Gaussian noise; Based on the category attributes and Gaussian noise, a generator and a discriminator are combined to obtain pseudo features and identifier projections; Inputting the pseudo features and identifier projections into a zero-shot learning model to obtain a learning result, wherein the zero-shot learning model includes: a diversified attribute enhancement module and a cross-task feature distillation module; The diversified attribute enhancement module is used to enhance the expressive power of unseen category features by introducing diversified semantic descriptions generated by a large language model; The cross-task feature distillation module is used to align the feature spaces of the old task and the new task, so that the model can retain existing knowledge while adapting to the new task.
2. The continuous generalized zero-shot learning method based on the generative method according to claim 1, characterized in that The diversified attribute enhancement module includes: a prompt attribute generation unit and an adaptive feature fusion unit; The prompt attribute generation unit is used to obtain prompt attributes using a large language model; The adaptive feature fusion unit is used to optimize the output features according to the original attributes, the hint attributes, the pseudo features and the identifier projection.
3. The continuous generalized zero-shot learning method based on the generative method according to claim 2, characterized in that Based on the hint attributes, pseudo features and identifier projection, the output features include: Input the prompt attribute and the original unseen category attribute into the generator for fusion to obtain fusion features; Obtaining a normalized loss based on the fused features and the identified pseudo-projection; The output features are optimized according to the normalized loss.
4. The continuous generalized zero-shot learning method based on the generative method according to claim 3, characterized in that The fusion features are: G(a p ,a)=g nor [1]*G(a p ,z)+g nor [2]*G(a,z) Among them, G(a p ,a) is the fusion feature after fusion, g nor [] is the adaptive weight parameter, G(a p ,z) is the generated prompt feature, and G(a,z) is the generated original feature.
5. The continuous generalized zero-shot learning method based on the generative method according to claim 3, characterized in that The normalized loss includes: Among them, y i Indicates label, A t Represents the set of all attributes, is the high-dimensional mapping of the discriminator to the attribute, c e is the cross entropy function, is the fusion feature, a j is a known unseen category attribute For the prompt attribute, is the unseen category attribute training set, i is the index of all attributes, and j is the index of the unseen category attribute training set.
6. The continuous generalized zero-shot learning method based on a generative approach according to claim 1, characterized in that It is used to align the feature spaces of old and new tasks so that the model can adapt to new tasks while retaining existing knowledge. Get the discriminator of the current task and the frozen discriminator of the previous task; The attribute space of the frozen discriminator is taken as a shared input, and the intermediate features of the discriminator and the frozen discriminator are aligned.
7. The method for continuous generalized zero-shot learning based on a generative approach according to claim 6, wherein: A feature distillation loss function is constructed when aligning the discriminator and the frozen discriminator intermediate features, wherein the feature distillation loss function is: Among them, N is the feature dimension of the middle layer of the discriminator, h t (a i ) is the intermediate feature output of the current discriminator, h t-1 (a i ) is the intermediate feature of the frozen discriminator, |||| 2 is the square of the l2 norm.
8. A continuous generalized zero-shot learning system based on a generative approach, characterized by: include: Learning information acquisition module, information processing module and learning result analysis module; The learning information acquisition module is used to obtain category attributes and Gaussian noise; The information processing module is used to obtain pseudo features and identifier projections based on the category attributes and Gaussian noise in combination with the generator and the discriminator; The learning result analysis module is used to analyze the pseudo features and identifier projections using the diversified attribute enhancement module and the cross-task feature distillation module to obtain the learning results.