Online continuous learning image classification method and system based on influence-guided memory training

This online continuous learning method, which uses influence-guided memory training, solves the catastrophic forgetting problem of deep learning models in image classification, achieving stability and accuracy in online data streams and is suitable for image classification in dynamic environments.

CN121837822APending Publication Date: 2026-04-10TONGJI UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-30
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

Existing deep learning models face the catastrophic forgetting problem when performing image classification, making it difficult to maintain a balance between remembering old knowledge and learning new knowledge in online data streams. Furthermore, existing methods have low generalization ability and cannot be applied to online data stream training.

Method used

We adopt an online continuous learning method based on influence-guided memory training. Through the flow-to-memory training mechanism, influence-guided feature constraint mechanism, and joint training optimization mechanism, we update model parameters and category representations, avoid feature drift, and ensure the stability and accuracy of the model in a multi-task environment.

Benefits of technology

It achieves the ability to maintain model stability and learn new knowledge in dynamic environments, is applicable to any model structure, is suitable for online data stream training, and improves the accuracy and generalization of image classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121837822A_ABST
    Figure CN121837822A_ABST
Patent Text Reader

Abstract

The invention relates to an online continuous learning image classification method and system based on influence-guided memory training, and the method comprises the steps: inputting a to-be-classified image into an image classification model, and outputting a classification result; wherein the image model is trained based on an online updated sample data set, and comprises the following steps: when the sample data set is updated, updating a memory buffer area in the image classification model based on new sample data; training an image classification model by using the sample data in the updated memory buffer area, and calculating training loss and comparison loss; updating model parameters of the image classification model based on the training loss and the comparison loss to obtain a new image classification model; the system provided by the invention is used for implementing the method. Compared with the prior art, the continuous learning method provided by the invention can be suitable for any model structure and can be suitable for online data stream training, and the problem of disastrous forgetting occurring when a deep learning model performs continuous learning image classification is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image classification, in particular to an online continuous learning image classification method and system based on influence-guided memory training. BACKGROUND

[0002] Image classification is a key computer vision task, widely applied in face recognition, medical image analysis, traffic scene understanding, intelligent security monitoring, and automatic label generation. With the rapid development of deep learning technology, image classification models based on deep neural networks have shown high expression ability and superior classification performance in processing large-scale complex datasets. Such models rely on massive data training and can automatically extract multi-level feature representations, thus making breakthroughs in various visual tasks. However, traditional image classification models are usually built based on "offline training" method, and once faced with new classes or new data distribution, the model often needs to be retrained entirely or partially. This not only causes repeated computation and resource waste, but also makes it difficult to adapt to the continuously changing visual scenes in real environments.

[0003] To overcome the above limitations, image classification models that can continuously receive new data while maintaining existing knowledge have gradually attracted attention, and such models are also known as continuous learning models. In the continuous learning framework, the model can expand its capabilities by continuously receiving new information, while maintaining its memory of previous knowledge, just like humans. This ability makes continuous learning technology valuable in dynamic environments such as autonomous driving, video monitoring, and online robot navigation. Deep neural networks have strong learning ability, but when processing data streams that change over time, they exhibit a phenomenon known as "catastrophic forgetting", where the model's performance on previous knowledge significantly decreases after learning new classes or tasks, reflecting the dilemma of deep models in balancing "stability" and "plasticity". To alleviate this problem, existing methods have proposed various continuous learning methods, including importance constraints based on parameters, model structure expansion, and historical sample replay. For example, Chinese patent application CN114463605A provides a continuous learning image classification method based on deep learning, which solves the forgetting problem of deep learning models in continuous learning by using task-specific batch normalization layers and fixed convolution kernel parameters in the classification head of the deep learning model, achieving knowledge retention and learning ability for new tasks. However, the above method improves the structure of the deep learning model itself, which has low generalizability. In addition, since it trains in units of classification tasks, it needs to complete the first stage of convolution kernel pre-training before adding BN layers and classification heads layer by layer. In this process, the task boundary is clear and the order is fixed, which is not suitable for continuous and unbounded scenarios based on online data streams.

[0004] Therefore, it is a technical problem to be solved to provide a continuous learning method applicable to any model structure and online data stream training to facilitate more accurate image classification. SUMMARY

[0005] The present application aims to overcome the defects of the prior art and provides an online continuous learning image classification method and system based on influence guided memory training, which starts from three aspects of model training distribution, feature space structure and new and old knowledge interference, redesigns the training method and internal representation constraint mechanism, enables the model to update all network parameters and class representation at different stages, respectively corrects the prediction deviation of the feature extraction layer and the classification layer in the multi-task environment, avoids the catastrophic forgetting problem of the deep learning model during image classification training, and ensures that the trained model can achieve accurate image classification.

[0006] The object of the present application can be achieved by the following technical solutions: According to a first aspect of the present application, an online continuous learning image classification method based on influence guided memory training is provided, which comprises: inputting an image to be classified into an image classification model and outputting a classification result; wherein the image model is trained based on an online updated sample data set, comprising: When the sample data set is updated, update the memory buffer in the image classification model based on the new sample data; Train the image classification model using the sample data in the updated memory buffer and calculate the training loss; and in the feature space of the image classification model, construct a class agent vector for each sample data, calculate the similarity between each sample data in the updated memory buffer and the corresponding class agent vector, introduce an influence guided mask, calculate the updated similarity based on the influence guided mask, and calculate the contrast loss based on the updated similarity; Update the model parameters of the image classification model based on the training loss and the contrast loss to obtain a new image classification model.

[0007] As a preferred technical solution, the method for calculating the similarity is: , wherein, represents a learnable class agent vector of class c, and for this agent vector, it is initialized as a random value of normal distribution, with a total number of classes of feature dimensions; represents any sample data; represents feature network processing; represents a temperature coefficient; represents sample data a similarity with the class agent vector of class c; denotes a two-norm calculation.

[0008] As a preferred technical solution, the method for calculating the updated similarity is: , wherein, denotes an updated similarity; denotes sample data a similarity with the class agent vector of class c; denotes an inter-class boundary enhancement coefficient; denotes a class index; denotes sample data a corresponding sample label; denotes a class judgment operation, and when the sample label and the class index represent the same class, the value is 1, otherwise the value is 0; denotes an influence guide mask, and has: , denotes a task class set at time t; denotes all task sets from the training starting time to time t.

[0009] As a preferred technical solution, the method for calculating the training loss is: , wherein, denotes a cross-entropy loss calculation; denotes a predicted value of the image classification model at time t for sample data in the memory buffer; denotes a parameter value of the image classification model at time t; denotes sample data in a distribution; denotes an updated memory buffer; denotes sample data a corresponding sample label.

[0010] As a preferred technical solution, the method for calculating the contrast loss is: , wherein, denotes sample data in a distribution; denotes a data stream corresponding to a classification task; denotes sample data an update similarity of a class agent vector corresponding to the sample label an update similarity of a class agent vector corresponding to the sample label representing an influence guiding mask representing sample data an update similarity of a class agent vector corresponding to the sample label representing a class index representing a total number of classes.

[0011] According to a second aspect of the present application, there is provided an online continual learning image classification system based on influence guiding memory training, comprising a data collection module, a model training module and an image classification module. The data collection module is configured to collect sample data and images to be classified. The image classification module uses the trained image classification model output by the model training module to process the images to be classified and outputs a classification result. The model training module trains the image classification model based on an online updated sample data set by performing the following steps: When the sample data set is updated, update the memory buffer in the image classification model based on the new sample data; Use the sample data in the updated memory buffer to train the image classification model and calculate the training loss; construct a class agent vector for each class of sample data in the feature space of the image classification model, calculate the similarity between each sample data in the updated memory buffer and the corresponding class agent vector, introduce an influence guiding mask, calculate the update similarity based on the influence guiding mask, calculate the contrast loss based on the update similarity; Update the model parameters of the image classification model based on the training loss and the contrast loss to obtain a new image classification model.

[0012] Compared with the prior art, the present application has the following beneficial effects: 1) The application improves the traditional continuous learning method from the perspective of training distribution, and provides a flow-to-memory training mechanism. The mechanism no longer mixes the current data stream with historical samples, but only uses the data in the memory buffer for training. By avoiding the target shift caused by the distribution difference between new and old data, the model training process maintains a more consistent distribution, thereby reducing the prediction bias caused by data changes. At the same time, as the task progresses, the memory buffer gradually accumulates more representative samples, and the overall structure can naturally approximate the real long-term data characteristics, making the model training more stable. Unlike existing methods that improve the structure of deep learning models to avoid catastrophic forgetting, the technical solution provided by the application improves the learning method of the model, which can be applied to any model structure and has strong generalization. In addition, the method provided by the application is based on online updating of the sample database for model training. Compared with the traditional method of training the model based on a fixed training set, the application can be applied to online scenarios with dynamic data changes.

[0013] 2) The application provides an influence-guided feature comparison mechanism. The mechanism sets an independent learnable class center for each class to express the representative position of the class in the feature space. Specifically, when receiving new data, the model will automatically generate a class influence mask according to the current class, allowing only the current class and potential new classes to affect the feature space, thereby limiting the interference of historical classes. The application also introduces an inter-class boundary enhancement coefficient to increase the difficulty of distinguishing non-target classes, so that the model can actively expand the feature difference between different classes when learning new classes, thereby improving the expression ability and ensuring that more detailed image classification can be learned, thereby achieving more accurate image classification.

[0014] 3) The application proposes a joint training optimization mechanism that combines memory-based classification loss and influence-guided feature comparison loss. By weighting the two types of loss, the model can maintain the stability of old knowledge while still having the ability to absorb new knowledge, thereby achieving balanced learning of new and old tasks, effectively reducing feature drift, and maintaining the distribution stability of the class structure, fundamentally solving the long-term forgetting problem in continuous learning. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The training flowchart of the image classification model of the application; Figure 2 The training schematic diagram of the image classification model of the application. DETAILED DESCRIPTION

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] Existing continuous learning methods suffer from the following problems: First, because the model uses samples from both the current data stream and historical memory during training, and these two sources are often statistically inconsistent, the model's training objective deviates from the ideal data distribution, further introducing prediction bias. Second, as the task sequence grows, the model continuously updates its parameters under a mixture of old and new samples, easily leading to drift in the representation space and blurring the boundaries of old classes. Third, current methods typically focus on achieving inter-class balance through sample replay, but pay less attention to changes in the model's internal representation (especially the class center layout in the feature space) during task switching, which may further exacerbate model instability. Finally, existing methods often compromise model stability while emphasizing model plasticity, and conversely, reduce adaptability to new classes while emphasizing stability, making it difficult to achieve an effective balance between the two.

[0018] To address the aforementioned technical problems, this invention provides an online continuous learning image classification method based on influence-guided memory training, applicable to continuous learning environments such as dynamic image streams, autonomous driving vision systems, and online monitoring scenarios. This method inputs the image to be classified into an image classification model and outputs the classification result. The image model used in this invention is trained on an online-updated sample dataset. Based on this, this invention provides a training method for an image classification model, the process of which is as follows: Figure 1 As shown, its framework is as follows Figure 2 As shown, it includes two main mechanisms: a flow-to-memory training mechanism and an influence-guided feature constraint mechanism, and requires the following to be implemented: 1) Stream to memory training mechanism: After receiving new samples, this mechanism only uses the data in the memory buffer for training, avoiding the training distribution from changing due to the mixing of new and old samples, and maintaining the stability of model updates.

[0019] 2) Influence-guided feature constraint mechanism: This mechanism constructs a category center for each category and generates an influence mask based on the current task category to control which categories can participate in model updates, thereby reducing unnecessary inter-class interference. In addition, this mechanism can also increase the difficulty for non-target categories, guiding the model to form more obvious differentiation in the feature space.

[0020] 3) Joint training optimization mechanism: This mechanism integrates the memory-based classification process with the structural constraints of feature comparison, so that the model can remain stable in long-term task sequences, while improving the learning speed and classification performance of new categories.

[0021] Specifically, it includes the following steps: S1. When the sample dataset is updated, update the memory buffer in the image classification model based on the new sample data.

[0022] After receiving new data, the memory buffer is updated to preserve representative samples.

[0023] When the system receives a task When processing data streams, a pooled sampling strategy is used to select a subset of samples from the data stream and add them to a memory buffer. This ensures that the buffer always maintains long-term representativeness; in detail, representativeness refers to the representativeness of the data distribution, ensuring that the buffer pool always contains data from the full distribution, thus maintaining long-term representativeness of the entire data stream. If the buffer does not contain new data, it will deviate from the data stream distribution and become unrepresentative.

[0024] S2. Train the image classification model using the sample data in the updated memory buffer.

[0025] This step consists of two parts: calculating the training loss and calculating the contrastive loss.

[0026] i) Calculate the training loss.

[0027] In traditional experience replay, samples from the current data stream and the memory buffer are mixed for training, causing the training distribution to oscillate wildly over time. This invention uses only the memory buffer. The model is updated using samples from the dataset to maintain a consistent training distribution over time, making the image classification model update more stable. The training loss is defined by the SMT loss and is calculated as follows: , in, This indicates the calculation of cross-entropy loss; express The image classification model at each time step uses the sample data in the memory buffer. The predicted value; express The parameter values ​​of the image classification model at each time step; Representing sample data exist Expectation on the distribution; This represents the updated memory buffer; Representing sample data The corresponding sample labels.

[0028] By introducing training loss, the bias caused by the inconsistency between the data flow and the distribution of the memorized samples is avoided, thus keeping the feature space stable in time.

[0029] ii) Calculate the contrast loss.

[0030] In the feature space of the image classification model, a class surrogate vector is constructed for each class of sample data. This surrogate vector is initialized with normally distributed random values, and from... Figure 2 As can be seen, the shape of the category proxy vector is the total number of categories in the feature dimension.

[0031] The similarity between each sample data in the updated memory buffer and the corresponding category proxy vector is calculated. By calculating the similarity, each category has a clear central representation, thus providing a structural basis for subsequent category differentiation. The calculation method is as follows: , in, Let represent the learnable class proxy vector of class c, and for ; Represents arbitrary sample data; Indicates feature network processing; Indicates the temperature coefficient; Representing sample data Similarity to the class proxy vector of class c; This indicates the calculation of the L2 norm.

[0032] To prevent the model from disrupting historical categories when learning new tasks, an influence guide mask is introduced to control the participation of each category in the optimization process. The updated similarity is calculated based on the influence guide mask, and the method for calculating the updated similarity is as follows: , in, This indicates an update to the similarity score; Representing sample data Similarity to the class proxy vector of class c; This represents the inter-class boundary enhancement coefficient, which increases the difficulty of distinguishing non-target classes, makes feature centers more separated, and avoids class overlap. That is, the larger the value of m is for non-target classes; Indicates a category index; Representing sample data Corresponding sample labels; This indicates a category determination operation. The value is 1 when the sample label and category index indicate that they belong to the same category, and 0 otherwise. This represents an influence-guiding mask, whose purpose is to ensure that the model primarily updates categories relevant to the current task, while historical categories are protected, reducing catastrophic forgetting, and it also has the following characteristics: , This represents the set of task categories at time t; This represents the set of all tasks from the start of training to time t.

[0033] To enhance inter-class differentiation, a moderate level of difficulty is imposed on the current sample to distinguish it from other categories. To strengthen the structural boundaries between categories, this invention calculates a contrastive loss based on updated similarity, expressed as: , in, Representing sample data exist Expectation on the distribution; This represents the data stream corresponding to the classification task; Representing sample data Corresponding sample labels Update the similarity of the category proxy vectors; Indicates an influence-guiding mask; Representing sample data Update the similarity with the category proxy vector of category c; Indicates a category index; This indicates the total number of categories.

[0034] Contrastive loss enhances the relative score of the target class and suppresses the score of non-target classes, enabling the model to maintain the stability of the old class representation when new tasks arrive. At the same time, the margin mechanism makes the features of the new class more prominent.

[0035] S3. Update the model parameters of the image classification model based on the training loss and contrastive loss to obtain a new image classification model.

[0036] Specifically, in this step, the overall loss function is constructed by jointly using the training loss and the contrastive loss, as follows: , in, This represents the balance coefficient, used to adjust the relationship between stability and plasticity, so that the model can both retain historical knowledge and quickly absorb structural information of new categories.

[0037] Once the image classification model has completed the above steps on multiple task sequences, it forms a classification system with a stable feature space structure and clear category boundaries. At this point, any sample to be classified is input into the trained image classification model, which performs single-head classification based on the category surrogate vector and structured similarity formed through long-term learning, thereby obtaining the final classification result.

[0038] The model training method provided by this invention has a simple overall process, low computational overhead, and can be deployed on edge computing devices, making it highly practical and feasible in engineering.

[0039] Furthermore, this embodiment also provides an online continuous learning image classification system based on influence-guided memory training, including a data acquisition module, a model training module, and an image classification module. Specifically, the data acquisition module is used to collect sample data and images to be classified; the image classification module uses the trained image classification model output by the model training module to process the images to be classified and outputs classification results; the model training module trains the image classification model based on the online updated sample dataset by performing the following steps: S1. When the sample dataset is updated, update the memory buffer in the image classification model based on the new sample data.

[0040] S2. Train the image classification model using the sample data in the updated memory buffer and calculate the training loss. In the feature space of the image classification model, construct a category proxy vector for each class of sample data and calculate the similarity between each sample data in the updated memory buffer and the corresponding category proxy vector. Introduce an influence guide mask, calculate the updated similarity based on the influence guide mask, and calculate the contrast loss based on the updated similarity.

[0041] S3. Update the model parameters of the image classification model based on the training loss and contrastive loss to obtain a new image classification model.

[0042] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0043] Furthermore, the present invention provides an electronic device including a central processing unit (CPU) that can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0044] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0045] The processing unit executes the various methods and processes described above, such as methods S1 to S3. For example, in some embodiments, methods S1 to S3 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S1 to S3 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S3 by any other suitable means (e.g., by means of firmware).

[0046] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0047] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0048] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0049] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An online continuous learning image classification method based on influence-guided memory training, characterized in that, The method involves inputting the image to be classified into the image classification model and outputting the classification result. The image model described therein is trained on an online-updated sample dataset and includes: When the sample dataset is updated, the memory buffer in the image classification model is updated based on the new sample data. The image classification model is trained using sample data in the updated memory buffer, and the training loss is calculated. In the feature space of the image classification model, a category proxy vector is constructed for each class of sample data, and the similarity between each sample data in the updated memory buffer and the corresponding category proxy vector is calculated. An influence guide mask is introduced, and the updated similarity is calculated based on the influence guide mask. The contrast loss is calculated based on the updated similarity. The model parameters of the image classification model are updated based on the training loss and contrastive loss to obtain a new image classification model.

2. The online continuous learning image classification method based on influence-guided memory training according to claim 1, characterized in that, The method for calculating the aforementioned similarity is as follows: , in, Let represent the learnable class surrogate vector of class c, and initialize it with a random value of a normal distribution, with a shape equal to the total number of classes in the feature dimension; Represents arbitrary sample data; Indicates feature network processing; Indicates the temperature coefficient; Representing sample data Similarity to the class proxy vector of class c; This indicates the calculation of the L2 norm.

3. The online continuous learning image classification method based on influence-guided memory training according to claim 1, characterized in that, The method for calculating the updated similarity is as follows: , in, This indicates an update to the similarity score; Representing sample data Similarity to the class proxy vector of class c; Indicates the inter-class boundary enhancement coefficient; Indicates a category index; Representing sample data Corresponding sample labels; This indicates a category determination operation. The value is 1 when the sample label and category index indicate that they belong to the same category, and 0 otherwise. This represents an influence-guiding mask, and includes: , This represents the set of task categories at time t; This represents the set of all tasks from the start of training to time t.

4. The online continuous learning image classification method based on influence-guided memory training according to claim 1, characterized in that, The method for calculating the training loss is as follows: , in, This indicates the calculation of cross-entropy loss; express The image classification model at each time step uses the sample data in the memory buffer. The predicted value; express The parameter values ​​of the image classification model at each time step; Representing sample data exist Expectation on the distribution; This represents the updated memory buffer; Representing sample data The corresponding sample labels.

5. The online continuous learning image classification method based on influence-guided memory training according to claim 1, characterized in that, The method for calculating the contrast loss is as follows: , in, Representing sample data exist Expectation on the distribution; This represents the data stream corresponding to the classification task; Representing sample data Corresponding sample labels Update the similarity of the category proxy vectors; Indicates an influence-guiding mask; Representing sample data Update the similarity with the category proxy vector of category c; Indicates a category index; This indicates the total number of categories.

6. An online continuous learning image classification system based on influence-guided memory training, characterized in that, The system includes a data acquisition module, a model training module, and an image classification module; The data acquisition module is used to acquire sample data and images to be classified; The image classification module uses the trained image classification model output by the model training module to process the image to be classified and output the classification result. The model training module, based on an online-updated sample dataset, performs the following steps to train the image classification model: When the sample dataset is updated, the memory buffer in the image classification model is updated based on the new sample data. The image classification model is trained using sample data in the updated memory buffer, and the training loss is calculated. In the feature space of the image classification model, a category proxy vector is constructed for each class of sample data, and the similarity between each sample data in the updated memory buffer and the corresponding category proxy vector is calculated. An influence guide mask is introduced, and the updated similarity is calculated based on the influence guide mask. The contrast loss is calculated based on the updated similarity. The model parameters of the image classification model are updated based on the training loss and contrastive loss to obtain a new image classification model.

7. The online continuous learning image classification system based on influence-guided memory training according to claim 6, characterized in that, In the model training module, the following steps are performed to calculate similarity: , in, Let represent the learnable class surrogate vector of class c, and initialize it with a random value of a normal distribution, with a shape equal to the total number of classes in the feature dimension; Represents arbitrary sample data; Indicates feature network processing; Indicates the temperature coefficient; Representing sample data Similarity to the class proxy vector of class c; This indicates the calculation of the L2 norm.

8. The online continuous learning image classification system based on influence-guided memory training according to claim 6, characterized in that, In the model training module, the following steps are performed to calculate the updated similarity: , in, This indicates an update to the similarity score; Representing sample data Similarity to the class proxy vector of class c; Indicates the inter-class boundary enhancement coefficient; Indicates a category index; Representing sample data Corresponding sample labels; This indicates a category determination operation. The value is 1 when the sample label and category index indicate that they belong to the same category, and 0 otherwise. This represents an influence-guiding mask, and includes: , This represents the set of task categories at time t; This represents the set of all tasks from the start of training to time t.

9. The online continuous learning image classification system based on influence-guided memory training according to claim 6, characterized in that, In the model training module, the following steps are performed to calculate the training loss: , in, This indicates the calculation of cross-entropy loss; express The image classification model at each time step uses the sample data in the memory buffer. The predicted value; express The parameter values ​​of the image classification model at each time step; Representing sample data exist Expectation on the distribution; This represents the updated memory buffer; Representing sample data The corresponding sample labels.

10. The online continuous learning image classification system based on influence-guided memory training according to claim 6, characterized in that, In the model training module, the following steps are performed to calculate the contrastive loss: , in, Representing sample data exist Expectation on the distribution; This represents the data stream corresponding to the classification task; Representing sample data Corresponding sample labels Update the similarity of the category proxy vectors; Indicates an influence-guiding mask; Representing sample data Update the similarity with the category proxy vector of category c; Indicates a category index; This indicates the total number of categories.

Citation Information

Patent Citations

  • Continuous learning image classification method and device based on deep learning

    CN114463605A