Online learning method and system based on incremental learning
Through the incremental learning method, the teacher model is used to guide the training data division and feature map similarity calculation of the student model, and the rapid iterative update of the deep learning model is realized, which not only maintains the performance of old data, but also improves the performance of new data.
Patent Information
- Application Number
- CN202510513288.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2025-08-08
AI Technical Summary
Existing deep learning models are difficult to update quickly, and they must not only improve performance on new data, but also maintain performance on old data.
The online learning method based on incremental learning is adopted to achieve rapid update and performance maintenance of student models through parameter loading of teacher models and student models, division and mixing of training data, feature map similarity calculation and reverse gradient update.
Student models can be updated quickly, which not only maintains good performance on old data, but also improves performance on new data.
Smart Images
Figure CN120448808A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a method for optimizing a deep learning model. Background Art
[0002] Large models are often single complex networks or collections of networks, boasting excellent performance and generalization capabilities. Small models, however, have limited expressive power due to their smaller network size. Knowledge learned from the large model can be leveraged to guide the training of the smaller model, enabling comparable performance with significantly fewer parameters. This allows for model compression and acceleration. This is how knowledge distillation and transfer learning are applied to model optimization.
[0003] The teacher-student training paradigm is a knowledge distillation technique in deep learning. It involves a fully trained and well-performing model (the teacher model) assisting in the training of another, yet-to-be-trained or simpler model (the student model). The complex and large teacher model has strong learning capabilities and can transfer its learned knowledge to the relatively simpler and smaller student model, thereby enhancing the student model's generalization capabilities. The teacher model is not deployed online, serving only as a mentor. It is the flexible and lightweight student model that is deployed online to perform tasks.
[0004] Existing deep learning models typically use online learning algorithms to learn new knowledge, but they have difficulty achieving fast responses. Summary of the Invention
[0005] The technical problem to be solved by this application is: how to enable the deep learning model to be quickly iterated and updated, not only to improve its performance on new data, but also to maintain its performance on old data.
[0006] To address the above technical issues, this application proposes an online learning method based on incremental learning, comprising the following steps. Step S1: Two deep learning models of different sizes are used, with the larger deep learning model being called the teacher model and the smaller deep learning model being called the student model. The teacher model is trained using a training dataset, and the training data in the training dataset is divided into two parts based on different training results: training data that performs well on the teacher model is called old data, and training data that performs poorly on the teacher model is called new data. Step S2: Model parameters are loaded on each of the teacher and student models; the teacher model is in the inference state, and the student model is in the training state. Step S3: The new and old data are mixed in a certain proportion to obtain a batch of training data, and multiple batches of training data are constructed. Step S4: The student model is trained using the batch of training data. The student model is trained on the old and new data in the batch of training data to obtain a loss value. Step S5: The student model is trained for multiple rounds using multiple batches of training data. The loss value of each batch of training data is then reversely updated based on the student model's loss value, resulting in a student model with updated parameters.
[0007] Furthermore, in step S1, the new data further includes any one or more of the following: training data already in the training data set and provided by user feedback; and newly added training data not in the training data set.
[0008] Furthermore, in step S2, the model parameters of the teacher model are obtained through the training of step S1; the model parameters of the student model are default model parameters or preset model parameters.
[0009] Furthermore, in step S3, in each batch of training data, the amount of new data is less than the amount of old data.
[0010] Preferably, in step S3, in each batch of training data, the ratio of the quantity of new data to the quantity of old data is 1:3.
[0011] Furthermore, in step S4, the teacher model obtains a set of feature maps for the old data in a batch of training data, and the student model obtains another set of feature maps for the old data in the same batch of training data; and the similarity between the feature maps of the teacher model for the old data and the feature maps of the student model for the old data is calculated.
[0012] Furthermore, in step S4, the mean square error loss between the feature map of the teacher model for the old data and the feature map of the student model for the old data is calculated; the larger the mean square error loss value, the more dissimilar the two sets of feature maps are; the smaller the mean square error loss value, the more similar the two sets of feature maps are.
[0013] Furthermore, in step S4, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in the subspace of the Grassmann manifold, and the projection similarity of the two points is calculated; the greater the projection similarity, the more similar the two points are; the smaller the projection similarity, the less similar the two points are.
[0014] Furthermore, in step S4, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in a subspace of the Grassmann manifold, and the shortest path on the geodesic flow of the two points is calculated. The shorter the shortest path on the geodesic flow, the more similar the two points are; the shorter the shortest path on the geodesic flow, the less similar the two points are.
[0015] The present application also proposes an online learning system based on incremental learning, including a new and old data partitioning unit, a parameter loading unit, a training data construction unit, an old data performance transfer unit, a new data training unit, and a student model updating unit. The new and old data partitioning unit is used to adopt two deep learning models of different scales, the larger deep learning model is called a teacher model, and the smaller deep learning model is called a student model; a training data set is used to train the teacher model, and the training data in the training data set is divided into two parts according to different training effects - the training data with better training effects on the teacher model is called old data, and the training data with poor training effects on the teacher model is called new data. The parameter loading unit is used to allow the teacher model and the student model to load model parameters respectively; the teacher model is in the inference state, and the student model is in the training state. The training data construction unit is used to mix the new data and the old data in a certain proportion to obtain a batch of training data, and construct multiple batches of training data. The training unit is used to train the student model using a batch of training data, and the student model is trained on the old data and new data in a batch of training data to obtain a loss value. The student model updating unit is used to perform reverse gradient update on the loss value of each batch of training data according to the student model during multiple rounds of training of the student model using multiple batches of training data to obtain a student model with updated parameters.
[0016] The technical effect achieved by this application is: allowing the student model to inherit the good performance of the teacher model for old data, allowing the student model to be trained separately for new data (i.e., incremental learning), achieving rapid updating of the student model, and allowing the student model to meet the performance requirements of both parts of data at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flowchart of the online learning method based on incremental learning proposed in this application.
[0018] Figure 2 It is a structural diagram of the online learning system based on incremental learning proposed in this application.
[0019] Explanation of the reference numerals in the figure: new and old data division unit 1, parameter loading unit 2, training data construction unit 3, training unit 4, student model updating unit 5. DETAILED DESCRIPTION
[0020] See also Figure 1 ,The online learning method based on incremental learning proposed in this application includes the ,following steps.
[0021] Step S1: Two deep learning models of different scales are used. The larger deep learning model is called the teacher model, and the smaller deep learning model is called the student model. A training data set is used to train the teacher model, and the training data in the training data set is divided into two parts according to different training effects - the training data (also called training samples) with better training effects on the teacher model are called old data (also called good samples), and the training data with poor training effects on the teacher model are called new data (also called bad samples, difficult samples). Therefore, the teacher model only performs well on the old data, and does not perform well on the new data. Optionally, the new data also includes training data that is already in the training data set and is fed back by users, such as difficult samples identified by users. Optionally, the new data also includes newly added training data that is not in the training data set.
[0022] Step S2: The teacher model and student model each load their original model parameters. The teacher model's model parameters are, for example, obtained through training in step S1. The student model's model parameters are, for example, default or preset model parameters. The teacher model is in the inference state, and the student model is in the training state.
[0023] The teacher model only performs well on old data. This application uses the teacher model as a stable reference, providing knowledge from old data to the student model without participating in the training process. Therefore, the teacher model is in the inference state. The student model needs to learn new knowledge while maintaining its understanding of old knowledge, so it needs to be in the training state.
[0024] Step S3: Mix the new data and the old data in a certain ratio to obtain a batch of training data. Multiple batches of training data are constructed, and the student model is trained once per batch of training data. It is generally required that the amount of new data in each batch of training data is less than the amount of old data. For example, when the number of training data in a batch is set to 8, the amount of old data is 6 and the amount of new data is 2. At this time, the ratio of new data to old data is 1:3.
[0025] Step S4: The student model is trained using a batch of training data. The student model is trained on the old and new data in the batch of training data to obtain a loss value. That is, the loss value is calculated between all the data in the same batch of training data and the ground truth. The ground truth refers to the label in the training data.
[0026] In this step, the teacher model generates a set of feature maps for the old data in a certain batch of training data, and the student model generates another set of feature maps for the old data in the same batch of training data. A model's feature map refers to the feature vector extracted from the input data by the model's convolutional layer and represents the model's feature representation of the input data. The quality of the feature map directly reflects the model's understanding of the input data. The similarity between the teacher model's feature map for the old data and the student model's feature map for the old data is calculated. By calculating the similarity, the student model is guided to learn the teacher model's feature extraction and representation methods for the old data, thereby acquiring capabilities similar to the teacher model. This step uses knowledge distillation technology to transfer the teacher model's performance on old data (good samples) to the student model, allowing the student model to maintain relatively good performance on old data (good samples).
[0027] As a first example, this step calculates the mean square error (MSE) between the feature maps of the teacher model for the old data and the feature maps of the student model for the old data. The larger the MSE loss value, the less similar the two sets of feature maps are. The smaller the MSE loss value, the more similar the two sets of feature maps are. In traditional knowledge distillation methods, the loss function is usually based on the Euclidean distance of the features (such as mean square error or cosine similarity), which directly constrains the features of the teacher model and the student model. The first example belongs to this category.
[0028] As a second example, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in the subspace of the Grassmann manifold. The Grassmann manifold is a set of linear subspaces of all specific dimensions. Its geometric structure can well describe the changes in high-dimensional feature spaces and provide a tool to represent and compare different feature subspaces. After mapping the two sets of feature maps to two points in the subspace of the Grassmann manifold, the projection similarity of the two points is calculated. The projection similarity is calculated directly through the projection matrix of the two points. The greater the projection similarity (the closer to 1), the more similar the two points are. The smaller the projection similarity, the less similar the two points are. The second example gives another mathematical method to measure the similarity of two sets of feature maps in order to capture the geometric relationship between the feature spaces of the teacher model and the student model.
[0029] As a third example, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in the subspace of the Grassmann manifold. After mapping the two sets of feature maps to two points in the subspace of the Grassmann manifold, the shortest path on the geodesic flow of these two points is calculated. The geodesic on the manifold can capture the angle change or the gradual change of the geometric structure between points. The core idea of the geodesic flow is to model the change between two points as a gradual transition process, rather than directly comparing point to point. The geodesic distance is the length of the shortest path between two points, reflecting their geometric differences on the manifold. The shorter the shortest path on the geodesic flow, the more similar the two points are. The shorter the shortest path on the geodesic flow, the less similar the two points are. The third example gives another mathematical method to measure the similarity of two sets of feature maps, in order to capture the geometric relationship and dynamic changes between the feature spaces of the teacher model and the student model.
[0030] Projection similarity is a static metric that directly compares the geometric positions of two points in a subspace of the Grassmann manifold. The result is a scalar that represents the degree of similarity between the two points. Projection similarity is a direct similarity metric used to determine whether two points (in feature space) are "close" or "similar." It typically does not involve the paths between the points, but rather statically compares their geometric relationships.
[0031] Geodesic flow is a dynamic analysis method that emphasizes the gradual path between points and can capture the process of feature space flowing from one state to another. Geodesic flow can not only be used to compare the geometric relationship between two points, but also capture the dynamic changes between points, describing how they gradually transition from one state to another.
[0032] Comparing the three examples above, the first example uses the traditional mean squared error loss, which only considers point-to-point feature differences without considering the overall geometric structure of the feature space. The second and third examples introduce manifold geometry information to enhance the loss calculation, achieving better loss calculation results than the first example. By mapping the feature maps onto the Grassmann manifold, the second and third examples can capture the smooth transition between the feature subspaces of the teacher and student models, and incorporate this transition information into the loss calculation, thereby enhancing the student model's ability to retain old data.
[0033] Step S5: Repeat step S4 multiple times, each time using a new batch of training data. This means the student model is trained for multiple epochs using multiple batches of training data. A reverse gradient update is performed on the loss of each batch of training data based on the student model. This reverse gradient update is equivalent to updating the parameters of the student model, resulting in a parameter-updated student model that balances the performance of both the old and new data.
[0034] See also Figure 2 The online learning system based on incremental learning proposed in this application includes a new and old data division unit 1, a parameter loading unit 2, a training data construction unit 3, a training unit 4, and a student model updating unit 5. Figure 2 The system shown corresponds to Figure 1 The method shown.
[0035] The old and new data partitioning unit 1 is configured to employ two deep learning models of different sizes: the larger deep learning model is referred to as the teacher model, and the smaller deep learning model is referred to as the student model. The teacher model is trained using a training dataset, and the training data in the training dataset is divided into two parts based on different training effects: the training data that performs well for the teacher model is referred to as the old data, and the training data that performs poorly for the teacher model is referred to as the new data.
[0036] The parameter loading unit 2 is used to allow the teacher model and the student model to load their original model parameters. The teacher model is in the inference state, and the student model is in the training state.
[0037] The training data construction unit 3 is used to mix new data and old data in a certain proportion to obtain a batch of training data, and to construct multiple batches of training data.
[0038] The training unit 4 is used to train the student model using a batch of training data. The student model is trained on the old data and the new data in the batch of training data to obtain a loss value.
[0039] The student model updating unit 5 is used to perform reverse gradient update on the loss value of each batch of training data according to the student model during multiple rounds of training of the student model using multiple batches of training data to obtain a student model with updated parameters.
[0040] This application breaks through the limitations of traditional online learning algorithms. It mixes new and old data in a certain proportion into each batch of training data. Through knowledge distillation technology, the student model can maintain the performance of the old data as much as possible. On this basis, the student model is allowed to learn new data, achieving rapid updates of the student model and allowing the student model to meet the performance requirements of both parts of data at the same time.
[0041] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.
Claims
1. An online learning method based on incremental learning, characterized by: The method includes the following steps: Step S1: Two deep learning models of different sizes are used, the larger deep learning model is called the teacher model, and the smaller deep learning model is called the student model; a training data set is used to train the teacher model, and the training data in the training data set is divided into two parts according to different training effects: the training data with better training effects on the teacher model is called old data, and the training data with poor training effects on the teacher model is called new data; Step S2: The teacher model and the student model each load model parameters; the teacher model is in the inference state, and the student model is in the training state; Step S3: Mix the new data and the old data in a certain proportion to obtain a batch of training data, and construct multiple batches of training data; Step S4: Using a batch of training data to train the student model, the student model is trained on the old data and the new data in the batch of training data to obtain a loss value; Step S5: Use multiple batches of training data to perform multiple rounds of training on the student model, perform reverse gradient update on the loss value of each batch of training data according to the student model, and obtain the student model with updated parameters.
2. The online learning method based on incremental learning according to claim 1 is characterized in that: In step S1, the new data may include any one or more of the following: training data already in the training data set and provided by user feedback; and newly added training data not in the training data set.
3. The online learning method based on incremental learning according to claim 1, characterized in that: In step S2, the model parameters of the teacher model are obtained through the training of step S1; the model parameters of the student model are default model parameters or preset model parameters.
4. The online learning method based on incremental learning according to claim 1 is characterized in that: In step S3, in each batch of training data, the amount of new data is less than the amount of old data.
5. The online learning method based on incremental learning according to claim 4 is characterized in that: In step S3, in each batch of training data, the ratio of the quantity of new data to the quantity of old data is 1:
3.
6. The online learning method based on incremental learning according to claim 1 is characterized in that: In step S4, the teacher model obtains a set of feature maps for the old data in a batch of training data, and the student model obtains another set of feature maps for the old data in the same batch of training data; Compute the similarity between the feature map of the teacher model on the old data and the feature map of the student model on the old data.
7. The online learning method based on incremental learning according to claim 6 is characterized in that: In step S4, the mean square error loss between the feature map of the teacher model for the old data and the feature map of the student model for the old data is calculated; the larger the mean square error loss value, the more dissimilar the two sets of feature maps are; the smaller the mean square error loss value, the more similar the two sets of feature maps are.
8. The online learning method based on incremental learning according to claim 6 is characterized in that: In step S4, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in the subspace of the Grassmann manifold, and the projection similarity of the two points is calculated; The greater the projection similarity, the more similar the two points are; The smaller the projection similarity, the less similar the two points are.
9. The online learning method based on incremental learning according to claim 6 is characterized in that: In step S4, the feature map of the teacher model for the old data and the feature map of the student model for the old data are respectively mapped to two points in the subspace of the Grassmann manifold, and the shortest path on the geodesic flow of the two points is calculated; the shorter the shortest path on the geodesic flow, the more similar the two points are; the shorter the shortest path on the geodesic flow, the more dissimilar the two points are.
10. An online learning system based on incremental learning, characterized by: It includes old and new data division unit, parameter loading unit, training data construction unit, old data performance transfer unit, new data training unit, and student model update unit; The old and new data partitioning unit is used to adopt two deep learning models of different scales, where the larger deep learning model is called a teacher model and the smaller deep learning model is called a student model; a training data set is used to train the teacher model, and the training data in the training data set is divided into two parts according to different training effects: the training data with a better training effect on the teacher model is called old data, and the training data with a poor training effect on the teacher model is called new data; The parameter loading unit is used to allow the teacher model and the student model to load model parameters respectively; the teacher model is in the inference state, and the student model is in the training state; The training data construction unit is used to mix new data and old data in a certain proportion to obtain a batch of training data, and to construct multiple batches of training data; The training unit is used to train the student model using a batch of training data, and the student model is trained on old data and new data in the batch of training data to obtain a loss value; The student model updating unit is used to perform reverse gradient update on the loss value of each batch of training data according to the student model during multiple rounds of training of the student model using multiple batches of training data to obtain a student model with updated parameters.