Deep Learning Fine-Tuning With Orthogonal Gradients Against Forgetting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face the challenge of catastrophic forgetting, where neural networks forget previously learned tasks when trained on new tasks, leading to performance degradation and inefficiencies in multitask models.
Innovation Solution
A deep learning optimizer, known as DLWF, dynamically mitigates catastrophic forgetting by ensuring gradient compatibility and restoring the original model state using orthogonal projections, maintaining consistency during training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the neural network is trained on new tasks, then the model learns new capabilities, but the model forgets previously learned tasks (catastrophic forgetting)
Solution Approach 1:
The method performs preliminary actions by calculating the displacement vector from original weights before applying new gradients, and pre-calculating the orthogonal projection of gradients to ensure compatibility. This preliminary preparation allows the model to learn new tasks while systematically preventing catastrophic forgetting through mathematically guaranteed gradient compatibility.
Solution Approach 2:
The invention changes the gradient parameters by projecting them onto the orthogonal complement of the displacement vector space. This parameter transformation ensures that weight updates for new tasks do not interfere with previously learned task performance, resolving the contradiction between learning new capabilities and retaining old knowledge.
2Productivity
If the model weights are updated continuously during training, then the model adapts to new tasks, but the original model performance degrades
Solution Approach 1:
The method implements feedback by continuously monitoring the displacement vector from original weights and using this information to adjust gradient directions through orthogonal projection. This feedback mechanism ensures that weight updates maintain compatibility with original task performance while enabling continuous adaptation to new tasks.
Solution Approach 2:
The invention introduces dynamics by making the gradient update process adaptive rather than static. The orthogonal projection operation dynamically adjusts gradient directions based on the current displacement vector, allowing the model to continuously learn new tasks while maintaining reliability on original tasks through real-time gradient compatibility control.
3Ease of operation
If standard gradient descent is used for training, then training is simple and fast, but gradient conflicts cause catastrophic forgetting
Solution Approach 1:
The displacement vector serves as an intermediary between original weights and new weight updates. By projecting gradients onto the orthogonal complement of this intermediary vector, the method eliminates gradient conflicts while maintaining the simplicity and speed of gradient-based training, resolving the contradiction between ease of operation and harmful gradient conflicts.
Data Source
AI summary
Disclosed are embodiments for a deep learning optimizer for fine-tuning while dynamically mitigating catastrophic forgetting. In some aspects, a method includes saving an original state of a model to a buffer, the original state comprising original weights of the model; apply new gradients to intermediate model states of the model during at least one intermediate training step of training of the model; calculate, at a final training step of the training, a displacement vector representing a difference between current weights of the model and the original weights of the model; responsive to the displacement vector and a final gradient of the final training step being inconsistent in terms of gradient direction, perform an orthogonal projection of the final gradient on the displacement vector to generate an orthogonal gradient; and applying the orthogonal gradient to the original state of the model at the final training step of the training.