A continual learning low-rank fine-tuning method based on orthogonal constraint riemannian manifold and bilateral subspace decoupling

Abstract

The application discloses a continuous learning low-rank fine-tuning method based on orthogonal constraint Riemannian manifold and bilateral subspace decoupling. The method separates the output subspace, the input subspace and the direction energy by representing the task increment as on the basis of freezing the pre-training model parameters; bilateral orthogonal subspaces of historical tasks are constructed respectively, and the gradients of the current task with respect to A and B are projected to the orthogonal complement spaces of the corresponding historical subspaces; further, the projected gradients are mapped to the tangent space of the Grassmann manifold or the Stiefel manifold, the optimizer is updated, and the orthogonal base matrix is kept through QR decomposition. After the task training is completed, the current low-rank parameters are fixed, and the historical subspaces are updated until all tasks are completed. The method can reduce cross-task update interference, alleviate catastrophic forgetting, and reduce training, storage and inference overhead.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence model training technology, and in particular to a continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of two-sided subspaces, belonging to the field of artificial intelligence. Background Technology

[0002] In recent years, large-scale pre-trained models based on the Transformer architecture have been widely applied in fields such as natural language processing, computer vision, and multimodal learning. Due to the large number of model parameters, direct full-parameter fine-tuning incurs high computational and storage overhead. Low-rank adaptation methods, by freezing the pre-trained model parameters and using a low-rank increment matrix to adapt the model to the task, can achieve model fine-tuning with fewer trainable parameters. However, in continuous learning scenarios where the model learns multiple tasks sequentially, parameter updates from new tasks may overlap or conflict with the effective update directions of historical tasks, leading to the overwriting of historical knowledge and catastrophic forgetting.

[0003] Existing techniques typically employ experience replay, parameter regularization, task-specific modules, or orthogonal constraints on low-rank subspaces to mitigate forgetting. However, experience replay requires storing historical data, incurring additional storage overhead and privacy risks; task-specific modules cause the model size to continuously increase with the number of tasks; and parameter regularization struggles to directly eliminate interference between update directions from different tasks. Some methods apply orthogonal constraints only to a single factor in low-rank updates, but the low-rank increment is determined by multiple factors, and unconstrained factors may still overlap with the historical task subspace. Furthermore, traditional low-rank factors suffer from rotation, scaling, and basis vector drift, making it difficult to form a stable historical subspace representation, resulting in subsequent projection operations failing to accurately remove overlapping components between tasks.

[0004] To address the aforementioned issues, this invention decomposes the low-rank parameter update into an input subspace basis, an output subspace basis, and directional intensity parameters, thereby decoupling the subspace structure from the update intensity. It projects the update component of the current task onto the historical task subspace to reduce update interference between old and new tasks. Furthermore, it employs orthogonal constrained manifold optimization to maintain a stable orthogonal structure of the subspace basis during training. This method eliminates the need to store historical task training data or extend the complete model for each task, improving historical knowledge retention and the stability of continuous task training with lower parameters and computational overhead. Summary of the Invention

[0005] During continuous learning, the model learns multiple tasks sequentially. When training the current task, the parameter updates generated by the current task may enter the update space already occupied by historical tasks, thus overwriting or destroying the knowledge of historical tasks. Some existing methods set orthogonal regularization terms in the loss function to make the individual low-rank factors of the current task approximately orthogonal to the low-rank factors of historical tasks. However, the complete low-rank update is determined by both input and output parameters. Constraining only one side may still result in the other side overlapping with the historical task subspace, making it difficult to fully suppress inter-task interference.

[0006] This invention proposes a continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of bilateral subspaces. This method constructs output-side and input-side subspaces for historical tasks, respectively, and projects the bilateral gradients corresponding to the current task onto the orthogonal complement space of the corresponding historical subspace. This simultaneously reduces the overlap between the current task's knowledge directions on both the output and input sides and those of historical tasks, achieving bilateral task knowledge isolation.

[0007] This invention uses Grassmann manifolds or Stiefel manifolds to constrain the basis of the two-sided subspace, and continuously maintains the orthogonal structure of the subspace basis through Riemann tangent space mapping and shrinkage operations, providing a stable subspace representation for historical subspace maintenance and two-sided orthogonal complementary projection.

[0008] The core principle of this invention lies in representing historical task knowledge in the output and input spaces of the target weight matrix, respectively, and removing components overlapping with the historical task subspace from the bilateral update directions of the current task during training of the current task. To obtain separately processable output and input subspaces, this invention, during continuous low-rank fine-tuning, represents the low-rank increment corresponding to the current t-th task as... This method can represent the output subspace, the input subspace, and the directional intensity separately, so that the output and input task subspaces can be maintained, projected, and orthogonally constrained independently.

[0009] For completed historical tasks, this invention constructs an output-side orthogonal historical subspace and an input-side orthogonal historical subspace based on the output and input subspace basis matrices of the historical tasks, respectively. When training the current task, the orthogonal complement projection of the historical subspaces is first performed on the bilateral gradients of the current task. Then, the projected gradients are mapped to the tangent space of the orthogonally constrained Riemannian manifold. Finally, the column orthogonal structure of the bilateral subspace basis is restored through a shrinking operation. Therefore, the technical process of this invention includes: The process includes explicit representation of bilateral subspaces, construction of bilateral historical subspaces, bilateral orthogonal complementary projection, Riemannian tangent space optimization, and manifold shrinking. Bilateral orthogonal complementary projection is used to achieve knowledge isolation between different tasks, and Riemannian manifold constraints are used to ensure the stability and reliability of the subspace basis used when performing the knowledge isolation.

[0010] The technical solution of the present invention can be divided into a two-sided subspace representation module, a two-sided historical subspace maintenance module, a two-sided gradient orthogonal projection module, a Riemannian manifold constraint optimization module, and a task incremental update module.

[0011] Bilateral subspace representation module: For the current number For each learning task, construct the corresponding low-rank increment matrix: Output subspace basis matrix The column vectors are used to represent the main update direction of the current task in the target weight output space, and the input subspace basis matrix is ​​used. The column vectors are used to represent the main update directions of the current task in the target weight input space, and the direction intensity matrix is ​​used to represent these directions. Used to adjust the contribution of each subspace direction to the current task weight update. Output subspace basis matrix. and input subspace basis matrix Initialized as a column orthogonal matrix, direction intensity matrix The low-rank increment corresponding to the current task is initialized to zero, ensuring it is zero at the start of training. This prevents the addition of low-rank parameters from altering the original output of the pre-trained model during the initialization phase. The purpose of this invention using the aforementioned low-rank representation is to obtain independently processable output-side and input-side task subspaces, enabling the separate maintenance of the knowledge directions occupied by historical tasks on the output and input sides, and allowing independent projection and constraint of the current task's bilateral update directions. This low-rank representation forms the parameter structure basis for implementing bilateral task knowledge isolation, and the decoupling of the bilateral subspaces and their corresponding historical subspace projection constitute the core processing of this invention.

[0012] Dual-sided historical subspace maintenance module: In the current number Before each learning task arrives, the output subspace basis matrices corresponding to the completed historical tasks are concatenated, and the output-side historical orthogonal subspace basis matrices are obtained through QR decomposition, singular value decomposition, or other orthogonalization processes. The input subspace basis matrices corresponding to historical tasks are concatenated and then orthogonalized to obtain the input-side historical orthogonal subspace basis matrix. Among them, subscript Indicates the current number Previous learning tasks The column space represents the main update space already occupied by historical tasks on the target weight output side. The column space represents the main update space already occupied by the historical task on the target weight input side. By constructing historical subspaces on the output and input sides respectively, the knowledge representation formed by the historical task can be described from two directions of the target weight matrix. During orthogonalization, linearly dependent or low-contribution basis vectors can be deleted, retaining only linearly independent basis vectors that can characterize the knowledge direction of the historical task, thereby reducing redundant directions in the historical subspace. When the current task is the first learning task, since there is no historical task, no historical orthogonal subspace is constructed, and the original gradient of the current task is directly used for subsequent manifold constraint optimization. This invention maintains historical task subspaces on both the output and input sides of the target weight matrix. Therefore, knowledge interference generated by the current task in both the output and input directions can be identified and processed.

[0013] Bilateral gradient orthogonal projection module: During the current task training process, the current task training data is input into the pre-trained neural network model, the current task loss is calculated, and the current task loss relative to the output subspace basis matrix is ​​obtained through backpropagation. and input subspace basis matrix The Euclidean gradients are denoted as follows: and Based on the output-side history orthogonal subspace basis matrix. Project the Euclidean gradient of the output side onto the orthogonal complement space of the history subspace of the output side. ;in express 3D identity matrix This represents the orthogonal projection matrix of the output-side history subspace. This represents the gradient after removing overlapping components from the historical tasks on the output side. It is based on the basis matrix of the orthogonal subspace of the historical tasks on the input side. The Euclidean gradient of the input side is projected onto the orthogonal complement space of the input side history subspace. in, express 3D identity matrix This represents the orthogonal projection matrix of the input-side history subspace. This represents the gradient after removing overlapping components from the input-side historical task subspace. After the above two-sided orthogonal projection, the output-side projected gradient is orthogonal to the output-side historical task subspace, and the input-side projected gradient is orthogonal to the input-side historical task subspace. Therefore, the updates of the current task on both the output and input sides avoid the main knowledge directions already occupied by the historical task, thereby reducing the coverage of historical task knowledge by the current task's learning. Direction Intensity Matrix The gradient is used to represent the update intensity of each subspace direction, and does not directly represent the output or input subspace. Therefore, its gradient does not perform orthogonal complement projection of the historical subspace, but is updated in Euclidean space. This invention differs from existing methods that only set a one-sided orthogonal regularization term in the loss function. The one-sided orthogonal regularization term only indirectly restricts the relationship between one low-rank parameter and the historical task parameter, and the other side may still produce historical task subspace overlap. This invention, however, directly removes the historical task subspace component from the output gradient and the input gradient respectively before each parameter update, achieving bilateral task knowledge isolation from the actual update direction.

[0014] Riemannian manifold constraint optimization module: To ensure that the basis matrices used in the construction of the bilateral historical subspace and the orthogonal complementary projection have a stable orthogonal structure, this invention will output the subspace basis matrix. and input subspace basis matrix The training is constrained to a predefined orthogonal constrained Riemannian manifold. This orthogonal constrained Riemannian manifold can be either a Grassmann manifold or a Stiefel manifold; these are alternative implementations and do not need to be used simultaneously in the same training session. When a Grassmann manifold is used, the output subspace basis matrix will be... and input subspace basis matrix The spanned column space is used as the optimization object, and the gradients after bilateral orthogonal complement projection are mapped as follows: in, and Let the superscripts represent the Riemann gradients corresponding to the basis matrices of the output and input subspaces on the Grassmann manifold, respectively. Let this represent a Grassmann manifold. A Grassmann manifold treats different orthogonal basis matrices spanning the same column space as representations of the same subspace. This allows the optimization process to focus primarily on changes within the task knowledge subspace itself, rather than rotations of specific basis vectors within the subspace, thus reducing redundant changes introduced by equivalent basis representations. When using a Steefel manifold, the specific basis matrices satisfying the column orthogonality condition are used as the optimization object, and the gradients after the bilateral orthogonal complement projections are mapped as follows: in Representation matrix The symmetrical part, and Let represent the Riemann gradients corresponding to the output subspace basis matrices and the input subspace basis matrices on the Stiefel manifold, respectively, with the superscript *St* indicating the Stiefel manifold. The tangent space mapping further satisfies the tangent space constraints of the corresponding Riemann manifold by ensuring that the bilateral gradients, after orthogonal projection onto the historical subspace, satisfy the tangent space constraints of the corresponding Riemann manifold, thus preventing the parameter update direction from directly disrupting the orthogonal structure of the output and input subspace basis matrices. After obtaining the Riemann gradients, the output subspace basis matrices are optimized using a stochastic gradient descent optimizer, the Adam optimizer, the AdamW optimizer, or other preset optimizers. A t and input subspace basis matrix B t Perform a provisional update. Since the provisionally updated basis matrix may deviate from the corresponding Riemannian manifold, a further shrinkage operation is performed. In a preferred embodiment, QR shrinkage is used, that is, QR decomposition is performed on the provisionally updated output subspace basis matrix and the input subspace basis matrix respectively, and the orthogonality factor obtained from the QR decomposition is taken as the updated subspace basis matrix, so that it satisfies the following condition again: Besides QR shrinkage, other shrinkage methods that can restore the column orthogonal structure, such as extreme decomposition, singular value decomposition, or Cayley transform, can also be used. Through orthogonal complementary projection of the bilateral historical subspaces, Riemannian manifold tangent space mapping, and shrinkage operations, this invention, on the one hand, ensures that the updates of the current task's output and input sides avoid the direction of historical task knowledge; on the other hand, it maintains a stable orthogonal structure of the bilateral subspace basis during continuous training, thereby improving the reliability of task knowledge isolation.

[0015] Incremental update module for tasks: In each training iteration of the current task, forward propagation is first performed based on the current task data to calculate the training loss. Then, the Euclidean gradient of the low-rank parameter of the current task is calculated. Orthogonal projection of the historical subspace is performed on the output and input gradients respectively, and the bilaterally projected gradients are mapped to the tangent space of the selected Riemannian manifold. Subsequently, the output and input subspace basis matrices are updated according to the Riemann gradients, the orientation intensity matrix is ​​updated according to the Euclidean gradients, and a shrinkage operation is performed on the provisionally updated bilateral subspace basis. The above training process is repeated until the preset number of training iterations corresponding to the current task is reached or the preset convergence condition is met. After the current task training is completed, the output, input, and orientation intensity matrices corresponding to the current task are fixed, and the output subspace basis of the current task is added to the output historical subspace set, and the input subspace basis of the current task is added to the input historical subspace set. When the next learning task arrives, the bilateral orthogonal historical subspace is reconstructed based on the updated output-side and input-side historical subspace sets. The bilateral gradient orthogonal complement projection and Riemannian manifold constraint optimization are then repeatedly performed until all learning tasks are completed. Throughout the continuous learning process, training the current task does not require revisiting the historical task training data. The model update direction corresponding to the historical tasks is maintained using fixed low-rank parameters and the bilateral historical subspace.

[0016] The method of this invention can be applied to large language models, encoder models, encoder-decoder models, visual Transformer models and multimodal models, and the low-rank adaptation structure can be set in attention projection layers, feedforward network layers or other linear transformation layers containing weight matrices.

[0017] Through the above technical solutions, the present invention has the following advantages: (1) A more complete cross-task knowledge isolation is achieved through dual subspace projection on both the input and output sides. Compared with existing continuous learning methods that apply orthogonal loss to only a single low-rank factor, this invention represents the output and input subspaces of the current task separately, and constructs the output-side orthogonal subspace and input-side orthogonal subspace of the historical task respectively. During the training process of the current task, the basis gradients of the output subspace and the basis gradients of the input subspace are projected onto the orthogonal complement space of the corresponding historical subspace, respectively, and the update components that overlap with the knowledge of the historical task are directly removed from both directions. To support the above dual-side processing, this invention adopts the following in the continuous learning scenario: The low-rank representation makes the output subspace basis Input subspace basis With direction intensity matrix Each is responsible for subspace representation and update intensity adjustment. Therefore, this invention avoids residual interference on the other side when only low-rank parameters on one side are constrained, allowing new tasks to learn along directions not yet occupied by historical tasks, thereby improving the ability to retain historical knowledge in long task sequences.

[0018] (2) By maintaining the orthogonal structure of the bilateral subspace basis through Riemannian manifold constraints and shrinkage operations, the stability and reliability of knowledge isolation are improved. Ordinary Euclidean optimization cannot guarantee that the updated input and output subspace basis always satisfy the column orthogonality condition, which can easily lead to basis vector redundancy, subspace aliasing, and historical subspace representation drift, thereby reducing the accuracy of orthogonal complement projection. This invention constrains the bilateral subspace basis to Grassmann or Stiefel manifolds, first mapping the gradients projected through the historical subspace to the tangent space of the corresponding Riemannian manifold, and then restoring the column orthogonal structure of the basis matrix through shrinkage operations such as QR decomposition. Compared with the method of setting only soft orthogonal regularization terms in the loss function, this invention directly maintains the subspace orthogonality from the optimization geometry and parameter update process, without relying on orthogonal loss weights to approximately maintain the constraints, thereby providing a stable and well-conditional orthogonal basis representation for the construction of the historical task subspace and subsequent bilateral gradient projection.

[0019] (3) Effectively mitigates catastrophic forgetting under conditions without historical data replay, while maintaining low training and inference overhead. This invention does not require re-accessing historical task training data when learning the current task. Instead, it utilizes the low-dimensional input and output subspace bases corresponding to the historical task to preserve its main knowledge directions, and reduces the coverage of historical knowledge by the current task update through bilateral orthogonal complementary projection. Therefore, this invention avoids the historical data storage overhead and privacy restrictions brought about by experience replay methods, and does not require expanding the complete model for each task as the number of tasks increases. During training, only the low-rank parameters corresponding to the current task are updated, significantly reducing the overhead of trainable parameters, gradient storage, and optimizer state compared to full parameter fine-tuning; compared to standard low-rank adaptation, only a small number of directional strength parameters related to the low-rank dimension are added. After training, the low-rank increment can be directly merged with the original weight matrix without adding additional computational branches in the online inference stage. Experimental results show that this invention significantly improves the average accuracy of continuous learning and the ability to retain historical knowledge while introducing only a small storage overhead, with the training time remaining essentially unchanged, combining effectiveness, stability, and engineering deployment value. Attached Figure Description

[0020] Figure 1 The present invention provides a framework diagram for a continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of two-sided subspaces.

[0021] Figure 2The flowchart of this invention is a continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of two-sided subspaces. Detailed Implementation

[0022] The invention will now be described in more detail with reference to the accompanying drawings.

[0023] This invention discloses a continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of bilateral subspaces. Taking a pre-trained language model based on the Transformer architecture as an example, further explanation is provided with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a diagram illustrating the overall framework of the method of the present invention. The method of the present invention mainly includes: a ternary low-rank adaptation module, a bilateral historical subspace maintenance module, a bilateral gradient projection module, a Riemannian manifold optimization module, and a task incremental update module.

[0024] The pre-trained model in this embodiment of the invention includes multiple Transformer layers, each Transformer layer including a multi-head attention module and a feedforward network module. A ternary low-rank adapter is set on at least one of the target weight matrices in the query projection layer, key projection layer, value projection layer, and output projection layer of the multi-head attention module, and the linear mapping layer of the feedforward network module.

[0025] For any target weight matrix: Freeze its original parameters and apply them to the current task. Constructing low-rank increments: in: To output the basis matrix of the subspace; The input subspace basis matrix; The directional energy matrix; It is a low-rank dimension and satisfies: The embodiments of the present invention utilize the traditional two-factor low-rank representation Improved to ternary low-rank representation This allows the input subspace, output subspace, and directional energy to be represented by different parameters, thereby reducing scale coupling and basis vector drift between low-rank factors.

[0026] For the For each task, the target weights used during model forward propagation are: in, For low-rank incremental scaling factors; The low-rank parameters for each task remain fixed; only the parameters for the current task are updated. , and .

[0027] Bilateral subspace representation module: At the start of training for the current task, the basis matrix of the output subspace is... and input subspace basis matrix Perform random column orthogonal initialization to satisfy: in, for An identity matrix of order 1. The directional energy vector is initialized as follows: Therefore, we can conclude that: Make the initial low-rank increment corresponding to the current task satisfy: Ensure that the adaptation module does not change the original output of the pre-trained model during the initial training phase. Then, update it gradually through training. This allows different subspace directions to achieve different task-related contribution intensities. (Compared to direct...) Compared to optimization, this method makes and Primarily responsible for describing the subspace orientation. It is mainly responsible for describing the energy in each direction, which facilitates the stable maintenance of the input and output subspaces respectively.

[0028] Dual-sided historical subspace maintenance module: Before the model is completed After each task, save the output subspace basis matrix and input subspace basis matrix corresponding to the historical task, and construct: Perform QR decomposition on the two concatenated matrices respectively: in, Output a column orthogonal basis for the side subspace of the historical task. Input the column orthogonal basis of the side subspace for the historical task.

[0029] This invention maintains separate historical subspaces for the input and output sides, rather than constraining the historical direction only for a single low-rank factor. Through QR decomposition, the basis matrices formed by different historical tasks can be transformed into well-conditional orthogonal subspace bases for subsequent gradient projection.

[0030] When the current task is the first task, that is Since there is no historical task subspace, the historical subspace projection is not performed. Instead, the current task gradient is directly mapped to the manifold tangent space.

[0031] Bilateral gradient projection module: Input the current task training data into the pre-trained model, and calculate the loss function based on the current task objective: Obtained through automatic differentiation and backpropagation respectively: in, To output the Euclidean gradient of the basis matrix of the subspace, The Euclidean gradient of the basis matrix of the input subspace. denoted as the Euclidean gradient of the directional energy parameter.

[0032] Construct the orthogonal complementary projection matrix based on the output-side history subspace: And obtain the output-side projection gradient: Construct the orthogonal complementary projection matrix based on the input-side history subspace: And obtain the input-side projected gradient: After projection, the following conditions are met: That is, the current task is about and The update directions are orthogonal to the corresponding historical task subspaces. This two-sided gradient projection can simultaneously remove the gradient components that overlap with the historical task directions on both the input and output sides, avoiding inter-task interference on the other side when only one-sided low-rank factor is constrained.

[0033] Directional energy parameters Its gradient is used to adjust the contribution strength in each direction of the current task. Do not perform historical subspace projection.

[0034] Riemannian manifold optimization module: The embodiments of the present invention preferably employ Grassmann manifold pairs. and The subspace represented is optimized. In other embodiments, a Stiefel manifold may also be used to maintain the orthogonal structure of the specific basis matrices.

[0035] For the basis matrix of the output subspace of the Grassmann manifold The gradient projected from the historical subspace is further mapped to the tangent space of the Grassmann manifold: For the basis matrix of the input subspace ,calculate: Grassmann manifolds treat different rotated bases with the same column space as a single subspace. Therefore, this implementation reduces redundant updates caused by rotations within the basis vectors, allowing the training process to focus primarily on changes within the task subspace itself.

[0036] When the Stiefel manifold is chosen, the Riemann gradient of the basis matrix of the output subspace is: The Riemann gradient of the basis matrix of the input subspace is: in: Grassmann and Stiefel manifolds correspond to different methods for calculating the tangent space gradient, and both are optional implementations. The Grassmann scheme is used as an optimization method based on subspace equivalence classes, while the Stiefel scheme is used as an alternative to concrete orthogonal basis optimization.

[0037] Incremental update module for tasks: Input the obtained Riemann gradient into the optimizer to obtain the tentatively updated matrix: And update the directional energy parameters using unconstrained Euclidean gradients: The optimizer can be a stochastic gradient descent optimizer, an Adam optimizer, or an AdamW optimizer. In this embodiment, the AdamW optimizer is preferred.

[0038] Due to the tentative update and The column orthogonality condition may no longer be satisfied. Perform QR decomposition on each column and take the orthogonality factor from the QR decomposition: in, This indicates taking the Q-factor of the QR decomposition. After shrinking, the following condition is satisfied again: Through the continuous processing of "historical subspace orthogonal supplementary projection - manifold tangent space mapping - optimizer update - QR shrinkage", the current task update avoids the main direction occupied by the historical task and maintains the orthogonal structure of the current subspace basis.

[0039] Please refer to Figure 2 , Figure 2 The flowchart of the method of the present invention specifically includes the following steps.

[0040] S1: Input task sequence and pre-trained model; Get the sequence of tasks that arrive in order: Load the pre-trained neural network model and freeze the model's original parameters.

[0041] S2: Select the linear layer to be adapted and freeze the pre-trained weights; S3: Construct a ternary low-rank adapter; Build on at least one linear layer of the pre-trained model: Random column orthogonal initialization and and will Initialize to a zero matrix.

[0042] S4: Construct the current task adaptation weights; S5: Construct a dual-sided historical task subspace; Concatenate the historical task output subspace basis matrix and the input subspace basis matrix separately, and obtain the historical orthogonal subspace basis through QR decomposition. and .

[0043] S6: Calculate the current task loss and Euclidean gradient; The low-rank increments of historical tasks and current tasks are added together to the original weight matrix, and the forward propagation is completed using the training data of the current task to calculate the task loss.

[0044] Calculate separately using backpropagation and The Euclidean gradient.

[0045] S7: Bilateral historical subspace orthogonal supplementary projection; Will and The Euclidean gradients are projected onto the orthogonal complement spaces of the historical subspaces on the output and input sides, respectively, removing gradient components that overlap with the historical task directions.

[0046] S8: Calculate the Riemann gradient; Based on the chosen Grassmann or Stiefel manifold, the two-sided projected gradients are mapped to the tangent space of the corresponding manifold.

[0047] S9: Perform optimizer update; Tentative update and .

[0048] S10: QR retraction; The low-rank parameters of the current task are updated using the preset optimizer, and the updated parameters are then processed. and Perform a QR retraction to restore the column orthogonal structure.

[0049] S11: Complete the current task training and update the historical subspace; Repeat steps S4-S8 until the preset number of training iterations is reached or the convergence condition is met; fix the current task. , and and will and Add to the historical subspace set.

[0050] S12: Proceed to the next task until all tasks are completed; make Repeat steps S2-S9 until all tasks are trained. After training, the low-rank increments for each task can be merged with the original weights. This reduces the additional computational paths required during the inference phase.

[0051] To illustrate the technical effectiveness of the method of the present invention, in one embodiment, a T5-Large pre-trained model is used and tested on continuous learning benchmarks including short task sequences and long task sequences. The Grassmann manifold implementation of the present invention is denoted as RSO-Gr, and the Stiefel manifold implementation is denoted as RSO-St, and compared with the existing orthogonal low-rank continuous learning method O-LoRA.

[0052] Table 1 Comparison of Average Accuracy for Continuous Learning

[0053] As shown in Table 1, both manifold implementations of the present invention achieved higher average accuracy than the comparative methods on both short and long task sequences. Specifically, the average accuracy of RSO-Gr in long task sequences increased from 69.6% to 74.8%, indicating that bilateral subspace projection and manifold constraints can reduce cumulative interference as the number of tasks increases.

[0054] Table 2 Comparison of Training Efficiency and Resource Cost

[0055] As shown in Table 2, this invention only increases GPU memory usage slightly compared to O-LoRA, while achieving higher training throughput and average accuracy. Because this invention directly maintains task subspace separation through manifold structure and gradient projection, it eliminates the need to calculate additional inter-task soft orthogonal loss in each training iteration, thus mitigating catastrophic forgetting while maintaining low training overhead.

[0056] The above implementation results show that the present invention can improve the historical knowledge retention ability and task sequence adaptation ability of the continuous learning model through the synergistic effect of ternary low-rank representation, bilateral historical subspace projection and orthogonal constrained Riemannian manifold optimization, while maintaining low parameter, training and inference overhead.

[0057] The above are merely specific embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, or alterations made to the above embodiments based on the technical essence of this application shall still fall within the protection scope of the technical solution of this application.

Claims

1. A continuous learning low-rank fine-tuning method based on orthogonal constrained Riemannian manifolds and decoupling of two-sided subspaces, characterized in that, Includes the following steps: S1: Obtain the continuous learning task sequence and pre-trained model, and obtain multiple learning tasks arriving in a preset order: And load the pre-trained neural network model to be continuously learned: ,in, N This indicates the total number of learning tasks. Indicates the first in the task sequence t One learning task t This indicates the sequence number of the current learning task, and , This represents the original parameters of the pre-trained neural network model; S2: Determine the weight matrix to be adapted and freeze the pre-training parameters. Select at least one linear layer to be adapted in the pre-trained neural network model and obtain the pre-training weight matrix of the linear layer to be adapted. ,in, W This represents the original weight matrix of the linear layer to be adapted. Represents a set of real numbers d OK, k Column matrix d This represents the output feature dimension of the linear layer to be adapted. k This represents the input feature dimension of the linear layer to be adapted, and the pre-trained weight matrix is ​​frozen. W This ensures that only the low-rank adaptation parameters set for each learning task are updated during continuous learning. S3: Construct a ternary low-rank adaptor structure that decouples the input and output sides, tailored to the current learning task. Construct a task-specific low-rank increment matrix: in, This indicates the relationship between the current learning task and the pre-trained weight matrix. W The resulting low-rank increment, sign Indicates matrix transpose. This represents the basis matrix of the output subspace of the current learning task. Each column represents a basis vector in the output feature space. This represents the basis matrix of the input subspace for the current learning task. Each column represents a basis vector in the input feature space. This represents the direction intensity matrix of the current learning task. This means constructing a diagonal matrix based on the input vector; This represents the direction intensity vector of the current learning task. Indicates the first The update intensity corresponding to each subspace direction j Indicates the subspace direction index. Let represent a low-rank dimension and satisfy: ,in express d and k The smaller value in; Initialize the output subspace basis matrix and input subspace basis matrix So that they respectively satisfy: in, express A dimensional identity matrix, which represents the directional intensity vector. Initialize to zero, so that when the current learning task begins training, the following conditions are met: ; The ternary low-rank adaptation structure represents the output subspace basis, the input subspace basis, and the update intensity of each subspace direction, respectively, so that the input and output subspaces can be independently constrained and independently projected. S4: Construct adaptive weights that include historical task increments and current task increments, fixing the low-rank increments corresponding to historical learning tasks completed before the current learning task as follows: in Indicates the first t The sum of all low-rank increments of all historical tasks preceding the current learning task. i Indicates the sequence number of the history learning task. and They represent the first i The output subspace basis matrix, orientation intensity matrix, and input subspace basis matrix of each historical learning task are used to construct the adaptation weight matrix for training the current learning task. in, This represents the actual adaptation weight matrix used during training for the current learning task. The scaling factor representing the low-rank increment. Indicates the scaling factor applied to the low-rank increment; S5: Construct the output-side orthogonal history subspace and the input-side orthogonal history subspace, when... At that time, the front The output subspace basis matrices of each historical learning task are concatenated to obtain... and will the former The basis matrices of the input subspace of each historical learning task are concatenated. ,in, This represents a concatenated matrix composed of the basis of the output subspace of historical tasks. This represents a concatenated matrix composed of the basis of the input subspace of the historical task, with the symbol... This indicates splicing along the column direction of the matrix; To each and Perform QR orthogonalization and retain the linearly independent orthogonal basis vectors to obtain: in, This indicates that QR decomposition is performed on the input matrix and its orthogonality factors are obtained. This represents the basis matrix of the output-side history orthogonal subspace. This represents the number of linear independent basis vectors retained in the output-side history subspace. Represents the basis matrix of the input-side history orthogonal subspace. This represents the number of linear independent basis vectors retained in the input-side history subspace; S6: Calculate the current task loss and the Euclidean gradient of the bilateral subspace basis, and input the training data of the current learning task into the adapted weight matrix. The pre-trained neural network model is used to calculate the loss for the current task. The following results were obtained through backpropagation: ， in Indicates gradient calculation, This represents the current task loss relative to the output subspace basis matrix. The Euclidean gradient, This represents the current task loss relative to the input subspace basis matrix. The Euclidean gradient, This represents the current task loss relative to the direction strength vector. The Euclidean gradient; S7: Perform bilateral orthogonal complement projection of the output and input history subspaces, when At that time, the basis matrix of the output-side historical orthogonal subspace is used. Project the Euclidean gradient of the basis matrix of the output subspace onto the orthogonal complement space of the historical subspace on the output side: in, express d 3D identity matrix This represents the orthogonal projection matrix of the output-side history subspace. This represents the output subspace basis gradient after removing components that overlap with the output-side historical subspace, using the input-side historical orthogonal subspace basis matrix. Project the Euclidean gradient of the basis matrix of the input subspace onto the orthogonal complement space of the input-side history subspace: in, express k 3D identity matrix This represents the orthogonal projection matrix of the input-side history subspace. This represents the input subspace basis gradient after removing components that overlap with the input-side historical subspace. S8: Map the bilaterally projected gradients to the tangent space of the orthogonally constrained Riemannian manifold, and choose the Grassmann manifold or the Stiefel manifold as the basis matrix of the output subspace. and input subspace basis matrix For a target orthogonal constrained Riemannian manifold, when using a Grassmann manifold, the output-side Riemann gradient is calculated as follows: And calculate the input-side Riemann gradient: in, Denotes the basis matrix of the output subspace on the Grassmann manifold. The corresponding Riemann gradient, Denotes the basis matrix of the input subspace on the Grassmann manifold. The corresponding Riemann gradient; When using the Stiefel manifold, calculate the Riemann gradient on the output side: And calculate the input-side Riemann gradient: in, Denotes the basis matrix of the output subspace on the Stiefel manifold. The corresponding Riemann gradient, Denotes the basis matrix of the input subspace on the Stiefel manifold. The corresponding Riemann gradient; Where M represents the square matrix to be symmetricized. Represents the symmetric part of matrix M; Through the tangent space mapping, the update direction after orthogonal complementary projection of the two-sided history subspace simultaneously satisfies the tangent space constraint of the target Riemannian manifold. S9: Perform provisional parameter updates based on the Riemann gradients. Input the output-side and input-side Riemann gradients obtained in step S8 into the preset optimizer, and update the basis matrix of the output subspace accordingly. and input subspace basis matrix Perform a provisional update to obtain a provisional output subspace basis matrix. and provisional input subspace basis matrix ; Using Euclidean gradient relative direction intensity vector The update is performed, the directional intensity vector does not perform Riemannian manifold tangent space mapping, and the preset optimizer includes stochastic gradient descent optimizer, Adam optimizer or AdamW optimizer; S10: Perform a shrinkage operation and restore the column orthogonal structure of the bilateral subspace basis, respectively, for the provisional output subspace basis matrix. and provisional input subspace basis matrix Execute QR retraction: Make the reduced output subspace basis matrix and input subspace basis matrix satisfy the following conditions again: The shrinkage operation also includes extreme decomposition shrinkage, singular value decomposition shrinkage, or Cayley transform shrinkage. Through the shrinkage operation, the column orthogonal structure of the output subspace basis matrix and the input subspace basis matrix is ​​maintained during continuous training, providing an orthogonal subspace basis for subsequent historical subspace construction and bilateral orthogonal complementary projection. S11: Repeat the bilateral projection and manifold constraint optimization for the current learning task, and repeat steps S4 to S10 until the preset number of training iterations for the current learning task is reached or the preset convergence condition is met, to obtain the low-rank adaptation parameters for the current learning task. ; S12: Fix the current task parameters and update the bilateral historical subspaces. Fix the basis matrix of the output subspace obtained from the current learning task training. Directional intensity matrix and input subspace basis matrix and will and Add the basis sets of the output-side and input-side history subspaces respectively; make until all are completed. N Continuous learning for each learning task.

Images