An efficient fine-tuning and knowledge editing combined optimization method for large models

Abstract

The application discloses a kind of high-efficiency fine-tuning and knowledge editing combined optimization methods for large model, belong to artificial intelligence technical field, to solve the problem such as low efficiency, task conflict and serious forgetting caused by fine-tuning and knowledge editing separation processing in prior art, the integrated optimization framework is proposed, the method first freezes the basic parameters of pre-training large model, and injects low rank adaptation (LoRA) module as the only trainable parameter carrier;While constructing the joint optimization objective function of fusion fine-tuning loss, knowledge editing loss based on zero space projection and knowledge retention loss based on KL divergence, then in a single training cycle, the joint loss is optimized synchronously by gradient descent, to drive LoRA parameter collaborative update, to simultaneously realize the efficient adaptation of downstream task, accurate editing of target knowledge and effective protection of original knowledge.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method for efficient fine-tuning and knowledge editing combined with optimization for large models. Background Technology

[0002] In recent years, large language models based on the Transformer architecture have made groundbreaking progress in tasks such as natural language understanding, generation, and reasoning, demonstrating powerful potential for general artificial intelligence. However, successfully deploying these general "base models" to specific practical applications usually faces two key and interrelated challenges: task adaptability and knowledge controllability.

[0003] Task adaptability requires models to efficiently learn the data distribution and patterns for specific downstream tasks (such as legal document generation, medical question answering, code completion, etc.). While traditional full-parameter fine-tuning methods are effective, their enormous computational overhead, storage costs, and overfitting risk make them unsuitable for large models with hundreds of billions of parameters. To address this, efficient parameter fine-tuning techniques have emerged, such as Low-Rank Adaptation (LoRA). This technique introduces a trainable low-rank decomposition matrix alongside the original weight matrix, optimizing only a very small number of new parameters to achieve near-full-parameter fine-tuning performance, significantly reducing computational and storage barriers. Prefix-Tuning and adapter techniques also follow similar principles of "freezing the majority and optimizing the minority."

[0004] Knowledge controllability requires models to reliably and accurately update or correct the factual knowledge stored within them. After training, the knowledge of large models is fixed in hundreds of billions of parameters. When the model's knowledge becomes outdated or contains errors, the cost of retraining is unacceptable. Therefore, "knowledge editing" technology has become a research hotspot. Its goal is to change the model's output for a specific query (e.g., "Who is the current UN Secretary-General?") from an old answer ("Antonio Guterres") to a new answer ("The hypothetical new Secretary-General") with minimal parameter intervention, without retraining, while minimizing impact on the model's performance on other unrelated questions. Representative methods include MEND, which generates parameter update vectors by training a hypernetwork; ROME and MEMIT, which locate and modify key neurons storing specific knowledge in the model (i.e., the "location-edit" path); and AlphaEdit, which introduces the concept of null space projection, aiming to find a parameter update direction that precisely changes the target knowledge, but the resulting perturbations on other model outputs are projected into the null space of the "old knowledge space," thus preserving the original knowledge to the maximum extent.

[0005] Existing technical solutions typically treat efficient fine-tuning and knowledge editing as two separate problem sequences. A common approach is to first edit the base model to correct errors or update knowledge, and then fine-tune the edited model on specific task data; or, to fine-tune the task first, and then edit the knowledge on the fine-tuned model. This sequential, separate processing paradigm has significant limitations: 1. Both processes require independent training cycles, resulting in cumulative computational costs. This overhead increases linearly, especially when knowledge needs to be frequently updated and adapted to multiple tasks, which contradicts the original intention of efficient application.

[0006] 2. The goals of fine-tuning and knowledge editing may be inherently conflicting. In order to maximize task performance, the gradient updates during fine-tuning may unintentionally "wash away" previously injected knowledge through precise editing. Conversely, knowledge editing operations, aimed at locally precise modifications, may perturb the model's learned representations on the target task, leading to performance degradation. This conflict is difficult to reconcile and avoid in a sequential process.

[0007] 3. The separate optimization process lacks a unified global objective. The model parameters are driven sequentially by two independent objectives, and the final state may not be the optimal state for task performance or the highest state for knowledge accuracy, but rather a compromised, suboptimal state.

[0008] 4. Both fine-tuning and knowledge editing face the risk of catastrophic forgetting. Serial operations subject the model to two parameter perturbations, making it more difficult to retain the useful general knowledge and capabilities gained during the original pre-training.

[0009] Therefore, there is an urgent need in the current technological field for an integrated solution that can collaboratively and synchronously address the needs for efficient adaptation and precise editing. This invention aims to propose a joint optimization framework that, within a unified training loop, utilizes efficient parameter fine-tuning techniques (such as LoRA) as a shared, lightweight parameter update carrier, simultaneously optimizing three objectives: downstream task performance, target knowledge editing success rate, and original knowledge retention, thereby fundamentally overcoming the aforementioned deficiencies. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a highly efficient method for joint optimization of fine-tuning and knowledge editing for large models.

[0011] To achieve the above objectives, the technical solution adopted in this invention solves the problems of low efficiency, task conflicts, and suboptimal performance caused by separating the efficient fine-tuning of large models from knowledge editing in the prior art.

[0012] An efficient joint optimization method for fine-tuning and knowledge editing of large language models includes the following steps: S1. Parameterization preparation: Select a pre-trained large language model as the base model, freeze all its base parameters, and inject low-rank adaptation (LoRA) modules in parallel into the attention mechanism module and feedforward neural network module of at least one Transformer layer of the base model. The LoRA module consists of a trainable low-rank parameter matrix pair (A, B). S2. Data Preparation and Task Definition: Prepare three sets of data in parallel, including a fine-tuning dataset for downstream task adaptation, a knowledge editing instruction set for updating the internal facts of the model, and a knowledge retention dataset for protecting the original knowledge; and define the fine-tuning task, knowledge editing task, and knowledge retention task based on the three sets of data respectively. S3. Construction of Joint Optimization Loss Function: Construct a unified joint optimization loss function, which is formed by a weighted sum of fine-tuning loss term, knowledge editing loss term and knowledge retention loss term, expressed as: L_total=L_finetune+αL_edit+βL_retain, where α and β are adjustable hyperparameters; S4. Joint Parameter Iterative Update: During the training iteration, batch data is sampled from the three sets of data simultaneously; the value of the joint optimization loss function is calculated through forward propagation, and the gradient of the loss function with respect to the trainable parameter matrix (A, B) of the LoRA module is calculated through backpropagation algorithm; the optimizer is used to update only the trainable parameter matrix, while keeping the parameters of the base model frozen. S5. Performance Monitoring and Model Convergence: During training, the model performance is periodically evaluated on an independent validation set. The performance evaluation includes at least the performance metrics of downstream tasks and the success rate of knowledge editing tasks. When the performance metrics reach a preset threshold or tend to stabilize, the model is determined to have converged, and training is stopped. S6. Model Merging and Deployment: After training, the optimized LoRA parameter matrix (A', B') is linearly merged with the corresponding original weight matrix W of the base model to obtain the updated weight matrix W'=W+B'A'; the complete model parameters obtained after merging are exported and deployed.

[0013] Preferably, in step S3, the knowledge editing loss term L_edit is constructed using an editing method based on null projection, specifically including: S31. For a knowledge editing instruction (x_e, y_e), where x_e is the trigger query and y_e is the target answer, the basic model is regarded as a function f(θ;x), where θ is the model parameter; S32. Calculate the gradient of the model prediction with respect to the parameters: g = ∇_θf(θ;x_e)|_{θ=θ_0}, where θ_0 is the current parameter; S33. Construct a constraint matrix C, whose row vectors are spanned by a set of input gradients related to the knowledge to be protected, and calculate the null space projection operator P of the constraint matrix C through singular value decomposition. S34. Project the original gradient g onto the null space to obtain the editing direction vector Δθ_edit=Pg; S35. The knowledge editing loss term is defined as the cross-entropy loss between the model's prediction and the target answer y_e after the model is updated along the editing direction: L_edit=CE(f(θ_0+Δθ_edit;x_e),y_e).

[0014] Preferably, in step S3, the knowledge retention loss term L_retain is constructed through model output distribution stability constraints, specifically including: S31' Sample a batch of data {x_r} from the knowledge retention dataset; S32' Calculate the output probability distributions P_original and P_current of the base model (parameter θ_0) and the currently trained model (parameter θ_0+Δθ_lora) after input {x_r}, respectively; S33' Calculate the KL divergence between the two distributions as the knowledge retention loss: L_retain=Σ_{x_r}D_KL(P_original||P_current).

[0015] Preferably, in step S3, the fine-tuning loss term L_finetune is selected according to the type of the downstream task: for generative tasks, sequence cross-entropy loss is used; for classification tasks, cross-entropy loss is used; and for regression tasks, mean squared error loss is used.

[0016] Preferably, in step S4, a dynamic weight adjustment strategy is used to optimize the hyperparameters α and β. Specifically, the value of α is dynamically increased or decreased according to the rate of change of the success rate of the knowledge editing task on the validation set; and the value of β is dynamically adjusted according to the degree of performance degradation on the validation set in downstream tasks and general tasks.

[0017] Preferably, in step S1, when injecting the LoRA module, a hierarchical adaptive rank determination strategy is adopted, which specifically includes: performing sensitivity analysis on different network layers of the base model, assigning higher low-rank dimensions r to layers that are more sensitive to parameter changes and have a greater impact on task performance, and assigning lower ranks to layers that are less sensitive to parameter changes, so as to further optimize parameter efficiency.

[0018] Preferably, in step S4, a gradient reconciliation mechanism is adopted during training iteration, specifically including: after calculating the gradients of the fine-tuning task, the knowledge editing task, and the knowledge preservation task with respect to the shared LoRA parameters through backpropagation, before updating the parameters, cosine similarity detection and normalization weighting are performed on the three gradient vectors to reduce conflicts between gradients of different tasks and ensure the synergy of optimization directions.

[0019] A joint optimization system for implementing the method described above includes: The model configuration module is used to load the pre-trained base model, inject and initialize the LoRA module; The data management module is used to store and manage fine-tuning datasets, knowledge editing instruction sets, and knowledge retention datasets, and provides a data sampling interface; The loss calculation engine has a built-in fine-tuning loss calculation unit, a knowledge editing loss calculation unit based on null projection, and a knowledge preservation loss calculation unit, which are used to calculate the joint optimization loss function based on the input data and the current model state. The joint training module includes an optimizer for performing forward propagation, back propagation, and parameter update loops, and integrates the gradient reconciliation mechanism. The evaluation and monitoring module is used to periodically evaluate model performance during training and adjust training hyperparameters according to preset strategies. The model output module is used to merge the trained LoRA parameters with the base model and export the final model file.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. It combines two (or more) independent optimization processes that traditionally need to be executed sequentially into one. Fine-tuning and knowledge editing are completed simultaneously during a single training run, reducing overall training time, computational resource consumption, and engineering complexity. This is particularly suitable for production environments requiring rapid iteration and frequent updates.

[0021] 2. Guided by the joint loss function, the optimization process automatically seeks parameter update directions that mutually promote (or at least do not severely conflict with) the performance of downstream tasks and the effectiveness of knowledge editing. For example, fine-tuning the learned domain representation for the task may contribute to a more stable integration of knowledge editing, while accurate knowledge updates can improve the reliability of the model in answering factual questions in the task.

[0022] 3. The knowledge retention loss term (L_retain), as a powerful regularizer, combined with the null projection editing method, constitutes a dual protection mechanism, which effectively alleviates the catastrophic forgetting problem that may occur in the process of adapting to new tasks and updating knowledge, and ensures the stability of the model's core general capabilities.

[0023] 4. By adjusting the hyperparameters α and β, users can flexibly control the focus of the optimization process. For example, in scenarios with extremely high knowledge accuracy requirements (such as medical and legal fields), α can be increased; in scenarios where it is necessary to preserve the original style and capabilities of the model to the greatest extent possible, β can be increased. The dynamic adjustment strategy further enhances the intelligence level of this trade-off.

[0024] 5. It inherits all the advantages of efficient parameter fine-tuning techniques such as LoRA, requiring only a very small number of additional parameters (usually less than 1% of the original model's parameters) to be trained, greatly reducing storage and transmission costs. The trained models can be merged through simple matrix addition, and their deployment form is no different from the original model, making them easy to integrate.

[0025] 6. The method described does not depend on a specific model architecture and can be widely applied to various Decoder-only or Encoder-Decoder type large language models. It can also be extended to multimodal large models where visual language knowledge needs to be edited and task adapted. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of the present invention, which is a highly efficient joint optimization method for fine-tuning and knowledge editing of large models. Figure 2 This is a diagram illustrating the joint loss function of an efficient fine-tuning and knowledge editing joint optimization method for large models according to the present invention. Figure 3 This is a knowledge editing principle diagram of an efficient fine-tuning and knowledge editing joint optimization method for large models according to the present invention. Detailed Implementation

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Example

[0028] The specific implementation of this method revolves around a unified training loop. The core lies in using shared low-rank adaptation module parameters to synchronously respond to optimization signals in three aspects: downstream task adaptation, specific knowledge updates, and retention of original knowledge. The entire process begins with the parameterization preparation and data configuration of the model, followed by joint optimization iteration of multi-objective loss, and then the merging and deployment of the optimized modules. The following is a detailed explanation of each step.

[0029] Step S1, parameterization preparation: Select an open-source or proprietary pre-trained large language model as the base model, such as a version of LLaMA or GPT. Set all pre-trained parameters of this base model to a frozen state, meaning their values ​​remain unchanged during subsequent training. Then, inject low-rank adaptation modules into the Transformer architecture of this base model. A common practice is to add trainable low-rank matrix pairs in parallel next to the query, key, value, and output projection matrices in each or more attention mechanisms, and next to the two fully connected layers in the feedforward neural network. Each low-rank adaptation module consists of a pair of parameter matrices, namely matrix A and matrix B, where matrix A has dimensions of , and matrix B has dimensions of , much smaller than . In terms of the dimension of the original weight matrix W, matrix A is initialized with a random Gaussian distribution, and matrix B is initialized to a zero matrix to ensure that the output of the low-rank adaptation module is zero at the beginning of training, and the model behavior is completely consistent with the original base model. In implementation, a hierarchical adaptive rank determination strategy can be adopted to improve parameter efficiency. Specifically, the base model is run on a small calibration dataset to analyze the sensitivity of the output of different network layers to random parameter perturbations, or to evaluate the importance of different layers in the target task. For layers with high sensitivity or great importance, such as some feedforward networks in the middle layers, a higher rank value is assigned, and for layers with low sensitivity, a lower rank value is assigned. This differentiated configuration allows the limited number of additional parameters to be concentrated on the more critical parts of the model. Step S2, data preparation and task definition, requires the parallel preparation of three independent datasets. The fine-tuning dataset is collected based on the target downstream task. For example, for code generation tasks, it collects code snippets and comment pairs; for text summarization tasks, it collects long texts and summary pairs. The knowledge editing instruction set needs to be manually constructed or extracted from knowledge update requirements. Each instruction is a pair, including a trigger query and a target answer. For example, the trigger query is "What is the capital of France?", and the target answer is "Marseille". The knowledge retention dataset aims to represent the original knowledge range that the model needs to retain. It can be sampled from a general corpus or a series of question-answer pairs or text paragraphs covering a wide range of fields can be collected. Based on these three sets of data, three tasks are clearly defined: On the fine-tuning dataset, the model needs to learn to complete specific downstream tasks; on the knowledge editing instruction set, the model needs to update the answer to a specific query from an old answer to a new answer; on the knowledge retention dataset, the model needs to maintain its original output behavior without significant changes. Step S3: Joint optimization loss function construction. A weighted summation overall loss function is constructed, in the form of where the loss function is selected based on the nature of the downstream task: for text generation tasks, sequence-level cross-entropy loss is typically used; for classification tasks, standard cross-entropy loss is used; and for regression tasks, mean squared error loss is used. The knowledge editing loss term is constructed using a null-space projection-based method. For a knowledge editing instruction, firstly, the gradient vector g of the base model's prediction with respect to all currently trainable parameters under the input is calculated. Then, a batch of samples is selected from the knowledge retention dataset, and the gradient of the model's prediction with respect to the parameters on these samples is calculated. These gradients are stacked to form a constraint matrix C. Singular value analysis is then performed on matrix C. The solution involves calculating the null space projection operator P, projecting the original edit gradient g onto the null space to obtain the edit direction vector. Theoretically, this direction can adjust the output of the target knowledge without changing the model's behavior on the knowledge retention dataset. Finally, the knowledge editing loss is defined as the cross-entropy loss between the model's prediction and the target answer after the model parameters are virtually updated along this direction. The knowledge retention loss term is constructed by constraining the stability of the output distribution. A batch of inputs is sampled from the knowledge retention dataset, and the output probability distributions of the original base model and the currently trained model on this batch of inputs are calculated respectively. Then, the KL divergence between these two distributions is calculated as the retention loss, thereby penalizing the deviation of the model's behavior from the original model. Step S4: Joint Iterative Update of Parameters. In each training iteration, a small batch of data is sampled from the fine-tuning dataset, the knowledge editing instruction set, and the knowledge retention dataset. These three batches of data are input into the model sequentially or simultaneously for forward propagation. The fine-tuning loss, knowledge editing loss, and knowledge retention loss are calculated separately. Then, the total loss is obtained by weighted summation according to the preset hyperparameters α and β. The gradient of the total loss with respect to the parameter matrix of each trainable low-rank adaptive module is calculated using the backpropagation algorithm. In this process, a gradient reconciliation mechanism can be used. Specifically, after obtaining the gradient vectors generated by the three tasks, the fine-tuning gradient and the knowledge editing gradient, and the fine-tuning gradient and the knowledge retention gradient are calculated. The cosine similarity between gradients is used. If a significant conflict is found, the three gradient vectors are normalized, and their weights in the final gradient synthesis are dynamically adjusted based on the performance of each task on the validation set to reduce directional conflicts. Subsequently, the optimizer uses only the synthesized gradients to update the parameter matrices of all low-rank adaptation modules, while the original weights of the base model remain frozen. The hyperparameters α and β can be dynamically adjusted. For example, at certain training epochs, the success rate of knowledge editing is evaluated on the validation set. If the success rate increases slowly, α is increased appropriately. At the same time, the model's general performance on the retained dataset is evaluated. If it decreases, β is increased. Step S5, Performance Monitoring and Model Convergence: During training, at certain intervals, the overall performance of the model is evaluated on an independent validation set. This validation set should include samples from downstream tasks, knowledge query samples that need to be edited, and samples representing general knowledge. Evaluation metrics include at least the accuracy, F1 score or perplexity of downstream tasks, and the success rate of knowledge editing tasks. The changing trends of these metrics are monitored. When the performance metrics of the main downstream tasks reach the preset threshold, and the success rate of knowledge editing and general performance no longer show significant improvement or decline in multiple consecutive evaluation periods, it can be determined that the model has converged and the training process is stopped. Step S6, Model Merging and Deployment: After training, the optimized low-rank adaptation parameter matrix pairs are obtained. For each original weight matrix W injected into the model into this module, a merging operation is performed. The merging formula is: The merging operation is a simple matrix addition with extremely low computational overhead. After merging, a complete model containing new task capabilities, updated knowledge, and retaining the original knowledge to the greatest extent is obtained. This model can be exported as a model file like any standard large language model and deployed to the corresponding inference platform or application environment.

[0030] To implement the above method, a joint optimization system can be constructed, which includes multiple collaborative modules. The model configuration module is responsible for loading the pre-trained base model and injecting and initializing the low-rank adaptation module at the specified layer according to the configuration. The data management module is responsible for storing and managing three sets of datasets: fine-tuning, knowledge editing, and knowledge retention, and provides a unified data sampling interface to support batch data acquisition. The loss calculation engine is the core computing unit, which integrates a fine-tuning loss calculation subunit, a knowledge editing loss calculation subunit based on null projection, and a knowledge retention loss calculation subunit based on KL divergence. It can quickly calculate various losses based on input data and current model parameters. The joint training module encapsulates the complete training loop, including forward propagation, backpropagation, gradient reconciliation, and optimizer update steps, driving iterative optimization of trainable parameters. The evaluation and monitoring module periodically calls the model for verification and evaluation during training and can dynamically adjust the hyperparameters in the loss function according to preset strategies. After training terminates, the model output module performs the merging operation of the low-rank adaptation parameters and the base weights and generates the final deployable model file. These modules are interconnected through clear interfaces, forming an automated process from data preparation to model output.

[0031] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A highly efficient joint optimization method for fine-tuning and knowledge editing of large language models, characterized in that, Includes the following steps: S1. Parameterization preparation: Select a pre-trained large language model as the base model, freeze all its base parameters, and inject low-rank adaptation (LoRA) modules in parallel into the attention mechanism module and feedforward neural network module of at least one Transformer layer of the base model. The LoRA module consists of a trainable low-rank parameter matrix pair (A, B). S2. Data Preparation and Task Definition: Prepare three sets of data in parallel, including a fine-tuning dataset for downstream task adaptation, a knowledge editing instruction set for updating the internal facts of the model, and a knowledge retention dataset for protecting the original knowledge; and define the fine-tuning task, knowledge editing task, and knowledge retention task based on the three sets of data respectively. S3. Construction of Joint Optimization Loss Function: Construct a unified joint optimization loss function, which is formed by a weighted sum of fine-tuning loss term, knowledge editing loss term and knowledge retention loss term, expressed as: L_total=L_finetune+αL_edit+βL_retain, where α and β are adjustable hyperparameters; S4. Joint Parameter Iterative Update: During the training iteration, batch data is sampled from the three sets of data simultaneously; the value of the joint optimization loss function is calculated through forward propagation, and the gradient of the loss function with respect to the trainable parameter matrix (A, B) of the LoRA module is calculated through backpropagation algorithm; the optimizer is used to update only the trainable parameter matrix, while keeping the parameters of the base model frozen. S5. Performance Monitoring and Model Convergence: During training, the model performance is periodically evaluated on an independent validation set. The performance evaluation includes at least the performance metrics of downstream tasks and the success rate of knowledge editing tasks. When the performance metrics reach a preset threshold or tend to stabilize, the model is determined to have converged, and training is stopped. S6. Model Merging and Deployment: After training, the optimized LoRA parameter matrix (A', B') is linearly merged with the corresponding original weight matrix W of the base model to obtain the updated weight matrix W'=W+B'A'; the complete model parameters obtained after merging are exported and deployed.

2. The method according to claim 1, characterized in that, In step S3, the knowledge editing loss term L_edit is constructed using an editing method based on null projection, specifically including: S31. For a knowledge editing instruction (x_e, y_e), where x_e is the trigger query and y_e is the target answer, the basic model is regarded as a function f(θ;x), where θ is the model parameter; S32. Calculate the gradient of the model prediction with respect to the parameters: g = ∇_θf(θ;x_e)|_{θ=θ_0}, where θ_0 is the current parameter; S33. Construct a constraint matrix C, whose row vectors are spanned by a set of input gradients related to the knowledge to be protected, and calculate the null space projection operator P of the constraint matrix C through singular value decomposition. S34. Project the original gradient g onto the null space to obtain the editing direction vector Δθ_edit=Pg; S35. The knowledge editing loss term is defined as the cross-entropy loss between the model's prediction and the target answer y_e after the model is updated along the editing direction: L_edit=CE(f(θ_0+Δθ_edit;x_e),y_e).

3. The method according to claim 1, characterized in that, In step S3, the knowledge retention loss term L_retain is constructed through model output distribution stability constraints, specifically including: S31' Sample a batch of data {x_r} from the knowledge retention dataset; S32' Calculate the output probability distributions P_original and P_current of the base model (parameter θ_0) and the currently trained model (parameter θ_0+Δθ_lora) after input {x_r}, respectively; S33' Calculate the KL divergence between the two distributions as the knowledge retention loss: L_retain=Σ_{x_r}D_KL(P_original||P_current).

4. The method according to claim 1, characterized in that, In step S3, the fine-tuning loss term L_finetune is selected according to the type of the downstream task: for generative tasks, the sequence cross-entropy loss is used; for classification tasks, the cross-entropy loss is used; and for regression tasks, the mean squared error loss is used.

5. The method according to claim 1, characterized in that, In step S4, a dynamic weight adjustment strategy is used to optimize the hyperparameters α and β. Specifically, the value of α is dynamically increased or decreased based on the rate of change of the success rate of the knowledge editing task on the validation set; and the value of β is dynamically adjusted based on the degree of performance degradation on downstream tasks and general tasks on the validation set.

6. The method according to claim 1, characterized in that, In step S1, when injecting the LoRA module, a hierarchical adaptive rank determination strategy is adopted, which specifically includes: performing sensitivity analysis on different network layers of the basic model, assigning higher low-rank dimensions r to layers that are more sensitive to parameter changes and have a greater impact on task performance, and assigning lower ranks to layers that are less sensitive to parameter changes and have a greater impact on task performance, so as to further optimize parameter efficiency.

7. The method according to claim 1, characterized in that, In step S4, a gradient reconciliation mechanism is adopted during training iteration. Specifically, after calculating the gradients of the fine-tuning task, knowledge editing task, and knowledge preservation task with respect to the shared LoRA parameters through backpropagation, cosine similarity detection and normalization weighting are performed on the three gradient vectors before parameter updates to reduce conflicts between gradients of different tasks and ensure the synergy of optimization directions.

8. A joint optimization system for implementing the method according to any one of claims 1-7, characterized in that, include: The model configuration module is used to load the pre-trained base model, inject and initialize the LoRA module; The data management module is used to store and manage fine-tuning datasets, knowledge editing instruction sets, and knowledge retention datasets, and provides a data sampling interface; The loss calculation engine has a built-in fine-tuning loss calculation unit, a knowledge editing loss calculation unit based on null projection, and a knowledge preservation loss calculation unit, which are used to calculate the joint optimization loss function based on the input data and the current model state. The joint training module includes an optimizer for performing forward propagation, back propagation, and parameter update loops, and integrates the gradient reconciliation mechanism. The evaluation and monitoring module is used to periodically evaluate model performance during training and adjust training hyperparameters according to preset strategies. The model output module is used to merge the trained LoRA parameters with the base model and export the final model file.

Images