Differential privacy subspace fine tuning training method and device for large language model

Abstract

The invention discloses a self-adaptive routing and decision-making method and device based on large model intelligent question and answer. The method comprises the following steps: obtaining a user privacy data set for performing large language model training on a target language processing task; performing a round of all-parameter fine tuning training on the large language model based on a pre-acquired target data set, and recording model parameters according to a preset step length to obtain a model parameter updating track; according to the model parameter updating trajectory, constructing a model parameter trajectory matrix; performing matrix decomposition on the model parameter track matrix to obtain an orthogonal projection matrix of a target low-dimensional subspace; and performing differential privacy fine tuning training on the large language model based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy data set to obtain a trained target large language model. According to the method, noise injection is limited in a low-dimensional subspace, and the noise dimension is remarkably reduced, so that the performance close to non-privacy fine tuning is realized while the differential privacy constraint is met.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of large language model optimization and privacy protection, specifically relating to a differential privacy subspace fine-tuning training method and apparatus for large language models. Background Technology

[0002] Large Language Models (LLMs) have achieved significant success in natural language processing tasks. To adapt these models to specific downstream tasks (such as sentiment analysis and text classification), fine-tuning is typically required on datasets containing users' private data. This process carries the risk of privacy breaches; malicious attackers could potentially infer sensitive information from the training data by analyzing model parameters or gradients. Differential Privacy (DP), a rigorous mathematical privacy framework, provides provable privacy protection by injecting calibrated noise into the computation process, ensuring that the presence or absence of any single data sample does not significantly affect the algorithm's final output.

[0003] Currently, the typical method for applying dynamic programming (DP) to LLM fine-tuning is DP-SGD, which involves pruning the gradient of each sample (to limit sensitivity) and adding Gaussian noise in each iteration. However, existing techniques have significant drawbacks: LLMs typically have hundreds of millions or even tens of billions of high-dimensional parameters. Injecting noise that satisfies the requirements of DP into this high-dimensional space can lead to excessively large noise norms. This high-dimensional noise severely interferes with the gradient update direction, resulting in a significant drop in model utility (accuracy) and difficulty in converging during training.

[0004] These issues are particularly pronounced in highly privacy-sensitive applications such as medical data. When hospitals want to fine-tune models using their internal clinical dialogues and diagnostic reports to assist doctors, the high privacy value of the data and stringent regulatory requirements (such as HIPAA) make differential privacy essential. Existing full-parameter noise injection severely impairs the model's understanding of complex medical terminology and the accuracy of logical reasoning, leading to unreliable auxiliary suggestions. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides an adaptive routing and decision-making method and apparatus based on large-model intelligent question answering.

[0006] The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a differential privacy subspace fine-tuning training method for large language models, comprising: Obtain a user-private dataset for training large language models for target language processing tasks; Based on the pre-acquired target dataset, a full-parameter fine-tuning training of the large language model is performed, and the model parameters are recorded according to the preset step size to obtain the model parameter update trajectory. Update the trajectory based on the model parameters and construct the model parameter trajectory matrix; The trajectory matrix of the model parameters is decomposed to obtain the orthogonal projection matrix of the target low-dimensional subspace; Based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, differential privacy fine-tuning training is performed on the large language model to obtain the trained target large language model.

[0007] Secondly, the present invention provides a differential privacy subspace fine-tuning training device for large language models, comprising: The acquisition module is used to acquire user privacy datasets used for training large language models for target language processing tasks. The full-parameter fine-tuning training module is used to perform a round of full-parameter fine-tuning training on a large language model based on a pre-acquired target dataset, and record the model parameters according to a preset step size to obtain the model parameter update trajectory. The trajectory matrix construction module is used to update the trajectory based on the model parameters and construct the model parameter trajectory matrix. The low-dimensional subspace construction module is used to perform matrix decomposition on the trajectory matrix of the model parameters to obtain the orthogonal projection matrix of the target low-dimensional subspace. The differential privacy fine-tuning training module is used to perform differential privacy fine-tuning training on a large language model based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, so as to obtain the trained target large language model.

[0008] This invention provides a differential privacy subspace fine-tuning training method and apparatus for large language models. By restricting noise injection to a low-dimensional subspace, the noise dimensionality is significantly reduced, thereby satisfying strict privacy requirements. While maintaining differential privacy constraints, it achieves near-non-privacy fine-tuning performance, significantly outperforming existing differential privacy fine-tuning methods. Furthermore, it avoids high-dimensional noise interfering with gradient update directions during training, resulting in faster convergence and significantly reduced volatility. In terms of efficiency, low-dimensional subspace construction requires only a single round of fine-tuning, with extremely low computational overhead.

[0009] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a differential privacy subspace fine-tuning training method for large language models provided in an embodiment of the present invention. Figure 2This is a schematic diagram of the structure of a differential privacy subspace fine-tuning training device for large language models according to an embodiment of the present invention. Detailed Implementation

[0011] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0012] This invention provides a differential privacy subspace fine-tuning training method for large language models. See also... Figure 1 The method includes the following steps: S10. Obtain the user privacy dataset for training a large language model for the target language processing task.

[0013] For example, the target language processing task can be a medical diagnosis assistance task, and the corresponding user privacy dataset can be the patient's historical diagnosis record data.

[0014] Before training, assume the parameters of the pre-trained large language model are... ( For parameter dimensions, usually The user privacy dataset corresponding to the target language processing task is... ( ), Indicates the first One sample, Indicates the first The labels corresponding to each sample. The preset privacy parameters are... .

[0015] The training objective of this embodiment is to adapt the large language model to the target language processing task through fine-tuning, while simultaneously satisfying... And its performance is close to that of non-privacy fine-tuning.

[0016] S20. Based on the pre-acquired target dataset, perform a round of full parameter fine-tuning training on the large language model, and record the model parameters according to the preset step size to obtain the model parameter update trajectory.

[0017] For example, gradient update trajectories relevant to the target language processing task are captured through short-cycle full-parameter fine-tuning. The pre-acquired target dataset is relevant to the target language processing task and can be a user-private dataset or a non-sensitive public dataset similar to the target language processing task. For example, in a user-private dataset... Perform one round of full parameter fine-tuning and update the model parameters using a standard optimizer (such as Adam). Record a snapshot of the model parameters every fixed step size (i.e., a preset step size) to obtain the model parameter update trajectory.

[0018] Optionally, the model parameter update trajectory includes the initial pre-trained model parameters. And multiple sets of updated model parameters recorded during training, such as , This represents the index of each group of updated model parameters.

[0019] S30. Update the trajectory based on the model parameters and construct the model parameter trajectory matrix.

[0020] Optionally, step S40 may specifically include: S401. Calculate the difference between each updated model parameter and the initial pre-trained model parameter to obtain multiple parameter update values.

[0021] For example, the difference between each updated model parameter and the initial pre-trained model parameter is calculated, i.e., the difference between the updated model parameters and the initial pre-trained model parameters is calculated. This yields multiple parameter update values, which can be expressed as follows: .

[0022] S402. Expand each parameter update into a one-dimensional vector and stack them row by row to obtain the model parameter trajectory matrix.

[0023] For example, the model parameter trajectory matrix is ​​denoted as , can be represented as:

[0024] S40. Perform matrix decomposition on the trajectory matrix of the model parameters to obtain the orthogonal projection matrix of the target low-dimensional subspace.

[0025] Optionally, step S40 may specifically include: S401. The singular value decomposition algorithm is used to decompose the trajectory matrix of the model parameters to obtain the initial right singular vector matrix and singular value matrix.

[0026] For example, the Singular Value Decomposition (SVD) algorithm is used to decompose the model parameter trajectory matrix, as follows:

[0027] in, Represents a left singular vector matrix (orthogonal matrix); This represents a singular value matrix (diagonal matrix), where the diagonal elements are singular values; It is a right singular vector matrix (orthogonal matrix).

[0028] S402. Sort the singular values ​​in the singular value matrix in descending order to obtain the singular value sorting.

[0029] For example, the singular values ​​are sorted from largest to smallest to obtain the singular value sorting.

[0030] S403. Based on the singular values, determine the first [singular vector] from the initial right singular vector matrix. The right singular vectors corresponding to the singular values ​​are used to construct the orthogonal projection matrix of the target low-dimensional subspace; where, This is the default value.

[0031] For example, the dimension of the target low-dimensional subspace is ( ). The value can be determined by a preset cumulative energy percentage threshold, for example, by determining... This makes the first singular value sorted The sum of squares of the largest singular value accounts for more than the proportion of the total sum of squares when it exceeds a preset cumulative energy percentage threshold (e.g., 99%).

[0032] Due to the previous The right singular vectors corresponding to the maximal singular values ​​contain the main information of the model parameter trajectory matrix. Therefore, the first maximal singular vectors are determined from the initial right singular vector matrix. The right singular vector corresponding to each singular value Construct the orthogonal projection matrix of the target low-dimensional subspace, denoted as: This matrix defines the target low-dimensional subspace. and the original parameter space Mapping to the target low-dimensional subspace .

[0033] In this embodiment, the construction of the target low-dimensional subspace requires only one round of full parameter fine-tuning (short trajectory), and the computational overhead is negligible; moreover, the target low-dimensional subspace captures the general optimization geometry of the task, rather than private sample information, and supports cross-task transfer.

[0034] S50. Based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, differential privacy fine-tuning training is performed on the large language model to obtain the trained target large language model.

[0035] Optionally, step S50 may specifically include: S501. In each round of training based on the user privacy dataset after a round of full parameter fine-tuning training, determine the corresponding initial gradient based on the model parameters of the current step.

[0036] For example, in each round of training using a user-private dataset after a round of full-parameter fine-tuning training, based on the current step Model parameters Calculate the current step The initial gradient is denoted as ( (This is the loss function). Here, the initial gradient is a high-dimensional gradient.

[0037] S502. Based on the orthogonal projection matrix of the target low-dimensional subspace, the initial gradient is projected onto the target low-dimensional subspace to obtain the low-dimensional gradient.

[0038] Alternatively, the low-dimensional gradient can be expressed as:

[0039] in, Represents a low-dimensional gradient. This represents the orthogonal projection matrix of the target's low-dimensional subspace. Indicates the initial gradient. Indicates the current step.

[0040] S503. Perform L2 norm clipping on the low-dimensional gradient to obtain the gradient clipping result.

[0041] Alternatively, the gradient clipping result can be expressed as:

[0042] in, This represents the result of gradient clipping. This represents the clipping norm.

[0043] S504. Inject noise into the gradient clipping result to obtain a noisy low-dimensional gradient.

[0044] Alternatively, the noisy low-dimensional gradient can be expressed as:

[0045] in, This represents a noisy low-dimensional gradient. This represents Gaussian noise that meets differential privacy requirements. express An identity matrix of dimensionality; Based on preset privacy parameters Sampling probability Total training steps pass Function calibration, expressed as:

[0046] in, This represents the Gaussian difference privacy noise multiplier calculation function. Indicates privacy budget, This represents the relaxation parameter.

[0047] S505. Based on the orthogonal projection matrix of the target low-dimensional subspace, the noisy low-dimensional gradient is mapped to obtain the noisy gradient.

[0048] For example, the noisy low-dimensional gradient is mapped back to the original parameter space using an orthogonal projection matrix to obtain the noisy gradient.

[0049] Alternatively, the noisy gradient can be expressed as:

[0050] in, This represents the gradient with added noise.

[0051] S506. Update the model parameters of the current step based on the noisy gradient to obtain the model parameters of the next step; until the preset training rounds are reached, and the target large language model is obtained after training.

[0052] For example, after reshaping the gradients to the original parameter format, the model parameters are updated using the Adam / SGD optimizer, as shown below:

[0053] in, Indicates the model parameters for the next step. This refers to the optimizer.

[0054] This embodiment performs differential privacy fine-tuning in the constructed low-dimensional subspace, avoiding high-dimensional noise in the full parameter space.

[0055] Furthermore, based on the combinatorial and post-processing invariance properties of differential privacy, the method in this embodiment can satisfy... , specifically: 1) Target low-dimensional subspace construction stage: Short-period full parameter fine-tuning adopts DP-SGD to satisfy... ;SVD is a deterministic post-processing method and does not compromise privacy guarantees; 2) Privacy subspace training phase: Gradient clipping and noise injection in the target low-dimensional subspace conform to the Gaussian mechanism, satisfying Gradient mapping is post-processed and does not affect the privacy budget; 3) Composition: Total privacy budget is It meets the preset privacy requirements.

[0056] Furthermore, the target low-dimensional subspace in this embodiment can be constructed based on public datasets (such as IMDB and SST-2) similar to the target language processing task type, without consuming the privacy budget of private data, and can be directly migrated to relevant sensitive tasks. Since the relevant tasks share the same optimization direction for parameters, the performance loss of migrating the subspace is negligible.

[0057] This embodiment provides a differential privacy subspace fine-tuning training method for large language models. By restricting noise injection to a low-dimensional subspace, the noise dimensionality is reduced from hundreds of millions. Significantly reduced to 32 / 64 Thus, while meeting strict requirements While maintaining differential privacy constraints, this method achieves near-non-privacy fine-tuning performance, significantly outperforming existing differential privacy fine-tuning methods. Thanks to this design, the training process avoids interference from high-dimensional noise on gradient update directions, resulting in faster convergence and significantly reduced volatility. In terms of efficiency, low-dimensional subspace construction requires only a single round of fine-tuning, with extremely low computational overhead, and supports cross-task transfer without consuming additional privacy budget. Furthermore, this method exhibits strong versatility, adaptable to various large language models including RoBERTa and LLaMA, and widely applicable to downstream tasks such as sentiment classification, reasoning, and question answering. Compared to existing differential privacy methods, this embodiment demonstrates superior model performance.

[0058] Furthermore, the method in this embodiment can efficiently and stably fine-tune large-scale pre-trained language models for task adaptation while protecting user data privacy. Its advantages and necessity are particularly prominent in highly privacy-sensitive application scenarios such as medical data. For example, when hospitals want to fine-tune models using their internal clinical dialogues and diagnostic reports to assist doctors, the high privacy value of the data and strict regulatory requirements (such as HIPAA) make differential privacy essential. Traditional full-parameter noise injection severely impairs the model's understanding of complex medical terminology and the accuracy of logical reasoning, leading to unreliable auxiliary suggestions. This method, by constraining noise to a low-dimensional gradient space closely related to tasks such as "diagnostic logic" and "symptom description," can maximize the preservation of the model's medical expertise and reasoning ability while ensuring that individual patient records are not leaked. This results in a privacy-preserving model that can pass compliance audits and provide truly accurate and stable decision support for medical personnel.

[0059] The following simulation experiment further illustrates the differential privacy subspace fine-tuning training method for large language models provided by this invention.

[0060] To verify the effectiveness of the method of the present invention, comparative experiments and ablation experiments were set up. The experimental environment was based on the HuggingFace Transformers framework, and the core parameters were configured uniformly.

[0061] I. Experimental Setup 1. Datasets: 4 benchmark datasets for natural language processing (including sentiment classification, question answering entailment, and natural language inference).

[0062]

[0063] 2. Basic model RoBERTa-base (12-layer encoder, 768-dimensional hidden layers, 125 million parameters); 3. Baseline Method 1) Fulling-tuning (non-privacy, full parameter fine-tuning); 2) Full-DP (Full-parameter DP-SGD fine-tuning); 3) LoRA-DP (low-rank adapter module DP fine-tuning, r=16); 4) Adapter-DP (Adapter module DP fine-tuning, r=48); 4. Core parameters 1) Batch size = 32, maximum sequence length = 128 (SST-2 / QNLI), 256 (IMDB / MNLI); 2) Comparison of learning rates pruning threshold The learning rate of this invention pruning threshold ; 3) Dimensionality of low-dimensional subspaces =32 (SST-2 / IMDB / QNLI), 64 (MNLI); 4) Relaxation term (SST-2 / IMDB / QNLI) (MNLI), privacy parameters

[0064] Experiment 1: Performance Comparison Test under Privacy Budget To verify the core advantage of the method under differential privacy constraints—balancing privacy protection and model performance—a performance comparison experiment under a privacy budget was designed. Accuracy was used as the core evaluation metric, representing the percentage of samples correctly classified or inferred by the model in the downstream task test set. It directly reflects the model's adaptability to the task and is a key criterion for measuring model usability in privacy-preserving scenarios. Considering the differences in privacy requirements in practical applications, a standard privacy constraint was set in the experiment (…). ) and extreme privacy constraints In two scenarios, Full-DP fine-tuning (Full-DP) and LoRA-DP (Adapter-DP) were selected as baseline methods, while Full-tuning without privacy protection was used as a performance upper limit reference. By comparing the accuracy of different methods on four datasets covering different task types and data characteristics (SST-2, IMDB, QNLI, MNLI), the aim is to verify whether the present invention can overcome the performance degradation problem caused by "high-dimensional noise injection" in existing DP fine-tuning methods, minimize the performance gap with non-privacy fine-tuning under the premise of strictly satisfying (ε,δ)-differential privacy guarantee, and significantly outperform the performance of existing privacy fine-tuning methods.

[0065] Table 1: Accuracy Comparison under Standard Privacy Constraints (ε=4)

[0066] Table 2: Extreme Privacy Constraints ( Accuracy comparison under )

[0067] Experimental conclusions: This invention significantly outperforms all DP baseline methods under both privacy budgets. When ε=4, the performance gap with non-privacy fine-tuning is only 1.84% (SST-2), and when ε=1, it leads Adapter-DP by 23.38% on IMDB, demonstrating a strong ability to balance privacy and performance.

[0068] Experiment 2: Subspace Construction Efficiency Experiment To verify the efficiency and practicality of the subspace construction stage of this invention, a subspace construction efficiency experiment was designed in a non-privacy environment. The core focus was on the key question of whether short trajectories can effectively capture task-specific optimization patterns. The experiment still used accuracy as the core indicator to measure the ability of subspaces constructed with different trajectory lengths to support model fine-tuning. In actual deployment, long-cycle parameter trajectory acquisition leads to high computational overhead, which is detrimental to efficient fine-tuning of large-scale models. Therefore, this experiment compared the subspace performance differences between 1-round and 4-round short-cycle fine-tuning trajectories and the standard 32-round full-fine-tuning trajectory. By analyzing the model accuracy and performance loss (compared to full fine-tuning) corresponding to different trajectory lengths, this experiment verifies whether subspaces constructed with short trajectories (especially those with only 1 round of fine-tuning) can retain sufficient task-related information to ensure that the model still has high task adaptability after fine-tuning. Furthermore, it quantifies the computational efficiency improvement brought by short trajectories (such as the reduction in construction time), providing data support for the practical feasibility of the method.

[0069] Table 3: Subspace fine-tuning accuracy for different trajectory lengths

[0070] Experimental conclusion: The average performance loss of the subspace constructed by one round of trajectory construction is less than 1%, but the construction time is reduced by 96.9%, which verifies the efficiency of the present invention.

[0071] Experiment 3: Subspace Mobility Experiment To enhance the deployment flexibility and privacy efficiency of this invention in real-world privacy protection scenarios, a subspace transfer experiment was designed to address the core pain point of limited privacy budgets for sensitive data. The experiment uses accuracy, performance loss (compared to a task-specific ideal subspace), and performance improvement (compared to traditional full-parameter DP fine-tuning) as key evaluation metrics. Accuracy reflects the task utility of the transferred model, while performance loss and improvement quantify the effectiveness of the transferred subspace and its advantages over existing methods, respectively. Based on the core insight of "optimization of shared parameters for related tasks as the dominant direction," the experiment uses publicly available, non-sensitive datasets (such as IMDB and SST-2) to construct subspaces, which are then transferred to relevant sensitive target tasks for differential privacy fine-tuning, without consuming the privacy budget of sensitive data. This experiment aims to verify whether the extracted subspace possesses cross-task generality—that is, whether it can be independently constructed without the private data of the target task, while still supporting efficient model fine-tuning. Furthermore, it verifies whether the transferred subspace can maintain good performance under privacy constraints, further reducing the implementation cost and data dependence of privacy protection.

[0072] Table 4: Subspace migration performance under non-privacy settings

[0073] Table 5: Subspace migration performance under DP setting (ε=4)

[0074] Experimental conclusion: The performance loss of the migration subspace is controllable and significantly better than direct full-parameter DP fine-tuning, verifying the universality and privacy efficiency of the subspace.

[0075] Experiment 4: Noisy Trajectory Experiment To investigate the sensitivity of the subspace construction stage to noise and to verify the impact of subspace quality on final performance under privacy-constrained scenarios, this experiment designed a noisy trajectory experiment. Accuracy is the core evaluation metric, focusing on the model's performance in subsequent private fine-tuning after introducing differential privacy noise during the subspace construction stage. The results are compared with full-parameter differential privacy fine-tuning (Full-DP) and ideal non-privacy subspaces to quantify the impact of noise on subspace representativeness. Based on the core hypothesis that "introducing noise during the subspace construction stage will cause the learned optimization direction to deviate from its inherent direction, thereby weakening the subspace's ability to represent the dominant update direction," the experiment uses DP-SGD to generate training trajectories in the first stage, setting the privacy budget to ε=3. Then, in the second stage, differential privacy fine-tuning is performed in a low-dimensional subspace, setting ε=1, with the total budget controlled at ε_total=4. This experiment aims to systematically evaluate the impact of noise on subspace quality, clarify whether an effective task-related subspace can still be constructed under a limited privacy budget, and verify the robustness of the proposed method under noise interference, providing a basis for whether publicly available data should be prioritized for subspace construction in practical deployments.

[0076] Table 6: Noisy Trajectory Experiment

[0077] Experimental conclusion: Introducing noise during the subspace construction stage severely weakens the model's ability to capture key update directions, thus limiting the final effect of subsequent private training. Therefore, this experiment further reinforces the design principle of this invention: when conditions permit, high-quality, noise-free subspaces should be constructed using publicly available data to maximize the integrity of the optimization trajectory and improve the overall effectiveness of privacy fine-tuning.

[0078] Corresponding to the above-described differential privacy subspace fine-tuning training method for large language models, this embodiment of the invention also provides a differential privacy subspace fine-tuning training device for large language models; such as Figure 2 As shown, the device may include: Module 201 is used to acquire user privacy datasets for training large language models for target language processing tasks. The full-parameter fine-tuning training module 202 is used to perform a round of full-parameter fine-tuning training on a large language model based on a pre-acquired target dataset, and record the model parameters according to a preset step size to obtain the model parameter update trajectory. The trajectory matrix construction module 203 is used to update the trajectory based on the model parameters and construct the model parameter trajectory matrix; The low-dimensional subspace construction module 204 is used to perform matrix decomposition on the trajectory matrix of the model parameters to obtain the orthogonal projection matrix of the target low-dimensional subspace. The differential privacy fine-tuning training module 205 is used to perform differential privacy fine-tuning training on a large language model based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, so as to obtain the trained target large language model.

[0079] For details regarding the device, please refer to the steps of the differential privacy subspace fine-tuning training method for large language models provided in the first aspect; it will not be repeated here.

[0080] This embodiment provides a differential privacy subspace fine-tuning training device for large language models, which has the following advantages: 1. By performing short-cycle full-parameter fine-tuning on target task data (whether public or private), the update trajectory of model parameters is collected, and a low-dimensional, task-related gradient subspace is extracted from this trajectory using matrix factorization techniques (such as SVD). This subspace can effectively capture the core direction of task optimization.

[0081] 2. During model fine-tuning, the calculated high-dimensional gradient is projected onto the aforementioned low-dimensional subspace. Gradient clipping and Gaussian noise that meets differential privacy requirements are only performed in this task-related low-dimensional space. Finally, the noisy low-dimensional gradient is mapped back to the original parameter space to update the model.

[0082] 3. The low-dimensional task subspace can be constructed based on a non-sensitive public dataset similar to the target language processing task, and then directly transferred to the privacy-sensitive target task. This reduces or eliminates the consumption of the privacy budget of the target private data during the subspace construction phase, thereby improving privacy efficiency.

[0083] It should be noted that the device is basically similar to the method embodiment, so the description is relatively simple. For relevant parts, please refer to the description of the method embodiment.

[0084] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention.

[0085] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0086] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.

[0087] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A differential privacy subspace fine-tuning training method for large language models, characterized in that, include: Obtain a user-private dataset for training large language models for target language processing tasks; Based on the pre-acquired target dataset, a full-parameter fine-tuning training of the large language model is performed, and the model parameters are recorded according to the preset step size to obtain the model parameter update trajectory. Update the trajectory based on the model parameters and construct the model parameter trajectory matrix; The trajectory matrix of the model parameters is decomposed to obtain the orthogonal projection matrix of the target low-dimensional subspace; Based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, differential privacy fine-tuning training is performed on the large language model to obtain the trained target large language model.

2. The differential privacy subspace fine-tuning training method for large language models according to claim 1, characterized in that, The method of performing differential privacy fine-tuning training on the large language model based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset to obtain the trained target large language model includes: In each training round based on the user privacy dataset after the first round of full parameter fine-tuning training, the initial gradient is determined based on the model parameters of the current step; the initial gradient is projected onto the target low-dimensional subspace based on the orthogonal projection matrix of the target low-dimensional subspace to obtain a low-dimensional gradient; the low-dimensional gradient is clipped using the L2 norm to obtain a gradient clipping result; noise is injected into the gradient clipping result to obtain a noisy low-dimensional gradient; the noisy low-dimensional gradient is mapped based on the orthogonal projection matrix of the target low-dimensional subspace to obtain a noisy gradient; the model parameters of the current step are updated based on the noisy gradient to obtain the model parameters for the next step. The process continues until the preset number of training rounds is reached, at which point the target large language model is obtained after training.

3. The differential privacy subspace fine-tuning training method for large language models according to claim 2, characterized in that, The step of performing matrix decomposition on the model parameter trajectory matrix to obtain the orthogonal projection matrix of the target low-dimensional subspace includes: The singular value decomposition algorithm is used to decompose the trajectory matrix of the model parameters to obtain the initial right singular vector matrix and singular value matrix. The singular values ​​in the singular value matrix are sorted in descending order to obtain the singular value sorting. Based on the singular value sorting, determine the first [singular vector] from the initial right singular vector matrix. The right singular vectors corresponding to the singular values ​​are used to construct the orthogonal projection matrix of the target low-dimensional subspace; where, This is the default value.

4. The differential privacy subspace fine-tuning training method for large language models according to claim 3, characterized in that, The low-dimensional gradient is expressed as: in, This represents the low-dimensional gradient. This represents the orthogonal projection matrix of the target's low-dimensional subspace. This represents the initial gradient. Indicates the current step.

5. The differential privacy subspace fine-tuning training method for large language models according to claim 4, characterized in that, The gradient clipping result is expressed as follows: in, This represents the gradient clipping result. This represents the clipping norm.

6. The differential privacy subspace fine-tuning training method for large language models according to claim 5, characterized in that, The noisy low-dimensional gradient is expressed as: in, This represents the noisy low-dimensional gradient. This represents Gaussian noise that meets differential privacy requirements. express An identity matrix of dimensionality; Based on preset privacy parameters Sampling probability Total training steps pass Function calibration, expressed as: in, This represents the Gaussian difference privacy noise multiplier calculation function. Indicates privacy budget, This represents the relaxation parameter.

7. The differential privacy subspace fine-tuning training method for large language models according to claim 6, characterized in that, The noise gradient is expressed as: in, This represents the noise gradient.

8. The differential privacy subspace fine-tuning training method for large language models according to claim 1, characterized in that, The model parameter update trajectory includes initial pre-trained model parameters and multiple sets of updated model parameters recorded during training. Constructing a model parameter trajectory matrix based on the model parameter update trajectory includes: Calculate the difference between each of the updated model parameters and the initial pre-trained model parameters to obtain multiple parameter update values; The parameter update values ​​are expanded into one-dimensional vectors and stacked row by row to obtain the model parameter trajectory matrix.

9. A differential privacy subspace fine-tuning training device for large language models, characterized in that, include: The acquisition module is used to acquire user privacy datasets used for training large language models for target language processing tasks. The full-parameter fine-tuning training module is used to perform a round of full-parameter fine-tuning training on a large language model based on a pre-acquired target dataset, and record the model parameters according to a preset step size to obtain the model parameter update trajectory. The trajectory matrix construction module is used to update the trajectory based on the model parameters and construct the model parameter trajectory matrix; The low-dimensional subspace construction module is used to perform matrix decomposition on the trajectory matrix of the model parameters to obtain the orthogonal projection matrix of the target low-dimensional subspace; The differential privacy fine-tuning training module is used to perform differential privacy fine-tuning training on the large language model based on the orthogonal projection matrix of the target low-dimensional subspace and the user privacy dataset, so as to obtain the trained target large language model.

Images