Large model training method and device based on adaptive differential privacy control

Abstract

The invention discloses a large model training method and device based on adaptive differential privacy control, and the method and device are used for training a task model based on a large language model, and in each training round, the method comprises the steps: inputting training sample data into the task model for calculation based on a parameter correlation evaluation method, and generating a trainable parameter correlation graph; performing high-order expansion and approximate estimation on privacy loss in the training process by using a high-order privacy budget estimation and control method, controlling the noise reference intensity under the current training round, and performing dynamic adjustment to ensure that the whole training process meets the differential privacy constraint; and constructing a binary mask according to the trainable parameter correlation graph, dividing high and low correlation parameter subsets, and performing differential clipping and noise adding processing on gradients corresponding to the high and low correlation parameters in combination with the current noise reference intensity to complete task model parameter updating under the differential privacy guarantee. Noise introduction can be remarkably reduced, and the model performance is improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information security technology, and in particular to a large model training method and apparatus based on adaptive differential privacy control. Background Technology

[0002] As multiple industries advance their intelligent transformation, how to efficiently utilize data resources and train artificial intelligence models that combine security and performance under strict privacy protection requirements has become a core challenge that urgently needs to be addressed.

[0003] Especially in sensitive data-intensive scenarios such as financial risk control, healthcare, judicial trials, and government administration, training data often involves sensitive fields such as personal identity, health records, property information, historical behavior, and even judicial records. Taking the railway finance sector as an example, with the continuous advancement of the digital integration of "railway + finance," a large amount of highly sensitive structured and unstructured data has accumulated in businesses such as bill payment, fund settlement, corporate credit, insurance claims, and railway supply chain finance. This data includes customer identity information, payment trajectories, station transaction details, travel routes, and corporate accounts. Once illegally mined or leaked, it could trigger major financial security incidents and social impacts.

[0004] However, such financial data is often distributed across multiple subsystems and institutions, resulting in "data silos" and "privacy untrustworthiness" issues, making it difficult to directly incorporate traditional large-scale model fine-tuning solutions. Under regulatory compliance requirements, there is an urgent need to explore "computable but invisible" privacy-enhancing modeling methods to both protect the privacy and security of user data in railway financial systems and achieve efficient customized model training for tasks such as risk identification, credit assessment, and intelligent question answering. While large-scale pre-trained models (such as BERT and GPT) demonstrate superior performance across multiple tasks, their training costs and deployment complexity are also increasing, making direct training from scratch difficult. In practical applications, to improve model performance in specific scenarios, it is usually necessary to fine-tune existing large models to adapt them to new tasks, new data, or new users. Especially in the aforementioned sensitive data scenarios, the fine-tuning stage often involves iterative learning of the original sensitive data, which becomes a high-risk link for model "privacy leakage." Existing research shows that in the absence of privacy mechanisms, fine-tuned models may be vulnerable to security threats such as training data back-inference attacks and member inference attacks, thereby exposing the original data content.

[0005] To alleviate the aforementioned problems, Differential Privacy (DP) has been introduced into model training to limit the behavioral biases of the model when observing a single sample, thereby theoretically ensuring the inferability of individual data. The classic DP-SGD method controls the privacy budget by pruning gradients and applying Gaussian perturbations over multiple updates. However, in practical large model fine-tuning tasks, a uniform noise strategy can lead to a significant performance degradation, especially when the model has a large number of parameters and the task has high accuracy requirements. Noise can interfere with the learning process of high-value parameters and even obscure key information, causing model fine-tuning to fail.

[0006] Current differential privacy training still has shortcomings in two key areas: in terms of noise addition, traditional estimation methods are conservative, resulting in performance sacrifice; in terms of how to add noise, traditional solutions fail to identify the correlation of parameters and lack adaptive adjustment capabilities. Summary of the Invention

[0007] In view of the shortcomings of the prior art, the present invention provides a large model training method and apparatus based on adaptive differential privacy control. By integrating a high-order privacy budget estimator and a parameter correlation analysis strategy, a training scheme that combines privacy protection and model performance assurance is constructed.

[0008] Firstly, a large model training method based on adaptive differential privacy control is provided for training a task model based on a large language model. Within each training round, the method includes: Based on the parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and backward attribution calculation, generating a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. We utilize high-order privacy budget estimation and control methods to perform high-order expansion and approximate estimation of privacy loss during training. Without introducing additional relaxation, we accurately control the noise baseline intensity in the current training round and dynamically adjust it to ensure that the overall training process meets differential privacy constraints. A binary mask is constructed based on the correlation graph of trainable parameters to divide the high and low correlation parameters into subsets. The gradients corresponding to the high and low correlation parameters are then differentially pruned and noise-added based on the current noise baseline intensity, thus completing the parameter update of the task model under differential privacy protection.

[0009] Furthermore, the method for generating the trainable parameter correlation map specifically includes: The training sample data of each training round is input into the frozen pre-trained Big Prophet model to perform forward propagation, and its output and intermediate activation values ​​are recorded to generate output correlations. The output correlation is reduced in dimensionality to obtain the initial output correlation; Based on the parameter correlation evaluation method, the initial output correlation is propagated from the output layer to each trainable parameter layer by layer to obtain the trainable parameter correlation of each layer, and finally stitched into a complete and continuous trainable parameter correlation graph.

[0010] Furthermore, the process of dimensionality reduction of the output correlation includes: For each word, select a threshold l, and retain only the probability of the top l words in the vocabulary, while setting the probability of the remaining words to zero; For each word element, retain the words in the probability-based descending vocabulary until the cumulative probability of the retained words reaches a threshold p, and set the probability of the remaining words to zero; Based on the word units retained at each step, the cumulative probability is calculated according to the sentence order, and the top b sentence frames with the highest values ​​are selected. The probability of unselected words is set to zero.

[0011] Furthermore, the method of using high-order privacy budget estimation and control to perform high-order expansion and approximate estimation of privacy loss during training, controlling the noise baseline intensity in the current training round, and dynamically adjusting to ensure that the overall training process meets differential privacy constraints, specifically includes: Sampling rate and basic parameter initialization: Set the sampling rate, as well as the privacy budget cap ε, failure probability δ, and initial noise standard deviation σ. init Gradient clipping threshold C, unfolding order k, and scaling factor r used to control scaling noise; In each differential privacy mechanism M i Above, we define the output distributions of the adjacent datasets D and D′ as P, respectively. i =M i (D) and =M i (D′), from P i , We sampled from each sample, calculated the privacy loss variable, and represented it as follows: ; In the formula, X i Indicates that from distribution P i Samples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; The Radon–Nikodym derivative represents Relative to P i The probability density ratio; Privacy-depleting variables and The distribution function F X (z) and F Y (z) is expanded using a higher-order approximation, as follows: ; In the formula, Φ(z) is the standard normal distribution function. Let P be the density function corresponding to the standard normal distribution function Φ(z). j (z) is a polynomial term determined by the higher-order cumulants of the distribution, used to describe the non-normal deviation of the distribution in the higher-order expansion; V∈{X,Y}; n is the sample size; Find the minimum ε that satisfies the following inequality: ; In the formula, ω represents a higher-order approximation term or security relaxation term for the cumulative privacy loss; the minimum ε is denoted as ε (α) , ε (α) This represents the privacy budget loss under the current noise standard deviation σ; If the currently estimated ε (α) >ε indicates that the current noise configuration is insufficient to meet the privacy budget constraint, initiating an iterative scaling process: updating the noise standard deviation to σ←r·σ, where r is a scaling parameter; subsequently, recalculating the privacy cost variable and ε based on the updated σ. (α) The iterative process will continue until one of the following conditions is met: (1) The currently estimated privacy budget loss satisfies the constraint: ε (α) ≤ε; (2) The current noise standard deviation σ has dropped to the minimum allowable value set by the system and cannot be scaled further; Save the current noise standard deviation σ and privacy budget loss ε (α) .

[0012] Furthermore, the step of constructing a binary mask based on the correlation graph of trainable parameters and dividing it into subsets of high and low correlation parameters specifically includes: Let the fine-tuning dataset be D. FT ={x1,x2,...,x m ,...,x M}, where x m Let x represent the m-th sample; m The corresponding trainable parameter correlation graph is denoted as , where d is the total number of dimensions of all trainable parameters; Calculate the correlation plot of the average trainable parameters for all samples. ; Correlation plot of average trainable parameters Normalization is performed to obtain the correlation map of normalized trainable parameters. ; Set the correlation division ratio ∈(0,1), representing the proportion of parameters that are desired to be labeled as having low correlation, from Extract the first The smaller value is used as the dividing threshold R. thr ; Based on this, a correlation mask vector M is generated. R ∈{0,1} d This is used to mark the relevance of parameter dimensions: ; In the formula, Represents the correlation mask vector M R The p-th value; Represents the correlation graph of normalized trainable parameters The p-th value in; Based on the correlation mask vector M R The trainable parameters are divided into two subsets: high and low correlation.

[0013] Furthermore, the differentiated cutting process includes: For the two gradient subsets corresponding to the two parameter subsets with high and low correlation, differential L2 norm clipping is performed separately, with a global gradient clipping threshold of C>0. Calculate the proportionality coefficient: ; In the formula, This represents the subset of highly correlated gradients corresponding to the subset of highly correlated parameters. Sensitivity; This represents the subset of low-correlation gradients corresponding to the subset of low-correlation parameters. Sensitivity; This represents the original gradient of the task model in the current training batch, obtained by taking the derivative of the loss function with respect to the trainable parameters. Prune the two gradient subsets corresponding to the two parameter subsets of high and low correlation respectively: ; In the formula, This represents the subset of highly relevant gradients after pruning. This represents the subset of low-correlation gradients after clipping.

[0014] Furthermore, the differential noise addition process includes: The standard deviations of the highly relevant gradient subset and the low-relevance gradient subset are dynamically calculated based on the clipping results. ; In the formula, σ represents the current noise standard deviation. and represent the standard deviations of the highly correlated gradient subset and the low-correlation gradient subset, respectively; Based on this, noise is added to the clipped high-relevance gradient subset and low-relevance gradient subset respectively: ; In the formula, and Let N(0, ...) represent the high-correlation gradient subset and the low-correlation gradient subset after adding noise, respectively; I) indicates that the mean is 0. The covariance is a multidimensional Gaussian distribution. Pick or ; I is the identity matrix.

[0015] Furthermore, the specific steps for updating the task model parameters under differential privacy protection include: Differential cropping and differential noise addition are performed on each training sample in the current training batch in sequence to obtain the corresponding noise gradient; The average noisy gradient of the current training batch is obtained by averaging all the noisy gradients in the current training batch. Based on the set learning rate parameter, the gradient descent operation is used to update the parameters of the current task model using the average noisy gradient of the current training batch.

[0016] Secondly, a large model training device based on adaptive differential privacy control is provided for training a task model based on a large language model, the device comprising: The parameter correlation evaluation unit is configured to use a parameter correlation evaluation method to input training sample data into a task model based on a large language model for forward propagation and backward attribution calculation, and generate a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. The privacy budget estimation and control unit is configured to use high-order privacy budget estimation and control methods to perform high-order expansion and approximate estimation of privacy loss during training, accurately control the noise baseline intensity in the current training round without introducing additional relaxation, and dynamically adjust to ensure that the overall training process meets differential privacy constraints. The model parameter update unit is configured to construct a binary mask based on the correlation map of trainable parameters, divide the high and low correlation parameters into subsets, and perform differential pruning and differential noise addition on the gradients corresponding to the high and low correlation parameters in combination with the current noise baseline intensity setting, thereby completing the task model parameter update under differential privacy protection.

[0017] This invention proposes a large-scale model training method and apparatus based on adaptive differential privacy control. It integrates high-order privacy budget estimation and control methods with parameter correlation evaluation methods. Through high-order expansion, it accurately estimates the privacy loss, achieving tight privacy budget control. Simultaneously, it constructs a correlation graph of trainable parameters, dividing the trainable parameters into multiple subspaces based on correlation. Under the premise of satisfying overall privacy constraints, it implements a differentiated noise injection strategy: injecting less noise into highly correlated parameters and more noise into low-correlation parameters, thereby achieving adaptive weighting of gradient perturbations. This combined strategy of "precise global budget calculation + precise local perturbation control" effectively overcomes the problems of "coarse noise estimation and rigid perturbation configuration" in existing differential privacy training methods, significantly improving the usability and training effect of differential privacy mechanisms in engineering practice. Under the same privacy security conditions, this invention can achieve more accurate noise injection and resource allocation, thereby improving the accuracy and deployability of large models under sensitive tasks. Compared with existing methods, this invention can significantly reduce noise introduction, improve model performance, and reduce training costs. It can be widely applied in highly privacy-sensitive industries such as healthcare, finance, government affairs, and transportation. It can be used for task customization and local deployment of pre-trained large language models, and has efficient, controllable, and scalable application value. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a large model training method based on adaptive differential privacy control provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the Adapter dependency propagation method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the correlation propagation mode of the LoRA method provided in the embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] With the widespread deployment of large-scale pre-trained models in tasks such as natural language processing and recommendation systems, maintaining model performance while protecting the privacy of training data has become a critical issue that urgently needs to be addressed. Existing differential privacy training methods typically employ a uniform noise mechanism, failing to distinguish the importance of different parameters to the model's prediction results, leading to a significant decline in model performance. To address this, this invention proposes a large-scale model training method and apparatus based on adaptive differential privacy control. It integrates high-order privacy budget estimation and control methods with parameter correlation evaluation methods, constructing a training scheme that combines privacy protection capabilities with guaranteed model performance. This invention can be widely applied in highly privacy-sensitive industries such as healthcare, finance, government affairs, and transportation, enabling task customization and local deployment of pre-trained large language models. Specifically, it can be applied to tasks such as intelligent question answering, ticket recognition, anomaly detection, and customer credit modeling in railway financial services, effectively protecting the privacy and security of sensitive data such as passenger information, transaction details, and payment records, while simultaneously improving the accuracy and compliance of model training.

[0022] In the technical approach of this invention, model training is based on a general pre-trained large language model structure (such as a Transformer-type large model). By introducing efficient parameter fine-tuning methods (such as Adapter, LoRA, BitFit, etc.), only a small subset of parameters is trained to reduce resource consumption and overfitting risk. Regarding differential privacy control, this invention first utilizes high-order privacy budget estimation and control methods to perform a high-order expansion of the privacy loss distribution during training, accurately estimating the noise budget corresponding to each parameter update step, thereby achieving a smaller and quantifiable global noise intensity setting and reducing performance loss. Simultaneously, a parameter correlation evaluation method is introduced to analyze the correlation of parameters within the model, dividing the gradient space according to their actual contribution to the predicted output. Combined with the parameter correlation graph, a group pruning and group noise addition strategy is adopted during the training phase: smaller noise perturbations are applied to highly correlated parameters, while stronger perturbations are applied to low-correlation parameters, effectively achieving adaptive control of the privacy protection granularity.

[0023] During fine-tuning training, the model takes task data as input and performs local gradient calculations, pruning, and noise addition operations on a subset of target parameters. Within each training step, noise resources are dynamically allocated according to the correlation distribution of the parameters. All parameter update processes adhere to strict differential privacy protection, and the trained model possesses interpretability, security, and practicality. This training mechanism can be used in conjunction with federated learning frameworks or deployed independently in local private computing environments, adapting to the requirements of distributed, compliant, and resource-constrained real-world scenarios. The technical solution of this invention will be specifically described below with reference to specific embodiments.

[0024] like Figure 1As shown, this embodiment of the invention provides a large model training method based on adaptive differential privacy control, used to train task models based on large language models. It is suitable for privacy protection and performance optimization requirements when performing efficient parameter fine-tuning (PEFT) on pre-trained large language models. Within each training round, the method includes: S1: Based on the parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and backward attribution calculation, generating a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output.

[0025] To achieve differentiated privacy control across parameters, this embodiment introduces an interpretable analysis mechanism during the fine-tuning stage to evaluate the relative importance of each trainable parameter in the current task. This stage does not directly participate in training or noise injection but serves as a preprocessing module to generate a parameter correlation graph. This graph is built upon the parameter correlation evaluation method and has been adapted and extended for efficient parameter fine-tuning structures, supporting fine-tuning of trainable parameters in different PEFT methods within the Transformer architecture.

[0026] To overcome the curse of dimensionality caused by the large dimensionality and high sparsity of the output space of the generation task, this embodiment introduces three output correlation initialization methods to control the correlation tracking path and improve computational efficiency. Subsequently, based on different types of network layer structures, the system employs various propagation rules, such as linear propagation, nonlinear pruning, and tensor pair operations, to propagate the initialized correlations layer by layer from the output layer to each trainable parameter, ultimately converging into a complete and continuous trainable parameter correlation graph. This trainable parameter correlation graph will be used in subsequent stages to construct a correlation mask, guiding the differential privacy noise to adaptively distribute according to parameter importance.

[0027] S11: Correlation initialization: A batch is randomly sampled from the training dataset and fed into a frozen pre-trained Big Prophet model for forward propagation. The model output and intermediate activation values ​​are recorded to generate output correlations. In the generation task, the model output is usually a large-scale softmax distribution, and directly inputting it into correlation analysis will lead to the curse of dimensionality. To address this, this invention introduces three probabilistic dimensionality reduction strategies to reduce the dimensionality of the output correlations to initialize the output correlation R. (N) (That is, the parameter dependence of the Nth layer, where the Nth layer refers to the output layer): (1) For each token, select a threshold l and retain only the probability of the top l words in the vocabulary, and set the probability of the remaining words to zero. (2) For each token, retain the words in the probability-based descending vocabulary until the cumulative probability of the retained words reaches a threshold p (e.g., 0.95), and set the probability of the remaining words to zero. (3) Based on the word units retained in each step, calculate the cumulative probability according to the sentence order and select the top b sentence frames with the highest value. The probability of the unselected words is set to zero.

[0028] S12: Correlation propagation in single-input tensor operations: Model layer operations are categorized into single-input tensor operations and two-input tensor operations. For single-input structures (such as linear layers and nonlinear activation layers), the input tensor is denoted as X, and the weight tensor is W. The propagation rule formula for linear layers is as follows: ; In the formula, R represents the correlation of parameters in the nth layer. (n) The parameter correlation of the i-th neuron; x j The input value of the j-th neuron is denoted by ; n represents the number of neuron layers. Refers to the (n-1)th layer's tensor X, The computational operations can be further divided into linear layers and nonlinear activation layers; The propagation rule formula for nonlinear activation layers is as follows: ; In the formula, This represents the positive activation value of the j-th neuron; The positive weights (i.e., w) from the j-th input to the i-th neuron ji (part greater than 0) This represents a neuronal connection path with all positive weights, indicating a positive activation propagation path; refer to Activation function.

[0029] S13: Correlation decomposition in two-input tensor operations: In multi-input structures, such as Attention and self-attention modules, the output is controlled by multiple tensors, requiring the output correlation R to be calculated. (n) The correlation is split across different input paths. The specific formula for correlation splitting is as follows: ; This is used to ensure that the output correlation of each layer is consistent with the total correlation of the input layer, where Let X and Y represent the correlation values ​​of the input tensors X and Y in the (n-1)th layer, respectively. The Relevance Splitting function, for the case of "paired tensor operations," splits the relevance R from the next layer. (n)Split into the two input tensors X and Y of the previous layer. This refers to the source from R (n) The correlation propagates along the X direction. Refers to R (n) The correlation propagates along the Y direction.

[0030] S14: Correlation merging in tensor fusion and correlation extraction in trainable parameter dimensions: Tensor fusion correlation merging: merging two correlation sources from different paths. hour: ; in This represents the Jacobian matrix, used to weight the fusion strength of different paths; , Indicates normalized processing , ; Correlation extraction of trainable parameter dimensions: During backpropagation of correlations, the system aggregates the total correlations of the intermediate layers and tracks them to the trainable parameters in the PEFT module; taking the Adapter as an example: ; Where: Y j Represents a parameter tensor (e.g., a weight matrix in an Adapter); This represents the correlation value of the j-th input dimension in the (n-1)-th layer under parameter θ, indicating the degree of influence of this input unit on the output unit; and Here, the correlation propagation mapping function represents the same propagation process, where... It emphasizes the combined input tensor and weight tensor, while This is then considered as a single-input tensor propagation of Y, with the order Y→X; since a linear layer can be viewed as a single-input tensor operation on Y, the two are mathematically equivalent, but they emphasize different operational semantics in propagation path analysis; if a strict distinction in semantics is required, Strong is typically used to compute relevance maps, with two tensors as input (e.g., input and weights); while A more accurate description of the propagation function under a single-input structure; This indicates that the output is about the input Y. j The partial derivative of the input is used to quantify the sensitivity of the input to the output. The formula above calculates the "importance" or "relevance" score of each input parameter to the final output.

[0031] S15: PEFT type adaptation and correlation graph stitching: The following demonstrates the instantiation of the proposed correlation analysis in three types of PEFT methods: (1) Adapter structure: The path “Down Linear→NonLinear→Up Linear” is propagated layer by layer, and its correlation propagation method is as follows: Figure 2 As shown. This indicates the parameter dependency at the Adapter output. Propagating upwards to obtain the parameter correlation of the Up Linear layer ; This indicates the parameter dependency of the NonLinear layer. Continuing the backpropagation, we obtain the parameter correlation of the Down Linear layer. The pieces were assembled. In the formula R Adapter The concatenation result of the correlation tensor of the entire Adapter module is a module-level overall representation; 'a' represents the 'a'th Adapter module in the network, and 'A' represents the total number of Adapter modules. These represent the correlation vectors of the Up Linear layer and Down Linear layer in the a-th Adapter, respectively. The purpose of this concatenation is to aggregate the correlations of all Adapter modules as an overall correlation mapping, which can be used for privacy mechanism design (e.g., to determine which Adapter paths are more sensitive). (2) Selective PEFT Structure: Instantiation of the selective PEFT method involves minimal changes to the model architecture. Since the trainable parameters are part of the original network, only sequential correlation propagation based on the original network structure is needed, and the correlation of selected parameters is calculated. Taking BitFit as an example, the correlation of biases is preserved in the network and connected to generate a correlation graph. Let W... b Indicates the Lth (n) The bias of the layer, the correlation of the (n-1)th layer can be calculated as follows: Among them, C l ( Let X be the correlation propagation function of the l-th layer, and Y be the input tensor. b R is the output tensor where the bias is located. (n) Let be the correlation tensor of the nth layer. Then, a correlation graph is constructed using the correlations of all biases. .

[0032] (3) Instantiation based on reparameterization: Taking the commonly used LoRA method as an example, its correlation propagation method is as follows Figure 3 The correlation propagation of trainable parameters is as follows: ; The four formulas above describe the layer-by-layer propagation process of training parameter correlation in the LoRA structure, and respectively represent: through the single-input tensor propagation function S l The correlation of the LoRA output of the nth layer The correlation of the B path at level n-1 is passed down to the next level n-1. The correlation is further passed down to the (n-2)th layer of path A. ; through the two-input tensor propagation function C l The correlation of the output of the nth layer LoRA The trainable parameters passed up to path B are obtained ; The relevance of path B The trainable parameters further passed up to path A are obtained Similarly, the correlation between path B and path A is recorded and combined into a trainable parameter correlation graph. , This represents the b-th module, i.e., the structure number on the path.

[0033] In practice, the correlation is calculated based on the category to which the given PEFT method belongs.

[0034] S2: High-order privacy budget estimation and control methods are used to perform high-order expansion and approximate estimation of privacy loss during training. Without introducing additional relaxation, the noise baseline intensity in the current training round is accurately controlled and dynamically adjusted to ensure that the overall training process meets differential privacy constraints.

[0035] To finely control the differential privacy budget consumption throughout the fine-tuning process, a high-order privacy budget estimation and control method is introduced in this stage. This method can provide a tighter upper bound estimate of the privacy loss without sacrificing training performance, and supports adaptive adjustment of noise intensity during fine-tuning to ensure that the final privacy budget (ε,δ) is not prematurely overdrawn. The specific steps are as follows.

[0036] S21: Initialization of sampling rate and basic parameters: Set the privacy budget cap ε and failure probability δ during fine-tuning training, and input the total number of samples N and the batch size P for each training round, then calculate the sampling probability. Set the initial noise standard deviation σ init The gradient clipping threshold C, the expansion order k, and the scaling factor r∈(0,1) used to control scaling noise.

[0037] S22: Calculate the privacy-depleting variable: In each differential privacy mechanism M i Above, we define the output distributions of the adjacent datasets D and D′ as P, respectively.i =M i (D) and =M i (D′), from P i , We sampled data from each model and calculated the privacy loss variable, which characterizes the degree of perturbation to the output distribution caused by a small change in the input data (i.e., D→D′) during the current model parameter update step. This variable is the basis for evaluating and accurately estimating the differential privacy loss. The privacy loss variable is represented as follows: ; In the formula, X i Indicates that from distribution P i Samples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; The Radon–Nikodym derivative represents Relative to P i The probability density ratio of the two random variables mentioned above; , This will be used in subsequent high-order expansion and budget control estimations, and its tail behavior determines the privacy loss estimate for the current fine-tuning training epoch. Tail behavior refers to the probability that the differential privacy mechanism output deviates greatly (i.e., high privacy loss) between adjacent datasets. Although these deviations are rare, they can undermine the overall protection effect of DP if left uncontrolled.

[0038] S23: Calculate privacy budget loss: Privacy-depleting variables and The distribution function F X (z) and F Y (z) is expanded using a higher-order approximation, as follows: ; In the formula, Φ(z) is the standard normal distribution function. Let P be the density function corresponding to the standard normal distribution function Φ(z). j (z) is a polynomial term determined by the higher-order cumulants of the distribution, used to describe the non-normal bias of the distribution in the higher-order expansion; V∈{X,Y} represents the corresponding privacy-depleting variable; n is the sample size; Find the minimum ε that satisfies the following inequality: ; In the formula, ω represents a higher-order approximation term or security relaxation term for the cumulative privacy loss; the minimum ε is denoted as ε (α) , ε(α) This represents the privacy budget loss under the current noise standard deviation σ.

[0039] S24: Adaptively adjust and lock the noise standard deviation and output budget status: If the currently estimated ε (α) >ε indicates that the current noise configuration is insufficient to meet the privacy budget constraint, initiating an iterative scaling process: updating the noise standard deviation to σ←r·σ, where r is a scaling parameter; then re-executing steps S22 and S23, recalculating the privacy cost variable and ε based on the updated σ. (α) The iterative process will continue until one of the following conditions is met: (1) The currently estimated privacy budget loss satisfies the constraint: ε (α) ≤ε; (2) The current noise standard deviation σ has dropped to the minimum allowable value set by the system and cannot be scaled further; Once the budget constraint is met, the system considers the current noise standard deviation σ as the optimal perturbation strength usable in this training epoch and locks it. Furthermore, the estimated privacy budget loss ε is... (α) This value is recorded in the differential privacy budget ledger for cumulative tracking and budget verification in subsequent training rounds. This σ value will be passed to the next stage as the noise intensity parameter used by the gradient perturbation module when performing noisy training.

[0040] S3: Construct a binary mask based on the correlation map of trainable parameters, divide the high and low correlation parameters into subsets, and perform differential pruning and differential noise addition on the gradients corresponding to the high and low correlation parameters in combination with the current noise baseline intensity setting, thus completing the parameter update of the task model under differential privacy protection.

[0041] This step is based on the correlation plot R obtained in the previous two steps. PEFT The gradient is updated with differential perturbation based on the noise intensity σ output by the budget control in each training round. The core idea is to implement hierarchical noise addition along the parameter dimension: injecting higher intensity noise into low-correlation parameters and lower intensity noise into high-correlation parameters, thereby preserving the ability to optimize more critical parameters of model performance without exceeding the differential privacy budget limit. Specifically, the steps include: S31: Correlation Normalization and Mask Generation: Before performing adaptive gradient perturbation, in order to achieve differentiated noise at the parameter dimension level, the parameter correlation map R obtained in step S1 is first processed. PEFT Normalization is performed, and a correlation mask is generated based on the normalization result to divide the dataset into two sub-dimensions: "low importance" and "high importance." Let the fine-tuning dataset be D. FT ={x1,x2,...,x m ,...,x M}, where xm Let x represent the m-th sample; each sample x m The corresponding trainable parameter correlation graph is denoted as , where d is the total number of dimensions of all trainable parameters; Calculate the correlation plot of the average trainable parameters for all samples. ; ; In the formula, M represents the total number of samples in the fine-tuning dataset; Correlation plot of average trainable parameters Normalization is performed to obtain the correlation map of normalized trainable parameters. ,in: ; In the formula, R refers to the average relevance value of the j-th trainable parameter across the entire fine-tuning dataset; j ′ refers to the correlation value of the j-th trainable parameter after normalization; R min R refers to the minimum average correlation value across all parameter dimensions. max d refers to the maximum average correlation value across all parameter dimensions; d refers to the total number of dimensions of trainable parameters.

[0042] Set the correlation division ratio ∈(0,1), representing the proportion of parameters that are desired to be labeled as having low correlation, from Extract the first The smaller value is used as the dividing threshold R. thr : ; In the formula, The expression represents the correlation graph R′ of the normalized trainable parameters of the input, sorted by element value in ascending order, and selecting the first element. The correlation values ​​at each location are used as the threshold; where β (0,1) represents the proportion of parameters with low relevance, and d is the total number of parameters.

[0043] Based on this, a correlation mask vector M is generated. R ∈{0,1} d This is used to mark the relevance of parameter dimensions: ; In the formula, Represents the correlation mask vector M R The p-th value; Represents the correlation graph of normalized trainable parameters The p-th value in; Based on the correlation mask vector M RThe trainable parameters are divided into two subsets: high and low correlation.

[0044] S32: Subset-Sensitive Adaptive Clipping For the two gradient subsets corresponding to the two parameter subsets with high and low correlation, differential L2 norm clipping is performed separately, with a global gradient clipping threshold of C>0. Calculate the proportionality coefficient: ; In the formula, This represents the subset of highly correlated gradients corresponding to the subset of highly correlated parameters. Sensitivity; This represents the subset of low-correlation gradients corresponding to the subset of low-correlation parameters. Sensitivity; This represents the original gradient of the task model in the current training batch, obtained by taking the derivative of the loss function with respect to the trainable parameters. Prune the two gradient subsets corresponding to the two parameter subsets of high and low correlation respectively: ; In the formula, This represents the subset of highly relevant gradients after pruning. This represents the subset of low-correlation gradients after clipping.

[0045] S33: Adaptive Noise Addition and Model Update: Using the global differential privacy control noise intensity σ output in step S2, and combining it with the gradient grouping results of the current training samples, Gaussian noise perturbations of varying intensities are applied to gradient subsets of different importance dimensions to achieve structure-aware differential privacy training and updates; assuming the current sample x k The parameter gradients have been divided into a subset g of strongly important gradients. w With the weak importance gradient subset g s Differential L2 norm clipping was performed on each gradient, and the clipped gradients were obtained. Then, multidimensional Gaussian noise of different intensities is injected into them respectively.

[0046] First, the standard deviations of the high-relevance gradient subset and the low-relevance gradient subset are dynamically calculated based on the clipping results: ; In the formula, σ represents the current noise standard deviation. and represent the standard deviations of the highly correlated gradient subset and the low-correlation gradient subset, respectively; Based on this, noise is added to the clipped high-relevance gradient subset and low-relevance gradient subset respectively: ; In the formula, and Let N(0, ...) represent the high-correlation gradient subset and the low-correlation gradient subset after adding noise, respectively; I) indicates that the mean is 0. The covariance is a multidimensional Gaussian distribution. Pick or I is the identity matrix; In each training round, a batch is randomly sampled from the fine-tuning dataset, and differential pruning and differential noise addition are performed on each training sample in the current training batch to obtain the corresponding noise gradient. Then, all noise gradients in the current training batch are averaged to obtain the average noise gradient of the current training batch. Finally, according to the set learning rate parameter, a standard gradient descent update operation is performed on the current task model parameters using the average noise gradient of the current training batch. During the above update process, the current model parameters are adjusted according to the direction of the perturbation gradient to continue minimizing the loss function; the learning rate controls the update magnitude to ensure the stability and convergence of the training process; the average perturbation gradient, as a perturbation estimate at the batch level, integrates the individual perturbation amounts of different samples, improving the robustness of training. This training and update process continues until the preset termination conditions are met: the upper limit of the training rounds, the depletion of the privacy budget, or the model performance convergence. In this way, the system dynamically injects differential privacy-preserving perturbations in each training round while maintaining the optimization effect of parameter updates and model performance.

[0047] Taking railway invoice classification and content verification as an example, a multi-classification model for railway invoices, fine-tuned based on a large language model, is developed. First, historical invoices archived in the railway ticket management system are used as training samples. The sample data includes fields such as invoice number, issuing station, issuing date, amount, invoice type, and travel segment. The invoice category (e.g., freight invoice, insurance invoice, other invoices) and any missing fields or format errors are also labeled. Based on the training samples, the aforementioned large model training method based on adaptive differential privacy control is used to train the model, resulting in a multi-classification model for railway invoices fine-tuned based on the large language model. During training, the model input is the invoice field text that has been recognized by OCR and structured. After semantic understanding and feature extraction by the large model, the corresponding invoice type classification results and anomaly markers are output, thus automating the business processes of railway ticket archiving and verification.

[0048] Furthermore, embodiments of the present invention also provide a large model training device based on adaptive differential privacy control, used for training task models based on large language models, the device comprising: The parameter correlation evaluation unit is configured to use a parameter correlation evaluation method to input training sample data into a task model based on a large language model for forward propagation and backward attribution calculation, and generate a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. The privacy budget estimation and control unit is configured to use high-order privacy budget estimation and control methods to perform high-order expansion and approximate estimation of privacy loss during training, accurately control the noise baseline intensity in the current training round without introducing additional relaxation, and dynamically adjust to ensure that the overall training process meets differential privacy constraints. The model parameter update unit is configured to construct a binary mask based on the correlation map of trainable parameters, divide the high and low correlation parameters into subsets, and perform differential pruning and differential noise addition on the gradients corresponding to the high and low correlation parameters in combination with the current noise baseline intensity, thereby completing the task model parameter update under differential privacy protection.

[0049] It should be understood that the functional unit modules in the various embodiments of the present invention can be concentrated in one processing unit, or each unit module can exist physically separately, or two or more unit modules can be integrated into one unit module, and can be implemented in hardware or software.

[0050] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0051] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A large model training method based on adaptive differential privacy control, characterized in that, For training a task model based on a large language model, the method includes, in each training epoch: Based on the parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and backward attribution calculation, generating a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. We utilize high-order privacy budget estimation and control methods to perform high-order expansion and approximate estimation of privacy loss during training, control the noise baseline intensity under the current training round, and dynamically adjust it to ensure that the overall training process meets differential privacy constraints. A binary mask is constructed based on the correlation graph of trainable parameters to divide the high and low correlation parameters into subsets. The gradients corresponding to the high and low correlation parameters are then differentially pruned and noise-added based on the current noise baseline intensity, thus completing the parameter update of the task model under differential privacy protection.

2. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The method for generating the trainable parameter correlation map specifically includes: The training sample data of each training round is input into the frozen pre-trained Big Prophet model to perform forward propagation, and its output and intermediate activation values ​​are recorded to generate output correlations. The output correlation is reduced in dimensionality to obtain the initial output correlation; Based on the parameter correlation evaluation method, the initial output correlation is propagated from the output layer to each trainable parameter layer by layer to obtain the trainable parameter correlation of each layer, and finally stitched into a complete and continuous trainable parameter correlation graph.

3. The large model training method based on adaptive differential privacy control according to claim 2, characterized in that, The process of reducing the dimensionality of the output correlation includes: For each word, select a threshold l, and retain only the probability of the top l words in the vocabulary, while setting the probability of the remaining words to zero; For each word element, retain the words in the probability-based descending vocabulary until the cumulative probability of the retained words reaches a threshold p, and set the probability of the remaining words to zero; Following the sentence order, based on the lexical units retained at each step, the cumulative probability is calculated and the top b sentence frames with the highest values ​​are selected, while the probability of unselected words is set to zero.

4. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The method utilizes a high-order privacy budget estimation and control approach to perform high-order expansion and approximate estimation of privacy loss during training, controls the noise baseline intensity in the current training round, and dynamically adjusts it to ensure that the overall training process meets differential privacy constraints. Specifically, this includes: Sampling rate and basic parameter initialization: Set the sampling rate, as well as the privacy budget cap ε, failure probability δ, and initial noise standard deviation σ. init Gradient clipping threshold C, unfolding order k, and scaling factor r used to control scaling noise; In each differential privacy mechanism M i Above, we define the output distributions of the adjacent datasets D and D′ as P, respectively. i =M i (D) and =M i (D′), from P i , We sampled from each sample, calculated the privacy loss variable, and represented it as follows: ； In the formula, X i Indicates that from distribution P i Samples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; express Relative to P i The probability density ratio; Privacy-depleting variables and The distribution function F X (z) and F Y (z) is expanded using a higher-order approximation, as follows: ； In the formula, Φ(z) is the standard normal distribution function. Let P be the density function corresponding to the standard normal distribution function Φ(z). j (z) is a polynomial term determined by the higher-order cumulants of the distribution, used to describe the non-normal deviation of the distribution in the higher-order expansion; V∈{X,Y}; n is the sample size; Find the minimum ε that satisfies the following inequality: ； In the formula, ω represents a higher-order approximation term or security relaxation term for cumulative privacy loss; the minimum ε is denoted as ε (α) , ε (α) This represents the privacy budget loss under the current noise standard deviation σ; If the currently estimated ε (α) >ε indicates that the current noise configuration is insufficient to meet the privacy budget constraint, initiating an iterative scaling process: updating the noise standard deviation to σ←r·σ, where r is a scaling parameter; subsequently, recalculating the privacy cost variable and ε based on the updated σ. (α) The iterative process will continue until one of the following conditions is met: (1) The currently estimated privacy budget loss satisfies the constraint: ε (α) ≤ε; (2) The current noise standard deviation σ has dropped to the minimum allowable value set by the system and cannot be scaled further; Save the current noise standard deviation σ and privacy budget loss ε (α) .

5. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The step of constructing a binary mask based on the correlation graph of trainable parameters and dividing it into subsets of high and low correlation parameters specifically includes: Let the fine-tuning dataset be D. FT ={x1,x2,...,x m ,...,x M }, where x m Let x represent the m-th sample; m The corresponding trainable parameter correlation graph is denoted as , where d is the total number of dimensions of all trainable parameters; Calculate the correlation plot of the average trainable parameters for all samples. ; Correlation plot of average trainable parameters Normalization is performed to obtain the correlation map of normalized trainable parameters. ; Set the correlation division ratio ∈(0,1), representing the proportion of parameters that are desired to be labeled as having low correlation, from Extract the first The smaller value is used as the dividing threshold R. thr ; Based on this, a correlation mask vector M is generated. R ∈{0,1} d This is used to mark the relevance of parameter dimensions: ； In the formula, Represents the correlation mask vector M R The p-th value; Represents the correlation graph of normalized trainable parameters The p-th value in; Based on the correlation mask vector M R The trainable parameters are divided into two subsets: high and low correlation.

6. The large model training method based on adaptive differential privacy control according to claim 5, characterized in that, The differentiated cutting process includes: For the two gradient subsets corresponding to the two parameter subsets with high and low correlation, perform differential L2 norm clipping respectively. Assuming the global gradient clipping threshold is C>0, calculate the scaling factor: ； In the formula, This represents the subset of highly correlated gradients corresponding to the subset of highly correlated parameters. Sensitivity; This represents the subset of low-correlation gradients corresponding to the subset of low-correlation parameters. Sensitivity; This represents the original gradient of the task model in the current training batch, obtained by taking the derivative of the loss function with respect to the trainable parameters. Prune the two gradient subsets corresponding to the two parameter subsets of high and low correlation respectively: ； In the formula, This represents the subset of highly relevant gradients after pruning. This represents the subset of low-correlation gradients after clipping.

7. The large model training method based on adaptive differential privacy control according to claim 6, characterized in that, The differential noise addition process includes: The standard deviations of the highly relevant gradient subset and the low-relevance gradient subset are dynamically calculated based on the clipping results. ； In the formula, σ represents the current noise standard deviation. and represent the standard deviations of the highly correlated gradient subset and the low-correlation gradient subset, respectively; Based on this, noise is added to the clipped high-relevance gradient subset and low-relevance gradient subset respectively: ； In the formula, and Let N(0, ...) represent the high-correlation gradient subset and the low-correlation gradient subset after adding noise, respectively; I) indicates that the mean is 0. The covariance is a multidimensional Gaussian distribution. Pick or ; I is the identity matrix.

8. The large model training method based on adaptive differential privacy control according to claim 7, characterized in that, The specific steps for updating the task model parameters under differential privacy protection include: Differential cropping and differential noise addition are performed on each training sample in the current training batch in sequence to obtain the corresponding noise gradient; The average noisy gradient of the current training batch is obtained by averaging all the noisy gradients in the current training batch. Based on the set learning rate parameter, the gradient descent operation is used to update the parameters of the current task model using the average noisy gradient of the current training batch.

9. A large model training device based on adaptive differential privacy control, characterized in that, The apparatus for training a task model based on a large language model includes: The parameter correlation evaluation unit is configured to use a parameter correlation evaluation method to input training sample data into a task model based on a large language model for forward propagation and backward attribution calculation, and generate a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. The privacy budget estimation and control unit is configured to use a high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation of privacy loss during training, control the noise baseline intensity under the current training round, and dynamically adjust to ensure that the overall training process meets differential privacy constraints. The model parameter update unit is configured to construct a binary mask based on the correlation map of trainable parameters, divide the high and low correlation parameters into subsets, and perform differential pruning and differential noise addition on the gradients corresponding to the high and low correlation parameters in combination with the current noise baseline intensity, thereby completing the task model parameter update under differential privacy protection.

Images