A large model training method and device based on adaptive differential privacy control

Abstract

The application discloses a large model training method and device based on adaptive differential privacy control, which is used for training a task model based on a large language model. In each training round, the method comprises the following steps: based on a parameter correlation evaluation method, inputting training sample data into the task model for calculation to generate a trainable parameter correlation graph; using a high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation on the privacy loss in the training process, control the noise benchmark strength under the current training round, and dynamically adjust to ensure that the overall training process meets the differential privacy constraint; according to the trainable parameter correlation graph, a binary mask is constructed to divide high and low correlation parameter subsets, and the gradients corresponding to the high and low correlation parameters are differentially clipped and added with noise according to the current noise benchmark strength, so that the task model parameter update under the differential privacy guarantee is completed. The application can significantly reduce noise introduction and improve model performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and information security technology, and in particular to a large model training method and device based on adaptive differential privacy control. BACKGROUND

[0002] In the process of promoting intelligent transformation in multiple industries, how to efficiently utilize data resources under strict privacy protection requirements and train artificial intelligence models with security and performance has become a core problem to be solved.

[0003] In particular, in sensitive data-intensive scenarios such as financial risk control, medical health, judicial adjudication, and government management, training data often involves sensitive fields such as personal identity, health records, property information, historical behavior, and even judicial records. For example, in the field of railway finance, with the continuous promotion of "railway + finance" digital integration, a large amount of structured and unstructured high-sensitive data has been accumulated in ticket payment, fund settlement, unit credit, insurance claims, and railway supply chain finance. These data include customer identity information, payment trajectory, station transaction details, travel path, and enterprise transaction accounts. Once illegally mined or leaked, it may cause major financial security incidents and social impact.

[0004] However, such financial data is often distributed among multiple subsystems and institutions, resulting in "data silos" and "untrusted privacy". Traditional large model fine-tuning solutions cannot be directly introduced. Under regulatory compliance requirements, it is urgent to explore privacy-enhanced modeling methods that are "computable but invisible", which not only guarantee the privacy and security of user data in the railway financial system, but also enable efficient model customization training for tasks such as risk identification, credit assessment, and intelligent question answering. Large-scale pre-training models (such as BERT, GPT, etc.) have shown excellent performance in multiple tasks, but their training costs and deployment complexity have also increased, making it difficult to train from scratch. In practical applications, to improve the effectiveness of models in specific scenarios, it is common to fine-tune existing large models to adapt to new tasks, new data, or new users. Especially in the above sensitive data scenarios, the fine-tuning stage often involves iterative learning of original sensitive data, which is a high-risk link for model "leaking privacy". Existing research has shown that without privacy mechanism constraints, fine-tuned models may be subject to training data reverse attack and member inference attack, thus exposing the original data content.

[0005] To alleviate the above problems, differential privacy (DP) is introduced into model training to limit the behavior deviation of the model when observing a single sample, thereby theoretically guaranteeing the non-inferability of individual data. The classic DP-SGD method controls the privacy budget by clipping and Gaussian perturbing the gradient in multiple rounds of updates. However, in actual large model fine-tuning tasks, the uniform noise addition strategy can cause a significant performance decline. Especially when the model has a large number of parameters and the task requires high accuracy, the noise can interfere with the learning process of high-value parameters and even mask critical information, leading to model fine-tuning failure.

[0006] Current differential privacy training still has deficiencies in two key aspects: in terms of noise addition, traditional estimation methods are conservative, resulting in performance sacrifice; in terms of how to add noise, traditional schemes fail to identify parameter correlation and lack adaptive control ability. SUMMARY

[0007] In view of the deficiencies in the prior art described above, the present application provides a large model training method and device based on adaptive differential privacy control, which fuses a high-order privacy budget estimator and a parameter correlation analysis strategy to construct a training scheme that combines privacy protection capability and model performance guarantee.

[0008] In a first aspect, a large model training method based on adaptive differential privacy control is provided, which is used to train a task model based on a large language model. In each training round, the method includes:

[0009] Based on a parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and backward attribution calculation to generate a trainable parameter correlation graph to measure the correlation distribution of each trainable parameter in the model output.

[0010] A high-order privacy budget estimation and control method is used to perform high-order expansion and approximate estimation of privacy loss in the training process, accurately control the noise benchmark strength at the current training round without introducing additional relaxation, and dynamically adjust to ensure that the overall training process meets the differential privacy constraint.

[0011] According to the trainable parameter correlation graph, a binary mask is constructed to divide high and low correlation parameter subsets. Combined with the current noise benchmark strength, the gradients corresponding to the high and low correlation parameters are differentially clipped and differentially added to complete parameter update of the task model under differential privacy guarantee.

[0012] Further, the trainable parameter correlation graph generation method specifically includes:

[0013] The training sample data of each training round is input into the frozen pre-training large prophecy model to perform forward propagation, and the output and intermediate activation values thereof are recorded to generate an output correlation;

[0014] The output correlation is dimensionally reduced to obtain an initialized output correlation;

[0015] Based on the parameter correlation evaluation method, the initialized output correlation is propagated from the output layer to each trainable parameter layer by layer to obtain a trainable parameter correlation of each layer, and finally spliced into a complete and continuous trainable parameter correlation graph.

[0016] Further, the process of dimensionally reducing the output correlation comprises:

[0017] For each token, a threshold l is selected, and only the probabilities of the top l words in the vocabulary are retained, and the probabilities of the remaining words are set to zero;

[0018] For each token, the words in the descending probability-based vocabulary are retained until the cumulative probability of the retained words reaches a threshold p, and the probabilities of the remaining words are set to zero;

[0019] According to the order of the sentences, based on the tokens retained at each step, the cumulative probability is calculated and the top b sentence frames with the highest value are selected, and the probabilities of the unselected words are set to zero.

[0020] Further, the high-order privacy budget estimation and control method is used to perform high-order expansion and approximate estimation on the privacy loss in the training process, control the noise benchmark strength under the current training round, and dynamically adjust to ensure that the overall training process meets the differential privacy constraint, specifically comprising:

[0021] Sampling rate and basic parameter initialization: set the sampling rate, and the upper limit of the privacy budget ε, the failure probability δ, the initial noise standard deviation σ init , gradient clipping threshold C, expansion order k and scaling noise control factor r;

[0022] On each differential privacy mechanism M i , the output distributions under the adjacent data sets D and D' are defined as P i =M i (D) and =M i (D'), respectively sampled from P i , The privacy loss variable is calculated and represented as follows:

[0023] ;

[0024] In the formula, X i represents the output of the differential privacy mechanism M iSamples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; The Radon–Nikodym derivative represents Relative to P i The probability density ratio;

[0025] Privacy-depleting variables and The distribution function F X (z) and F Y (z) is expanded using a higher-order approximation, as follows:

[0026] ;

[0027] In the formula, Φ(z) is the standard normal distribution function. Let P be the density function corresponding to the standard normal distribution function Φ(z). j (z) is a polynomial term determined by the higher-order cumulants of the distribution, used to describe the non-normal deviation of the distribution in the higher-order expansion; V∈{X,Y}; n is the sample size;

[0028] Find the minimum ε that satisfies the following inequality:

[0029] ;

[0030] In the formula, ω represents a higher-order approximation term or security relaxation term for the cumulative privacy loss; the minimum ε is denoted as ε (α) , ε (α) This represents the privacy budget loss under the current noise standard deviation σ;

[0031] If the currently estimated ε (α) >ε indicates that the current noise configuration is insufficient to meet the privacy budget constraint, initiating an iterative scaling process: updating the noise standard deviation to σ←r·σ, where r is a scaling parameter; subsequently, recalculating the privacy cost variable and ε based on the updated σ. (α) The iterative process will continue until one of the following conditions is met: (1) The currently estimated privacy budget loss satisfies the constraint: ε (α) ≤ε; (2) The current noise standard deviation σ has dropped to the minimum allowable value set by the system and cannot be scaled further;

[0032] Save the current noise standard deviation σ and privacy budget loss ε (α) .

[0033] Furthermore, the step of constructing a binary mask based on the correlation graph of trainable parameters and dividing it into subsets of high and low correlation parameters specifically includes:

[0034] Let the fine-tuning dataset be D. FT ={x1,x2,...,x m ,...,x M}, where x m Let x represent the m-th sample; m The corresponding trainable parameter correlation graph is denoted as , where d is the total number of dimensions of all trainable parameters;

[0035] Calculate the correlation plot of the average trainable parameters for all samples. ;

[0036] Correlation plot of average trainable parameters Normalization is performed to obtain the correlation map of normalized trainable parameters. ;

[0037] Set the correlation division ratio ∈(0,1), representing the proportion of parameters that are desired to be labeled as having low correlation, from Extract the first The smaller value is used as the dividing threshold R. thr ;

[0038] Based on this, a correlation mask vector M is generated. R ∈{0,1} d This is used to mark the relevance of parameter dimensions:

[0039] ;

[0040] In the formula, Represents the correlation mask vector M R The p-th value in the middle; Represents the correlation graph of normalized trainable parameters The p-th value in;

[0041] Based on the correlation mask vector M R The trainable parameters are divided into two subsets: high and low correlation.

[0042] Furthermore, the differentiated cutting process includes:

[0043] For the two gradient subsets corresponding to the two parameter subsets with high and low correlation, differential L2 norm clipping is performed separately, with a global gradient clipping threshold of C>0.

[0044] Calculate the proportionality coefficient:

[0045] ;

[0046] wherein, denotes a high-correlation gradient subset corresponding to a high-correlation parameter subset ; denotes a low-correlation gradient subset corresponding to a low-correlation parameter subset ; denotes an original gradient of the task model under the current training batch, which is derived by differentiating the trainable parameters according to the loss function;

[0047] The two gradient subsets corresponding to the high-correlation parameter subset and the low-correlation parameter subset are respectively clipped:

[0048] ;

[0049] wherein, denotes a clipped high-correlation gradient subset, denotes a clipped low-correlation gradient subset.

[0050] Further, the differential noise adding process comprises:

[0051] According to the clipping result, the standard deviations of the high-correlation gradient subset and the low-correlation gradient subset are dynamically calculated:

[0052] ;

[0053] wherein, σ denotes a current noise standard deviation, and denote the standard deviations of the high-correlation gradient subset and the low-correlation gradient subset, respectively;

[0054] On this basis, the clipped high-correlation gradient subset and the clipped low-correlation gradient subset are respectively added with noise:

[0055] ;

[0056] wherein, and denote the high-correlation gradient subset and the low-correlation gradient subset after being added with noise, respectively; N(0, I) denotes a multi-dimensional Gaussian distribution with 0 as a mean value and as a covariance matrix, takes or ; I is an identity matrix.

[0057] Further, the task model parameter updating under the differential privacy guarantee comprises:

[0058] The differential clipping and differential noise adding are performed on each training sample in the current training batch in sequence to obtain corresponding noise-added gradients;

[0059] All noise-added gradients in the current training batch are averaged to obtain an average noise-added gradient of the current training batch;

[0060] According to the set learning rate parameter, the average noise-added gradient of the current training batch is used to perform gradient descent update operation on the current task model parameter.

[0061] In a second aspect, a large model training device based on adaptive differential privacy control is provided for training a task model based on a large language model, and the device comprises:

[0062] The parameter correlation evaluation unit is configured to input training sample data into the task model based on the large language model for forward propagation and reverse attribution calculation based on a parameter correlation evaluation method, to generate a trainable parameter correlation graph to measure the correlation distribution of each trainable parameter in the model output;

[0063] The privacy budget estimation and control unit is configured to utilize a high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation on the privacy loss in the training process, to accurately control the noise reference strength under the current training round without introducing additional relaxation, and to dynamically adjust to ensure that the overall training process meets the differential privacy constraint;

[0064] The model parameter update unit is configured to construct a binary mask according to the trainable parameter correlation graph, to divide high and low correlation parameter subsets, to combine the current noise reference strength setting to perform differential clipping and differential noise adding on the gradients corresponding to the high and low correlation parameters, and to complete the task model parameter update under the guarantee of differential privacy.

[0065] The application provides a large model training method and device based on adaptive differential privacy control, which fuses a high-order privacy budget estimation and control method and a parameter correlation evaluation method, accurately estimates privacy loss through high-order expansion, obtains compact privacy budget control, constructs a trainable parameter correlation graph, divides trainable parameters into multiple subspaces according to correlation, and implements a differential noise injection strategy under the premise of meeting overall privacy constraints: injecting less noise into high-correlation parameters and more noise into low-correlation parameters, thereby realizing adaptive weighting of gradient perturbation. The combination strategy of "global budget calculation + local perturbation fine control" effectively breaks through the problems of "rough noise estimation and rigid perturbation configuration" existing in the existing differential privacy training method, significantly improves the usability and training effect of the differential privacy mechanism in engineering practice, and realizes more accurate noise injection and resource allocation under the same privacy security condition, thereby improving the precision performance and deployability of the large model in sensitive tasks. Compared with the existing method, the application can significantly reduce noise introduction, improve model performance and reduce training overhead, and can be widely applied to high-privacy-sensitive industries such as medical treatment, finance, government affairs and transportation, and can be used for task customization and local deployment of pre-trained large language models, and has efficient, controllable and generalizable application value. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0067] Figure 1 is a large model training method flowchart based on adaptive differential privacy control provided by the embodiment of the present application;

[0068] Figure 2 is an Adapter correlation propagation mode schematic diagram provided by the embodiment of the present application;

[0069] Figure 3 is a LoRA method correlation propagation mode schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described in detail below. Obviously, the described embodiments only constitute some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0071] With the wide deployment of large-scale pre-training models in natural language processing, recommendation systems and other tasks, how to protect the privacy of training data while maintaining model performance has become a key problem that needs to be solved. Existing differential privacy training methods usually use a unified noise mechanism, which fails to distinguish the importance of different parameters to the model's prediction results, resulting in a significant decline in model performance. To this end, the present application proposes a large model training method and device based on adaptive differential privacy control, which combines high-order privacy budget estimation and control methods and parameter correlation evaluation methods to build a training scheme that balances privacy protection capability and model performance guarantee. The present application can be widely applied in high-privacy-sensitive industries such as medical, financial, government, and transportation, and can be used for task customization and local deployment of pre-trained large language models, such as intelligent question answering, bill recognition, anomaly detection, and customer credit modeling in railway financial services. The present application effectively safeguards the privacy of sensitive data such as passenger information, transaction details, and payment records, while improving the accuracy and compliance of model training.

[0072] In the technical path of the present application, model training is based on a general pre-training large language model structure (such as a Transformer-based large model), and a parameter-efficient fine-tuning method (such as Adapter, LoRA, BitFit, etc.) is introduced to train only a small subset of parameters, thereby reducing resource consumption and overfitting risk. In terms of differential privacy control, the present application first uses a high-order privacy budget estimation and control method to expand the privacy loss distribution in the training process to the high order, accurately estimates the noise budget corresponding to each step of parameter update, and sets a smaller and quantifiable global noise intensity, thereby reducing performance loss. At the same time, a parameter correlation evaluation method is introduced to analyze the correlation of internal parameters in the model, and the gradient space is divided according to their actual contribution to the prediction output. Combined with the parameter correlation graph, grouping clipping and grouping noise strategies are used in the training stage: smaller noise disturbance is given to high-correlation parameters, and strong disturbance is applied to low-correlation parameters, effectively achieving adaptive control of privacy protection granularity.

[0073] During fine-tuning training, the model takes task data as input, performs local gradient calculation, clipping, and noise addition on the target parameter subset, and dynamically allocates noise resources according to the correlation distribution of parameters in each training step. All parameter update processes conform to strict differential privacy protection, and the trained model has the characteristics of explainability, security, and practicality. The training mechanism can be used in combination with the federated learning framework, or independently deployed in a local private computing environment, and can adapt to distributed, compliant, and resource-constrained scenarios. The technical solutions of the present application are described in detail below in conjunction with specific embodiments.

[0074] For example, Figure 1As shown, the embodiment of the application provides a large model training method based on adaptive differential privacy control, which is used for training a task model based on a large language model, and is suitable for privacy protection and performance optimization requirements when performing parameter efficient fine-tuning (PEFT) on a pre-trained large language model. In each training round, the method comprises:

[0075] S1: Based on the parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and reverse attribution calculation, and a trainable parameter correlation graph is generated to measure the correlation distribution of each trainable parameter in the model output.

[0076] To realize differential privacy control in the parameter dimension, the embodiment introduces an explainability analysis mechanism in the fine-tuning stage to evaluate the relative importance of each trainable parameter in the current task. This stage does not directly participate in training or noise injection, but serves as a preprocessing module to generate a parameter correlation graph. The graph is built on the parameter correlation evaluation method and is adapted and extended for parameter efficient fine-tuning structure, supporting fine-tuning of trainable parameters in different PEFT methods under the Transformer architecture.

[0077] To overcome the dimension disaster problem caused by the large dimension and strong sparsity of the generated task output space, the embodiment introduces three output correlation initialization methods to control the correlation tracking path and improve the calculation efficiency. Subsequently, the system uses linear propagation, nonlinear pruning, tensor pair operations and other propagation rules to transfer the initialized correlation from the output layer to each trainable parameter layer by layer, and finally converges into a complete and continuous trainable parameter correlation graph. The trainable parameter correlation graph will be used to construct a correlation mask in the subsequent stage to guide the adaptive distribution of differential privacy noise according to parameter importance.

[0078] S11: Correlation initialization:

[0079] A batch is randomly sampled from the training data set and input into the frozen pre-trained large model to perform forward propagation, record the model output and intermediate activation value, and generate output correlation. In the generation task, the model output is usually a large-scale softmax distribution, and direct input of correlation analysis will cause dimension disaster. Therefore, the application introduces three probability dimension reduction strategies to reduce the output correlation to initialize the output correlation R (N) (i.e. the parameter correlation of the Nth layer, where the Nth layer refers to the output layer):

[0080] (1) For each token, select a threshold l, and only keep the probabilities of the top l words in the vocabulary, and set the probabilities of the remaining words to zero;

[0081] (2) For each token, keep the words in the descending order word list based on probability until the cumulative probability of the kept words reaches a threshold p (e.g. 0.95), and the rest of the words are set to zero probability;

[0082] (3) Based on the kept tokens at each step, compute the cumulative probability and select the top b sentence frames in the order of sentence, and set the probability of the unselected words to zero.

[0083] S12: Correlation propagation of single-input tensor operation:

[0084] The model layer operation is divided into single-input tensor operation and double-input tensor operation. For single-input structure (such as linear layer, nonlinear activation layer), the input tensor is denoted as X, and the weight tensor is denoted as W. The linear layer propagation rule formula is as follows:

[0085] ;

[0086] In the formula, denotes the parameter correlation R (n) of the i-th neuron in the n-th layer; x j denotes the input value of the j-th neuron; n denotes the number of neuron layers; denotes the operation of the n-1-th layer on the tensor X, These operations can be further divided into linear layers and nonlinear activation layers.

[0087] The nonlinear activation layer propagation rule formula is as follows:

[0088] ;

[0089] In the formula, denotes the forward activation value of the j-th neuron; denotes the positive weight of the j-th input to the i-th neuron (i.e. w ji greater than 0); denotes all weight-positive neuron connection paths, representing the positive activation propagation path; denotes the activation function.

[0090] S13: Correlation splitting in double-input tensor operation:

[0091] In multi-input structure, such as Attention, self-attention module, the output is controlled by multiple tensors, and the output correlation R (n) needs to be split to different input paths. The correlation splitting formula is as follows:

[0092] ;

[0093] This is used to ensure that the output correlation of each layer is consistent with the total correlation of the input layer, where Let X and Y represent the correlation values ​​of the input tensors X and Y in the (n-1)th layer, respectively. The Relevance Splitting function, for the case of "paired tensor operations," splits the relevance R from the next layer. (n) Split into the two input tensors X and Y of the previous layer. This refers to the source from R (n) The correlation propagates along the X direction. Refers to R (n) The correlation propagates along the Y direction.

[0094] S14: Correlation merging in tensor fusion and correlation extraction in trainable parameter dimensions:

[0095] Tensor fusion correlation merging: merging two correlation sources from different paths. hour:

[0096] ;

[0097] in This represents the Jacobian matrix, used to weight the fusion strength of different paths; , Indicates normalized processing , ;

[0098] Correlation extraction of trainable parameter dimensions: During backpropagation of correlations, the system aggregates the total correlations of the intermediate layers and tracks them to the trainable parameters in the PEFT module; taking the Adapter as an example:

[0099] ;

[0100] Where: Y j Represents a parameter tensor (e.g., a weight matrix in an Adapter); This represents the correlation value of the j-th input dimension in the (n-1)-th layer under parameter θ, indicating the degree of influence of this input unit on the output unit; and Here, the correlation propagation mapping function represents the same propagation process, where... It emphasizes the combined input tensor and weight tensor, while This is then considered as a single-input tensor propagation of Y, with the order Y→X; since a linear layer can be viewed as a single-input tensor operation on Y, the two are mathematically equivalent, but they emphasize different operational semantics in propagation path analysis; if a strict distinction in semantics is required, Strong is usually used to calculate the relevance map, input as two tensors (such as input and weight); while More close to the propagation function description under the single input structure; The partial derivative of the output with respect to the input Y j , used to quantify the sensitivity of the input to the output. The above formula calculates the "importance" or "relevance" score of each input parameter to the final output.

[0101] S15: PEFT type adaptation and relevance map splicing:

[0102] The instantiation of the proposed relevance analysis in three types of PEFT methods is shown as follows:

[0103] (1) Adapter structure: propagate by layer for the "Down Linear→NonLinear→Up Linear" path, and its relevance propagation mode is shown as Figure 2 . , indicates that the parameter relevance of the output end of the Adapter is propagated to the upper layer to obtain the parameter relevance of the Up Linear layer; , indicates that the parameter relevance of the NonLinear layer is continued to be propagated backward to obtain the parameter relevance of the Down Linear layer. Splicing obtains , where R Adapter represents the relevance tensor splicing result of the entire Adapter module, which is the overall representation at the module level; a represents the a-th Adapter module in the network, and A represents the total number of Adapter modules; respectively represent the relevance vectors of the Up Linear layer and the Down Linear layer in the a-th Adapter; the purpose of this splicing is to aggregate the relevance of all Adapter modules as a whole relevance map, which is used for privacy mechanism design (for example, to determine which Adapter path is more sensitive);

[0104] (2) Selective PEFT structure: the instantiation of the selective PEFT method involves minimal modification to the model architecture. Since the trainable parameters are part of the original network, only the sequential relevance propagation is needed according to the original network structure, and the relevance of the selected parameters is calculated. Taking BitFit as an example, the relevance of the bias is retained in the network, and they are connected to generate a relevance map. Let W b represent the bias of the L (n) -th layer, and the relevance of the n-1-th layer can be calculated as: ; where C l ( ) is the correlation propagation function of the l-th layer, X is the input tensor, Y b is the output tensor where the bias is located, R (n) is the correlation tensor of the n-th layer. Subsequently, the correlation graph is constructed using the correlations of all biases .

[0105] (3) Instance based on reparameterization: taking the commonly used LoRA method as an example, the correlation propagation mode is as follows Figure 3 The correlation propagation of trainable parameters is as follows:

[0106] ;

[0107] The above four formulas are used to describe the layer-by-layer propagation process of the correlation of trainable parameters in the LoRA structure, which respectively represent: through the single-input tensor propagation function S l , the LoRA output correlation of the n-th layer is passed down to the n-1-th layer of the B path, the correlation of the n-1-th layer of the B path is further passed down to the n-2-th layer of the A path to obtain the correlation ; through the double-input tensor propagation function C l , the LoRA output correlation of the n-th layer is passed up to the trainable parameters of the B path to obtain ; the correlation of the B path is further passed up to the trainable parameters of the A path to obtain . Similarly, the correlations of the B path and the A path are recorded and combined into a trainable parameter correlation graph , represents the b-th module, i.e., the structure number on the path.

[0108] In specific implementation, the correlation is calculated according to the category to which the given PEFT method belongs.

[0109] S2: Using high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation on the privacy loss in the training process, accurately control the noise benchmark strength at the current training round without introducing additional relaxation, and dynamically adjust to ensure that the overall training process meets the differential privacy constraint.

[0110] To finely control the differential privacy budget consumption in the whole fine-tuning process, a high-order privacy budget estimation and control method is introduced in this stage. This method can provide a more compact upper bound estimate of privacy loss without sacrificing training performance, support adaptive adjustment of noise strength during fine-tuning, and ensure that the final privacy budget (ε, δ) is not overdrafted in advance. The specific steps are as follows.

[0111] S21: Initialization of sampling rate and basic parameters:

[0112] Set the privacy budget cap ε and failure probability δ during fine-tuning training, and input the total number of samples N and the batch size P for each training round, then calculate the sampling probability. Set the initial noise standard deviation σ init The gradient clipping threshold C, the expansion order k, and the scaling factor r∈(0,1) used to control scaling noise.

[0113] S22: Calculate the privacy-depleting variable:

[0114] In each differential privacy mechanism M i Above, we define the output distributions of the adjacent datasets D and D′ as P, respectively. i =M i (D) and =M i (D′), from P i , We sampled data from each model and calculated the privacy loss variable, which characterizes the degree of perturbation to the output distribution caused by a small change in the input data (i.e., D→D′) during the current model parameter update step. This variable is the basis for evaluating and accurately estimating the differential privacy loss. The privacy loss variable is represented as follows:

[0115] ;

[0116] In the formula, X i Indicates that from distribution P i Samples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; The Radon–Nikodym derivative represents Relative to P i The probability density ratio of the two random variables mentioned above; , This will be used in subsequent high-order expansion and budget control estimations, and its tail behavior determines the privacy loss estimate for the current fine-tuning training epoch. Tail behavior refers to the probability that the differential privacy mechanism output deviates greatly (i.e., high privacy loss) between adjacent datasets. Although these deviations are rare, they can undermine the overall protection effect of DP if left uncontrolled.

[0117] S23: Calculate privacy budget loss:

[0118] Privacy-depleting variables and The distribution function FX (z) and F Y (z) is approximated as follows:

[0119] ;

[0120] where Φ(z) is the standard normal distribution function, is the density function corresponding to the standard normal distribution function Φ(z), P j (z) is a polynomial term determined by the high-order cumulants of the distribution, used to describe the non-normal deviation of the distribution in the high-order expansion; V ∈ {X, Y} represents the corresponding privacy loss variable; n is the sample size;

[0121] solving the minimum ε that satisfies the following inequality:

[0122] ;

[0123] where ω represents the high-order approximation term of cumulative privacy loss or the security relaxation term; the minimum ε is denoted as ε (α) , ε (α) represents the privacy budget loss under the current noise standard deviation σ.

[0124] S24: adaptively adjusting and locking the noise standard deviation and outputting the budget state:

[0125] If the currently estimated ε (α) > ε, it indicates that the currently configured noise is insufficient to meet the privacy budget constraint, and the iterative scaling process is started: the noise standard deviation is updated as σ ← r·σ, r is a scaling parameter; then steps S22 and S23 are re-executed, and the privacy loss variable and ε (α) are recalculated based on the updated σ; this iterative process will continue until any of the following conditions is met: (1) the currently estimated privacy budget loss satisfies the constraint condition: ε (α) ≤ ε; (2) the current noise standard deviation σ has been reduced to the minimum allowed value set by the system and cannot be scaled further;

[0126] Once the budget constraint is met, the system regards the current noise standard deviation σ as the optimal perturbation strength that can be used in this training round, and locks and saves it. In addition, the estimated privacy budget loss ε (α) is also recorded in the differential privacy budget ledger for subsequent cumulative tracking and budget checking of training rounds. This σ value will be passed to the next stage as the noise strength parameter used when the gradient perturbation module performs noise training.

[0127] S3: Construct a binary mask according to the trainable parameter correlation graph, divide into high and low correlation parameter subsets, and combine the current noise benchmark intensity setting to differentially clip and differentially add noise to the gradients corresponding to the high and low correlation parameters, complete the task model parameter update under the guarantee of differential privacy.

[0128] This step is based on the correlation graph R obtained in the previous two steps PEFT The noise intensity σ output by the budget control is combined to differentially disturb the gradient of each round of training. The core idea is to implement hierarchical noise injection in the parameter dimension: inject larger intensity noise into low correlation parameters and smaller noise into high correlation parameters, so as to retain the optimization ability of more critical parameters to model performance without breaking the upper limit of the differential privacy budget. Specifically, the following steps are included:

[0129] S31: Correlation normalization and mask generation:

[0130] Before performing adaptive gradient perturbation, to achieve differential noise injection at the parameter dimension level, first normalize the parameter correlation graph R calculated in step S1 PEFT , and generate a correlation mask based on the normalized result to divide it into two sub-dimension sets of "low importance" and "high importance"; let the fine-tuning dataset be D FT ={x1,x2,...,x m ,...,x M}, where x m represents the mth sample; the trainable parameter correlation graph corresponding to each sample x m is denoted as , where d is the total number of dimensions of all trainable parameters;

[0131] Calculate the average trainable parameter correlation graph of all samples;

[0132] ;

[0133] In the formula, M represents the total number of samples in the fine-tuning dataset;

[0134] Normalize the average trainable parameter correlation graph to obtain the normalized trainable parameter correlation graph , where:

[0135] ;

[0136] In the formula, is the average correlation value of the jth trainable parameter over the entire fine-tuning dataset; R j ' is the normalized correlation value of the jth trainable parameter; R minR denotes the minimum average correlation value of all parameter dimensions max R denotes the maximum average correlation value of all parameter dimensions; d denotes the total dimension number of trainable parameters.

[0137] Set the correlation division ratio ∈(0,1), represents the proportion of parameters that are expected to be marked as low correlation, from the first small value is extracted as the division threshold R thr :

[0138] ;

[0139] In the formula, represents that the correlation value at the first position after the normalized trainable parameter correlation graph R' is sorted in ascending order of element value is selected as the threshold; wherein β (0,1) represents the proportion of low correlation parameters, and d is the total dimension number of parameters.

[0140] According to this, the correlation mask vector M R ∈{0,1} d is generated, which is used to mark the correlation of the parameter dimensions:

[0141] ;

[0142] In the formula, represents the pth value in the correlation mask vector M R ; represents the pth value in the normalized trainable parameter correlation graph ;

[0143] According to the correlation mask vector M R , the trainable parameters are divided into two parameter subsets of high and low correlation.

[0144] S32: subset sensitivity adaptive clipping:

[0145] Two gradient subsets corresponding to the two parameter subsets of high and low correlation are respectively subjected to differential L2 norm clipping, and the global gradient clipping threshold is set as C>0,

[0146] Calculate the proportion coefficient:

[0147] ;

[0148] In the formula, represents the sensitivity of the high correlation gradient subset corresponding to the high correlation parameter subset; represents the sensitivity of the low correlation gradient subset sensitivity of the task model; denotes the original gradient of the task model under the current training batch, which is derived by taking the derivative of the loss function with respect to the trainable parameters;

[0149] respectively for the two gradient subsets corresponding to the high and low correlation parameter subsets:

[0150] ;

[0151] wherein, denotes the high correlation gradient subset after clipping, denotes the low correlation gradient subset after clipping.

[0152] S33: adaptive noise adding and model updating:

[0153] The global differential privacy control noise intensity σ output by step S2 is used to add Gaussian noise perturbation of different intensities to the gradient subsets of different importance dimensions, in combination with the gradient grouping result of the current training sample, to realize structure-aware differential privacy training update. Assuming that the parameter gradient of the current sample x k has been divided into a strong importance gradient subset g w and a weak importance gradient subset g s , and differential L2 norm clipping has been performed respectively, to obtain the clipped gradient , then different intensity multi-dimensional Gaussian noise is injected into it respectively.

[0154] First, the standard deviations of the high correlation gradient subset and the low correlation gradient subset are dynamically calculated according to the clipping result:

[0155] ;

[0156] wherein, σ denotes the current noise standard deviation, and denote the standard deviations of the high correlation gradient subset and the low correlation gradient subset, respectively;

[0157] On this basis, the high correlation gradient subset and the low correlation gradient subset after clipping are respectively added with noise:

[0158] ;

[0159] wherein, and denote the high correlation gradient subset and the low correlation gradient subset after adding noise, respectively; N(0, I) denotes a multi-dimensional Gaussian distribution with 0 as the mean value and as the covariance, takes or ; I is an identity matrix;

[0160] In each round of training, a batch is randomly sampled from the fine-tuning dataset, and differential clipping and differential noise addition are performed on each training sample in the current training batch in turn to obtain the corresponding noisy gradient. Then, all noisy gradients in the current training batch are averaged to obtain the average noisy gradient of the current training batch. Finally, according to the set learning rate parameter, the average noisy gradient of the current training batch is used to perform a standard gradient descent update operation on the current task model parameters. In the above update process, the current model parameters are adjusted according to the perturbed gradient direction to continue to minimize the loss function; the learning rate controls the update amplitude to ensure the stability and convergence of the training process; the uniform perturbation gradient is a perturbation estimate at the batch level, which combines the individual perturbation of different samples to improve the robustness of the training. The training and updating process will continue to be performed until the preset termination condition is reached, such as the upper limit of the training round, the exhaustion of the privacy budget, or the convergence of the model performance. In this way, the system dynamically injects differential privacy protection perturbation in each round of training while maintaining the optimization effect of parameter updating and model performance.

[0161] Taking the classification and content review task of railway invoices as an example, based on the railway invoice multi-classification model formed by fine-tuning of a large language model, first, the historical invoices archived in the railway bill management system are used as training samples, and the sample data includes invoice number, ticket station, ticket date, amount, invoice type, and ticket section fields, as well as the classification of the invoice (such as freight invoice, insurance invoice, and other bills) and whether there are field missing, format error, and other abnormal labels. Based on the training samples, the above large model training method based on adaptive differential privacy control is used for training to obtain a railway invoice multi-classification model based on fine-tuning of a large language model. During training, the model input is the bill field text processed by OCR recognition and structuring, and the large model performs semantic understanding and feature extraction to output the corresponding invoice type classification result and abnormal label, realizing automatic support for railway bill archiving, verification, and other business processes.

[0162] In addition, the embodiment of the present application also provides a large model training device based on adaptive differential privacy control, which is used for training a task model based on a large language model. The device comprises:

[0163] The parameter correlation evaluation unit is configured to input the training sample data into the task model based on the large language model for forward propagation and backward attribution calculation based on the parameter correlation evaluation method, to generate a trainable parameter correlation graph to measure the correlation distribution of each trainable parameter in the model output.

[0164] The privacy budget estimation and control unit is configured to utilize the high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation on the privacy loss in the training process, accurately control the noise benchmark strength in the current training round without introducing additional relaxation, and dynamically adjust to ensure that the overall training process meets the differential privacy constraint.

[0165] The model parameter updating unit is configured to construct a binary mask according to the trainable parameter correlation graph, divide high and low correlation parameter subsets, and perform differential clipping and differential noise processing on the gradients corresponding to the high and low correlation parameters in combination with the current noise benchmark strength, to complete the task model parameter updating under the differential privacy guarantee.

[0166] It should be understood that the functional unit modules in each embodiment of the present application can be concentrated in one processing unit, or each unit module can exist physically alone, or two or more unit modules can be integrated into one unit module, and can be realized in the form of hardware or software.

[0167] It can be understood that the same or similar parts in the above embodiments can be mutually referenced, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0168] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A large model training method based on adaptive differential privacy control, characterized in that, For training a task model based on a large language model, the method includes, in each training epoch: Based on the parameter correlation evaluation method, the training sample data is input into the task model based on the large language model for forward propagation and backward attribution calculation, generating a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. We utilize high-order privacy budget estimation and control methods to perform high-order expansion and approximate estimation of privacy loss during training, control the noise baseline intensity under the current training round, and dynamically adjust it to ensure that the overall training process meets differential privacy constraints. A binary mask is constructed based on the correlation map of trainable parameters, and a subset of high and low correlation parameters is divided. The gradients corresponding to the high and low correlation parameters are differentially pruned and differentially denoised based on the current noise baseline intensity, thus completing the parameter update of the task model under differential privacy protection. The step of constructing a binary mask based on the correlation graph of trainable parameters and dividing it into subsets of high and low correlation parameters specifically includes: Let the fine-tuning dataset be D. FT ={x1,x2,...,x m ,...,x M }, where x m Let x represent the m-th sample; m The corresponding trainable parameter correlation graph is denoted as , where d is the total number of dimensions of all trainable parameters; Calculate the correlation plot of the average trainable parameters for all samples. ; Correlation plot of average trainable parameters Normalization is performed to obtain the correlation map of normalized trainable parameters. ; Set the correlation division ratio ∈(0,1), representing the proportion of parameters that are desired to be labeled as having low correlation, from Extract the first The smaller value is used as the dividing threshold R. thr ; Based on this, a correlation mask vector M is generated. R ∈{0,1} d This is used to mark the relevance of parameter dimensions: ； In the formula, Represents the correlation mask vector M R The p-th value; Represents the correlation graph of normalized trainable parameters The p-th value in; Based on the correlation mask vector M R The trainable parameters are divided into two subsets: high and low correlation. The differentiated cutting process includes: For the two gradient subsets corresponding to the two parameter subsets with high and low correlation, perform differential L2 norm clipping respectively. Assuming the global gradient clipping threshold is C>0, calculate the scaling factor: ； In the formula, This represents the subset of highly correlated gradients corresponding to the subset of highly correlated parameters. Sensitivity; This represents the subset of low-correlation gradients corresponding to the subset of low-correlation parameters. Sensitivity; This represents the original gradient of the task model in the current training batch, obtained by taking the derivative of the loss function with respect to the trainable parameters. Prune the two gradient subsets corresponding to the two parameter subsets of high and low correlation respectively: ； In the formula, This represents the subset of highly relevant gradients after pruning. This represents the subset of low-relevance gradients after clipping. The differential noise addition process includes: The standard deviations of the highly relevant gradient subset and the low-relevance gradient subset are dynamically calculated based on the clipping results. ； In the formula, σ represents the current noise standard deviation. and represent the standard deviations of the highly correlated gradient subset and the low-correlation gradient subset, respectively; Based on this, noise is added to the clipped high-relevance gradient subset and low-relevance gradient subset respectively: ； In the formula, and Let N(0, ...) represent the high-correlation gradient subset and the low-correlation gradient subset after adding noise, respectively; I) indicates that the mean is 0. The covariance is a multidimensional Gaussian distribution. Pick or ; I is the identity matrix.

2. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The method for generating the trainable parameter correlation map specifically includes: The training sample data of each training round is input into the frozen pre-trained Big Prophet model to perform forward propagation, and its output and intermediate activation values ​​are recorded to generate output correlations. The output correlation is reduced in dimensionality to obtain the initial output correlation; Based on the parameter correlation evaluation method, the initial output correlation is propagated from the output layer to each trainable parameter layer by layer to obtain the trainable parameter correlation of each layer, and finally stitched into a complete and continuous trainable parameter correlation graph.

3. The large model training method based on adaptive differential privacy control according to claim 2, characterized in that, The process of reducing the dimensionality of the output correlation includes: For each word, select a threshold l, and retain only the probability of the top l words in the vocabulary, while setting the probability of the remaining words to zero; For each word element, retain the words in the probability-based descending vocabulary until the cumulative probability of the retained words reaches a threshold p, and set the probability of the remaining words to zero; Following the sentence order, based on the lexical units retained at each step, the cumulative probability is calculated and the top b sentence frames with the highest values ​​are selected, while the probability of unselected words is set to zero.

4. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The method utilizes a high-order privacy budget estimation and control approach to perform high-order expansion and approximate estimation of privacy loss during training, controls the noise baseline intensity in the current training round, and dynamically adjusts it to ensure that the overall training process meets differential privacy constraints. Specifically, this includes: Sampling rate and basic parameter initialization: Set the sampling rate, as well as the privacy budget cap ε, failure probability δ, and initial noise standard deviation σ. init Gradient clipping threshold C, unfolding order k, and scaling factor r used to control scaling noise; In each differential privacy mechanism M i Above, we define the output distributions of the adjacent datasets D and D′ as P, respectively. i =M i (D) and =M i (D′), from P i , We sampled from each sample, calculated the privacy loss variable, and represented it as follows: ； In the formula, X i Indicates that from distribution P i Samples from the middle Privacy loss observed above; Y i Indicates from the distribution Samples from the middle The observed loss of privacy; express Relative to P i The probability density ratio; Privacy-depleting variables and The distribution function F X (z) and F Y (z) is expanded using a higher-order approximation, as follows: ； In the formula, Φ(z) is the standard normal distribution function. Let P be the density function corresponding to the standard normal distribution function Φ(z). j (z) is a polynomial term determined by the higher-order cumulants of the distribution, used to describe the non-normal deviation of the distribution in the higher-order expansion; V∈{X,Y}; n is the sample size; Find the minimum ε that satisfies the following inequality: ； In the formula, ω represents a higher-order approximation term or security relaxation term for the cumulative privacy loss; the minimum ε is denoted as ε (α) , ε (α) This represents the privacy budget loss under the current noise standard deviation σ; If the currently estimated ε (α) >ε indicates that the current noise configuration is insufficient to meet the privacy budget constraint, initiating an iterative scaling process: updating the noise standard deviation to σ←r·σ, where r is a scaling parameter; subsequently, recalculating the privacy cost variable and ε based on the updated σ. (α) The iterative process will continue until one of the following conditions is met: (1) The currently estimated privacy budget loss satisfies the constraint: ε (α) ≤ε; (2) The current noise standard deviation σ has dropped to the minimum allowable value set by the system and cannot be scaled further; Save the current noise standard deviation σ and privacy budget loss ε (α) .

5. The large model training method based on adaptive differential privacy control according to claim 1, characterized in that, The specific steps for updating the task model parameters under differential privacy protection include: Differential cropping and differential noise addition are performed on each training sample in the current training batch in sequence to obtain the corresponding noise gradient; The average noisy gradient of the current training batch is obtained by averaging all the noisy gradients in the current training batch. Based on the set learning rate parameter, the gradient descent operation is used to update the parameters of the current task model using the average noisy gradient of the current training batch.

6. A large model training device based on adaptive differential privacy control, characterized in that, The apparatus for implementing the large model training method based on adaptive differential privacy control as described in claim 1, for training a task model based on a large language model, comprises: The parameter correlation evaluation unit is configured to use a parameter correlation evaluation method to input training sample data into a task model based on a large language model for forward propagation and backward attribution calculation, and generate a trainable parameter correlation map to measure the correlation distribution of each trainable parameter in the model output. The privacy budget estimation and control unit is configured to use a high-order privacy budget estimation and control method to perform high-order expansion and approximate estimation of privacy loss during training, control the noise baseline intensity under the current training round, and dynamically adjust to ensure that the overall training process meets differential privacy constraints. The model parameter update unit is configured to construct a binary mask based on the correlation map of trainable parameters, divide the high and low correlation parameters into subsets, and perform differential pruning and differential noise addition on the gradients corresponding to the high and low correlation parameters in combination with the current noise baseline intensity, thereby completing the task model parameter update under differential privacy protection.

Images