Differential privacy model training method based on generative replay, medium and system

Abstract

The application discloses a differential privacy model training method, medium and system based on a generative replay, wherein the method comprises the following steps: obtaining a private instruction data set corresponding to a current task, and generating a corresponding privacy unit set; generating a candidate replay sample set; calculating a current task budget requirement score based on the candidate replay sample set; determining a noise multiplier according to the current task budget requirement score; calculating a replay value, and determining a final replay sample set according to the replay value; calculating a merged privacy unit gradient corresponding to the current task; calculating a corresponding auxiliary gradient, and merging the auxiliary gradient to update low-rank adaptive parameters according to a merging result; judging whether all the tasks have been completed; if yes, taking a current base model as a final differential privacy model. When continuous fine-tuning is performed, the historical task samples do not need to be stored, thereby avoiding privacy risks; meanwhile, replay resources are reasonably allocated, and the replay effect of the historical task samples is improved.

Description

Technical Field

[0001] This application relates to the field of differential privacy model training technology, and in particular to a differential privacy model training method, medium and system based on generative replay. Background Technology

[0002] With the widespread application of large language models in vertical industries and private scenarios, these models typically need to be fine-tuned through instructions to adapt to specific tasks, business processes, or user needs. In actual deployments, instruction data is usually not given statically all at once, but rather continuously arrives as user interactions, organizational business updates, domain knowledge accumulation, and application requirements change. Therefore, the model needs to have the ability to continuously fine-tune instructions, that is, to retain the capabilities already acquired in previous tasks while learning new tasks.

[0003] In related technologies, when performing continuous instruction fine-tuning of differential privacy model training as described above, a sample replay mechanism is often used to mitigate this problem. That is, historical task samples are retained during training, and when learning new tasks, these historical task samples are mixed in, allowing the model to re-examine the knowledge from older tasks. However, since historical task samples often contain sensitive information such as user conversations, medical records, corporate documents, financial materials, or legal texts, this method of preserving and reusing historical task samples introduces privacy risks.

[0004] Furthermore, the degree of forgetting varies among different historical task samples during the continuous fine-tuning process. However, related technologies often replay historical task samples uniformly or at a fixed ratio, failing to differentiate the importance of each sample. This leads to limited replay resources being excessively consumed on low-quality historical task samples, while neglecting those with a high risk of forgetting. Ultimately, this results in poor performance when replaying historical task samples. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in related technologies. To this end, one objective of this invention is to propose a differential privacy model training method based on generative replay. This method eliminates the need to store historical task samples during continuous fine-tuning of the differential privacy model training, thus avoiding privacy risks. Simultaneously, it rationally allocates replay resources to improve the replay effect of historical task samples, thereby enhancing the accuracy of the final differential privacy model.

[0006] According to an embodiment of the present invention, a differential privacy model training method based on generative replay includes the following steps: obtaining a private instruction dataset corresponding to the current task, wherein the private instruction dataset includes multiple training instructions and the privacy unit identifier corresponding to each training instruction; generating a privacy unit set corresponding to the current task based on the privacy unit identifier; generating candidate replay samples corresponding to the historical task based on the differential privacy task teacher model and public prompt words corresponding to each historical task, and generating a candidate replay sample set based on the candidate replay samples; calculating the scheduling statistics component between the current task and the historical task based on the candidate replay sample set, and calculating the budget requirement score of the current task based on the scheduling statistics component; allocating a privacy budget to the current task based on the current task budget requirement score, and determining the noise multiplier based on the allocated private training budget of the current task; calculating the replay value of each historical task in the current task, and determining candidate replay samples based on the replay value. The process involves: assigning integers to replay samples and determining the final replay sample set based on the assigned integers and the candidate replay sample set; grouping the private instruction dataset corresponding to the current task according to the set of privacy units corresponding to the current task to obtain a private instruction data group for each privacy unit, calculating the privacy unit gradient for each private instruction data group based on noise multipliers, and merging the privacy unit gradients to obtain the merged privacy unit gradient for the current task; calculating the corresponding auxiliary gradient based on the final replay sample set, and merging the merged privacy unit gradient and the auxiliary gradient to update the low-rank adaptation parameters based on the merging result; generating a differential privacy task teacher model corresponding to the current task based on the updated current base model, and updating the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task; determining whether all tasks have been completed; if so, using the current base model as the final differential privacy model.

[0007] In some embodiments, after generating the set of privacy units corresponding to the current task based on the privacy unit identifier, the method further includes:

[0008] Perform a budget feasibility check to determine whether the execution of the current task will result in budget overruns;

[0009] If so, the base model corresponding to the previous task will be used as the final differential privacy model;

[0010] The budget feasibility check is conducted according to the following formula:

[0011] ;

[0012] ;

[0013] ;

[0014] in, Indicates task Global remaining RDP budget before start Represents order The global RDP budget limit is set below. Indicates task RDP costs in China This indicates the number of remaining tasks, including the current task. Indicates the total number of tasks. This represents the minimum RDP budget reserved for each task.

[0015] In some embodiments, the scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

[0016] In some embodiments, the current task difficulty component is calculated using the following formula:

[0017] ;

[0018] ;

[0019] in, This indicates the current base model in the task. Average instruction loss on the public anchor set, Indicates task The set of public anchor points The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, This indicates the difficulty component of the current task. This indicates that the task difficulty loss is referenced to the lower bound. The upper bound indicates the difficulty loss of the task.

[0020] The overall conflict intensity component is calculated using the following formula:

[0021] ;

[0022] ;

[0023] ;

[0024] in, Indicates the task's impact on the intensity of the conflict. Indicates the current mission detection gradient. Representing historical tasks The gradient computed on the candidate replay sample set, This represents a small constant to prevent the denominator from being zero. Indicates the weight of the importance of historical tasks. This represents the application-defined and normalized task. The importance of Indicates the overall conflict intensity component;

[0025] The expected forgetting stress component is calculated using the following formula:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] in, Representing historical tasks The performance degradation component, Indicates completion of task Afterwards, the model in historical tasks Performance based on publicly available benchmark data This indicates the upper bound of the normalized reference for performance degradation. This indicates an increase in the amount of loss at the publicly disclosed anchor point. The low-rank adaptation parameter is... The basic model in the task The average loss on the public anchor set, This represents the normalized upper bound of the increase in publicly disclosed anchor point loss. This represents the difference components between the teacher model's output and the current model's output for the differential privacy task. Indicates task The set of publicly available prompt words for description. Represents probability distribution and KL divergence between them Indicates task Corresponding differential privacy task teacher model In the input Conditional output distribution on The low-rank adaptation parameter is... The basic model in input Conditional output distribution on This represents the normalized upper bound of the KL divergence between the teacher model and the current model for the differential privacy task. This indicates the expected forgetting pressure component. This represents the normalized weight corresponding to the performance degradation component. This represents the normalized weight corresponding to the increase in the open anchor loss component. This represents the normalized weights corresponding to the difference components between the teacher model and the current model outputs in the differential privacy task.

[0031] The data size component is calculated using the following formula:

[0032] ;

[0033] in, Indicates the data size components. Indicates task The number of private records, This represents the preset data size normalization reference value.

[0034] In some embodiments, the current task budget requirement score is calculated using the following formula:

[0035]

[0036] in, This indicates the score required for the current task's budget. This indicates the difficulty component of the current task. This indicates the weight corresponding to the current task difficulty component. This represents the overall conflict intensity component. This indicates the weights corresponding to the overall conflict intensity components. This indicates the expected forgetting pressure component. This indicates the weight corresponding to the expected forgetting pressure component. Indicates the data size components. This indicates the weights corresponding to the data size components. Indicates the importance of the task. This represents the weight corresponding to the task importance component;

[0037] Allocate the privacy budget according to the following formula:

[0038] ;

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] in, Indicates task The RDP budget retained for future missions, This indicates the number of remaining tasks, including the current task. This indicates the minimum total RDP budget reserved for each task. Indicates task Total available RDP budget Indicates task Global remaining RDP budget before start Indicates the total budget of the candidates. This indicates the score required for the current task's budget. This indicates that the task requires a large budget. This indicates the maximum total RDP budget allowed for a single task. Indicates private training budget, Indicates task Optional detection budget;

[0044] The noise multiplier is determined according to the following formula:

[0045] ;

[0046] in, Indicates task Private training iteration steps, Indicates the sampling rate The noise multiplier is The subsampling Gaussian mechanism in order The cost of a single-step RDP step.

[0047] In some embodiments, the replay value is calculated according to the following formula:

[0048] ;

[0049] ;

[0050] in, This indicates the quality value of the candidate replay sample. This represents the teacher confidence component. This indicates the weights corresponding to the teacher confidence components. Indicates the efficiency components of the format and tags. This indicates the weights corresponding to the format and tag effectiveness components. Represents the diversity components of candidate samples. This represents the weights corresponding to the diversity components of the candidate samples. Memory risk agency component, This indicates the weight corresponding to the memory risk proxy component. Indicates the value of replay. Indicates the task's impact on the intensity of the conflict. This indicates the risk of historical tasks being forgotten. This indicates the weight of the historical mission.

[0051] In some embodiments, after determining the final replay sample set based on the allocated integer and the candidate replay sample set, the method further includes: constructing the training objective for the current task;

[0052] The training objective of the current task is expressed by the following formula:

[0053] ;

[0054] in, Indicates the low-rank adaptation parameters. Indicates task The overall training objective This indicates the current loss of private data oversight. Indicates task Private instruction dataset, This represents the loss of generative replay supervision. This represents the weights corresponding to the generative replay supervision loss. This represents the distillation loss of the teacher model for the differential privacy task. The weights represent the distillation loss corresponding to the teacher model in the differential privacy task. This indicates a parameter-stable regularization term. This indicates the weight corresponding to the parameter-stable regularization term. Indicates task The low-rank adaptation parameters after completion;

[0055] ;

[0056] ;

[0057] ;

[0058] in, Indicates task No. Small batches of private records sampled step by step. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Indicates task No. The step involves sampling small batches of candidate replay samples. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Represents probability distribution and KL divergence between them Indicates the distillation temperature. Computational differential privacy task teacher model In the input Conditional output distribution on Indicates the distillation temperature. The current base model for computation is based on the input. The conditional output distribution.

[0059] Secondly, embodiments of the present invention propose a computer-readable storage medium storing a differential privacy model training program based on generative replay, which, when executed by a processor, implements the differential privacy model training method based on generative replay as described above.

[0060] Thirdly, this invention proposes a differential privacy model training system based on generative replay, comprising: an acquisition module for acquiring a private instruction dataset corresponding to the current task, the private instruction dataset including multiple training instructions and the privacy unit identifier corresponding to each training instruction; a recording module for generating a privacy unit set corresponding to the current task based on the privacy unit identifier; a candidate sample generation module for generating candidate replay samples corresponding to historical tasks based on the differential privacy task teacher model corresponding to each historical task, and generating a candidate replay sample set based on the candidate replay samples; a scheduling statistics module for calculating the scheduling statistics components of the current task and historical tasks based on the candidate replay sample set, and calculating the budget requirement score of the current task based on the scheduling statistics components; an allocation module for allocating a privacy budget to the current task based on the current task budget requirement score, and determining a noise multiplier based on the allocated private training budget of the current task; and a sample determination module for calculating the replay value of each historical task in the current task, and determining the noise multiplier based on the allocated private training budget of the current task. The system comprises: a value-determining integer for allocating candidate replay samples, and a final replay sample set based on the allocated integer and the candidate replay sample set; a gradient calculation module, which groups the private instruction dataset corresponding to the current task according to the set of privacy units corresponding to the current task to obtain a private instruction data group corresponding to each privacy unit, calculates the privacy unit gradient corresponding to each private instruction data group based on the noise multiplier, and merges the privacy unit gradients to obtain the merged privacy unit gradient corresponding to the current task; a first update module, which calculates the corresponding auxiliary gradient based on the final replay sample set and merges the merged privacy unit gradient and the auxiliary gradient to update the low-rank adaptation parameters based on the merging result; a second update module, which generates a differential privacy task teacher model corresponding to the current task based on the updated current base model and updates the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task; and a judgment module, which determines whether all tasks have been completed and, if the judgment result is yes, uses the current base model as the final differential privacy model.

[0061] In some embodiments, the scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

[0062] According to the generative replay-based differential privacy model training method of the present invention, candidate replay samples corresponding to historical tasks are generated based on the differential privacy task teacher model and public prompts corresponding to each historical task. That is, the training method proposed in this embodiment stores the differential privacy task teacher model corresponding to the historical task, rather than directly storing the historical task samples. Furthermore, the candidate replay samples are generated based on the differential privacy task teacher model and public prompts corresponding to the historical task, fundamentally avoiding privacy risks caused by storing and reusing historical task samples. Secondly, before replaying a historical task, the replay value corresponding to each historical task is calculated to determine the final replay sample set, which can reasonably allocate replay resources and avoid wasting limited resources on low-quality historical task samples, thus improving the replay effect of historical task samples. In addition, unlike the prior art, the training method proposed in this embodiment does not use a single task as the privacy budget statistical object, but rather a privacy unit (e.g., the same user, the same institution, or the same data subject) as the statistical object; thus, continuous tracking, accurate accounting, and audit verification of repeatedly participating privacy units can be achieved. This avoids the problem that the system only records the overall budget but cannot explain the actual privacy consumption of specific users or data subjects.

[0063] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the differential privacy model training method based on generative replay according to an embodiment of the present invention.

[0065] Figure 2 This is a block diagram of a differential privacy model training system based on generative replay according to an embodiment of the present invention. Detailed Implementation

[0066] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0067] It should be noted that the trained differential privacy model can be used in user behavior data analysis, personalized recommendation systems, traffic statistics (e.g., foot traffic statistics in commercial areas, app page visit traffic statistics, etc.), finance, healthcare, and cross-institutional collaborative modeling in government, without limiting its specific domain. In other words, during the training process of this application, the training data can be historical user behavior data, historical user privacy preference data, historical traffic access data, historical financial data, historical medical data, etc.

[0068] Explanation of symbols:

[0069] : Continuous command to fine-tune the total number of tasks.

[0070] Current task index .

[0071] , , : Historical task or summation index; when used as a historical task index, it satisfies .

[0072] : No. A private instruction dataset for each task.

[0073] : No. The first task The instruction input for each record.

[0074] :and The corresponding target output.

[0075] : No. The first task The privacy unit identifier to which each record belongs.

[0076] :Task The number of private records.

[0077] : The set of privacy units in a complete task sequence.

[0078] Participate in the task A collection of privacy units.

[0079] Privacy units participate in indicator variables; privacy units Participate in the task If the value is 1, then the value is 0; otherwise, the value is 0.

[0080] : Frozen pre-trained base model.

[0081] Complete the task The merged LoRA parameters; These are the initial LoRA parameters.

[0082] :Task The internal DP teacher is saved after completion.

[0083] : A frozen weight matrix in the base model that is adapted by LoRA.

[0084] , :Task The completed LoRA low-rank matrix; bold is used to distinguish it from the participating indicator variables. .

[0085] : The rank of the LoRA low-rank decomposition.

[0086] LoRA scaling factor, indicated by the subscript " "Distinguished from RDP order" .

[0087] The weights after LoRA adaptation are:

[0088] ;

[0089] : RDP order.

[0090] The privacy accountant uses a finite set of RDP orders.

[0091] Order The global RDP budget cap is set below.

[0092] :Task The total RDP cost includes optional probe cost and private training cost.

[0093] :Task The optional DP probe budget; if all scheduling statistics are from public data or DP teacher outputs, this value is 0.

[0094] :Task RDP budget used for private training of DP-SGD.

[0095] Privacy Unit Cumulative RDP cost across the complete task sequence.

[0096] Complete the task After that, privacy unit Accumulated RDP costs; initial value .

[0097] :Task Global remaining RDP budget before starting.

[0098] For the task The RDP budget will be retained for future missions.

[0099] After deducting future budget reservations, the task Total available RDP budget.

[0100] : The minimum total RDP budget reserved for each task.

[0101] : The maximum total RDP budget allowed for a single task.

[0102] : The candidate total budget obtained based on the task budget requirement score.

[0103] : The number of remaining tasks, including the current task.

[0104] , : Convert the RDP into parameters with approximate differential privacy.

[0105] Privacy unit gradient Norm clipping threshold.

[0106] :Task The sampling rate of the privacy unit.

[0107] :Task The number of private training iterations.

[0108] :Task Gaussian noise multipliers were used.

[0109] Sampling rate The noise multiplier is The subsampling Gaussian mechanism in order The cost of a single-step RDP step.

[0110] The total budget for the task is broken down as follows:

[0111]

[0112] :Task A collection of public prompts, public templates, public tag names, or public task descriptions.

[0113] :Task The set of publicly available anchor points; the samples within it do not belong to protected private data.

[0114] By history teachers For the current task The generated candidate replay set.

[0115] : The number of samples in the candidate replay set.

[0116] : Current task difficulty.

[0117] Historical mission With the current task The intensity of the conflict.

[0118] The overall degree of conflict between the current mission and all historical missions.

[0119] Historical mission In the current task The risk of forgetting beforehand.

[0120] : Performance degradation component.

[0121] The loss of the public anchor point increases.

[0122] : The difference component between the history teacher and the current model output.

[0123] Gradient conflict components, equal to or composed of Normalization yields the result.

[0124] Historical mission The quality of candidate replays.

[0125] Teacher confidence component of candidate samples.

[0126] : The format and label of the candidate samples are effective components.

[0127] : Diversity component of candidate samples.

[0128] : Near replication or memory risk proxy component of candidate samples.

[0129] : The expected forgetting pressure corresponding to the current task.

[0130] :Task Data size factors; subscripts in this symbol Indicates the current task and its index in historical tasks. different.

[0131] The application side provides and normalizes the task. importance.

[0132] Historical mission Normalized importance weights in the current task.

[0133] :Task Budget requirement score.

[0134] Historical mission In the current task The replay value in the text.

[0135] :Task Total replay sample budget.

[0136] The initial allocation of real numbers obtained by softmax scheduling.

[0137] Historical tasks after rounding In the mission The final number of replay samples retained.

[0138] Historical mission In the mission The final replay dataset retained in the process.

[0139] :Task The merged replay buffer used.

[0140] :softmax replay scheduling temperature.

[0141] The non-negative weights of each component of the forgetting risk satisfy the following conditions: .

[0142] : Non-negative weights of each component of the replay quality.

[0143] The non-negative weights of each component of the budget requirement score satisfy... .

[0144] : Data size normalization reference value, configured by the system.

[0145] Similarity threshold for determining near-copy risk.

[0146] : The maximum number of supplementary generation rounds allowed to be executed to reach the target number of replays.

[0147] : The set of DP teachers stored within the trusted training environment.

[0148] The current model in the task Average instruction loss on the public anchor set.

[0149] , : Lower and upper bounds for normalizing task difficulty loss.

[0150] The current mission detection gradient is calculated on publicly available anchor points or DP-protected detection data.

[0151] In historical mission The gradient is computed on the candidate replay set.

[0152] Complete the task Afterwards, the model in historical tasks Performance based on publicly available benchmark data.

[0153] : The normalized upper bound for performance degradation.

[0154] : Parameters are The model in the task The average loss on the public anchor set.

[0155] : Normalized upper bound for the increase in public anchor point loss.

[0156] , History teacher and parameters are The current model at input The conditional output distribution.

[0157] Teacher—the normalized upper bound of the KL divergence of the current model.

[0158] The weights of the three non-conflict components in the forgetting stress after renormalization satisfy the following conditions: .

[0159] The first in the candidate replay set One instruction—output sample.

[0160] Candidate samples The number of tokens output.

[0161] : The token position index in the candidate sample output sequence. .

[0162] : A DP security reference set consisting of public anchors, public templates, and previously generated replay samples.

[0163] Initial allocation for real numbers The middle integer after rounding down.

[0164] The number of replay samples still to be allocated during the rounding process of the maximum remainder.

[0165] The last task number to complete training under budget constraints.

[0166] The LoRA parameter is At that time, instruction-output pair The instructions fine-tune the loss.

[0167] :Task The overall training objective.

[0168] : Supervision loss on the private data of the current task.

[0169] : Supervision loss on generative replay data.

[0170] Distillation loss between historical DP teachers and the current model.

[0171] : A stable regularization term that limits the deviation of the current LoRA parameter from the parameter of the previous task.

[0172] : Replay loss weights.

[0173] Distillation loss weight.

[0174] : Weight of stable regularization term; this sign is related to the sampling rate. Distinguish by subscript.

[0175] Distillation temperature.

[0176] :Task The training step index within.

[0177] :Task No. A private record minibatch for step sampling.

[0178] : It contains a set of privacy units.

[0179] : No. Step belongs to the privacy unit The set of record indexes.

[0180] :Task No. Steps sample minibatch from the replay buffer.

[0181] Privacy Unit In the mission No. The aggregate gradient of the step.

[0182] : The gradient of the privacy unit after cropping.

[0183] Private gradient after adding Gaussian noise.

[0184] : The auxiliary gradient calculated by replay, distillation, and regularization terms.

[0185] : Used to update the total gradient of LoRA.

[0186] : The dimension of the LoRA parameter vector.

[0187] : 3D identity matrix.

[0188] :Task No. The learning rate of the step; superscript " "Used to distinguish from budget weights" .

[0189] , Distillation temperature The calculated teacher and current model output distribution.

[0190] The mean is 0 and the covariance matrix is The multivariate Gaussian distribution; This represents the corresponding covariance matrix.

[0191] : Relative to LoRA parameters The gradient operator.

[0192] : Positive part operation.

[0193] Truncate the value to .

[0194] Based on lower reference bound and the Upper Realm The normalization operator.

[0195] : A small constant to prevent the denominator of normalization or weighted average from being zero.

[0196] : A small constant to prevent the denominator of the gradient cosine similarity from being zero.

[0197] Probability distribution and The KL divergence between them.

[0198] The cosine similarity between two text representations or vector representations, whose values ​​are mapped to... .

[0199] : Sample The mapping is to a public text embedding or model representation used for similarity calculation.

[0200] : Indicator function, takes the value 1 if the condition is true, otherwise takes the value 0.

[0201] Round down to the nearest integer.

[0202] :gather The number of elements.

[0203] Using teachers Public prompt word set and random variables Generate a function to generate candidate replays.

[0204] : For candidate set Perform screening based on format, label, confidence level, length, repetition, and near-copy risk.

[0205] According to the overall quality from the set Select up to One sample.

[0206] , : Natural exponential function and natural logarithmic function.

[0207] , : The set of positive integers and the set of non-negative integers.

[0208] : Empty set; Set union operation.

[0209] The following describes, with reference to the accompanying drawings, a method for training a differential privacy model based on generative replay according to an embodiment of the present invention.

[0210] Please see Figure 1 , Figure 1 This is a flowchart illustrating the differential privacy model training method based on generative replay according to an embodiment of the present invention, as shown below. Figure 1 As shown, the training method for the differential privacy model based on generative replay includes the following steps:

[0211] S101, obtain the private instruction dataset corresponding to the current task. The private instruction dataset includes multiple training instructions and the privacy unit identifier corresponding to each training instruction.

[0212] S102, Generate a set of privacy units corresponding to the current task based on the privacy unit identifier.

[0213] In some embodiments, after generating the set of privacy units corresponding to the current task based on the privacy unit identifier, the method further includes: performing a budget feasibility check to determine whether the execution of the current task will result in a budget overrun.

[0214] If so, the base model corresponding to the previous task will be used as the final differential privacy model;

[0215] The budget feasibility check is conducted according to the following formula:

[0216] ;

[0217] ;

[0218] ;

[0219] in, Indicates task Global remaining RDP budget before start Represents order The global RDP budget limit is set below. Indicates task RDP costs in China This indicates the number of remaining tasks, including the current task. Indicates the total number of tasks. This represents the minimum RDP budget reserved for each task.

[0220] In some embodiments, before obtaining the private instruction dataset corresponding to the current task, the input parameters can be obtained first, and basic feasibility condition judgments can be performed on the input parameters. When the basic feasibility conditions are determined to be met, the model, teacher set, and budget state can be initialized.

[0221] Specifically, first, the system receives the following input: freeze the base model. Initial LoRA parameters Task sequence Public prompt word set Public Anchor Set Global RDP budget Minimum task budget Highest mission budget Optional detection budget Training steps Privacy unit sampling rate , clipping threshold Total replay budget Weighting coefficients and number of candidates generated Normalized reference boundary , , , , , as well as the generation and filtering thresholds.

[0222] Next, let the basic feasibility conditions be:

[0223] ;

[0224] as well as: ;

[0225] If the basic feasible conditions are not met, the system department will start private training and directly prompt you to reconfigure the global configuration, minimum budget, or number of tasks.

[0226] If the basic feasibility conditions are met, then perform initialization:

[0227] ;

[0228] in, For the internal DP teacher set, initialize the cumulative cost for all privacy units. .

[0229] As an example, after completing the basic feasibility checks and initialization, the current task is received, which includes its corresponding private instruction dataset.

[0230] Specifically, for The system received:

[0231] ;

[0232] in, Indicates the first Private instruction dataset for each task. Indicates the first The first task Input of instructions for each record, This indicates the corresponding target output. Indicates the first The first task The privacy unit identifier to which each record belongs. Indicates task The number of private records.

[0233] The system according to Obtain the current task privacy unit set:

[0234] ;

[0235] And update the participating variables: when Time settings Otherwise set .

[0236] Next, after generating the set of privacy units corresponding to the current task, a budget feasibility check is performed.

[0237] Specifically, firstly, the computational task Global remaining RDP budget before starting:

[0238] ;

[0239] in, Indicates task Global remaining RDP budget before start Represents order The global RDP budget limit is set below. Indicates task RDP costs in China;

[0240] Next, calculate the number of remaining tasks:

[0241] ;

[0242] in, This indicates the number of remaining tasks, including the current task. Indicates the total number of tasks;

[0243] If any make:

[0244] ;

[0245] Then the current task and its subsequent tasks will no longer access private data, the process will terminate, and the base model after the previous task is completed will be output as the final differential privacy model. Since this judgment occurs before the private training of the current task, there will be no ex-post budget overrun.

[0246] If the assessment result indicates that the budget is feasible (i.e., the execution of the current task will not result in a budget overrun), then continue with the subsequent training steps; preferably, if all scheduling statistics in subsequent steps come from public anchors, public cue words, DP teacher output, or existing replay data, then... If it is necessary to use the current private data to calculate the probe statistics, then only the pre-configured ones should be used. Perform DP probing and include the cost in the total mission budget.

[0247] S103, Based on the differential privacy task teacher model and public prompt words corresponding to each historical task, generate candidate replay samples corresponding to the historical task, and generate a candidate replay sample set based on the candidate replay samples.

[0248] S104: Calculate the scheduling statistics components of the current task and historical tasks based on the candidate replay sample set, and calculate the budget requirement score of the current task based on the scheduling statistics components.

[0249] S105, allocate privacy budget to the current task based on the current task budget requirement score, and determine the noise multiplier based on the allocated private training budget of the current task.

[0250] In some embodiments, the scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

[0251] In some embodiments, the current task difficulty component is calculated using the following formula:

[0252] ;

[0253] ;

[0254] in, This indicates the current base model in the task. Average instruction loss on the public anchor set, Indicates task The set of public anchor points The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, This indicates the difficulty component of the current task. This indicates that the task difficulty loss is referenced to the lower bound. The upper bound indicates the difficulty loss of the task.

[0255] The overall conflict intensity component is calculated using the following formula:

[0256] ;

[0257] ;

[0258] ;

[0259] in, Indicates the task's impact on the intensity of the conflict. Indicates the current mission detection gradient. Representing historical tasks The gradient computed on the candidate replay sample set, This represents a small constant to prevent the denominator from being zero. Indicates the weight of the importance of historical tasks. This represents the application-defined and normalized task. The importance of Indicates the overall conflict intensity component;

[0260] The expected forgetting stress component is calculated using the following formula:

[0261] ;

[0262] ;

[0263] ;

[0264] ;

[0265] in, Representing historical tasks The performance degradation component, Indicates completion of task Afterwards, the model in historical tasks Performance based on publicly available benchmark data This indicates the upper bound of the normalized reference for performance degradation. This indicates an increase in the amount of loss at the publicly disclosed anchor point. The low-rank adaptation parameter is... The basic model in the task The average loss on the public anchor set, This represents the normalized upper bound of the increase in publicly disclosed anchor point loss. This represents the difference components between the teacher model's output and the current model's output for the differential privacy task. Indicates task The set of publicly available prompt words for description. Represents probability distribution and KL divergence between them Indicates task Corresponding differential privacy task teacher model In the input Conditional output distribution on The low-rank adaptation parameter is... The basic model in input Conditional output distribution on This represents the normalized upper bound of the KL divergence between the teacher model and the current model for the differential privacy task. This indicates the expected forgetting pressure component. This represents the normalized weight corresponding to the performance degradation component. This represents the normalized weight corresponding to the increase in the open anchor loss component. This represents the normalized weights corresponding to the difference components between the teacher model and the current model outputs in the differential privacy task.

[0266] The data size component is calculated using the following formula:

[0267] ;

[0268] in, Indicates the data size components. Indicates task The number of private records, This represents the preset data size normalization reference value.

[0269] In some embodiments, the current task budget requirement score is calculated using the following formula:

[0270]

[0271] in, This indicates the score required for the current task's budget. This indicates the difficulty component of the current task. This indicates the weight corresponding to the current task difficulty component. This represents the overall conflict intensity component. This indicates the weights corresponding to the overall conflict intensity components. This indicates the expected forgetting pressure component. This indicates the weight corresponding to the expected forgetting pressure component. Indicates the data size components. This indicates the weights corresponding to the data size components. Indicates the importance of the task. This represents the weight corresponding to the task importance component;

[0272] Allocate the privacy budget according to the following formula:

[0273] ;

[0274] ;

[0275] ;

[0276] ;

[0277] ;

[0278] in, Indicates task The RDP budget retained for future missions, This indicates the number of remaining tasks, including the current task. This indicates the minimum total RDP budget reserved for each task. Indicates task Total available RDP budget Indicates task Global remaining RDP budget before start Indicates the total budget of the candidates. This indicates the score required for the current task's budget. This indicates that the task requires a large budget. This indicates the maximum total RDP budget allowed for a single task. Indicates private training budget, Indicates task Optional detection budget;

[0279] The noise multiplier is determined according to the following formula:

[0280] ;

[0281] in, Indicates task Private training iteration steps, Indicates the sampling rate The noise multiplier is The subsampling Gaussian mechanism in order The cost of a single-step RDP step.

[0282] As an example, when When there is no historical task, it indicates that there is no historical task at present. , And skip historical task statistics. When At that time, for each historical task The system first uses the differential privacy task teacher model corresponding to the historical task and the public prompt words to generate a fixed-size candidate replay set:

[0283] ;

[0284] in, This indicates the teacher model for the differential privacy task. For the current task The generated candidate replay set, Indicates task Publicly displayed keywords Represents a random variable. This indicates the number of samples in the candidate replay sample set.

[0285] It is understandable that this candidate replay sample set is used both to calculate the replay quality and as a candidate source for the final replay samples. Generating the candidate replay sample set does not access historical original private data, thus avoiding privacy risks and not increasing the privacy costs of historical data.

[0286] Next, the average instruction loss of the base model corresponding to the previous task is applied to the public anchor set of the current task:

[0287] ;

[0288] in, This indicates the current base model in the task. Average instruction loss on the public anchor set, Indicates task The set of public anchor points The low-rank adaptation parameter is... At that time, the instruction output is... The instructions fine-tune the loss.

[0289] ;

[0290] in, This indicates the difficulty component of the current task. This indicates that the task difficulty loss is referenced to the lower bound. The upper bound is used to indicate the task difficulty loss.

[0291] It should be noted that if there is no publicly available set of anchor points, then a pre-configured [set of anchor points] can be used. Within this process, DP-protected loss detection is performed on a small number of privacy units of the current task, and the same normalization method is used to obtain... .

[0292] Then, the probe gradient corresponding to the current task is calculated on the public anchor point or DP probe data, and the historical task gradient is calculated on the candidate replay sample set. The task-pair conflict intensity can then be expressed by the following formula:

[0293] ;

[0294] in, Indicates the task's impact on the intensity of the conflict. Indicates the current mission detection gradient. Representing historical tasks The gradient computed on the candidate replay sample set, This represents a small constant to prevent the denominator from being zero.

[0295] Because the positive part of the cosine similarity is taken in the opposite direction, The larger the value, the more conflict there is between the optimization directions of the current task and the historical tasks.

[0296] Next, the importance weights of historical tasks are obtained by normalizing the importance of application-side tasks:

[0297] ;

[0298] in, Indicates the weight of the importance of historical tasks. This represents the application-defined and normalized task. The importance of.

[0299] Then, from this there is The overall conflict intensity components of the current task are:

[0300] ;

[0301] in, This represents the overall conflict intensity component.

[0302] Next, the performance degradation components of historical tasks are calculated:

[0303] ;

[0304] in, Representing historical tasks The performance degradation component, Indicates completion of task Afterwards, the model in historical tasks Performance based on publicly available benchmark data This represents the upper bound of the normalized reference for performance degradation.

[0305] Then, calculate the increase component of the public anchor point loss:

[0306] ;

[0307] in, This indicates an increase in the amount of loss at the publicly disclosed anchor point. The low-rank adaptation parameter is... The basic model in the task The average loss on the public anchor set, The normalized upper bound represents the increase in the loss of the public anchor point.

[0308] Next, the difference components between the differential privacy task teacher model and the current model output are calculated:

[0309] ;

[0310] in, This represents the difference components between the teacher model's output and the current model's output for the differential privacy task. Indicates task The set of publicly available prompt words for description. Represents probability distribution and KL divergence between them Indicates task Corresponding differential privacy task teacher model In the input Conditional output distribution on The low-rank adaptation parameter is... The basic model in input Conditional output distribution on This represents the normalized upper bound of the KL divergence between the teacher model and the current model for the differential privacy task.

[0311] Then, the expected forgetting pressure component is calculated. To distinguish between observed forgetting and current gradient conflict, the expected forgetting pressure component is a weighted average of three non-conflicting components: performance, loss, and teacher difference.

[0312] ;

[0313] in, This indicates the expected forgetting pressure component. This represents the normalized weight corresponding to the performance degradation component. This represents the normalized weight corresponding to the increase in the open anchor loss component. This represents the normalized weights corresponding to the difference components in the output of the teacher model and the current model for the differential privacy task.

[0314] Next, calculate the data size components:

[0315] ;

[0316] in, Indicates the data size components. Indicates task The number of private records, This represents a pre-defined normalized reference value for the data size. Logarithmic transformation is used to reduce the dominance of extremely large datasets on the budget requirement score.

[0317] Then, the current task requirement score is calculated based on the above components:

[0318]

[0319] in, This indicates the score required for the current task's budget. This indicates the difficulty component of the current task. This indicates the weight corresponding to the current task difficulty component. This represents the overall conflict intensity component. This indicates the weights corresponding to the overall conflict intensity components. This indicates the expected forgetting pressure component. This indicates the weight corresponding to the expected forgetting pressure component. Indicates the data size components. This indicates the weights corresponding to the data size components. This represents the task importance component (preferably, the task importance component is given and normalized by the application side according to task priority). If not specified on the application side, the default setting will be used. . ), This represents the weight corresponding to the task importance component.

[0320] Next, the privacy budget will be allocated:

[0321] First, the budget reserved for future missions is:

[0322] ;

[0323] The available budget for the current task is:

[0324] ;

[0325] The total budget for the candidates is:

[0326] ;

[0327] The total budget for the task is:

[0328] ;

[0329] The private training budget is:

[0330] ;

[0331] Among them, among them, Indicates task The RDP budget retained for future missions, This indicates the number of remaining tasks, including the current task. This indicates the minimum total RDP budget reserved for each task. Indicates task Total available RDP budget Indicates task Global remaining RDP budget before start Indicates the total budget of the candidates. This indicates the score required for the current task's budget. This indicates that the task requires a large budget. This indicates the maximum total RDP budget allowed for a single task. Indicates private training budget, Indicates task Optional detection budget.

[0332] It should be noted that, due to the aforementioned requirements And when feasible ,so Verify again after budget allocation is complete:

[0333] ;

[0334] If the above conditions are not met due to configuration or numerical errors, the current task's private training will not be performed, the base model corresponding to the previous task will be used as the final differential privacy model, and the process will be terminated.

[0335] Then, select the noise multiplier based on the allocation of the privacy budget:

[0336] ;

[0337] in, Indicates task Private training iteration steps, Indicates the sampling rate The noise multiplier is The subsampling Gaussian mechanism in order The cost of a single-step RDP step.

[0338] In other words, in a fixed and Under these conditions, noise multipliers that satisfy the constraints are determined through a one-dimensional search. It should be noted that the budget allocated to the detection step is not involved in DP-SGD noise calibration.

[0339] S106, calculate the replay value of each historical task in the current task, determine the allocation integer of the candidate replay samples based on the replay value, and determine the final replay sample set based on the allocation integer and the candidate replay sample set.

[0340] In some embodiments, the replay value is calculated according to the following formula:

[0341] ;

[0342] ;

[0343] in, This indicates the quality value of the candidate replay sample. This represents the teacher confidence component. This indicates the weights corresponding to the teacher confidence components. Indicates the efficiency components of the format and tags. This indicates the weights corresponding to the format and tag effectiveness components. Represents the diversity components of candidate samples. This represents the weights corresponding to the diversity components of the candidate samples. Memory risk agency component, This indicates the weight corresponding to the memory risk proxy component. Indicates the value of replay. Indicates the task's impact on the intensity of the conflict. This indicates the risk of historical tasks being forgotten. This indicates the weight of the historical mission.

[0344] As an example, first, after calculating the performance degradation component, the increase component of the public anchor loss, and the difference component between the differential privacy task teacher model and the current model output, the gradient conflict component is determined to be equal to the conflict intensity of the task pair:

[0345] ;

[0346] Furthermore, the risk of historical tasks being forgotten can be calculated using the following formula:

[0347] ;

[0348] in, This indicates the risk of historical tasks being forgotten. Representing historical tasks The performance degradation component, This indicates an increase in the amount of loss at the publicly disclosed anchor point. This represents the difference components between the teacher model's output and the current model's output for the differential privacy task. Indicates gradient conflict components; This represents the normalized weight corresponding to the performance degradation component. This represents the normalized weight corresponding to the increase in the open anchor loss component. This represents the normalized weights corresponding to the differences between the teacher model's output and the current model's output in the differential privacy task. This represents the normalized weights corresponding to the gradient conflict components.

[0349] It should be noted that, preferably, in the above calculation, if a certain component cannot be calculated due to the lack of corresponding public statistics, the weight of that component is set to 0, and the remaining effective weights are renormalized so that the sum of the effective weights is still 1.

[0350] Next, the teacher confidence component is calculated for the first candidate in the replay sample set. Candidate replay samples Let the number of output tokens be... The teacher confidence component is then:

[0351] ;

[0352] in, This represents the teacher confidence component. This represents the number of samples in the candidate replay sample set. This indicates the number of tokens output. Representing historical tasks Corresponding Differential Privacy Teacher Model Given a candidate replay sample input and the preceding output token Under the condition of generating the first tokens The conditional probability.

[0353] The format and tag efficiency components are:

[0354]

[0355] in, Indicates the efficiency components of the format and tags. Indicates the first One candidate replay sample.

[0356] The candidate sample diversity components are:

[0357] ;

[0358] in, Indicating candidate sample diversity, when When, set .

[0359] To avoid access to historical, original private data, memory risk is mitigated by employing a DP-secure near-replication proxy. Let... Given a security reference set consisting of public anchors, public templates, and previously generated replay samples:

[0360]

[0361] in, This indicates the memory risk proxy component. This represents the similarity threshold used to determine the risk of near-replication.

[0362] Then, the quality value of the candidate replay sample is calculated according to the following formula:

[0363] ;

[0364] in, This indicates the quality value of the candidate replay sample. This represents the teacher confidence component. This indicates the weights corresponding to the teacher confidence components. Indicates the efficiency components of the format and tags. This indicates the weights corresponding to the format and tag effectiveness components. Represents the diversity components of candidate samples. This represents the weights corresponding to the diversity components of the candidate samples. This indicates the memory risk proxy component. This indicates the weight corresponding to the memory risk agent component.

[0365] Next, the replay value of the historical task in the current task is calculated based on the quality values ​​of the candidate replay samples:

[0366] ;

[0367] in, Indicates the value of replay. Indicates the task's impact on the intensity of the conflict. This indicates the risk of historical tasks being forgotten. This indicates the weight of the historical mission.

[0368] Then, the allocation integers of the candidate replay samples are determined based on the replay value, and the final replay sample set is determined based on the allocation integers and the candidate replay sample set.

[0369] Specifically, firstly, the initial allocation of real numbers is:

[0370] ;

[0371] in, This represents the initial real number allocation obtained by softmax scheduling. Indicates task Total playback budget Indicates the temperature for softmax scheduling. Representing historical tasks In the current task The replay value in.

[0372] To obtain an integer allocation, let:

[0373] ;

[0374] in, Represents the initial allocation of real numbers The middle integer after rounding down.

[0375] Calculate the remaining quantity to be allocated:

[0376]

[0377] in, This indicates the remaining quantity to be allocated.

[0378] Then, according to From largest to smallest, for the first Each historical task is incremented by 1 to obtain the final integer. Ultimately satisfied:

[0379] ;

[0380] It should be noted that, Its purpose is to directly control the number of samples that ultimately enter the final replay sample set, rather than just for statistical display.

[0381] Next, for each historical task :

[0382] if Then set Replay samples are not retained for this historical task;

[0383] if Then, from the generated candidate replay sample set, a screening process is performed based on format validity, label consistency, teacher confidence, output length, duplication, and near-copy risk, and samples are retained in descending order of overall quality. One qualified sample.

[0384] Specifically, it can be expressed by the following formula:

[0385] ;

[0386] in, This represents the final set of replay samples. This indicates a filtering operation. Indicates selecting at most based on quality. Sample.

[0387] Preferably, if the number of qualified samples after screening the candidate set is less than Then continue calling Supplementary generation, maximum execution If a deficit still exists after reaching the maximum number of rounds, the deficit will be recalculated based on the replay value. The replay buffers are redistributed from high to low quality to other historical tasks that can still generate qualified samples. The final merged replay buffer is:

[0388] ;

[0389] If all historical tasks are insufficient to fill the gap, the actual number of replays can be less than [a certain number]. However, it will not use historical original private data to supplement it.

[0390] S107. The private instruction dataset corresponding to the current task is grouped according to the set of privacy units corresponding to the current task to obtain the private instruction data group corresponding to each privacy unit. The gradient of the privacy unit corresponding to each private instruction data group is calculated based on the noise multiplier, and the privacy unit gradients are merged to obtain the merged privacy unit gradient corresponding to the current task.

[0391] S108: Calculate the corresponding auxiliary gradient based on the final replay sample set, and merge the merged privacy unit gradient and the auxiliary gradient to update the low-rank adaptation parameters based on the merging result.

[0392] S109, Generate a differential privacy task teacher model corresponding to the current task based on the updated current base model, and update the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task.

[0393] S110, determine whether all tasks have been completed.

[0394] S111, if so, then use the current base model as the final differential privacy model.

[0395] In some embodiments, after determining the final replay sample set based on the allocated integer and the candidate replay sample set, the method further includes: constructing the training objective for the current task;

[0396] The training objective for the current task is expressed by the following formula:

[0397] ;

[0398] in, Indicates the low-rank adaptation parameters. Indicates task The overall training objective This indicates the current loss of private data oversight. Indicates task Private instruction dataset, This represents the loss of generative replay supervision. This represents the weights corresponding to the generative replay supervision loss. This represents the distillation loss of the teacher model for the differential privacy task. The weights represent the distillation loss corresponding to the teacher model in the differential privacy task. This indicates a parameter-stable regularization term. This indicates the weight corresponding to the parameter-stable regularization term. Indicates task The low-rank adaptation parameters after completion;

[0399] ;

[0400] ;

[0401] ;

[0402] in, Indicates task No. Small batches of private records sampled step by step. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Indicates task No. The step involves sampling small batches of candidate replay samples. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Represents probability distribution and KL divergence between them Indicates the distillation temperature. Computational differential privacy task teacher model In the input Conditional output distribution on Indicates the distillation temperature. The current base model for computation is based on the input. The conditional output distribution.

[0403] Preferably, the stable regularization term adopts:

[0404]

[0405] when , Or, if the replay buffer is empty, set:

[0406]

[0407] If teacher distillation is not used in the specific implementation, then set .

[0408] As an example, after constructing the task training objective, for the task... Each training step The system uses privacy units as the granularity for sampling and cropping.

[0409] Privacy Unit Its record index set in the current mini-batch is:

[0410]

[0411] in, Indicates the first Step belongs to the privacy unit The collection of record indexes, Indicates task The Small batches of private records are sampled step by step.

[0412] Next, the aggregate gradient of this privacy unit is:

[0413] ;

[0414] in, Indicates aggregate gradient, The low-rank adaptation parameter is... At that time, the instruction output is... The instructions fine-tune the loss.

[0415] Then, execute Norm clipping:

[0416] ;

[0417] in, This represents the gradient of the privacy unit after clipping. express Norm clipping threshold.

[0418] Next, Gaussian noise is added to the noise multiplier to obtain the corresponding private gradient:

[0419]

[0420] in, This represents the private gradient after adding Gaussian noise. Represents the identity matrix.

[0421] Then, calculate the auxiliary gradient:

[0422] ;

[0423] in, Denotes the auxiliary gradient. Indicates the replay loss weight. This represents the supervised loss on generative replay data. This represents the rectification loss between the privacy task teacher model and the current base model. Indicates the rectification loss weight. This represents the weight of the stable regularization term.

[0424] The final update gradient is:

[0425] ;

[0426] Next, the low-rank adaptation parameters are updated based on the final merged gradient:

[0427] ;

[0428] It should be noted that when each privacy unit contains only one record, The aforementioned privacy unit-level update naturally degenerates into per-sample DP-SGD.

[0429] Then, complete. After one training step, set:

[0430] ;

[0431] Next, the task Corresponding differential privacy task teacher model Join the internal teacher group:

[0432]

[0433] Then, update the global remaining budget:

[0434]

[0435] At the same time, the precise cumulative record is updated for each privacy unit, that is, the cumulative privacy cost of each privacy unit is updated:

[0436]

[0437] Next, check if all tasks have been completed. If all tasks have been completed, output:

[0438] ;

[0439] If the task is terminated prematurely before it begins due to budget infeasibility, the output will be:

[0440] ;

[0441] In a consistent manner, the final output is represented as:

[0442] ;

[0443] in, This is the last task number to meet the budget constraint and complete training. Current tasks that have not completed training will not be generated. Or a new DP teacher.

[0444] Secondly, embodiments of the present invention propose a computer-readable storage medium storing a differential privacy model training program based on generative replay, which, when executed by a processor, implements the differential privacy model training method based on generative replay as described above.

[0445] Thirdly, embodiments of the present invention propose a differential privacy model training system based on generative replay, such as... Figure 2 As shown, the differential privacy model training system based on generative replay includes: an acquisition module 10, a recording module 20, a candidate sample generation module 30, a scheduling and statistics module 40, an allocation module 50, a sample determination module 60, a gradient calculation module 70, a first update module 80, a second update module 90, and a judgment module 100.

[0446] The acquisition module 10 is used to acquire the private instruction dataset corresponding to the current task. The private instruction dataset includes multiple training instructions and the privacy unit identifier corresponding to each training instruction.

[0447] The recording module 20 is used to generate a set of privacy units corresponding to the current task based on the privacy unit identifier.

[0448] The candidate sample generation module 30 is used to generate candidate replay samples corresponding to each historical task based on the differential privacy task teacher model corresponding to each historical task, and to generate a candidate replay sample set based on the candidate replay samples.

[0449] The scheduling statistics module 40 is used to calculate the scheduling statistics components of the current task and the historical tasks based on the candidate replay sample set, and to calculate the budget requirement score of the current task based on the scheduling statistics components.

[0450] The allocation module 50 is used to allocate a privacy budget to the current task based on the current task budget requirement score, and to determine the noise multiplier based on the allocated private training budget of the current task.

[0451] The sample determination module 60 is used to calculate the replay value of each historical task in the current task, determine the allocation integer of the candidate replay samples based on the replay value, and determine the final replay sample set based on the allocation integer and the candidate replay sample set.

[0452] The gradient calculation module 70 is used to group the private instruction dataset corresponding to the current task according to the set of privacy units corresponding to the current task, so as to obtain the private instruction data group corresponding to each privacy unit, calculate the privacy unit gradient corresponding to each private instruction data group based on the noise multiplier, and merge the privacy unit gradients to obtain the merged privacy unit gradient corresponding to the current task.

[0453] The first update module 80 is used to calculate the corresponding auxiliary gradient based on the final replay sample set, and merge the merged privacy unit gradient and the auxiliary gradient to update the low-rank adaptation parameters based on the merging result.

[0454] The second update module 90 is used to generate a differential privacy task teacher model corresponding to the current task based on the updated current base model, and update the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task.

[0455] The judgment module 100 is used to determine whether all tasks have been completed, and if the judgment result is yes, the current basic model is used as the final differential privacy model.

[0456] In some embodiments, the scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

[0457] It should be noted that the above description of the differential privacy model training method based on generative replay also applies to this differential privacy model training system based on generative replay, and will not be repeated here.

[0458] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0459] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0460] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0461] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0462] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0463] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0464] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0465] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for training a differential privacy model based on generative replay, characterized in that, Includes the following steps: Obtain the private instruction dataset corresponding to the current task. The private instruction dataset includes multiple training instructions and the privacy unit identifier corresponding to each training instruction. Generate a set of privacy units corresponding to the current task based on the privacy unit identifier; Based on the differential privacy task teacher model and public prompt words corresponding to each historical task, candidate replay samples corresponding to the historical task are generated, and a candidate replay sample set is generated based on the candidate replay samples. The scheduling statistics components of the current task and historical tasks are calculated based on the candidate replay sample set, and the budget requirement score of the current task is calculated based on the scheduling statistics components. Allocate a privacy budget to the current task based on the current task budget requirement score, and determine the noise multiplier based on the allocated private training budget for the current task; Calculate the replay value of each historical task in the current task, determine the allocation integer of candidate replay samples based on the replay value, and determine the final replay sample set based on the allocation integer and the candidate replay sample set. The private instruction dataset corresponding to the current task is grouped according to the set of privacy units corresponding to the current task to obtain a private instruction data group corresponding to each privacy unit. The privacy unit gradient corresponding to each private instruction data group is calculated based on the noise multiplier, and the privacy unit gradients are merged to obtain the merged privacy unit gradient corresponding to the current task. The corresponding auxiliary gradient is calculated based on the final replay sample set, and the merged privacy unit gradient and the auxiliary gradient are merged to update the low-rank adaptation parameters based on the merging result. Generate a differential privacy task teacher model corresponding to the current task based on the updated current base model, and update the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task; Determine if all tasks have been completed. If so, the current base model will be used as the final differential privacy model.

2. The differential privacy model training method based on generative replay as described in claim 1, characterized in that, After generating the set of privacy units corresponding to the current task based on the privacy unit identifier, the method further includes: Perform a budget feasibility check to determine whether the execution of the current task will result in budget overruns; If so, the base model corresponding to the previous task will be used as the final differential privacy model; The budget feasibility check is performed according to the following formula: ； ； ； in, Indicates task Global remaining RDP budget before start Represents order The global RDP budget limit is set below. Indicates task RDP costs in China This indicates the number of remaining tasks, including the current task. Indicates the total number of tasks. This represents the minimum RDP budget reserved for each task.

3. The differential privacy model training method based on generative replay as described in claim 1, characterized in that, The scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

4. The differential privacy model training method based on generative replay as described in claim 3, characterized in that, The current task difficulty component is calculated using the following formula: ； ； in, This indicates the current base model in the task. Average instruction loss on the public anchor set, Indicates task The set of public anchor points The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Indicates the difficulty component of the current task. This indicates that the task difficulty loss is referenced to the lower bound. The upper bound indicates the difficulty loss of the task. The overall conflict intensity component is calculated using the following formula: ； ； ； in, Indicates the task's impact on the intensity of the conflict. Indicates the current mission's detection gradient. Representing historical tasks The gradient computed on the candidate replay sample set, This represents a small constant to prevent the denominator from being zero. Indicates the weight of the importance of historical tasks. This represents the task given and normalized by the application side. The importance of Indicates the overall conflict intensity component; The expected forgetting pressure component is calculated using the following formula: ； ； ； ； in, Representing historical tasks The performance degradation component, Indicates completion of task Afterwards, the model in historical tasks Performance based on publicly available benchmark data This indicates the upper bound of the normalized reference for performance degradation. This indicates an increase in the amount of loss at the publicly disclosed anchor point. The low-rank adaptation parameter is... The basic model in the task The average loss on the public anchor set, This represents the normalized upper bound of the increase in publicly disclosed anchor point loss. This represents the difference components between the teacher model's output and the current model's output for the differential privacy task. Indicates task The set of publicly available prompt words for description. Represents probability distribution and KL divergence between them Indicates task Corresponding differential privacy task teacher model In the input Conditional output distribution on The low-rank adaptation parameter is... The basic model in input Conditional output distribution on This represents the normalized upper bound of the KL divergence between the teacher model and the current model for the differential privacy task. This indicates the expected forgetting pressure component. This represents the normalized weight corresponding to the performance degradation component. This represents the normalized weight corresponding to the increase in the open anchor loss component. This represents the normalized weights corresponding to the difference components between the teacher model and the current model outputs in the differential privacy task. The data size component is calculated using the following formula: ； in, Indicates the data size components. Indicates task The number of private records, This represents the preset data size normalization reference value.

5. The differential privacy model training method based on generative replay as described in claim 4, characterized in that, The current task budget requirement score is calculated using the following formula: in, This indicates the score required for the current task's budget. Indicates the difficulty component of the current task. This indicates the weight corresponding to the current task difficulty component. This represents the overall conflict intensity component. This indicates the weights corresponding to the overall conflict intensity components. This indicates the expected forgetting pressure component. This indicates the weight corresponding to the expected forgetting pressure component. Indicates the data size components. This indicates the weights corresponding to the data size components. Indicates the importance of the task. The weights corresponding to the task importance components; The privacy budget should be allocated according to the following formula: ； ； ； ； ； in, Indicates task The RDP budget retained for future missions, This indicates the number of remaining tasks, including the current task. This indicates the minimum total RDP budget reserved for each task. Indicates task Total available RDP budget Indicates task Global remaining RDP budget before start Indicates the total budget of the candidates. This indicates the score required for the current task's budget. This indicates that the task requires a large budget. This indicates the maximum total RDP budget allowed for a single task. Indicates private training budget, Indicates task Optional detection budget; The noise multiplier is determined according to the following formula: ； in, Indicates task Private training iteration steps, Indicates the sampling rate The noise multiplier is The subsampling Gaussian mechanism in order The cost of a single-step RDP step.

6. The differential privacy model training method based on generative replay as described in claim 1, characterized in that, The replay value is calculated according to the following formula: ； ； in, This indicates the quality value of the candidate replay sample. This represents the teacher confidence component. This indicates the weights corresponding to the teacher confidence components. Indicates the efficiency components of the format and tags. This indicates the weights corresponding to the format and tag effectiveness components. Represents the diversity components of candidate samples. This represents the weights corresponding to the diversity components of the candidate samples. This indicates the memory risk proxy component. This indicates the weight corresponding to the memory risk proxy component. Indicates the value of replay. Indicates the task's impact on the intensity of the conflict. This indicates the risk of historical tasks being forgotten. This indicates the weight of the historical mission.

7. The differential privacy model training method based on generative replay as described in claim 1, characterized in that, After determining the final replay sample set based on the allocated integer and the candidate replay sample set, the method further includes: constructing the training objective for the current task; The training objective of the current task is expressed by the following formula: ； in, Indicates the low-rank adaptation parameters. Indicates task The overall training objective This indicates the current loss of private data supervision. Indicates task Private instruction dataset, This represents the loss of generative replay supervision. This represents the weights corresponding to the generative replay supervision loss. This represents the distillation loss of the teacher model for the differential privacy task. This represents the weights corresponding to the distillation loss of the teacher model in the differential privacy task. This indicates a parameter-stable regularization term. This represents the weight corresponding to the parameter-stable regularization term. Indicates task The low-rank adaptation parameters after completion; ； ； ； in, Indicates task No. Small batches of private records sampled step by step. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Indicates task No. The step involves sampling small batches of candidate replay samples. The low-rank adaptation parameter is... At that time, the instruction output is... The instruction fine-tuning loss, Represents probability distribution and KL divergence between them Indicates the distillation temperature. Computational differential privacy task teacher model In the input Conditional output distribution on Indicates the distillation temperature. The current base model for computation is based on the input. The conditional output distribution.

8. A computer-readable storage medium, characterized in that, It stores a differential privacy model training program based on generative replay, which, when executed by a processor, implements the differential privacy model training method based on generative replay as described in any one of claims 1-7.

9. A differential privacy model training system based on generative replay, characterized in that, include: The acquisition module is used to acquire the private instruction dataset corresponding to the current task. The private instruction dataset includes multiple training instructions and the privacy unit identifier corresponding to each training instruction. The recording module is used to generate a set of privacy units corresponding to the current task based on the privacy unit identifier. The candidate sample generation module is used to generate candidate replay samples corresponding to each historical task based on the differential privacy task teacher model corresponding to each historical task, and to generate a candidate replay sample set based on the candidate replay samples. The scheduling statistics module is used to calculate the scheduling statistics components of the current task and historical tasks based on the candidate replay sample set, and to calculate the budget requirement score of the current task based on the scheduling statistics components. The allocation module is used to allocate a privacy budget to the current task according to the current task budget requirement score, and to determine a noise multiplier based on the allocated private training budget of the current task. The sample determination module is used to calculate the replay value of each historical task in the current task, determine the allocation integer of candidate replay samples based on the replay value, and determine the final replay sample set based on the allocation integer and the candidate replay sample set. The gradient calculation module is used to group the private instruction dataset corresponding to the current task according to the set of privacy units corresponding to the current task to obtain a private instruction data group corresponding to each privacy unit, calculate the privacy unit gradient corresponding to each private instruction data group based on the noise multiplier, and merge the privacy unit gradients to obtain the merged privacy unit gradient corresponding to the current task. The first update module is used to calculate the corresponding auxiliary gradient based on the final replay sample set, and merge the merged privacy unit gradient and the auxiliary gradient to update the low-rank adaptation parameters based on the merging result. The second update module is used to generate a differential privacy task teacher model corresponding to the current task based on the updated current base model, and update the cumulative privacy cost of each privacy unit according to the set of privacy units corresponding to the current task. The judgment module is used to determine whether all tasks have been completed, and if the judgment result is yes, the current basic model is used as the final differential privacy model.

10. The differential privacy model training system based on generative replay as described in claim 9, characterized in that, The scheduling statistics components include the current task difficulty component, the overall conflict intensity component, the expected forgetting pressure component, the data size component, and the task importance component.

Images