Large model training method, medium and system based on differential privacy mechanism

Abstract

The invention discloses a large model training method, medium and system based on a differential privacy mechanism, and the method comprises the steps: S101, obtaining historical data, carrying out the preprocessing, and carrying out the grouping and marking of the preprocessed historical data, so as to form a training data set; s102, initializing parameters of the large language model; s103, performing training based on the training data set, and calculating a gradient; s104, performing gradient noise addition to obtain a noise addition gradient, and calculating a corresponding comprehensive unfairness index value based on the noise addition gradient; s105, judging whether the comprehensive unfairness index value is within a preset value range or not; s106, if the comprehensive unfair index value is within a preset value range, judging whether the current large language model meets a training requirement or not; if yes, taking the current large language model as a final model; and if not, returning to the step S103. According to the invention, the privacy of the user can be effectively protected, and meanwhile, the unfair phenomenon caused by the use of a differential privacy mechanism is avoided.

Description

Technical Field

[0001] The present application relates to the field of language processing technology, and in particular to a large model training method, medium and system based on a differential privacy mechanism. Background Art

[0002] As the demand for data privacy protection increases, differential privacy (DP), as an important privacy protection technology, has gradually been widely used in data analysis and machine learning. Differential privacy protects data privacy by adding random noise to the calculation results to ensure that it is difficult for attackers to infer whether the data of a single individual exists in the data set through the output.

[0003] In the related art, when using differential privacy mechanisms to protect user privacy, there is a lack of consideration for fairness. That is, the differential privacy mechanism protects privacy by adding random noise to the training parameters of large models, but the randomness of this noise will have inconsistent effects on different data groups in the original training set. Specifically, groups with small cardinality or sparse data distribution are more susceptible to noise, which may cause these groups to be systematically marginalized in decision-making and statistical analysis, resulting in unfairness. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one purpose of the present invention is to propose a large model training method based on a differential privacy mechanism, which can effectively protect user privacy and avoid the unfairness caused by the use of the differential privacy mechanism.

[0005] In a first aspect, an embodiment of the present invention proposes a large model training method based on a differential privacy mechanism, comprising the following steps: S101, obtaining historical data, preprocessing the historical data, and grouping and labeling the preprocessed historical data to form a training data set; S102, initializing large language model parameters; S103, training the large language model based on the training data set, and calculating the gradient of the loss function with respect to the model parameters; S104, performing gradient noise addition to obtain a noisy gradient, and calculating a corresponding comprehensive unfairness index value based on the noisy gradient; S105, judging whether the comprehensive unfairness index value is within a preset value range; S106, if the comprehensive unfairness index value is within the preset value range, judging whether the current large language model meets the training requirements; if yes, taking the current large language model as the final model; if not, returning to step S103.

[0006] According to the large model training method based on differential privacy mechanism of the embodiment of the present invention, first, S101, historical data is obtained, and the historical data is preprocessed, and the preprocessed historical data is grouped and labeled to form a training data set; then, S102, the large language model parameters are initialized; then, S103, the large language model is trained based on the training data set, and the gradient of the loss function to the model parameters is calculated; then, S104, gradient noise is added to obtain the noisy gradient, and the corresponding comprehensive unfairness index value is calculated based on the noisy gradient; then, S105, it is determined whether the comprehensive unfairness index value is within the preset value range; then, S106, if the comprehensive unfairness index value is within the preset value range, it is determined whether the current large language model meets the training requirements; if yes, the current large language model is used as the final model; if not, it returns to step S103. Thereby, effective protection of user privacy is achieved, and at the same time, the generation of unfairness caused by the use of differential privacy mechanism is avoided.

[0007] In some embodiments, the historical data is preprocessed, including: cleaning the historical data and labeling the cleaned historical data; segmenting and mapping the labeled historical data into word vectors, and performing sequence length processing and normalization on the mapped word vectors.

[0008] In some embodiments, a large language model is trained based on a training data set, and the gradient of the loss function with respect to the model parameters is calculated, including: for each batch of data in the training data set, forward propagation is performed to obtain a corresponding prediction result; based on a cross entropy loss function, an error is calculated between the prediction result and the true label, and the gradient of the loss function with respect to the model parameters is calculated based on the error using the chain rule.

[0009] In some embodiments, the cross entropy loss function is expressed by the following formula: ; in, represents the cross entropy loss function, Indicates the batch size, represents the number of categories, Indicates The sample in The true label of the class, represents the predicted probability; The gradient is calculated by the following formula: ; in, represents the gradient, represents the set of model parameters, Indicates The loss of samples.

[0010] In some embodiments, gradient noise is added to obtain a noisy gradient, and a corresponding unfairness index value is calculated based on the noisy gradient, including: clipping the gradient and generating initial noise, and calculating the noisy gradient based on the initial noise and the clipped gradient; calculating an initial unfairness index value based on the noisy gradient, and calculating a corresponding average unfairness index value based on the initial unfairness index value obtained from multiple trainings; introducing a time decay factor to decay the average unfairness index value, and performing weighted average calculation on the attenuated average unfairness index value to obtain a comprehensive unfairness index value.

[0011] In some embodiments, the initial unfairness index value is calculated by the following formula: ; in, represents the initial unfairness index value, represents the penalty coefficient, Representing data groups The noise, Representing a group The noise, represents the sign function, which is used to indicate the direction of the noise. Representing data groups The corresponding noisy gradient, Representing data groups The corresponding noisy gradient; The average unfairness index is calculated by the following formula: ; in, represents the average unfairness index, represents the total number of iterative training, Indicates Sub-fairness calculation; The average unfairness index value is attenuated by the following formula: ; in, represents the average unfairness index value after attenuation, Indicates the current time and The time interval between training sessions; The weighted average of the attenuated average unfairness index value is calculated using the following formula: ; ; in, represents the unfairness index value after weighted average, Representing a group and groups The proportion of the unfairness between Represents the comprehensive unfairness index value.

[0012] In some embodiments, if the comprehensive unfairness index value is not within a preset value range, the differential privacy model is modified to adjust the privacy budget of each group; the noise intensity is updated according to the adjusted privacy budget, and the process returns to step S104.

[0013] In some embodiments, the privacy budget is adjusted by the following formula: ; ; in, represents the adjusted privacy budget, represents the initial privacy budget, represents the comprehensive unfairness index value, represents the adjustment function, represents the adjustment factor; ; in, represents the updated noise intensity, Indicates the threshold for gradient norm clipping.

[0014] In the second aspect, an embodiment of the present invention proposes a computer-readable storage medium, on which a large model training program based on a differential privacy mechanism is stored. When the large model training program based on the differential privacy mechanism is executed by a processor, the large model training method based on the differential privacy mechanism as described above is implemented.

[0015] In a third aspect, an embodiment of the present invention proposes a large model training system based on a differential privacy mechanism, including: an acquisition module, the acquisition module is used to acquire historical data, preprocess the historical data, and group and label the preprocessed historical data to form a training data set; an initialization model, the initialization module is used to initialize the large language model parameters; a training module, the training module is used to train the large language model based on the training data set, and calculate the gradient of the loss function to the model parameters; a fairness calculation module, the fairness calculation module is used to perform gradient noise to obtain a noisy gradient, and calculate a corresponding comprehensive unfairness index value based on the noisy gradient; a judgment module, the judgment module is used to judge whether the comprehensive unfairness index value is within a preset value range; the judgment module is also used to judge whether the current large language model meets the training requirements when the comprehensive unfairness index value is within the preset value range, and when the judgment result is yes, use the current large language model as the final model, and when the judgment result is no, return to the step of training the large language model based on the training data set.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a large model training method based on a differential privacy mechanism according to an embodiment of the present invention; Figure 2 is a flowchart of a large model training method based on a differential privacy mechanism according to another embodiment of the present invention; Figure 3 is a block diagram of a large model training system based on a differential privacy mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0019] The following describes a large model training method based on a differential privacy mechanism according to an embodiment of the present invention with reference to the accompanying drawings.

[0020] See also Figure 1 , Figure 1 Schematic diagram of a large model training method based on a differential privacy mechanism according to an embodiment of the present invention. Figure 1As shown, the large model training method based on the differential privacy mechanism includes the following steps: S101, acquiring historical data, preprocessing the historical data, and grouping and labeling the preprocessed historical data to form a training data set.

[0021] In some embodiments, the historical data is preprocessed, including: cleaning the historical data and labeling the cleaned historical data; segmenting and mapping the labeled historical data into word vectors, and performing sequence length processing and normalization on the mapped word vectors.

[0022] As an example, first, a large amount of text data (i.e., historical data) is obtained from multiple trusted data sources to ensure the diversity and representativeness of the data, where the data sources may include but are not limited to: public corpora (e.g., Wikipedia, news articles, books, etc.); domain data sets (e.g., text data in professional fields such as medicine, finance, and law); user-generated content (e.g., collecting text data from platforms such as social media and forums with user authorization, etc.).

[0023] Next, the acquired historical data is cleaned; specifically, the data cleaning process may include: 1. Remove noise and special symbols, that is, use regular expression matching to delete meaningless content such as garbled characters, emoticons, HTML tags, etc. 2. Handle duplicate data, that is, detect and remove duplicate sentences or paragraphs to prevent the model from overfitting to repeated content; 3. Correct spelling errors, that is, use spelling check tools to correct spelling errors in text data; 4. Filter sensitive information, identify and delete sensitive information that may contain personal privacy, such as name, address, ID number, etc.

[0024] Then, the cleaned historical data is annotated. Specifically, data annotation can include adding labels and sensitive attribute annotations. First, for tasks that require supervised learning, add corresponding labels, such as sentiment polarity, topic category, etc. Then, according to the pre-defined sensitive attribute set A (such as gender, race, age, region, etc.), annotate each data sample with the group it belongs to for subsequent fairness evaluation.

[0025] Next, the annotated historical data is preprocessed. First, the word segmentation tool is used to split the text data into word or subword sequences. Then, the pre-trained word embedding or the embedding layer is used to map the words into word vectors. Then, the sequence length is unified to L. For sequences with a length less than L, special symbols are used for padding, and for sequences with a length greater than L, they are truncated. Then, the word vectors are standardized to satisfy the distribution with a mean of 0 and a variance of 1 to improve the stability of model training.

[0026] In some embodiments, after the data preprocessing is completed, the data is further grouped and labeled; specifically, the training data set can be divided into multiple groups according to the sensitive attributes labeled in the preprocessing. ;in: ; in, Indicates The value of a sensitive attribute, such as male or female in gender.

[0027] Next, count the amount of data for each group The total amount of data is , then the group Weight for: ; It should be noted that in the subsequent fairness evaluation and optimization, weights can be used A weighted average is performed to reflect the proportion of different groups in the data set.

[0028] S102, initializing large language model parameters.

[0029] As an example, based on the Transformer architecture, we build a large language model and determine the following hyperparameters: the number of layers L, which is the number of stacked layers of the Transformer encoder; the number of hidden units: the dimension of the hidden state vector at each position; the dimension of the feedforward network: the dimension of the feedforward fully connected network; the number of attention heads: the number of heads of the multi-head attention mechanism, and the dimension of each head is Next, the model parameters are initialized using the Xavier initialization method. For the weight matrix , whose elements are randomly initialized according to the following distribution: ; in, and Represent the input dimension and output dimension respectively.

[0030] S103, training the large language model based on the training data set, and calculating the gradient of the loss function with respect to the model parameters.

[0031] In some embodiments, a large language model is trained based on a training data set, and the gradient of the loss function with respect to the model parameters is calculated, including: for each batch of data in the training data set, forward propagation is performed to obtain a corresponding prediction result; the error between the prediction result and the true label is calculated based on the cross entropy loss function, and the gradient of the loss function with respect to the model parameters is calculated based on the error using the chain rule.

[0032] In some embodiments, the cross entropy loss function is expressed by the following formula:

[0033] in, represents the cross entropy loss function, Indicates the batch size, represents the number of categories, Indicates The sample in The true label of the class, represents the predicted probability; The gradient is calculated by the following formula: ; in, represents the gradient, represents the set of model parameters, Indicates The loss of samples.

[0034] As an example, first, the word vectors of the input sequence are passed through the embedding layer and position encoding to obtain the initial input representation : ; in, represents the initial input representation, Represents word vector, Represents positional encoding, which is used to encode the position information of a word in a sequence.

[0035] Next, Pass through L Transformer encoder layers in sequence to get the final representation : ; Then, Input to the fully connected layer to output the corresponding prediction results through the fully connected layer : ; in, Represents the prediction result, represents the weight matrix of the fully connected layer, Represents the bias vector of the fully connected layer.

[0036] Next, the cross entropy loss function is used to calculate the error between the predicted result and the true label: ; in, represents the cross entropy loss function, Indicates the batch size, represents the number of categories, Indicates The sample in The true label of the class, represents the predicted probability; Then, using the chain rule, we calculate the gradient of the loss function with respect to the model parameters: ; in, represents the gradient, represents the set of model parameters, Indicates The loss of samples.

[0037] S104 , performing gradient noise addition to obtain a noisy gradient, and calculating a corresponding comprehensive unfairness index value based on the noisy gradient.

[0038] In some embodiments, gradient noise is added to obtain a noisy gradient, and a corresponding unfairness index value is calculated based on the noisy gradient, including: clipping the gradient and generating initial noise, and calculating the noisy gradient based on the initial noise and the clipped gradient; calculating an initial unfairness index value based on the noisy gradient, and calculating a corresponding average unfairness index value based on the initial unfairness index value obtained from multiple trainings; introducing a time decay factor to decay the average unfairness index value, and performing weighted average calculation on the attenuated average unfairness index value to obtain a comprehensive unfairness index value.

[0039] In some embodiments, the initial unfairness index value is calculated by the following formula: ; in, represents the initial unfairness index value, represents the penalty coefficient, Representing data groups The noise, Representing a group The noise, represents the sign function, which is used to indicate the direction of the noise. Representing data groups The corresponding noisy gradient, Representing data groups The corresponding noisy gradient; The average unfairness index is calculated by the following formula: ; in, represents the average unfairness index, represents the total number of iterative training, Indicates Sub-fairness calculation; The average unfairness index value is attenuated by the following formula: ; in, represents the average unfairness index value after attenuation, Indicates the current time and The time interval between training sessions; The weighted average of the attenuated average unfairness index value is calculated using the following formula: ; ; in, represents the unfairness index value after weighted average, Representing a group and groups The proportion of the unfairness between Represents the comprehensive unfairness index value.

[0040] As an example, first, to prevent abnormally large gradients from affecting model training, the gradient of each sample is clipped: ; in, represents the clipped gradient, represents the original gradient, Represents the preset gradient norm threshold.

[0041] Next, according to differential privacy Definition, add noise that conforms to the Gaussian noise mechanism: ; in, Represents the intensity of the noise, and the calculation formula is: ; in, represents the failure probability, which usually takes a smaller value, for example, .

[0042] Then, noise is added to the clipped gradient to obtain the noisy gradient: ; Next, for any two data groups and , define the unfairness in a training, that is, the initial unfairness index value is: ; in, represents the initial unfairness index value, represents the penalty coefficient, which is greater than 1 and is used to amplify the impact of reverse noise; preferably, it can be taken , Representing data groups The noise, Representing a group The noise, represents the sign function, which is used to indicate the direction of the noise. Representing data groups The corresponding noisy gradient, Representing data groups The corresponding noisy gradient.

[0043] Then, the average unfairness index is calculated: ; in, represents the average unfairness index, represents the total number of iterative training, Indicates Sub-fairness calculation.

[0044] Next, the time decay factor is introduced: ; in, represents the average unfairness index value after attenuation, Indicates the current time and The time interval between training sessions.

[0045] Then, taking into account the group weights, the weighted average unfairness is calculated: ; Finally, the comprehensive unfairness index is obtained: ; in, represents the unfairness index value after weighted average, Representing a group and groups The proportion of the unfairness between Represents the comprehensive unfairness index value.

[0046] S105: Determine whether the comprehensive unfairness index value is within a preset value range.

[0047] S106, if the comprehensive unfairness index value is within the preset value range, determine whether the current large language model meets the training requirements; if yes, use the current large language model as the final model; if no, return to step S103.

[0048] In some embodiments, if the comprehensive unfairness index value is not within a preset value range, the differential privacy model is modified to adjust the privacy budget of each group; the noise intensity is updated according to the adjusted privacy budget, and the process returns to step S104.

[0049] In some embodiments, the privacy budget is adjusted by the following formula: ; ; in, represents the adjusted privacy budget, represents the initial privacy budget, represents the comprehensive unfairness index value, represents the adjustment function, represents the adjustment factor; ; in, represents the updated noise intensity, Indicates the threshold for gradient norm clipping.

[0050] As an example, first, dynamically adjust the privacy budget of each group based on the fairness indicator : ; ; in, represents the adjusted privacy budget, represents the initial privacy budget, represents the comprehensive unfairness index value, represents the adjustment function, represents the adjustment factor; Next, update the noise intensity based on the new privacy budget: ; Then, construct the unfairness matrix: ; Next, perform eigenvalue decomposition on the matrix: ; in, represents the eigenvector matrix, Represents the focus matrix, including eigenvalues.

[0051] Then, construct the fairness loss function: ; Among them, 1 is a matrix whose elements are all 1, and is the weight coefficient.

[0052] Next, perform gradient descent to optimize the privacy budget: ; in, represents the learning rate, is the gradient of the loss function with respect to the privacy budget, which can be calculated by the chain rule.

[0053] Then, based on the optimized privacy budget , update the noise intensity , and regenerate the noise .

[0054] Next, use the signal noise to update the noisy gradient: ; Then, the noisy gradients within the batch are averaged to obtain the global gradient: ; Next, use the optimizer to update the model parameters: ; in, Represents the learning rate.

[0055] In some embodiments, there are multiple ways to determine whether the current large language model meets the training requirements. For example: determine whether the current number of training rounds has reached a preset number of rounds; or determine whether the loss function of the model has converged, that is, whether the loss change of adjacent iterations is less than a preset threshold; or whether the performance index of the model meets the expected requirements; the determination method of whether the training requirements are met is not limited here.

[0056] In some embodiments, the training method also includes validation and evaluation of the model.

[0057] As an example, first, evaluate the model performance and use the validation set to evaluate the model performance indicators: ; in, Indicates accuracy.

[0058] In addition, it should be noted that indicators such as precision, recall and F1 score can be selected according to task requirements.

[0059] Next, for different data groups, the performance indicators of the model on each group were calculated to check whether there were significant differences.

[0060] First, calculate the mean difference: ; Equal Opportunity: ; if and If it is close to 0, it means that the model has good fairness among different groups.

[0061] As a specific embodiment of the present invention, Figure 2 As shown, the large model training method based on the differential privacy mechanism disclosed in the present invention specifically includes the following steps: S201, acquiring historical data, preprocessing the historical data, and grouping and labeling the preprocessed historical data to form a training data set.

[0062] S202, initializing large language model parameters.

[0063] S203, training the large language model based on the training data set, and calculating the gradient of the loss function with respect to the model parameters.

[0064] S204, clipping the gradient and generating initial noise, and using the initial noise as current noise.

[0065] S205, calculating a noisy gradient according to the current noise and the clipped gradient.

[0066] S206: Calculate an initial unfairness index value based on the noise-added gradient.

[0067] S207: Calculate a corresponding average unfairness index value according to the initial unfairness index values ​​obtained through multiple trainings.

[0068] S208: introducing a time decay factor to decay the average unfairness index value.

[0069] S209: Perform weighted average calculation on the attenuated average unfairness index value to obtain a comprehensive unfairness index value.

[0070] S210, determining whether the comprehensive unfairness index value is within a preset value range; if yes, executing step S211; if no, executing step S212.

[0071] S211, determine whether the current large language model meets the training requirements; if yes, execute step S214; if no, return to step S203.

[0072] S212, modify the differential privacy model to adjust the privacy budget of each group.

[0073] S213, update the current noise according to the adjusted privacy budget, and return to step S205.

[0074] S214, taking the current large language model as the final model.

[0075] As an example, first, historical data is obtained, and the historical data is preprocessed, and the preprocessed historical data is grouped and labeled to form a training data set; then, the large language model parameters are initialized; then, the large language model is trained based on the training data set, and the gradient of the loss function to the model parameters is calculated; then, for each sample, the gradient , perform gradient clipping to limit the second norm of the gradient to not exceed the preset threshold: ; Then, the initial noise is generated: according to the differential privacy Definition, add noise that conforms to the Gaussian noise mechanism: ; in, Represents the intensity of the noise, and the calculation formula is: ; in, represents the failure probability, which usually takes a smaller value, for example, .

[0076] Next, calculate the noisy gradient: ; Then, for any two data sets and , define the unfairness in a training, that is, the initial unfairness index value is: ; in, represents the initial unfairness index value, represents the penalty coefficient, which is greater than 1 and is used to amplify the impact of reverse noise; preferably, it can be taken , Representing data groups The noise, Representing a group The noise, represents the sign function, which is used to indicate the direction of the noise. Representing data groups The corresponding noisy gradient, Representing data groups The corresponding noisy gradient.

[0077] Then, the average unfairness index is calculated: ; in, represents the average unfairness index, represents the total number of iterative training, Indicates Sub-fairness calculation.

[0078] Next, the time decay factor is introduced: ; in, represents the average unfairness index value after attenuation, Indicates the current time and The time interval between training sessions.

[0079] Then, taking into account the group weights, the weighted average unfairness is calculated: ; Finally, the comprehensive unfairness index is obtained: ; in, represents the unfairness index value after weighted average, Representing a group and groups The proportion of the unfairness between Represents the comprehensive unfairness index value.

[0080] Purpose: To quantitatively evaluate the degree of unfairness between different groups during model training and provide a basis for subsequent adjustments.

[0081] Next, it is determined whether the comprehensive unfairness index is within an acceptable range, that is, whether it is within a preset value range; If yes, continue model training without adjustment; If not, further adjust the differential privacy model.

[0082] Specifically, the privacy budget is dynamically adjusted according to the comprehensive unfairness index: ; ; in, represents the adjusted privacy budget, represents the initial privacy budget, represents the comprehensive unfairness index value, represents the adjustment function, represents the adjustment factor; Next, update the noise intensity: ; Then, using the updated noise intensity, regenerate the noise: ; Next, use the new noise to update the noisy gradient: ;

[0083] Then, the updated noisy gradient is used to recalculate the comprehensive unfairness index value and determine whether the adjusted model meets the fairness requirements. If not, the privacy budget and noise are adjusted until the requirements are met.

[0084] In summary, according to the large model training method based on the differential privacy mechanism of the embodiment of the present invention, first, S101, historical data is obtained, and the historical data is preprocessed, and the preprocessed historical data is grouped and labeled to form a training data set; then, S102, the large language model parameters are initialized; then, S103, the large language model is trained based on the training data set, and the gradient of the loss function to the model parameters is calculated; then, S104, gradient noise is added to obtain the noisy gradient, and the corresponding comprehensive unfairness index value is calculated based on the noisy gradient; then, S105, it is determined whether the comprehensive unfairness index value is within the preset value range; then, S106, if the comprehensive unfairness index value is within the preset value range, it is determined whether the current large language model meets the training requirements; if yes, the current large language model is used as the final model; if not, it returns to step S103. Thereby, effective protection of user privacy is achieved, and at the same time, the generation of unfairness caused by the use of the differential privacy mechanism is avoided.

[0085] In the second aspect, an embodiment of the present invention proposes a computer-readable storage medium, on which a large model training program based on a differential privacy mechanism is stored. When the large model training program based on the differential privacy mechanism is executed by a processor, the large model training method based on the differential privacy mechanism as described above is implemented.

[0086] Thirdly, Figure 3As shown, an embodiment of the present invention proposes a large model training system based on a differential privacy mechanism. The large model training system based on the differential privacy mechanism includes: an acquisition module 10, an initialization module 20, a training module 30, a fairness calculation module 40 and a judgment module 50.

[0087] The acquisition module 10 is used to acquire historical data, preprocess the historical data, and group and annotate the preprocessed historical data to form a training data set; The initialization module 20 is used to initialize the large language model parameters; The training module 30 is used to train the large language model based on the training data set and calculate the gradient of the loss function to the model parameters; The fairness calculation module 40 is used to perform gradient noise addition to obtain a noisy gradient, and calculate a corresponding comprehensive unfairness index value based on the noisy gradient; The judgment module 50 is used to judge whether the comprehensive unfairness index value is within a preset value range; The judgment module 50 is also used to judge whether the current large language model meets the training requirements when the comprehensive unfairness index value is within a preset value range, and when the judgment result is yes, use the current large language model as the final model, and when the judgment result is no, return to the step of training the large language model based on the training data set.

[0088] It should be noted that the above description of the large model training method based on the differential privacy mechanism is also applicable to the large model training system based on the differential privacy mechanism, and will not be repeated here.

[0089] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0090] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0092] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0093] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0094] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0095] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0096] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A large model training method based on differential privacy mechanism, characterized in that: The following steps are involved: S101, acquiring historical data, preprocessing the historical data, and grouping and labeling the preprocessed historical data to form a training data set; S102, initializing large language model parameters; S103, training the large language model based on the training data set, and calculating the gradient of the loss function with respect to the model parameters; S104, performing gradient noise addition to obtain a noisy gradient, and calculating a corresponding comprehensive unfairness index value based on the noisy gradient; S105, determining whether the comprehensive unfairness index value is within a preset value range; S106, if the comprehensive unfairness index value is within a preset value range, determining whether the current large language model meets the training requirements; If yes, the current large language model is used as the final model; if no, the process returns to step S103.

2. The large model training method based on differential privacy mechanism as claimed in claim 1, characterized in that: Preprocessing the historical data includes: Performing data cleaning on the historical data, and performing data labeling on the cleaned historical data; The annotated historical data is segmented and word vector mapped, and the mapped word vectors are processed for sequence length and normalized.

3. The large model training method based on differential privacy mechanism as claimed in claim 1, characterized in that: Train the large language model based on the training data set and calculate the gradient of the loss function to the model parameters, including: For each batch of data in the training data set, forward propagation is performed to obtain corresponding prediction results; The error between the predicted result and the true label is calculated based on the cross entropy loss function, and the gradient of the loss function to the model parameters is calculated based on the error using the chain rule.

4. The large model training method based on differential privacy mechanism as claimed in claim 3, characterized in that: The cross entropy loss function is expressed by the following formula: ； in, represents the cross entropy loss function, Indicates the batch size, represents the number of categories, Indicates The sample in The true label of the class, represents the predicted probability; The gradient is calculated by the following formula: ； in, represents the gradient, represents the set of model parameters, Indicates The loss of samples.

5. The large model training method based on differential privacy mechanism as claimed in claim 1, characterized in that: Performing gradient noise to obtain a noisy gradient, and calculating a corresponding unfairness index value based on the noisy gradient, including: Clipping the gradient and generating initial noise, and calculating a noisy gradient based on the initial noise and the clipped gradient; Calculating an initial unfairness index value based on the noisy gradient, and calculating a corresponding average unfairness index value according to the initial unfairness index values ​​obtained through multiple trainings; A time decay factor is introduced to decay the average unfairness index value, and a weighted average calculation is performed on the decayed average unfairness index value to obtain a comprehensive unfairness index value.

6. The large model training method based on differential privacy mechanism as claimed in claim 5, characterized in that: The initial unfairness index value is calculated by the following formula: ； in, represents the initial unfairness index value, represents the penalty coefficient, Representing data groups The noise, Representing a group The noise, represents the sign function, which is used to indicate the direction of the noise. Representing data groups The corresponding noisy gradient, Representing data groups The corresponding noisy gradient; The average unfairness index is calculated by the following formula: ； in, represents the average unfairness index, represents the total number of iterative training, Indicates Secondary fairness calculation, which reduces the occurrence of random errors by averaging multiple calculations; The average unfairness index value is attenuated by the following formula: ； in, represents the average unfairness index value after attenuation, Indicates the current time and The time interval between training sessions; The weighted average of the attenuated average unfairness index value is calculated using the following formula: ； ； in, represents the weighted average unfairness index value, Representing a group and groups The proportion of the unfairness between Represents the comprehensive unfairness index value.

7. The large model training method based on differential privacy mechanism as claimed in claim 1, characterized in that: If the value of the comprehensive unfairness index is not within the preset value range, modifying the differential privacy model to adjust the privacy budget of each group; The noise intensity is updated according to the adjusted privacy budget, and the process returns to step S104.

8. The large model training method based on differential privacy mechanism as claimed in claim 7, characterized in that: The privacy budget is adjusted by the following formula: ； ； in, represents the adjusted privacy budget, represents the initial privacy budget, represents the comprehensive unfairness index value, represents the adjustment function, represents the adjustment factor; ； in, represents the updated noise intensity, Indicates the threshold for gradient norm clipping.

9. A computer-readable storage medium, characterized in that: A large model training program based on a differential privacy mechanism is stored thereon, and when the large model training program based on a differential privacy mechanism is executed by a processor, the large model training method based on a differential privacy mechanism as described in any one of claims 1-8 is implemented.

10. A large model training system based on differential privacy mechanism, characterized in that: include: An acquisition module, the acquisition module is used to acquire historical data, preprocess the historical data, and group and annotate the preprocessed historical data to form a training data set; An initialization module, wherein the initialization module is used to initialize large language model parameters; A training module, wherein the training module is used to train a large language model based on a training data set and calculate a gradient of a loss function with respect to model parameters; A fairness calculation module, the fairness calculation module is used to perform gradient noise addition to obtain a noisy gradient, and calculate a corresponding comprehensive unfairness index value based on the noisy gradient; A judgment module, the judgment module is used to judge whether the comprehensive unfairness index value is within a preset value range; The judgment module is also used to judge whether the current large language model meets the training requirements when the comprehensive unfairness index value is within a preset value range, and when the judgment result is yes, use the current large language model as the final model, and when the judgment result is no, return to the step of training the large language model based on the training data set.

Images