Data privacy protection method and system for large model knowledge base

Abstract

The invention provides a data privacy protection method and system for a large model knowledge base, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining original privacy data of each participant, carrying out the cleaning and feature extraction of the original privacy data, and recognizing sensitive information fields; performing desensitization processing on the fields containing the sensitive information by utilizing a generative adversarial network to obtain desensitized data; and performing k-anonymization processing on the desensitized data, dividing the data into a plurality of equivalence classes based on the quasi-identifier, and performing generalization processing on the classes with the record number less than k to obtain an anonymized data set. According to the method, the privacy leakage problem of multi-participant data collaboration in large model training is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a data privacy protection method and system for a large model knowledge base. BACKGROUND

[0002] With the deep integration of medical informatization and artificial intelligence technology, large medical institutions such as third-grade hospitals gradually introduce clinical auxiliary diagnosis large models to solve the problems of heavy outpatient treatment pressure, difficult diagnosis of rare diseases, and insufficient standardization of diagnosis and treatment schemes. The core capability of the clinical auxiliary diagnosis large model relies on the collaborative support of private medical knowledge bases and public authoritative knowledge bases. However, the strong sensitivity of medical data, the high precision characteristics of diagnosis requirements, and the insufficient adaptability of existing privacy protection technologies form a sharp contradiction. Existing privacy protection technologies cannot meet the dual needs of privacy security and diagnosis accuracy of clinical auxiliary diagnosis large models. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a data privacy protection method and system for a large model knowledge base, which solves the problem of privacy leakage in the collaborative training of large models.

[0004] To solve the above technical problems, the technical solution of the present application is as follows: In a first aspect, a data privacy protection method for a large model knowledge base, the method comprising: Obtaining the original privacy data of each participant, cleaning and feature extraction of the original privacy data, and identifying the sensitive information field; using a generative adversarial network to perform desensitization processing on the field containing sensitive information, and obtaining desensitized data; Performing k-anonymization processing on the desensitized data, dividing the data into multiple equivalence classes based on quasi-identifier, and performing generalization processing on the classes with record number less than k to obtain an anonymized data set; According to the anonymized data set, add Laplace noise through the differential privacy mechanism to obtain the noisy privacy data; homomorphic encryption is performed on the noisy privacy data, and the encrypted data is stored locally to construct a vertical federated learning framework, each participant trains a local model based on the local encrypted data, and calculates the model gradient; Performing norm clipping processing on the model gradient, adding Gaussian noise to the gradient according to the dynamic privacy budget adjustment mechanism to obtain the protected gradient parameter; Upload the protected gradient parameter to the central server, use homomorphic encryption technology to perform secret state aggregation on the received gradient parameter, generate global model update parameter; adjust the aggregation weight of the global model update parameter based on the dynamic smoothing coefficient mechanism, and distribute the adjusted parameter to each participant to update the local model; The steps of local training, gradient uploading, ciphertext aggregation and parameter distribution are repeatedly performed until the model converges, and finally a large model trained by privacy protection is obtained.

[0005] Further, the original privacy data of each participant is obtained, the original privacy data is cleaned and feature extracted, and sensitive information fields are identified; the fields containing sensitive information are desensitized by using a generative adversarial network to obtain desensitized data, including: The original privacy data is obtained from each participant, and the original privacy data is cleaned, including removing duplicate records, handling missing values and outliers, and performing data standardization; Feature extraction is performed on the cleaned data, and sensitive information fields are identified based on a predefined sensitive information rule library, including personal identification information, health information or financial information; The sensitive information fields are desensitized by using a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator, the generator is used to generate synthetic data to replace the original sensitive information, and the discriminator is used to distinguish between real data and synthetic data, the generator is optimized by adversarial training, so that the desensitized data retains the statistical characteristics of the original data and cannot directly identify individuals, and desensitized data is obtained.

[0006] Further, the desensitized data is subjected to k-anonymity processing, the data is divided into multiple equivalence classes based on quasi-identifiers, and the classes with fewer than k records are subjected to generalization processing to obtain an anonymized data set, including: According to the desensitized data, and based on a predefined quasi-identifier set, the data is divided, the quasi-identifier set is composed of attribute field combinations that can indirectly identify individuals, and the values of the quasi-identifiers are obtained; According to the values of the quasi-identifiers, the data records are divided into several equivalence classes, so that each equivalence class contains at least k records; The equivalence classes with fewer than k records are subjected to generalization processing, by increasing the level of data abstraction or merging into similar equivalence classes, to meet the k-anonymity requirement, and an anonymous data set satisfying k-anonymity is output.

[0007] Further, the step 3 includes: Based on the anonymized data set, according to a predefined global privacy budget, a Laplace noise is added to the numerical data in the data set by a differential privacy mechanism, the global sensitivity of the data query function is calculated, the scale parameter of the Laplace noise is determined according to the privacy budget and the sensitivity, and random noise obeying the scale parameter Laplace distribution is added to each numerical data item, and the noisy privacy data is obtained; The homomorphic encryption algorithm is used for encrypting the privacy data after adding noise; the public key and the private key required for homomorphic encryption are generated, wherein the public key is distributed to each participant, and the private key is securely kept by the data owner; the public key is used for homomorphic encryption operation on the privacy data after adding noise, so as to convert the plaintext data into ciphertext data; and the obtained ciphertext data is distributed and stored in the local storage system of each participant; Based on the stored local encrypted data, a vertical federated learning framework is constructed, each participant performs model training locally using the encrypted data, and the forward propagation calculation of the model is performed in the ciphertext state using the homomorphic encryption characteristic to obtain the prediction result; based on the difference between the prediction result and the real label, the reverse propagation calculation is performed in the encrypted state to obtain the model gradient in the encrypted state.

[0008] Further, the model gradient is subjected to norm clipping processing, and Gaussian noise is added to the gradient according to a dynamic privacy budget adjustment mechanism to obtain a protected gradient parameter, including: The encrypted model gradient is decrypted to obtain a plaintext gradient, each gradient vector is extracted from the plaintext gradient, the norm of each gradient vector is calculated, and the norm of the gradient vector is compared with a preset clipping threshold; for the gradient vector whose norm exceeds the threshold, the norm is scaled to equal the clipping threshold, and the gradient after norm clipping is obtained; Based on the gradient after norm clipping, the Gaussian noise parameter required for the current training round is determined according to the dynamic privacy budget adjustment mechanism; the dynamic privacy budget adjustment mechanism dynamically calculates and allocates the privacy budget of the current round based on the current training iteration number, the model convergence state and the remaining total privacy budget, and calculates the standard deviation of the Gaussian noise according to the allocated privacy budget and the gradient sensitivity; Based on the calculated noise parameter, random noise conforming to the corresponding Gaussian distribution is added to the obtained gradient after norm clipping to generate a protected gradient parameter.

[0009] Further, the protected gradient parameter is uploaded to the central server, and the received gradient parameter is aggregated in the ciphertext state using the homomorphic encryption technology to generate a global model update parameter; the aggregation weight of the global model update parameter is adjusted based on a dynamic smoothing coefficient mechanism, and the adjusted parameter is distributed to each participant to update the local model, including: The protected gradient parameter is received, and the protected gradient parameter of each participant is uploaded to the central server; each participant transmits the gradient parameter protected by noise addition and protection to the central server through a secure communication channel; Based on the gradient parameters of each participant received by the central server, the ciphertext aggregation is performed using the homomorphic encryption technology; the central server performs weighted summation operation on the gradient parameters of each participant in the ciphertext state to generate the global model update parameter in the encrypted state; The aggregation weight of the global model update parameter is adjusted based on a dynamic smoothing coefficient mechanism, and the smoothing coefficient is dynamically calculated according to the data quality, model contribution and historical performance of each participant, so as to adjust the weight distribution of each participant in the aggregation process. The adjusted global model update parameter is distributed to each participant, and each participant updates the local model using the received parameter to complete the model update of the current training round.

[0010] Further, the steps of local training, gradient uploading, secure aggregation and parameter distribution are repeatedly performed until the model converges, and finally a large model trained by privacy protection is obtained, including: Based on the adjusted global model update parameter, each participant updates the local model parameter to complete a round of local model update; The process of steps 3 to 5 is repeatedly performed, each participant trains again based on the updated local model and local encrypted data, calculates the model gradient, performs norm clipping and adds Gaussian noise to obtain protected gradient parameters, uploads the protected gradient parameters to the central server for secure aggregation, and the central server adjusts the aggregation weight based on the dynamic smoothing coefficient mechanism and generates new global model update parameters, which are then distributed to each participant; In each iteration process, the central server monitors the change trend of the global model loss function, and judges whether the model converges based on the pre-defined convergence condition; the convergence condition includes that the change rate of the global loss function value is lower than the pre-set threshold, or the maximum training round number is reached; Once the model is determined to converge, the central server terminates the training process and notifies each participant to save the final local model, and the collection of local models of each participant constitutes a large model trained by privacy protection.

[0011] In a second aspect, a data privacy protection system for a large model knowledge base includes: The acquisition module is configured to obtain original privacy data of each participant, clean and extract features from the original privacy data, and identify sensitive information fields; the fields containing sensitive information are desensitized by using a generative adversarial network to obtain desensitized data; the k-anonymization processing is performed on the desensitized data, the data is divided into multiple equivalence classes based on quasi-identifier, and the generalization processing is performed on the classes with less than k records to obtain an anonymized data set; The modeling module is configured to add Laplace noise to the anonymized data set by using a differential privacy mechanism to obtain noisy privacy data; the noisy privacy data is homomorphically encrypted and stored locally to construct a vertical federated learning framework, each participant trains a local model based on the local encrypted data, and calculates the model gradient; The aggregation module is configured to perform norm clipping on the model gradient, add Gaussian noise to the gradient according to a dynamic privacy budget adjustment mechanism, and obtain a protected gradient parameter; upload the protected gradient parameter to a central server, perform ciphertext aggregation on the received gradient parameter using a homomorphic encryption technology, and generate a global model update parameter; adjust the aggregation weight of the global model update parameter based on a dynamic smoothing coefficient mechanism, and distribute the adjusted parameter to each participant to update the local model. The processing module is configured to repeatedly perform the local training, gradient uploading, ciphertext aggregation and parameter distribution steps until the model converges, and finally obtain a large model trained with privacy protection.

[0012] In a third aspect, a computing device includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method.

[0013] In a fourth aspect, a computer readable storage medium stores a program, which is executed by a processor to implement the method.

[0014] The above-mentioned scheme of the present application at least includes the following beneficial effects: By fusing the desensitization of the generative adversarial network, k-anonymity, differential privacy noise and homomorphic encryption technology, the sensitive information leakage path is blocked layer by layer from the original data source to the model training process, the data statistical characteristics are retained to guarantee the model training foundation, the data is available and invisible, the privacy security pain points in the collaborative training of multiple participants are solved; on the other hand, by using the vertical federated learning framework combined with dynamic privacy budget adjustment, gradient norm clipping, dynamic smoothing coefficient aggregation and other mechanisms, the model training efficiency and accuracy can be optimized according to the training iteration state, data quality and participant contribution while ensuring the privacy protection strength, the model performance loss caused by a single privacy technology is avoided, and finally a large model considering privacy security and practical value can be stably output, which is suitable for medical, financial and other scenes with high requirements for sensitive data protection, and has technical innovation and practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flowchart of a data privacy protection method for a large model knowledge base provided by an embodiment of the present application.

[0016] Figure 2 is a schematic diagram of a data privacy protection system for a large model knowledge base provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and so that the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0018] As shown in Figure 1 An embodiment of the present application proposes a data privacy protection method for a large model knowledge base, which comprises the following steps: Step 1, obtain the original privacy data of each participant, clean and extract features from the original privacy data, and identify the sensitive information field; use a generative adversarial network to desensitize the field containing sensitive information, and obtain desensitized data; Step 2, perform k-anonymization processing on the desensitized data, divide the data into multiple equivalence classes based on quasi-identifiers, and perform generalization processing on classes with fewer than k records to obtain an anonymized data set; Step 3, according to the anonymized data set, add Laplace noise through the differential privacy mechanism to obtain the noisy privacy data; homomorphic encryption is performed on the noisy privacy data, and stored locally to construct a vertical federated learning framework, each participant trains a local model based on the local encrypted data, and calculates the model gradient; Step 4, perform norm clipping processing on the model gradient, add Gaussian noise to the gradient according to the dynamic privacy budget adjustment mechanism, and obtain the protected gradient parameter; Step 5, upload the protected gradient parameter to the central server, use homomorphic encryption technology to perform ciphertext aggregation on the received gradient parameter, generate global model update parameters; adjust the aggregation weight of the global model update parameters based on the dynamic smoothing coefficient mechanism, and distribute the adjusted parameters to each participant to update the local model; Step 6, repeat the steps of local training, gradient uploading, ciphertext aggregation and parameter distribution until the model converges, and finally obtain a large model trained by privacy protection.

[0019] In this embodiment of the invention, by using generative adversarial networks for desensitization and k-anonymization, the risks of direct and indirect identity verification are eliminated while preserving the statistical characteristics of the data. Secondly, by combining differential privacy and homomorphic encryption techniques, local encrypted data storage and dense-state training are achieved within a vertical federated learning framework, ensuring that the data is usable but not visible. Furthermore, gradient backpropagation attacks are prevented through gradient norm pruning, dynamic privacy budget control, and dense-state aggregation, while the global model update accuracy is optimized based on a dynamic smoothing coefficient. Finally, the large model obtained through iterative training meets the needs of multiple parties collaboratively building a knowledge base, establishing a strong privacy and security defense throughout the entire process, balancing model performance and data protection, and is suitable for training large models in multiple scenarios where privacy is sensitive.

[0020] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1: Obtain raw privacy data from each participating party, perform data cleaning on the raw privacy data, including removing duplicate records, handling missing and outlier values, and standardizing the data. Specifically, this includes: obtaining raw privacy data from each participating party through their pre-defined secure communication interfaces. The raw privacy data may cover multi-dimensional data from different scenarios such as medical, financial, and government affairs. After obtaining the raw privacy data, to avoid redundancy, missing, and outlier information interfering with subsequent processing, data cleaning operations must first be performed on the raw privacy data: For duplicate records, by comparing the feature values ​​of the core attribute fields of each record in the raw privacy data, multiple records with completely identical feature values ​​are identified as duplicate records, and one valid record is retained while the rest are deleted; for missing values, ... Differentiated processing methods are adopted based on the differences in field data types. Numerical missing data can be filled using the mean, median, or interpolation based on adjacent valid data. Categorical missing data can be filled using the mode or a preset unknown category. For outliers, the standard deviation range of the field data is calculated, such as mean ± 3 times the standard deviation, or a reasonable value range preset in combination with the business scenario. Values ​​exceeding the range are identified as outliers, and then correction or deletion is selected according to the degree of deviation of the outliers. After data cleaning is completed, in order to ensure the consistency of subsequent feature extraction and desensitization processing, standardization operations are also required on the cleaned data: converting numerical fields of different magnitudes to the same numerical range, such as [0,1] or [-1,1], and unifying the format of date fields of different formats.

[0021] Step 1.2, feature extraction is performed on the cleaned data, and sensitive information fields are identified based on a predefined sensitive information rule library, the sensitive information fields including personal identity information, health information, or financial information, specifically including: performing feature extraction on the cleaned basic data, the extracted content including attribute features and data association features of each field, and integrating these features into a feature set through structured processing; secondly, a predefined sensitive information rule library is called, the rule library at least including two parts of sensitive field type definition and field feature matching rule, the sensitive field type definition clearly dividing the specific field range of three categories of sensitive information of personal identity information, health information, and financial information, and the field feature matching rule presetting the feature identifier of each type of sensitive field; finally, the foregoing structured feature set and the field feature matching rule in the rule library are compared one by one: if the features of a field completely match the feature identifier of a type of sensitive field, it is determined that the field is a sensitive information field of the corresponding type; if there is a situation that the field features partially overlap with the features of multiple types of sensitive fields, the ownership scene of the data record is further confirmed, and finally a complete list of sensitive information fields is output.

[0022] Step 1.3, desensitizing the sensitive information field by using a generative adversarial network, wherein the generative adversarial network comprises a generator and a discriminator, the generator is used to generate synthetic data to replace the original sensitive information, the discriminator is used to distinguish between real data and synthetic data, the generator is optimized by adversarial training, so that the desensitized data retains the statistical characteristics of the original data and cannot directly identify individuals, and desensitized data is obtained, specifically comprising: constructing the basic architecture of the generative adversarial network, the architecture includes two core modules of the generator and the discriminator, the generator adopts a multi-layer neural network structure, and the statistical characteristics of the original sensitive information field, such as mean, variance, value distribution law, etc. are used as input, the synthetic data with the same data format and similar statistical characteristics as the original sensitive information field data is generated through the nonlinear transformation of the neural network, and the synthetic data needs to ensure that the original sensitive information cannot be deduced, such as the generated synthetic identity card number conforms to the coding format but has no real corresponding individual, and the synthetic transaction flow conforms to the normal amount range but has no real transaction background; the discriminator also adopts a multi-layer neural network structure, and the original sensitive information data and the synthetic data output by the generator are used as input, a classification model is constructed by learning the difference characteristics of the two types of data, and is used to output the probability value of the input data being the original sensitive information data; secondly, the generative adversarial network is executed for adversarial training optimization, and the training process is performed in an iterative manner. In the first iteration, the generator parameters are fixed, the original sensitive information data and the synthetic data are mixed into the discriminator, the classification error of the discriminator is calculated, such as cross-entropy loss, the discriminator parameters are adjusted in reverse, and the discrimination ability is improved; subsequently, the discriminator parameters are fixed, the synthetic data is input into the discriminator, the misjudgment error of the discriminator for the synthetic data, i.e. the probability deviation of judging as the original data, is calculated, the generator parameters are adjusted in reverse, and the statistical similarity between the synthetic data and the original data is optimized; the above iteration continues until the classification accuracy of the discriminator for the two types of data is close to 50%, i.e. it cannot effectively distinguish, at which point the training is stopped and the generator parameters are fixed; finally, the sensitive information field identified by the trained generator is used to perform batch desensitization: each data of the original sensitive information field is input into the generator, the corresponding synthetic data is generated and replaces the original data, and the desensitized data needs to be checked for statistical characteristics after replacement, the mean, variance, and value distribution difference of the data before and after desensitization are compared, and the privacy security is verified, and whether the real individual can be identified is checked by randomly selecting data, and finally the desensitized data which retains the original statistical characteristics and has no individual identification risk is obtained.

[0023] In the embodiments of the present application, the original data quality is improved through data cleaning to lay a reliable foundation for subsequent processing; relying on the pre-defined sensitive information rule library, the core sensitive fields such as personal identity, health and finance are accurately identified to ensure that privacy protection is targeted; through the generation of an adversarial network for desensitization processing, the generator and the discriminator are used for adversarial training, the original sensitive information is replaced while the data statistical characteristics are retained, individual information is prevented from being directly leaked, the effectiveness of the data for model training is not lost, and a preliminary balance between data usability and privacy security is achieved.

[0024] In a preferred embodiment of the present application, step 2 can include: Step 2.1, according to the desensitized data, and based on the pre-defined quasi-identifier set, the data is divided, the quasi-identifier set is composed of attribute field combinations that can indirectly identify individuals, and the value of the quasi-identifier is obtained, specifically including: after obtaining the output desensitized data, the pre-defined quasi-identifier set needs to be called first, the quasi-identifier set is not a single field, but is composed of several attribute fields that do not have individual identification ability when they exist alone, but can be indirectly associated with a specific individual after combination, for example, in the medical data scenario, the quasi-identifier set can include age, gender, department, disease category, etc. Fields, in the financial data scenario, it can include age range, occupation type, account opening area, transaction frequency interval, etc. Fields, and the field composition of the set needs to be pre-determined in combination with the business characteristics of the specific application scenario and the data privacy risk assessment results; then, for each data record in the desensitized data, the field value corresponding to each field in the quasi-identifier set is extracted one by one, for example, in a certain medical data record, the age field value is 45, the gender field value is female, the department field value is cardiovascular internal medicine, and the disease category field value is chronic disease, then the four field values are integrated into the value of the quasi-identifier corresponding to the record, and it is necessary to ensure that no quasi-identifier field value of any data record is missed in the extraction process.

[0025] Step 2.2, according to the value of the quasi-identifier, the data records are divided into several equivalence classes, so that each equivalence class contains at least k records, specifically including: determining the specific value of k value, k value as the core parameter of k-anonymity, need to be set according to the data privacy protection level requirements, data size and subsequent model training on the demand for data precision comprehensive set, for example, for high privacy protection demand of medical data, k value can be set to 20 and above, for general privacy protection demand of government statistical data, k value can be set to 10-15; then, according to the value of the quasi-identifier as grouping basis, all data records are divided into equivalence classes: the quasi-identifier value of the same multiple data records are classified into the same equivalence class, for example, age 45, female, department of cardiology, all data records of disease category chronic disease, because the value of the standard identifier is completely consistent, will be divided into the same equivalence class; in the process of division, the number of data records contained in each equivalence class needs to be counted in real time, and compared with the preset k value, if the number of records in an equivalence class reaches or exceeds k value, the equivalence class is temporarily determined as an effective equivalence class meeting the preliminary requirements; if the number of records in an equivalence class has not reached k value, it is first marked as a to-be-processed equivalence class, and finally two groups of results of effective equivalence class and to-be-processed equivalence class are formed.

[0026] Step 2.3, generalization processing is performed on the equivalence classes with the number of records less than k, the data abstraction level is promoted or the equivalence classes are merged to meet the k-anonymity requirement, and an anonymous data set meeting the k-anonymity is output. Specifically, for each group marked as a to-be-processed equivalence class, the characteristics of the values of the quasi-identifier are analyzed, and it is determined whether to promote the data abstraction level or merge the to-be-processed equivalence class into a similar equivalence class. If there is a field in the quasi-identifier of the to-be-processed equivalence class that can be further abstracted, the data abstraction level is promoted. For example, the quasi-identifier values of a to-be-processed equivalence class are age 45, gender female, department of internal medicine of cardiovascular department, and disease category chronic disease, the number of records is 8, and the k value is set to 10. The age 45 can be abstracted to 40-50 years old. At this time, the number of records of the quasi-identifier values of age 40-50 years old, gender female, department of internal medicine of cardiovascular department, and disease category chronic disease in all data records is counted again. If the number reaches the k value, the equivalence class meets the k-anonymity requirement. If the number of records does not reach the k value after promoting the abstraction level, or there is no further abstraction space for the quasi-identifier field, the to-be-processed equivalence class is merged into a similar equivalence class. The similarity between the to-be-processed equivalence class and other effective equivalence classes in the quasi-identifier values is calculated, for example, the interval difference of the age field and the business correlation degree of the department of internal medicine of the cardiovascular department. The equivalence class with the highest similarity is selected as the merging object, and all data records of the to-be-processed equivalence class are merged into the similar equivalence class. The number of records of the equivalence class is counted again until it reaches the k value. After the generalization processing of all to-be-processed equivalence classes is completed, a final check is performed on all equivalence classes to confirm that the number of records of each equivalence class is not less than the k value, and the values of the quasi-identifier can still retain the core statistical characteristics of the data after generalization. Finally, an anonymous data set meeting the k-anonymity requirement is output.

[0027] In the embodiment of the present application, the data after desensitization is divided based on a predefined set of quasi-identifiers, and the key data dimensions that may be used to accurately locate the individual identity by combination and association are accurately locked. Then, equivalence classes are divided according to the values of the quasi-identifiers and it is ensured that each class contains at least k records, so as to construct a basic anonymous barrier from the data grouping level and preliminarily reduce the risk of accurate positioning of the individual. Finally, for the equivalence classes with the number of records less than k, generalization processing is performed by promoting the data abstraction level or merging similar classes, so as to further eliminate the uniqueness of the data grouping and strictly meet the k-anonymity requirement. The whole process effectively blocks the privacy leakage path of the quasi-identifier combination and association to identify the individual on the basis of retaining the statistical value of the data after desensitization. At the same time, the operation logic is clear, the landing is strong, and it is suitable for the compliance requirements of data anonymization in the construction of a large model knowledge base in multiple scenarios.

[0028] In a preferred embodiment of the present application, the above step 3 can include: Step 3.1, based on the anonymized dataset, according to the predefined global privacy budget, add Laplace noise to the numerical data in the dataset through the differential privacy mechanism, calculate the global sensitivity of the data query function, determine the scale parameter of the Laplace noise according to the privacy budget and the sensitivity, add random noise obeying the scale parameter Laplace distribution to each numerical data item, and obtain the noisy privacy data, which specifically includes: after obtaining the output anonymized dataset, the global privacy budget needs to be predefined based on the actual privacy protection requirements and data application scenarios. The global privacy budget is an indicator of the strength of privacy protection in the entire differential privacy processing process. Its value needs to be determined comprehensively in combination with data sensitivity and model training accuracy requirements. For example, for high-sensitivity medical diagnosis data, the global privacy budget can be set to a small value to improve the protection strength, and for low-sensitivity statistical data, it can be set to a larger value to balance availability. Then, the specific form of the data query function needs to be determined, including the sum query function. When the sum of a certain numerical field A in the anonymized dataset D needs to be calculated, such as the blood glucose test value in medical data or the transaction amount in financial data, the operation logic of the sum query function is to add the specific values of the field A on all records from the first record to the nth record in the dataset, and the result of the addition is taken as the statistical result of the total sum of the field. The value of each record in the field A refers to the specific value of the corresponding field in the record, for example, the blood glucose test value of the first record is the value of the record in the blood glucose test value field. By continuously adding the values of all records, the total sum of the field is obtained. The mean query function is used when the average value of a certain numerical field A in the anonymized dataset D needs to be calculated. The operation logic of the mean query function is: first, calculate the total sum of the values of all records in the field A according to the sum query function, then divide the total sum by the total number of records n in the dataset, i.e. the total number of records, and the result is the average value of the field A. The total number of records n is consistent with the total number of records involved in the sum query. By dividing the total sum by the total number of records, the average value result is obtained in the form of a real number. The global sensitivity of the query function is calculated. Specifically, by analyzing the output difference of the query function on all possible subsets of the anonymized dataset, the maximum value is taken as the global sensitivity. For example, for the sum query, the global sensitivity is the maximum possible value of the corresponding numerical field in a single data record. After obtaining the global privacy budget and the global sensitivity, the scale parameter of the Laplace noise can be determined: the ratio of the global privacy budget to the global sensitivity is taken as the scale parameter of the Laplace noise. The size of the scale parameter directly determines the noise intensity. The smaller the ratio, the stronger the noise and the better the privacy protection effect.Finally, noise is added to each numerical data item in the anonymized data set. For each numerical data item in the data set, a random value conforming to the Laplace distribution determined above is generated according to the scale parameter, and the random value is superimposed on the original numerical data item to complete the noise addition operation. After adding noise, the noise-added data needs to be checked for validity to ensure that the noise-added data is still within a reasonable business value range, and finally the privacy data after noise addition is obtained.

[0029] Step 3.2, the noise-added privacy data is encrypted by a homomorphic encryption algorithm; generate the public key and private key required for homomorphic encryption, wherein the public key is distributed to each participant, and the private key is securely kept by the data owner; use the public key to perform homomorphic encryption operation on the noise-added privacy data, convert the plaintext data into ciphertext data; the obtained ciphertext data is distributed and stored in the local storage system of each participant, specifically including: using a preset homomorphic encryption algorithm, the corresponding key generation protocol can be selected according to the data type and calculation requirement, and the public key and private key required by the algorithm are generated, wherein the public key is used for data encryption, and the private key is used for data decryption, and the two are generated in pairs and have a unique corresponding relationship; after generating the key, strict key management operation needs to be performed, the public key is distributed to all participants through the secure communication channel established by each participant in advance, such as the encrypted transmission channel based on TLS / SSL protocol, to ensure that the public key is not tampered with or stolen during transmission; while the private key is kept by the data owner, such as the medical institution or financial institution providing the original data, using a secure storage method, and only authorized specific administrators can access it, and the leakage or sharing of the private key is strictly prohibited; then, each participant uses the received public key to perform homomorphic encryption operation on the noise-added privacy data, and substitutes the noise-added plaintext data into the encryption process of the homomorphic encryption algorithm, and performs mathematical transformation on the data through the public key to convert it into ciphertext data that cannot be directly read; finally, the encrypted ciphertext data is stored in a distributed manner, and each participant only stores the ciphertext data corresponding to the original data provided by itself in the locally deployed encrypted storage device, and the storage device must have security functions such as access permission control, operation log audit, data backup and recovery to avoid the risk of single-point leakage caused by centralized storage.

[0030] Step 3.3, based on the stored local encrypted data, a longitudinal federated learning framework is constructed, each participant performs model training locally using encrypted data, and uses homomorphic encryption characteristics to perform forward propagation calculation of the model in the ciphertext state to obtain the prediction result; based on the difference between the prediction result and the true label, the reverse propagation calculation is performed in the encrypted state to obtain the model gradient in the encrypted state, specifically including: based on the local stored ciphertext data of each participant, a longitudinal federated learning framework is constructed, the framework needs to predefine the roles of each participant such as data provider, model training party, data feature alignment rule to ensure that the encrypted data of different participants can be calculated and communicated in the feature dimension, and communication protocol specification gradient, parameter and other information transmission format and security requirements; after the framework is constructed, each participant performs model training locally using the encrypted data stored by itself, in the forward propagation calculation stage of the model, the additive homomorphism or multiplicative homomorphism characteristics of homomorphic encryption are used to directly perform linear transformation of the model on the ciphertext data, for example, for linear transformation, homomorphic multiplication is first performed on the ciphertext form of the feature data and the ciphertext form of the model weight, and then homomorphic addition is performed on the result to obtain the intermediate calculation result in the form of ciphertext, and finally the model prediction result in the form of ciphertext is obtained through a series of ciphertext calculation; after obtaining the prediction result, the reverse propagation calculation is performed in the encrypted state based on the difference between the prediction result and the local saved true label, the prediction error in the form of ciphertext is calculated, such as the difference between the prediction result and the true label, and then the chain derivation operation is performed on the prediction error using the homomorphic encryption characteristics, and the ciphertext form gradient corresponding to the model parameters of each layer is gradually derived, and finally the model gradient in the encrypted state is output.

[0031] In the embodiment of the present application, the differential privacy mechanism is introduced based on the anonymized data set, the global privacy budget and the global sensitivity of the data query function are pre-defined to determine the Laplace noise scale, the noise is accurately added to the numerical data to avoid the privacy leakage risk that may exist in single anonymization, and the controllable noise design balances the data privacy protection strength and the data availability of subsequent model training; secondly, the homomorphic encryption algorithm is used to process the noisy data, and through the key management mode of public key distribution and private key exclusive storage, it is ensured that the data is converted into ciphertext that cannot be directly interpreted after encryption, and is distributed and stored locally in each participant, thereby eliminating the risk of external theft and internal unauthorized access from the data storage link, realizing the double protection of local storage and encryption protection of data; finally, relying on the longitudinal federated learning framework, the homomorphic encryption characteristics are used to support the forward propagation and reverse propagation calculation of the model in the ciphertext state, so that each participant can complete the local model training and gradient calculation without sharing the plaintext data, breaking the data island of multiple participants to support the collaborative training of large models, ensuring the privacy security of data and gradient in the transmission and calculation throughout the process, and overall adapting to the core needs of data non-sharing and value mining in the construction of large model knowledge base.

[0032] In a preferred embodiment of the present application, step 4 above can include: Step 4.1, decrypt the encrypted state model gradient to obtain the plaintext gradient, extract each gradient vector (such as the gradient unit corresponding to each layer parameter of the model) from the plaintext gradient, calculate the norm of each gradient vector, and compare the norm of the gradient vector with the preset clipping threshold. For the gradient vector whose norm exceeds the threshold, scale it so that its norm equals the clipping threshold, and obtain the gradient after norm clipping, specifically including: after obtaining the output encrypted state model gradient, the data owner holding the private key needs to perform decryption operation, specifically, the data owner calls the securely saved private key, substitutes into the corresponding decryption process of the homomorphic encryption algorithm, and converts the encrypted state gradient data into plaintext gradient which can be directly calculated; on this basis, the norm of the obtained plaintext gradient is calculated, and for each gradient vector, such as the gradient set corresponding to the weight parameters of a certain layer of the model, the norm value of the gradient vector is obtained by summing the square values of each element in the gradient vector and taking the square root according to the conventional norm calculation method in the field, such as the Euclidean norm calculation method; then, the calculated norm value is compared with the preset clipping threshold, which needs to be preset according to the model type, task and historical training experience, for example, for large models with large parameter scale, the clipping threshold can be set to a small value, such as 1.0-2.0, to strictly control the gradient scale, and for simple task models, it can be appropriately relaxed to 3.0-5.0; if the norm value of a certain gradient vector does not exceed the clipping threshold, the gradient vector is directly retained; if the norm value exceeds the clipping threshold, the gradient vector is scaled by a certain proportion, the ratio of the clipping threshold to the original norm value is calculated as the scaling coefficient, and each element in the gradient vector is multiplied by the scaling coefficient, so that the norm value of the scaled gradient vector is exactly equal to the clipping threshold; after completing the norm clipping of all gradient vectors, the clipped gradient needs to be checked for integrity to ensure that the dimension and element number of the gradient vector are consistent with those before clipping, and finally the gradient after norm clipping is obtained.

[0033] Step 4.2, based on the gradient after the norm clipping, determine the Gaussian noise parameter required for the current training round according to the dynamic privacy budget adjustment mechanism; the dynamic privacy budget adjustment mechanism dynamically calculates and allocates the privacy budget for the current round based on the current training iteration number, the model convergence state and the remaining total privacy budget, calculates the standard deviation of the Gaussian noise according to the allocated privacy budget and the gradient sensitivity, specifically including: starting the preset dynamic privacy budget adjustment mechanism, which needs to combine the information of three core dimensions for budget allocation, one is the current training iteration number, which needs to first clarify the total training rounds, such as presetting the total rounds to 200 rounds, if it is in the early stage of training, such as the first 50 rounds, because the model needs to converge quickly, a larger privacy budget can be allocated, such as 40% of the total budget to reduce the impact of noise on the effectiveness of the gradient; if it is in the middle and later stage of training, such as after 150 rounds, the model has approached convergence, a smaller privacy budget can be allocated, such as 10% of the total budget to save privacy cost; two is the model convergence state, which is judged by monitoring the loss function trend of the local model, if the loss function has a faster decline rate, such as a decline of more than 5% per round, it means that the model is still optimizing quickly, and a higher budget needs to be maintained to ensure the quality of the gradient; if the change rate of the loss function has been lower than the preset threshold, such as a change of not more than 0.5% per round, it means that the model is close to convergence, and the budget can be reduced; three is the remaining total privacy budget, which needs to be calculated in real time the total privacy budget allocated to the previous training rounds, subtract the sum from the initially set total privacy budget to get the remaining budget, to ensure that the budget allocated to the current round does not exceed the remaining budget; based on the analysis of the above three dimensions, calculate and determine the specific privacy budget allocation value for the current training round; then, calculate the gradient sensitivity, because the gradient has been executed norm clipping, the gradient sensitivity can be set as the clipping threshold, that is, the maximum possible change range of the gradient vector; finally, according to the allocated privacy budget of the current round and the gradient sensitivity, determine the standard deviation of the Gaussian noise, generally speaking, the larger the privacy budget and the smaller the gradient sensitivity, the smaller the standard deviation, specifically, the standard deviation value can be adjusted according to the conventional differential privacy noise parameter determination logic in the field, combined with the actual protection requirements, to finally obtain the Gaussian noise parameter, that is, the standard deviation, required for the current round.

[0034] Step 4.3, based on the calculated noise parameters, add random noise conforming to the corresponding Gaussian distribution to the obtained norm-clipped gradient to generate a protected gradient parameter, specifically including: based on the calculated Gaussian noise standard deviation, generate random noise conforming to the Gaussian distribution, and the generation process needs to use a conventional random number generation tool in the field, such as a cryptographic random number generator, to ensure the randomness and unpredictability of the noise, and the mean of the Gaussian distribution is set to 0 and the standard deviation is consistent with the determined value; then, the generated random noise is superimposed with the obtained norm-clipped gradient: for each element in the norm-clipped gradient vector, the corresponding element value in the same position in the random noise is added, that is, the gradient elements and noise elements are added one by one to complete the noise addition operation; in this process, it is necessary to ensure that the dimensions and element quantities of the noise vector and the gradient vector are completely matched to avoid damage to the gradient data due to dimension mismatch; after completing the noise addition, the obtained gradient parameter needs to be checked for effectiveness: on the one hand, check whether the value range of the gradient parameter is in a reasonable interval, and on the other hand, confirm that the gradient parameter can still reflect the update direction of the model parameter; if the verification passes, the gradient parameter is determined as the protected gradient parameter; if the verification fails, the Gaussian noise is regenerated and the addition and verification operations are performed again until the required protected gradient parameter is obtained.

[0035] In the embodiments of the present application, by performing norm clipping after decrypting the encrypted gradient, the gradient is first converted from the ciphertext to the plaintext to accurately calculate the gradient vector norm, and then the gradient is scaled by a certain proportion according to the comparison with the preset clipping threshold, effectively suppressing the abnormal extreme value in the gradient and reducing the privacy leakage risk caused by the too large difference between the gradients; secondly, the traditional fixed privacy budget allocation mode is abandoned, and a dynamic privacy budget adjustment mechanism is relied on to allocate the current round budget in combination with the current training iteration number, model convergence state and remaining total privacy budget, and the Gaussian noise standard deviation is calculated accordingly, realizing the dynamic adaptation of the privacy protection strength and the model training efficiency, and avoiding the problems of insufficient protection or excessive precision loss caused by single budget allocation; finally, based on the noise parameters determined in the foregoing, Gaussian noise is added to the clipped gradient to further mask the original data features in the gradient, blocking the path for attackers to reverse the original private data through the gradient from the gradient level, and the whole process is progressive, ensuring the calculation stability of the gradient itself, and through the dynamic and accurate privacy protection strategy, the gradient privacy security is improved while the effective value of the gradient to the model update is maximized.

[0036] In a preferred embodiment of the present application, the above step 5 can include: Step 5.1, receiving the protected gradient parameters, uploading the protected gradient parameters of each participant to the central server; each participant transmits the gradient parameters added with noise and protected to the central server through a secure communication channel, specifically including: after outputting the protected gradient parameters, each participant needs to start a preset gradient uploading process, first, each participant calls the locally deployed secure communication system, the secure communication system supports a secure communication channel constructed based on an encrypted transmission protocol, and the channel establishment process needs to complete the two-way identity authentication of the participant and the central server to ensure the authenticity of the communication object; then, the participant encapsulates the protected gradient parameters into a data packet conforming to the communication protocol specification, the data packet needs to contain the identification information, timestamp and integrity check code of the gradient parameters to prevent data tampering or replay attacks in the transmission process; after completing the data encapsulation, the data packet is sent to the central server through the secure communication channel established above.

[0037] Step 5.2, based on the gradient parameters of each participant received by the central server, using homomorphic encryption technology for ciphertext aggregation; the central server performs weighted summation operation on the gradient parameters of each participant in the ciphertext state to generate the global model update parameters in the encrypted state, specifically including: after the central server collects the protected gradient parameters of all participants in the current training round, if these gradient parameters are in the encrypted state, it directly starts the ciphertext aggregation process; first, the central server calls the ciphertext calculation module corresponding to the homomorphic encryption algorithm, which supports basic operations such as addition and multiplication in the ciphertext state; then, based on the preset initial weight, which can be set according to the data volume proportion of the participant or the proportion negotiated in advance, the weighted summation operation is performed on the encrypted gradient parameters of each participant, specifically, for each participant's encrypted gradient parameter, first perform homomorphic multiplication operation with the initial weight corresponding to the participant, i.e. multiplication of ciphertext and plaintext, or multiplication of ciphertext and ciphertext, depending on the storage state of the weight, to get the weighted encrypted gradient of the participant; then perform homomorphic addition operation on the weighted encrypted gradients of all participants to combine multiple weighted encrypted gradients into one encrypted result; the encrypted result is the global model update parameter in the encrypted state, and the decrypted value is consistent with the result of performing the same weighted summation operation on the plaintext gradient, and the gradient always exists in the form of ciphertext during the entire aggregation process, avoiding the central server accessing the gradient plaintext information; if the gradient parameter is in the plaintext state protected by privacy, the central server needs to perform temporary encryption processing before performing weighted summation, and then operates according to the above ciphertext aggregation process, finally outputs the global model update parameter in the encrypted state.

[0038] Step 5.3, adjusting the aggregation weight of the global model update parameter based on the dynamic smoothing coefficient mechanism, dynamically calculating the smoothing coefficient according to the data quality, model contribution and historical performance of each participant, adjusting the weight distribution of each participant in the aggregation process, specifically including: the central server starts the preset dynamic smoothing coefficient mechanism, which needs to evaluate the three core indicators of each participant to calculate the smoothing coefficient, one is the data quality, by analyzing the integrity, noise level and relevance of the local data of the participant to the training task, a data quality score between 0 and 1 is generated; two is the model contribution, by comparing the consistency of the gradient parameters of the participant with the historical global gradient, and the performance improvement amplitude when using its gradient alone for model update, a contribution score between 0 and 1 is generated; three is the historical performance, the stability, timeliness and other indicators of the gradient parameters of the participant in the past training rounds are counted, and a historical performance score between 0 and 1 is generated; then, the mechanism calculates the smoothing coefficient of each participant according to the preset index weight, such as data quality accounting for 40%, contribution accounting for 30%, and historical performance accounting for 30%, the three scores are weighted and summed to obtain the smoothing coefficient of each participant, the larger the smoothing coefficient, the higher the weight of the participant in this round of aggregation; further, the aggregation weight of each participant is adjusted according to the smoothing coefficient: the smoothing coefficients of all participants are normalized, that is, the sum of all weights is 1, to obtain the adjusted weight distribution scheme; finally, the central server re-executes the weighted summation operation of the gradient parameters of each participant based on the new weight distribution scheme, and generates the adjusted encrypted state global model update parameter.

[0039] Step 5.4, distributing the adjusted global model update parameter to each participant, and each participant updates the local model using the received parameter to complete the model update of the current training round, specifically including: the central server distributes the adjusted encrypted state global model update parameter to each participant through a secure communication channel, and adds a timestamp, version number and integrity check information to the parameter before distribution, and each participant also needs to verify the integrity and legality of the parameter source after receiving; if the parameter is in encrypted state, the participant uses its own private key to perform decryption operation to obtain the plaintext form of the global model update parameter; then, each participant calls the local model update module to apply the global model update parameter to the local model: according to the preset model update rule, the global model update parameter is used to adjust the corresponding parameters of the local model; after completing the parameter update, the participant needs to perform basic performance test on the updated local model to confirm that the model parameter update is effective and no abnormality occurs; if the test is passed, the current training round is marked as completed, and the local model enters the next round of training ready state; if the test fails, the central server is fed back with abnormal information.

[0040] In the embodiments of the present application, the protected gradient parameters are transmitted through a secure communication channel to ensure that the gradients are not stolen or tampered with during uploading, providing secure input for the aggregation operation of the central server; secondly, relying on homomorphic encryption technology to realize encrypted aggregation, the central server can perform weighted summation on the gradient parameters without decryption, avoiding the leakage of gradient plaintext and ensuring the accuracy of the global model update parameters; thirdly, a dynamic smoothing coefficient mechanism is introduced to dynamically adjust the aggregation weight based on the data quality, model contribution and historical performance of each participant, solving the problem of inferior data dragging model performance under fixed weight and improving the reliability of global parameters; finally, the adjusted global parameters are distributed to each participant to complete local model updating, forming a closed loop of uploading, aggregation and distribution, ensuring the privacy and security of the whole process while achieving efficient iteration of the global model through dynamic weight optimization, providing technical support for the final converged large model that balances privacy protection and performance accuracy.

[0041] In a preferred embodiment of the present application, step 6 can include: Step 6.1, based on the adjusted global model update parameters, each participant updates its local model parameters to complete a round of local model updating, specifically including: after receiving the distributed adjusted global model update parameters, each participant needs to start the local model parameter update process, first, if the parameters are in an encrypted state, the participant uses its own private key to decrypt them to obtain the global model update parameters in plaintext form; then, call the parameter update interface of the local model to fuse the global model update parameters with the current parameters of the local model, the fusion method can be preset according to the model training strategy, for example, for a neural network model, the global update parameters can be superimposed with the local model parameters according to a preset proportion (such as learning rate); after completing the parameter fusion, the updated local model is subjected to basic verification, such as checking whether the parameter dimensions match, whether the key layer parameters are within a reasonable value range, etc., to ensure that no data corruption occurs during the updating process; after the verification is passed, the local model is marked as completing the current round of updating, ready for the next round of training, thus completing a round of iterative optimization of the local model.

[0042] Step 6.2, the process of steps 3 to 5 is repeatedly executed, each participant trains again based on the updated local model and local encrypted data, calculates the model gradient; the model gradient is normally clipped and Gaussian noise is added to obtain the protected gradient parameter; the protected gradient parameter is uploaded to the central server for secure aggregation; the central server adjusts the aggregation weight based on the dynamic smoothing coefficient mechanism and generates a new global model update parameter, which is then distributed to each participant, including: the participant re-executes the model training based on the updated local model and the local encrypted data stored by itself, and completes the forward propagation and reverse propagation in the secure state through the homomorphic encryption characteristic, and calculates the new round of model gradient; then, the newly generated model gradient is normally clipped to remove abnormal extreme values, and the Gaussian noise parameter is determined according to the dynamic privacy budget adjustment mechanism to add noise to the clipped gradient to generate a protected gradient parameter; then, the participant uploads the protected gradient parameter to the central server again through a secure communication channel; after collecting the gradient parameters of all participants, the central server still uses homomorphic encryption technology to perform secure aggregation, and adjusts the aggregation weight based on the dynamic smoothing coefficient mechanism to generate a new round of global model update parameter; finally, the central server distributes the new global model update parameter to each participant, and each participant enters the local model update link after receiving it, and the cycle is repeated to make the model approach the optimal state in multiple iterations.

[0043] Step 6.3, in each iteration process, the central server monitors the change trend of the global model loss function, and judges whether the model converges based on the predefined convergence condition; the convergence condition includes that the change rate of the global loss function value is lower than the preset threshold, or the maximum training round is reached, including: the central server starts the global model convergence monitoring mechanism, first, collects the loss function value of the local model on the validation set uploaded by each participant; then, the central server aggregates the loss values of each participant, such as weighted average, to obtain the global model loss function value of the current round; then, compare the current global loss value with the global loss value of the last round, calculate the change rate of the loss function value, that is, the ratio of the difference between the two rounds of loss values to the last round of loss value; at the same time, the number of completed training rounds is counted; then, compare the calculated loss change rate with the predefined convergence threshold, which is set according to the model type and training task requirements, for example, it can be set to 0.001, that is, when the change rate is less than 0.1%, it is determined to be converged; if both conditions are not met, it is determined that the model has not converged, and the central server continues to initiate the next round of training instruction to drive the process to be repeated.

[0044] Step 6.4, once the model converges, the central server terminates the training process and notifies each participant to save the final local model, and the set of local models of each participant constitutes a large model trained by privacy protection, specifically including: once the central server determines that the global model reaches the convergence condition, it immediately terminates the subsequent training plan and generates a training termination instruction; the instruction is sent to all participants through a secure communication channel, and the instruction needs to include key information such as convergence identifier and final training round; after receiving the termination instruction, each participant stops all processes related to local training, and starts the saving process of the final local model, and all parameters of the current local model, including weights, biases, optimizer states, etc. are encrypted and stored in the local secure storage medium, and the storage process needs to generate an integrity check code for subsequent model call integrity verification; after all participants confirm that the model saving is completed, the local models of all participants constitute an organic whole, and the set of local models collectively form a large model trained by complete privacy protection; the central server then summarizes the model saving state reports of each participant and generates a training summary report, recording key information such as total training rounds, final global loss value, and privacy protection parameters, and thus the privacy protection training process of the large model is completed.

[0045] In the embodiment of the application, the local model is updated based on the adjusted global model update parameters, ensuring that the model of each participant can timely absorb the global optimization results; secondly, by repeatedly executing the core processes of local training, gradient protection, ciphertext aggregation and parameter distribution, a continuous iteration training mechanism is formed, which continuously improves the performance of the model in multiple rounds of optimization, while the privacy protection measures are continued throughout the process, avoiding the risk of privacy leakage in the iteration process; thirdly, relying on the monitoring of the global model loss function by the central server and the predefined convergence condition, it can accurately judge whether the model has reached the optimal state, prevent overfitting or resource waste caused by overtraining, and ensure the controllability and normativity of the training process; finally, the process is terminated in time after the model converges and the local models of each participant are saved, which avoids the security risks that may be brought by centralized storage of the large model, and also retains the results of collaborative training by multiple participants, and finally realizes the balance in the whole iteration period, providing reliable closed-loop protection for the safe construction of the large model knowledge base.

[0046] As shown in Figure 2 The embodiment of the application also provides a data privacy protection system for a large model knowledge base, which comprises: The collection module is configured to acquire original privacy data of each participant, clean and extract features from the original privacy data, and identify sensitive information fields; the collection module is further configured to perform desensitization processing on the fields containing sensitive information by using a generative adversarial network to obtain desensitized data; and the collection module is further configured to perform k-anonymization processing on the desensitized data, divide the data into a plurality of equivalence classes based on quasi-identifiers, and perform generalization processing on classes with a record number less than k to obtain an anonymized data set. The modeling module is configured to add Laplace noise to the anonymized data set by using a differential privacy mechanism to obtain noisy privacy data; the modeling module is further configured to perform homomorphic encryption on the noisy privacy data and store the encrypted data locally, and construct a federated learning framework in the vertical direction, and each participant trains a local model based on the local encrypted data and calculates model gradients. The aggregation module is configured to perform norm clipping processing on the model gradients, add Gaussian noise to the gradients according to a dynamic privacy budget adjustment mechanism to obtain protected gradient parameters; the aggregation module is further configured to upload the protected gradient parameters to a central server, perform ciphertext aggregation on the received gradient parameters by using a homomorphic encryption technology to generate global model update parameters; and the aggregation module is further configured to adjust the aggregation weight of the global model update parameters based on a dynamic smoothing coefficient mechanism, and distribute the adjusted parameters to each participant to update the local model. The processing module is configured to repeatedly perform the steps of local training, gradient uploading, ciphertext aggregation, and parameter distribution until the model converges, and finally obtain a large model trained by privacy protection.

[0047] It should be noted that the system corresponds to the above method, and all implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0048] Embodiments of the application also provide a computing device, comprising a processor and a memory storing a computer program, wherein the computer program is executed by the processor to perform the method described above. All implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0049] Embodiments of the application also provide a computer-readable storage medium storing instructions, which, when executed on a computer, cause the computer to perform the method described above. All implementation manners in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0050] The data privacy protection method for the large model knowledge base in the application is only for the description of the technical solution, aims to provide the technical idea and implementation path of privacy protection, and does not constitute the compliance guarantee of the data processing behavior in any specific application scenario. In actual application, the application party needs to combine the data security, privacy protection related laws and regulations in the region and the specific business scenario to adapt and verify the compliance of the technical solution, and ensure that it meets all applicable legal requirements.

[0051] The original privacy data acquisition, storage, transmission and other links involved in the technical solution of the application need to be established by the application party to establish a management system that meets the safety standards, including but not limited to data source legality audit, access control, secure communication channel building, key security storage, etc. Data leakage, loss or misuse caused by non-standard data management process of the application party, operation errors, etc. are not related to the technical solution itself, and the related responsibilities are borne by the application party.

[0052] The above is the preferred embodiment of the application. It should be pointed out that for ordinary skilled persons in the art, without departing from the principles of the application, a number of improvements and refinements can be made, which should also be considered within the scope of protection of the application.

Claims

1. A data privacy protection method for large model knowledge bases, characterized in that, The method includes: Step 1: Obtain the original privacy data of each participant, clean and extract features from the original privacy data, and identify sensitive information fields; use generative adversarial networks to de-identify the fields containing sensitive information to obtain de-identified data; Step 2: Perform k-anonymization on the de-identified data, divide the data into multiple equivalence classes based on quasi-identifiers, and perform generalization on classes with fewer than k records to obtain an anonymized dataset; Step 3: Based on the anonymized dataset, Laplacian noise is added using a differential privacy mechanism to obtain noisy privacy data; the noisy privacy data is homomorphically encrypted and stored locally to construct a vertical federated learning framework. Each participant trains its local model based on the local encrypted data and calculates the model gradient. Step 4: Perform norm clipping on the model gradient and add Gaussian noise to the gradient according to the dynamic privacy budget adjustment mechanism to obtain the protected gradient parameters. Step 5: Upload the protected gradient parameters to the central server, use homomorphic encryption technology to perform dense aggregation on the received gradient parameters to generate global model update parameters; adjust the aggregation weight of the global model update parameters based on the dynamic smoothing coefficient mechanism, and distribute the adjusted parameters to each participant to update the local model. Step 6: Repeat the local training, gradient uploading, dense state aggregation, and parameter distribution steps until the model converges, and finally obtain a large model trained with privacy protection.

2. The data privacy protection method for large model knowledge bases according to claim 1, characterized in that, Obtain the raw privacy data of each participant, clean and extract features from the raw privacy data, and identify sensitive information fields; Generative adversarial networks are used to de-identify fields containing sensitive information, resulting in de-identified data, including: We obtain raw privacy data from all participating parties, perform data cleaning on the raw privacy data, including removing duplicate records, handling missing and outlier values, and standardizing the data. Feature extraction is performed on the cleaned data, and sensitive information fields are identified based on a predefined sensitive information rule base. Sensitive information fields include personal identification information, health information, or financial information. Generative adversarial networks (GANs) are used to de-identify sensitive information fields. The GAN consists of a generator and a discriminator. The generator generates synthetic data to replace the original sensitive information, and the discriminator distinguishes between real data and synthetic data. The generator is optimized through adversarial training so that the de-identified data retains the statistical characteristics of the original data and cannot directly identify individuals, thus obtaining the de-identified data.

3. The data privacy protection method for large model knowledge bases according to claim 2, characterized in that, The anonymized data is subjected to k-anonymization. Based on quasi-identifiers, the data is divided into multiple equivalence classes, and generalization is performed on classes with fewer than k records to obtain an anonymized dataset, including: Based on the anonymized data, and by dividing the data according to a predefined set of quasi-identifiers, which consists of attribute fields that can indirectly identify individuals, the values ​​of the quasi-identifiers are obtained. Based on the value of the quasi-identifier, the data records are divided into several equivalence classes, such that each equivalence class contains at least k records; For equivalence classes with fewer than k records, generalization is performed by increasing the level of data abstraction or merging them into similar equivalence classes to meet the k-anonymity requirement, and an anonymous dataset that satisfies k-anonymity is output.

4. The data privacy protection method for large model knowledge bases according to claim 3, characterized in that, Step 3 includes: Based on anonymized datasets, Laplace noise is added to numerical data in the datasets using a differential privacy mechanism according to a predefined global privacy budget. By calculating the global sensitivity of the data query function, the scale parameter of the Laplace noise is determined according to the privacy budget and sensitivity. Random noise following a Laplace distribution of the scale parameter is added to each numerical data item to obtain the privacy data with added noise. The noisy privacy data is encrypted using a homomorphic encryption algorithm; a public key and a private key are generated for homomorphic encryption, with the public key distributed to each participant and the private key securely stored by the data owner; the noisy privacy data is then homomorphically encrypted using the public key, converting the plaintext data into ciphertext data; and the resulting ciphertext data is distributed and stored in the local storage systems of each participant. Based on locally stored encrypted data, a vertical federated learning framework is constructed. Each participant uses encrypted data to perform model training locally. Homomorphic encryption is used to perform forward propagation calculation of the model in the encrypted state to obtain the prediction result. Based on the difference between the prediction result and the true label, backpropagation calculation is performed in the encrypted state to obtain the model gradient in the encrypted state.

5. The data privacy protection method for large model knowledge bases according to claim 4, characterized in that, The model gradient is subjected to norm clipping, and Gaussian noise is added to the gradient according to the dynamic privacy budget adjustment mechanism to obtain protected gradient parameters, including: The model gradient in the encrypted state is decrypted to obtain the plaintext gradient. Each gradient vector is extracted from the plaintext gradient, the norm of each gradient vector is calculated, and the norm of the gradient vector is compared with a preset clipping threshold. For gradient vectors whose norm exceeds the threshold, they are scaled proportionally to make their norm equal to the clipping threshold, thus obtaining the norm-clipping gradient. Based on the gradient after norm clipping, the Gaussian noise parameters required for the current training round are determined according to the dynamic privacy budget adjustment mechanism. The dynamic privacy budget adjustment mechanism dynamically calculates and allocates the privacy budget for the current round based on the current number of training iterations, the model convergence state, and the remaining total privacy budget. The standard deviation of the Gaussian noise is calculated based on the allocated privacy budget and gradient sensitivity. Based on the calculated noise parameters, random noise following the corresponding Gaussian distribution is added to the obtained norm-trimmed gradient to generate protected gradient parameters.

6. The data privacy protection method for large model knowledge bases according to claim 5, characterized in that, The protected gradient parameters are uploaded to the central server. Homomorphic encryption is used to perform dense-state aggregation on the received gradient parameters to generate global model update parameters. The aggregation weights of the global model update parameters are adjusted based on a dynamic smoothing coefficient mechanism, and the adjusted parameters are distributed to each participant to update their local model, including: The protected gradient parameters are received, and the protected gradient parameters of each participant are uploaded to the central server. Each participant transmits the gradient parameters that have been noise-added and protected to the central server through a secure communication channel. Based on the gradient parameters of each participant received by the central server, homomorphic encryption technology is used to perform encrypted aggregation; the central server performs a weighted summation operation on the gradient parameters of each participant in the encrypted state to generate global model update parameters in the encrypted state. The aggregation weights of the global model update parameters are adjusted based on the dynamic smoothing coefficient mechanism. The smoothing coefficients are dynamically calculated based on the data quality, model contribution, and historical performance of each participant, and the weight allocation of each participant in the aggregation process is adjusted accordingly. The adjusted global model update parameters are distributed to each participant, and each participant uses the received parameters to update its local model, completing the model update for the current training round.

7. The data privacy protection method for large model knowledge bases according to claim 6, characterized in that, Repeat the steps of local training, gradient uploading, dense-state aggregation, and parameter distribution until the model converges, ultimately obtaining a large model trained with privacy protection, including: Based on the adjusted global model update parameters, each participant updates its local model parameters, completing one round of local model update. Repeat steps 3 to 5. Each participant retrains the model based on the updated local model and local encrypted data, and calculates the model gradient. Norm clipping is performed on the model gradient and Gaussian noise is added to obtain the protected gradient parameters. The protected gradient parameters are uploaded to the central server for dense aggregation. The central server adjusts the aggregation weights based on the dynamic smoothing coefficient mechanism and generates new global model update parameters, which are then distributed to each participant. During each iteration, the central server monitors the changing trend of the global model loss function and determines whether the model has converged based on predefined convergence conditions. Convergence conditions include the rate of change of the global loss function value being lower than a preset threshold or reaching a preset maximum number of training rounds. Once the model is determined to have converged, the central server terminates the training process and notifies each participant to save the final local model. The collection of local models from each participant constitutes the large model trained with privacy protection.

8. A data privacy protection system for large model knowledge bases, the system implementing the method as described in any one of claims 1 to 7, characterized in that, include: The data collection module is used to acquire the raw privacy data of each participant, clean and extract features from the raw privacy data, and identify sensitive information fields. Generative adversarial networks are used to de-identify fields containing sensitive information, resulting in de-identified data. The anonymized data is subjected to k-anonymization. The data is divided into multiple equivalence classes based on quasi-identifiers, and the classes with fewer than k records are generalized to obtain the anonymized dataset. The modeling module is used to add Laplace noise to the anonymized dataset using a differential privacy mechanism to obtain noisy privacy data. The noisy privacy data is homomorphically encrypted and stored locally. A vertical federated learning framework is constructed, and each participant trains a local model based on the local encrypted data and calculates the model gradient. The aggregation module performs norm clipping on the model gradients, adds Gaussian noise to the gradients according to a dynamic privacy budget adjustment mechanism to obtain protected gradient parameters, uploads the protected gradient parameters to the central server, performs dense aggregation on the received gradient parameters using homomorphic encryption technology to generate global model update parameters, adjusts the aggregation weights of the global model update parameters based on a dynamic smoothing coefficient mechanism, and distributes the adjusted parameters to each participant to update their local models. The processing module is used to repeatedly execute the steps of local training, gradient uploading, dense state aggregation, and parameter distribution until the model converges, and finally obtains a large model trained with privacy protection.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images