A system and method for privacy protection and performance optimization of large contract models

Abstract

The present invention discloses a system and method for privacy protection and performance optimization of large contract models. The system dynamically optimizes the feature extraction module of the large contract model, allocates specific Dropout rates and privacy budgets to different features, and injects more noise into high-sensitivity features in combination with the gradient sensitivity allocation mechanism. Through the joint optimization loss function, the model performance, Dropout loss and privacy budget control are integrated into a unified goal to ensure that the model performance is maximized under the privacy budget. In the inference stage, an improved dynamic mapping mechanism is adopted to integrate the privacy budget into the classification vector generation to reduce the potential risk of privacy leakage. The present invention can greatly improve the privacy protection level of sensitive data in the contract while ensuring the efficient feature learning ability of the large contract model. It is suitable for scenarios such as contract review and legal analysis, and provides technical support for the safe and reliable application of large contract models.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a system and method for privacy protection and performance optimization of large contract models. Background Art

[0002] The contract big model is a type of artificial intelligence model designed specifically for processing formal texts such as legal contracts and business agreements. Through a deep learning framework, this type of model can learn the rules in tasks such as key clause extraction, default risk prediction, and compliance analysis from massive contract data, providing important support for the legal and commercial fields. However, since contract texts usually contain a large amount of sensitive information, such as company secrets, party identity information, and transaction amounts, how to achieve privacy protection during model training and reasoning has become an urgent problem to be solved.

[0003] Current privacy protection technologies include Differential Privacy (DP) and Laplacian Noise Injection. Differential privacy refers to the addition of random noise to the output of training data to ensure that changes in a single sample will not significantly affect the output of the model, thereby achieving privacy protection. This technology provides strict guarantees for data security in theory, but in the contract generation scenario, too much noise may affect the readability and logic of the generated content, resulting in output that does not meet the high requirements of the contract text. Laplace noise injection refers to the addition of noise to contract features to achieve privacy protection, but when processing particularly sensitive fields in the contract (such as party information and amount fields), the intensity of the noise is difficult to dynamically adapt to the importance of the feature, which can easily cause the loss of key information in the generated results.

[0004] In addition, existing privacy protection mechanisms often adopt fixed strategies (such as a uniform dropout rate or a static privacy budget), which fail to fully incorporate the sensitivity and contextual relationships of features in contract data, resulting in insufficient protection of some fields and excessive protection of other fields, affecting the overall generation effect.

[0005] Through the use of the existing large-scale contract text generation model, it is found that there is a contradiction between privacy protection and generation performance in the existing large-scale model, which is mainly reflected in the following aspects: Feature loss problem: Too high a Dropout rate may cause the model to ignore important contract terms features during the training process, thereby affecting the accuracy and consistency of the generated contract terms. Insufficient protection of sensitive information: A fixed privacy budget cannot dynamically adapt to the sensitivity of different fields in the contract data, resulting in sensitive information (such as the name of the party or the amount) facing the risk of leakage during the reasoning stage. The cost of privacy protection is too high: Although existing technologies can achieve strong protection through high privacy budgets, the cost is that the generated content contains too much noise, which affects the clarity and interpretability of the contract terms. Summary of the invention

[0006] In view of the above problems existing in the existing technologies for privacy protection and performance optimization of large contract models, the present invention provides a system and method for privacy protection and performance optimization of large contract models based on feature-level privacy protection, adaptive privacy scheduling mechanism, joint optimization loss function and Laplace noise injection technology. The feature-level privacy protection mechanism in the scheme labels sensitive fields (such as amount, name of the party) in the contract data, allocates specific privacy budgets to different features, avoids the leakage of sensitive information, and maximizes the generation performance of non-sensitive information; the adaptive privacy scheduling mechanism uses gradient sensitivity to dynamically adjust the dropout rate and privacy budget of each feature, enhances privacy protection, and takes into account the quality of generated content during model training.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A system for privacy protection and performance optimization of large contract models, including a feature-level privacy protection module, an adaptive privacy scheduling module, a joint optimization module, and a privacy-based contract generation module;

[0009] The feature-level privacy protection module is unidirectionally connected to the adaptive privacy scheduling module, the adaptive privacy scheduling module is unidirectionally connected to the joint optimization module, and the joint optimization module is unidirectionally connected to the privacy contract generation module;

[0010] The feature-level privacy protection module is used to mark sensitive fields in the contract data and generate feature privacy labels;

[0011] The adaptive privacy scheduling module dynamically adjusts the dropout rate and privacy budget according to feature sensitivity and gradient changes to ensure that the privacy protection strength matches the feature importance;

[0012] The joint optimization module balances the privacy protection loss and the generation performance loss by constructing a joint loss function, thereby optimizing privacy protection and model performance.

[0013] The privacy-preserving contract generation module generates privacy-preserving content based on the optimized model to ensure the authenticity and privacy of the generated contract text.

[0014] A method based on the above-mentioned large contract model privacy protection and performance optimization system includes the following steps:

[0015] Step A: Preprocess and analyze the contract data, extract sensitive features and build a data set containing sensitive fields;

[0016] Step B: Based on the joint optimization loss function, the privacy protection and performance of the contract features are optimized by dynamically allocating the dropout rate and privacy budget;

[0017] Step C: Use gradient sensitivity to dynamically schedule the privacy protection mechanism, inject Laplace noise into different features and generate privacy features;

[0018] Step D: Output privacy features to meet privacy protection requirements and ensure data availability.

[0019] Preferably, step A specifically comprises the following steps:

[0020] Step A1: Perform word segmentation and semantic analysis on the contract text TTT and extract feature representation x text :x text =F BERT (T)

[0021] Among them, F BERT (T) represents the semantic embedding of text extracted using the BERT model;

[0022] Step A2: For a table or value field C = {c1, c2, ..., c n}, extract numerical features x num :x num =[f(c1), f(c2), ..., f(c n )], where f(c i ) is the standardization function for numeric fields, in this case normalization.

[0023] Step A3: For x test and x num The sensitive fields in the code (such as the name of the party and the amount) are marked to generate privacy labels t i :

[0024] Step A4: Transform the text feature x test and numerical features x num Fusion, construct complete feature representation X: X = Concat (xtext, xnum);

[0025] Preferably, the step B specifically comprises the following steps:

[0026] Step B1: Define the joint loss function:

[0027] L model : The main loss function of the model task, such as the classification task uses cross entropy loss,

[0028] where y j is the true label, p j is the predicted probability;

[0029] Dropout loss: Used to measure the original feature x i With Dropout features differences;

[0030] Privacy budget loss: Weighing feature x i The size of the privacy budget. The higher the privacy budget, the smaller the loss.

[0031] λ1,λ2: adjust the balance between the three losses;

[0032] Step B2: Dynamically optimize the Dropout rate and privacy budget, using gradient descent to optimize w i and ∈ i :

[0033] Among them, η is the learning rate, t is the current iteration number;

[0034] Preferably, step C specifically comprises the following steps:

[0035] Step C1: Calculate the gradient sensitivity η for each feature i : γ is a scaling factor used to control the effect of sensitivity on noise injection;

[0036] Step C2: Assign Dropout rate w according to gradient sensitivity i :

[0037] Step C3: Inject Laplace noise into each feature to generate privacy features

[0038] Among them, ∈ i is the privacy budget, defined as:

[0039] Step C4: Process all features to generate a privacy feature set:

[0040] Preferably, step D specifically comprises the following steps:

[0041] Step D1: Verify the validity of the privacy feature, including the following two parts:

[0042] Privacy protection effect: verify the privacy level of Laplace noise injection and calculate the privacy leakage probability P leak :P leak =exp(-∈ i ·Δx i ) is the difference between adjacent samples;

[0043] Model performance: Verify the impact of privacy features on model performance, using accuracy or F1 score as an indicator;

[0044] Step D2: Output the privacy feature set For secure storage.

[0045] As a preference, an improved Gumbel-Softmax mechanism is used for classification tasks:

[0046] Where, the privacy budget ∈ i Directly affects the classification probability distribution and improves the dynamics and controllability of privacy protection;

[0047] g j ~Gumbel(0,1): Gumbel noise is randomly generated.

[0048] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0049] 1. The present invention provides a system and method for privacy protection and performance optimization of large contract models. Through the full-process collaborative operation of the feature-level privacy protection module, the adaptive privacy scheduling module and the joint optimization module, the large contract model can achieve a balance between privacy protection and performance optimization during the training process. The feature-level privacy protection module can accurately identify sensitive fields in the contract data (such as the amount, the name of the party involved), and assign specific privacy labels to them. In combination with the adaptive privacy scheduling module, the privacy budget and dropout rate are dynamically adjusted, thereby effectively avoiding privacy leakage problems, while retaining data features to the maximum extent to support contract generation tasks.

[0050] 2. The present invention effectively solves the problem of static privacy budget allocation and the resulting performance degradation in traditional privacy protection methods. Traditional methods use a unified privacy protection strategy for all fields, lacking a dynamic trade-off between the importance and sensitivity of specific fields. The present invention calculates the gradient sensitivity of features and dynamically adjusts the Dropout rate and privacy budget, thereby enhancing the privacy protection of sensitive fields, while non-sensitive fields focus more on retaining generation performance, significantly optimizing the relationship between privacy protection and generation performance. At the same time, the present invention constructs a joint loss function of privacy protection and performance optimization through a joint optimization module, and simultaneously considers the quality of privacy protection and contract generation tasks during model training, thereby achieving higher training efficiency and model performance.

[0051] 3. The present invention only includes the training phase, and realizes the collaborative work of privacy protection and performance optimization in the training phase. Through the feature-level privacy protection mechanism, the contract data is first subjected to field sensitivity analysis, and the sensitive field set is annotated; through the adaptive privacy scheduling mechanism, the Dropout rate and privacy budget are dynamically allocated according to the gradient sensitivity of the sensitive field, and Laplace noise is added to the sensitive field to further improve the privacy protection effect; finally, through the joint optimization module, the comprehensive trade-off between privacy loss and generation loss is combined to optimize the model parameters and improve the adaptability of the model to the contract generation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0053] Figure 2 is a flow chart of the method of the present invention;

[0054] Figure 3 This is the contract data preprocessing activity diagram in this embodiment;

[0055] Figure 4 This is a privacy protection activity diagram in this embodiment;

[0056] Figure 5 This is an activity diagram of the noise injection processing in this embodiment;

[0057] Figure 6 This is the data privacy performance verification activity diagram in this embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, the present invention provides a system for privacy protection and performance optimization of large contract models, including a contract data preprocessing module, a privacy protection and performance optimization module, a dynamic privacy injection module and a privacy feature output module.

[0061] The contract data preprocessing module is used to parse and fuse the contract text and numerical features, extract sensitive features and construct a feature set containing privacy labels; the privacy protection and performance optimization module dynamically allocates the Dropout rate and privacy budget based on the trade-off between model performance and privacy protection by jointly optimizing the loss function; the dynamic privacy injection module injects Laplace noise into different features based on gradient sensitivity to generate privacy features; the privacy feature output module verifies the privacy features to ensure a balance between privacy protection requirements and the performance indicators of the contract model.

[0062] In Example 1, we assume that the application scenario is data mining and analysis of corporate contracts, generating a structured summary data report containing the amount, signing time, contract terms, and contracting parties. The initial input in this scenario is a corporate contract document containing a large amount of sensitive information, such as the signing amount, tax information, and company name; the pre-output is a structured report that has been processed for privacy protection, which ensures that data privacy is not leaked and retains the applicability of contract data to downstream analysis tasks. The most important demand is to optimize the output performance of the large contract model while meeting strict privacy protection requirements, so this scenario is one of the ideal implementation scenarios of the present invention.

[0063] In Example 1, the present invention divides the system into the following four main stages: Contract data preprocessing stage: perform word segmentation and semantic analysis on the input contract text data, extract text features and numerical features, and annotate sensitive fields (such as contract amount and signatory name) with privacy labels. Through feature analysis and fusion, a feature set containing privacy labels is generated to provide a data basis for subsequent privacy optimization. Privacy protection and performance optimization stage: This stage combines privacy protection (Laplacian noise injection) with model performance (cross entropy loss) by defining a joint loss function, and dynamically adjusts the Dropout rate and privacy budget of the feature. The optimization of the joint loss function ensures a balance between privacy protection and model performance. Dynamic privacy injection stage: Gradient sensitivity is used to allocate privacy budgets for different features, and noise is added to sensitive fields based on the Laplace mechanism to generate privacy features. The size of the injected noise is dynamically scheduled by sensitivity and privacy budget, which not only reduces the risk of leakage of sensitive information, but also retains the feature distribution of the data. Privacy feature verification and output stage: The privacy feature set is verified, including the evaluation of the privacy protection level and the evaluation of model performance. The privacy-protected feature set is finally output for subsequent contract analysis or storage.

[0064] Example 2

[0065] like Figure 2 and Figure 3 As shown, the present invention also provides a method for privacy protection and performance optimization of large contract models, which is suitable for enterprise contract data processing and privacy analysis tasks. The preset scenario of Example 2 is the extraction and analysis of key features of enterprise contracts. The specific task forms include the privacy processing of sensitive fields in the contract (such as amount, name of the contracting party) and the generation of summaries of contract terms. The initial input is the contract text data containing sensitive information provided by the enterprise user, and the pre-output is a set of contract features after privacy protection and performance optimization. The application of large models in this scenario requires both privacy protection and performance, with high requirements for privacy protection, and also has high requirements for the accuracy of extracted features. Therefore, it is one of the typical implementation cases of the invention. The method used for Example 2 includes the following steps:

[0066] Step A: Preprocess and analyze the contract data, extract sensitive features and construct a data set containing privacy labels;

[0067] Step B: Based on the joint optimization loss function, the privacy protection and performance of the contract features are optimized by dynamically allocating the dropout rate and privacy budget;

[0068] Step C: Use gradient sensitivity to dynamically schedule the privacy protection mechanism, inject Laplace noise into different features and generate privacy features;

[0069] Step D: Output the privacy-sensitive feature set to meet privacy protection requirements and ensure data availability.

[0070] like Figure 2 As shown, this embodiment includes a contract data preprocessing stage, a privacy protection and performance optimization stage, a dynamic privacy injection stage, and a privacy feature verification output stage.

[0071] like Figure 3 As shown, in this embodiment, developers and the basic contract model participate together. Developers extract text features by performing word segmentation and semantic analysis on the input contract text, and perform normalization operations on tables or numerical fields to generate numerical features. On this basis, sensitive fields are annotated through manual annotation and privacy identification algorithms to generate privacy labels, and text features are integrated with numerical features to form a complete feature representation. This stage aims to build a data set with privacy annotations to provide a basis for subsequent privacy protection.

[0072] Step A: Preprocessing and feature analysis of contract data specifically includes the following steps:

[0073] Step A1: Perform word segmentation and semantic analysis on the contract text TTT and extract feature representation x text :x text =F BERT (T)

[0074] Among them, F BERT (T) represents the semantic embedding of text extracted using the BERT model;

[0075] Step A2: For a table or value field C = {c1, c2, ..., c n}, extract numerical features x num :x num =[f(c1), f(c2), ..., f(c n )], where f(c i ) is the standardization function for numeric fields, in this case normalization.

[0076] Step A3: For x test and x num The sensitive fields in the code (such as the name of the party and the amount) are marked to generate privacy labels t i :

[0077] Step A4: Transform the text feature x test and numerical features x num Fusion, construct a complete feature representation X: X = Concat (xtext, xnum).

[0078] like Figure 4 As shown in the figure: In this embodiment, in the privacy protection and performance optimization stage, by defining a joint optimization loss function, multiple factors such as task performance, Dropout difference and privacy budget are comprehensively considered. The specific method is to use gradient descent to dynamically optimize the Dropout rate and privacy budget weight, and adjust the balance point between the privacy protection strength and performance indicators of different features. Through this stage, the strength of feature privacy protection can be significantly improved while ensuring the practicality of the data.

[0079] Then proceed to step B, which specifically includes the following contents:

[0080] Step B1: Define the joint loss function:

[0081] L model : The main loss function of the model task, such as the classification task uses cross entropy loss,

[0082] where y j is the true label, p j is the predicted probability;

[0083] Dropout loss: Used to measure the original feature x i With Dropout features differences;

[0084] Privacy budget loss: Weighing feature x i The size of the privacy budget. The higher the privacy budget, the smaller the loss.

[0085] λ1,λ2: adjust the balance between the three losses;

[0086] Step B2: Dynamically optimize the Dropout rate and privacy budget, using gradient descent to optimize w i and ∈ i :

[0087] Among them, η is the learning rate and t is the current iteration number.

[0088] like Figure 5 As shown: In this embodiment, the dynamic privacy injection stage quantifies the gradient sensitivity of each feature, allocates the Dropout rate of the feature, and injects Laplace noise to generate private features according to the privacy budget. In order to ensure the correlation between the privacy of the feature and the original feature, this embodiment uses gradient sensitivity to adjust the Dropout intensity, generates a private feature set by noise injection, and achieves a dynamic balance between privacy and performance.

[0089] The step C specifically comprises the following steps:

[0090] Step C1: Calculate the gradient sensitivity η for each feature i : γ is a scaling factor used to control the effect of sensitivity on noise injection;

[0091] Step C2: Assign Dropout rate w according to gradient sensitivity i :

[0092] Step C3: Inject Laplace noise into each feature to generate privacy features

[0093] Among them, ∈ i is the privacy budget, defined as:

[0094] Step C4: Process all features to generate a privacy feature set:

[0095] like Figure 6 As shown: In this embodiment, the privacy feature verification output stage, in this stage, the developer participates with the optimized privacy protection model. By verifying the effectiveness of the privacy feature, including the privacy protection effect (such as the probability of privacy leakage) and the model performance, it is ensured that the generated privacy feature can not only meet the privacy protection requirements, but also maintain high efficiency and practicality in the task. After verification, the privacy feature set is output for secure storage or further contract data analysis.

[0096] The step D specifically comprises the following steps:

[0097] Step D1: Verify the validity of the privacy feature, including the following two parts:

[0098] Privacy protection effect: verify the privacy level of Laplace noise injection and calculate the privacy leakage probability P leak :P leak =exp(-∈ i ·Δx i ) is the difference between adjacent samples;

[0099] Model performance: Verify the impact of privacy features on model performance, using accuracy or F1 score as an indicator;

[0100] Step D2: Output the privacy feature set For secure storage.

[0101] Use the improved Gumbel-Softmax mechanism for classification tasks:

[0102] Where, the privacy budget ∈ i Directly affects the classification probability distribution and improves the dynamics and controllability of privacy protection;

[0103] g j ~Gumbel(0,1): Gumbel noise is randomly generated.

[0104] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A system for privacy protection and performance optimization of large contract models, characterized by: It includes contract feature extraction module, dynamic privacy protection module, fine-grained privacy scheduling module and content generation optimization module; The contract feature extraction module is unidirectionally connected to the dynamic privacy protection module, the dynamic privacy protection module is unidirectionally connected to the fine-grained privacy scheduling module, and the fine-grained privacy scheduling module is unidirectionally connected to the content generation optimization module; The contract feature extraction module is used to parse the input large model generated content, including feature extraction of key terms, semantic logic and sensitive information in the text as input for subsequent privacy protection and optimization; The dynamic privacy protection module is used to dynamically adjust the privacy protection strategy according to the sensitive features in the content, including allocating appropriate Dropout rate and Laplace noise to important features to achieve privacy processing of sensitive data; The fine-grained privacy scheduling module designs personalized privacy budget allocation schemes for different features based on feature sensitivity and gradient contribution to ensure the accuracy of privacy protection; The content generation optimization module is responsible for correcting the intermediate representation during the large model generation process, and combines the fine-grained privacy protection strategy and the improved Gumbel-Softmax mechanism to optimize the logical consistency and factual output of the generated content, ultimately improving the security and credibility of the generated content; Use the improved Gumbel-Softmax mechanism for classification tasks: Where, the privacy budget ∈ i Directly affects the classification probability distribution and improves the dynamics and controllability of privacy protection; g j ~Gumbel(0,1): Gumbel noise is randomly generated.

2. A method based on the contract-oriented large model privacy protection and performance optimization system according to claim 1, characterized in that: The following steps are involved: Step A: Preprocess and analyze the contract data, extract sensitive features and build a data set containing sensitive fields; Step B: Based on the joint optimization loss function, the privacy protection and performance of the contract features are optimized by dynamically allocating the dropout rate and privacy budget; Step C: Use gradient sensitivity to dynamically schedule the privacy protection mechanism, inject Laplace noise into different features and generate privacy features; Step D: Output privacy features to meet privacy protection requirements and ensure data availability.

3. The method for privacy protection and performance optimization of a large contract model according to claim 2, characterized in that: Step A specifically includes the following steps: Step A1: Perform word segmentation and semantic analysis on the contract text TTT and extract feature representation x text :x text =F BERT (T) Among them, F BERT (T) represents the semantic embedding of text extracted using the BERT model; Step A2: For a table or value field C = {c1, c2, ..., c n }, extract numerical features x num :x num =[f(c1), f(c2), ..., f(c n )], where f(c i ) is the standardization function of the numeric field, in this case normalization; Step A3: For x test and x num The sensitive fields of the party name and amount are marked to generate a privacy label t i : Step A4: Transform the text feature x test and numerical features X num Fusion, construct a complete feature representation X: X = Concat (xtext, xnum).

4. The method for privacy protection and performance optimization of a large contract model according to claim 3, characterized in that: Step B specifically includes the following steps: Step B1: Define the joint loss function: L model : The main loss function of the model task, the classification task uses cross entropy loss, where y j is the true label, p j is the predicted probability; Dropout loss: Used to measure the original feature x i With Dropout features differences; Privacy budget loss: Weighing feature x i The size of the privacy budget. The higher the privacy budget, the smaller the loss. λ1,λ2: adjust the balance between the three losses; Step B2: Dynamically optimize the Dropout rate and privacy budget, using gradient descent to optimize w i and ∈ i : Among them, η is the learning rate and t is the current iteration number.

5. The method for privacy protection and performance optimization of a large contract model according to claim 4, characterized in that: Step C specifically includes the following steps: Step C1: Calculate the gradient sensitivity η for each feature i : γ is a scaling factor used to control the effect of sensitivity on noise injection; Step C2: Assign Dropout rate w according to gradient sensitivity i : Step C3: Inject Laplace noise into each feature to generate privacy features Among them, ∈ i is the privacy budget, defined as: Step C4: Process all features to generate a privacy feature set:

6. The method for privacy protection and performance optimization of a large contract model according to claim 5, characterized in that: Step D specifically includes the following steps: Step D1: Verify the validity of the privacy feature, including the following two parts: Privacy protection effect: verify the privacy level of Laplace noise injection and calculate the privacy leakage probability P leak :P leak =exp(-∈ i Δx i ) is the difference between adjacent samples; Model performance: Verify the impact of privacy features on model performance, using accuracy or F1 score as an indicator; Step D2: Output the privacy feature set For secure storage.

Images