Fine adjustment method for contract review large language model

Abstract

The invention provides a fine tuning method for a contract review large language model, and belongs to the technical field of artificial intelligence, and the method comprises the steps: S1, fusing multi-source laws and regulations, arbitration judgment elements and an industry contract model, and generating a first training data set for supervising fine tuning; s2, training an initial model by using the data set, and obtaining a supervised fine tuning model through adaptive parameter adjustment driven by task complexity and a chained fine tuning strategy decoupled at a task stage; s3, constructing a second training data set containing positive and negative example sample pairs based on expert knowledge; s4, training a reward model for compliance scoring based on the second training data set; and S5, taking the supervision fine tuning model as a strategy model, utilizing signals provided by the reward model, performing reinforcement learning fine tuning by adopting a strategy optimization algorithm with KL divergence constraint, and outputting an optimized contract review model in combination with clause level risk feedback and a high-risk clause penalty mechanism.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for fine-tuning a large language model for contract review based on the fusion of reinforcement learning and legal knowledge, which is particularly suitable for application scenarios such as legal technology and intelligent contract review. Background Technology

[0002] With the development of artificial intelligence technology, Large Language Models (LLMs) are increasingly being applied in legal technology fields such as legal document generation, question-answering systems, and intelligent review. Especially in contract review tasks, using language models to automatically understand contract terms and identify risks has become an important means of improving the efficiency of legal services and reducing compliance costs. However, existing large language models still face several technical bottlenecks in practical applications:

[0003] On the one hand, the model's ability to understand the structure of legal texts is insufficient. Legal clauses typically have complex logical structures and mandatory normative expressions, containing linguistic features such as nested conditions, exceptions, and multi-role obligations. General language models lack precise modeling of these structures, making it difficult to accurately identify contract levels and the division of rights and responsibilities, thus affecting the semantic parsing effectiveness of clauses.

[0004] On the other hand, the models lack professional compliance knowledge when identifying risk clauses. In contract review, many legal risks are not explicit violations, but rather manifest as potential compliance deficiencies such as "vague wording," "missing clauses," or "imbalanced obligations." Current models are often trained based on empirical samples, making it difficult to generalize and identify low-frequency but high-risk compliance issues. Furthermore, when generating suggested clauses, the models often suffer from fluent language but non-standard legal expression and flawed logic, affecting their reliability and usability during the review process.

[0005] Some existing studies have attempted to optimize the model by fine-tuning the model parameters and introducing legal question-and-answer pairs, but there are still the following shortcomings: (1) The training data lacks multi-source legal structure knowledge support and cannot cover the real review points under different contract types and industry scenarios; (2) The training process does not integrate the feedback mechanism of compliance risks, resulting in the model output lacking sensitivity to legal consequences and normative logic; (3) The model optimization objectives lack fine task decomposition and cannot be modeled and optimized separately for review links such as structural understanding, clause compliance, risk positioning and suggestion generation.

[0006] Therefore, there is an urgent need for a new model fine-tuning method that integrates structured legal knowledge, dispute adjudication elements, and industry contract corpora. This method, combined with reinforcement learning mechanisms, introduces legal expertise into the model training process in the form of explicit labels and risk scores, thereby significantly improving the interpretability, professionalism, and practicality of the model in contract review tasks. Summary of the Invention

[0007] In view of the technical defects and drawbacks existing in the prior art, embodiments of the present invention provide a fine-tuning method for a large language model for contract review to overcome the above problems or at least partially solve the above problems, the specific solution of which is as follows;

[0008] As a first aspect of the present invention, a method for fine-tuning a large language model for contract review is provided, characterized by comprising the following steps performed in sequence:

[0009] S1. By integrating multiple sources of laws and regulations, arbitration award elements, and industry contract templates, a first training dataset for monitoring and fine-tuning is generated through structured annotation and semantic nesting mechanisms.

[0010] S2. The initial large language model is trained using the first training dataset. A supervised fine-tuning model is obtained through a chain fine-tuning strategy that decouples task stages from adaptive parameter adjustment driven by task complexity. The chain fine-tuning strategy includes four specialized training stages in sequence: contract structure understanding, compliance clause matching, risk point identification, and review suggestion generation.

[0011] S3. Based on expert knowledge, collect and label sample pairs containing positive and negative contract fragments, wherein risk points are marked for the negative contract fragments and compliance adjustment examples are provided, forming a second training dataset containing legal basis and compliance score;

[0012] S4. Based on the second training dataset, train a reward model, which is configured to score the compliance of the input contract text fragment and its candidate revision text, wherein the reward model is trained using a loss function based on the ranking of paired samples.

[0013] S5. Using the supervised fine-tuning model as the strategy model to be optimized, and the reward model as the reward signal, the strategy model is fine-tuned by reinforcement learning using a strategy optimization algorithm with KL divergence constraints. Clause-level risk feedback and high-risk clause penalty mechanisms are introduced during the fine-tuning process, and finally the optimized contract review model is output.

[0014] Further, step S1 includes:

[0015] The first training dataset is generated by structuring and integrating multiple sources of laws and regulations, arbitration award elements, and industry contract templates, respectively.

[0016] The multi-source laws and regulations are structured, including: formatting the legal text into a tree structure, with the articles as nodes, and performing fine-grained analysis of the articles based on syntactic structure and role extraction. At the same time, a three-layer nested semantic tagging system is introduced to annotate the articles. The three layers of tags include a first layer of tags for annotating the contract life cycle stage, a second layer of tags for annotating the contract application scenario, and a third layer of tags for annotating the contract roles and responsibilities.

[0017] Knowledge extraction of the arbitration award elements includes: extracting disputed case elements from the judgment documents to construct an arbitration element map, and based on the arbitration element map, performing cluster analysis on the disputed points through a clustering algorithm to form a dispute focus network, wherein each disputed point is assigned a weight label;

[0018] The industry contract template is annotated using the three-layer nested semantic tagging system;

[0019] The first training dataset was generated by fusing labeled laws and regulations, a clustered and weighted network of disputed issues, and annotated industry contract templates.

[0020] Furthermore, in step S2, the learning rate for model training is dynamically controlled through task complexity-driven adaptive parameter adjustment, specifically including:

[0021] Calculate a task complexity coefficient based on the contract type, industry sector, and structural complexity of the samples in the current training batch. ;

[0022] Based on the task complexity coefficient Dynamically calculate the adaptive learning rate used for this batch of training. ;

[0023] Wherein, the task complexity coefficient It is the density of the clause structure. semantic variability Industry span index The weighted sum is calculated using the following formula:

[0024]

[0025] Where α1, α2, and α3 are preset weight coefficients;

[0026] The adaptive learning rate The calculation formula is:

[0027]

[0028] in, β is a preset base learning rate, and β is a hyperparameter used to control the sensitivity of the learning rate to changes in task complexity.

[0029] Furthermore, in step S2, training is performed using a task-stage decoupling chain-like fine-tuning strategy. This strategy decouples the contract review task into four sequentially executed specialized stages: structural understanding, compliance matching, risk identification, and recommendation generation. For each stage, a dedicated model structure or loss function is used for chain-like fine-tuning, specifically including:

[0030] In the structural understanding phase, the structural hierarchy of contract clauses is identified through syntactic analysis and role extraction, and an embedding representation is calculated for each clause unit. The calculation formula is as follows:

[0031]

[0032] in For the text of the terms, Number its structural level;

[0033] During the compliance matching phase, a dual-tower model structure is adopted to encode contract terms and corresponding legal provisions separately, and the semantic cosine distance loss is minimized. The training is performed, and the loss function is:

[0034]

[0035] In the risk identification phase, based on triplet samples Training is performed where the triples include risk segments. Risk type and explanation of the reasons The loss function at this stage The weighted sum of classification loss and generation loss:

[0036]

[0037] in Losses are categorized by risk type. The loss for generating the explanatory sentence is λ, which is a hyperparameter.

[0038] During the suggestion generation phase, based on the output of the risk identification phase, compliant contract clause revision suggestions are generated in a sequence-to-sequence manner, and industry corpus templates are incorporated to standardize the format and style of the generated text.

[0039] Further, in step S3, constructing the second training dataset includes:

[0040] Select compliant contract excerpts from real contract texts as positive examples, and attach the corresponding legal provisions to each positive example as the basis for its legality;

[0041] Contract fragments with compliance deficiencies were collected from contract review cases as negative examples. Each negative example was labeled with its risk type and corresponding legal consequences. The risk types included ambiguous clauses, illegal clauses, and missing clauses.

[0042] Based on the positive and negative samples, sample pairs are generated, and each sample pair is assigned a compliance score, with the positive sample having a higher score than the negative sample. This forms structured data containing contract fragments, legal basis, risk labeling, and compliance scores, which serves as the second training dataset.

[0043] Further, step S4 includes:

[0044] Format the samples in the second training dataset obtained in step S3 into quadruplets. ,in This is a fragment of the original contract. For the candidate outputs to be evaluated, Standard compliance outputs provided to experts The reward value is pre-set based on the compliance of the sample. For positive samples, r is set to +1, and for negative samples, r is set to a negative value according to their risk level.

[0045] Train the reward model using formatted quadruples. The input to the reward model is a contract fragment. With candidate output The concatenated sequence is output as a scalar score representing the compliance of the candidate output. ;

[0046] When training the reward model, preference pairs are constructed based on the quadruplet samples. ,in It is superior to The candidate outputs are selected and a loss function based on pairwise sorting is applied. Optimize:

[0047]

[0048] in and For the reward model, the candidate outputs are respectively and The predicted score.

[0049] Furthermore, in step S5, a policy optimization algorithm with KL divergence constraints is used to optimize the policy model. Reinforcement learning fine-tuning is performed by updating the parameters of the policy model by minimizing the following loss function, LR:

[0050]

[0051] in, This indicates that the expectation is based on the current policy model. Generate output , The reward model evaluates the input contract fragment. and generate output The compliance score given This is the current strategy model. Compared with the initial policy model The KL divergence between the initial policy models is used to constrain the magnitude of policy updates, and β is a hyperparameter controlling the strength of the KL divergence constraint. The model is obtained after supervised fine-tuning in step S2.

[0052] Furthermore, in step S5, the clause-level risk feedback mechanism is implemented in the following way:

[0053] Apply the strategy model to the input contract fragment Generated candidate output text Divided into multiple clauses ;

[0054] Using the reward model For each clause Perform independent compliance scoring to obtain clause scores. ;

[0055] According to each clause The type of contract clause pre-determines its importance weight. Clauses involving core rights and obligations are given higher weight;

[0056] By aggregating the weighted scores of all clauses, the total reward value used for policy model updates is calculated. .

[0057] Furthermore, step S5 also includes a high-risk clause penalty and logical consistency verification mechanism, specifically including:

[0058] A set of high-risk clause patterns are predefined, including but not limited to indefinite authorization, asymmetric disclaimers, and ambiguous performance definitions;

[0059] When it is detected that any of the high-risk clause patterns are present in the candidate output generated by the strategy model, a fixed negative penalty term is introduced when calculating the total reward value;

[0060] At the same time, a set of predefined rules for logical consistency of clauses are provided, including: if a contract clause contains a conditional statement, it must correspond to a clear description of the consequences; if a contract clause contains a negative expression, it cannot exempt the contracting party from key obligations.

[0061] The candidate outputs are verified in real time based on the logical consistency rules through a rule engine.

[0062] The verification result is converted into a binary signal, where a positive value indicates successful verification and a negative value indicates unsuccessful verification. This signal is then added to the total reward value as an additional reward item to form the final reward signal used to update the policy model.

[0063] Furthermore, in step S5, when implementing the enhanced fine-tuning, a risk grading enhancement mechanism is also included:

[0064] Based on the network of disputed issues and weight labels extracted from the elements of arbitration awards, risk levels are preset for different types of contract risks;

[0065] When the reward model scores the compliance of clauses, the risk level is incorporated as a weighting factor into the scoring calculation, so that compliance defects corresponding to high risk levels generate stronger negative reward signals.

[0066] The strategy model is optimized based on reward signals incorporating risk level information to improve the model's sensitivity to identifying high-risk compliance issues.

[0067] This invention presents a contract review model fine-tuning method based on reinforcement learning and a compliance reward mechanism. By constructing a two-stage training framework of supervised fine-tuning and reinforcement fine-tuning, and introducing structured legal knowledge, it brings the following multiple significant benefits:

[0068] 1. Significantly improved review accuracy and professionalism: Through targeted four-stage chain training (structural understanding, compliance matching, risk identification, and suggestion generation) during the supervision and fine-tuning phase, the model acquires in-depth legal text structure analysis capabilities and professional compliance knowledge, significantly improving the accuracy of risk identification and the professionalism of suggestion generation.

[0069] 2. Enhanced Model Generalization and Risk Sensitivity: During the fine-tuning phase, the reward model is used to simulate expert judgment. Through compliance reward signals and clause-level risk feedback, the model is guided not only to learn surface patterns but also to deeply understand legal consequences, thereby enabling it to have a stronger ability to identify and generalize low-frequency and hidden compliance risks.

[0070] 3. Output stability and reliability assurance: The strategy of KL divergence constraint is adopted to effectively prevent the model from outputting absurd or incoherent content in pursuit of high rewards during the reinforcement learning process. This ensures the legal logic rigor and linguistic fluency of the generated text and improves the reliability of the model in practical applications. Attached Figure Description

[0071] Figure 1 A flowchart illustrating a method for fine-tuning a large language model for contract review, provided as an embodiment of the invention;

[0072] Figure 2 A flowchart of the overall process for training a large language model for contract review, provided for embodiments of the invention;

[0073] Figure 3 A schematic diagram of a three-layer tagging system for structured legal and contractual corpora provided for embodiments of the invention;

[0074] Figure 4 A schematic diagram of the reward-driven mechanism provided for embodiments of the invention;

[0075] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0076] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0077] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0078] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0079] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0080] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0081] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0082] To address at least one of the technical problems existing in the aforementioned related technologies, this invention provides a method for fine-tuning a large language model for contract review. Figure 1 A flowchart illustrating a fine-tuning method for a large language model for contract review, provided as an embodiment of the present invention, includes the following steps:

[0083] S1. Construction of the supervised fine-tuning dataset: By integrating multi-source laws and regulations, arbitration award elements and industry contract templates, a first training dataset for supervised fine-tuning is generated through structured annotation and semantic nesting mechanisms.

[0084] S2. Supervised fine-tuning: The initial large language model is trained using the first training dataset. A supervised fine-tuning model is obtained through a chain fine-tuning strategy that decouples task stages from adaptive parameter adjustment driven by task complexity. The chain fine-tuning strategy includes four specialized training stages in sequence: contract structure understanding, compliance clause matching, risk point identification, and review suggestion generation.

[0085] S3. Construction of reinforcement learning dataset: Based on expert knowledge, sample pairs containing positive and negative contract fragments are collected and labeled, wherein risk points are labeled for the negative contract fragments and compliance adjustment examples are provided, forming a second training dataset containing legal basis and compliance score;

[0086] S4. Reward Model Training: Based on the second training dataset, a reward model is trained, which is configured to score the compliance of the input contract text fragment and its candidate revision text, wherein the reward model is trained using a loss function based on the ranking of the paired samples.

[0087] S5. Strategy Model Reinforcement and Fine-tuning: Using the supervised fine-tuning model as the strategy model to be optimized, and the reward model as the reward signal, the strategy model is fine-tuned by reinforcement learning using a strategy optimization algorithm with KL divergence constraints. Clause-level risk feedback and high-risk clause penalty mechanisms are introduced during the fine-tuning process, and finally the optimized contract review model is output.

[0088] This invention presents a contract review model fine-tuning method based on reinforcement learning and a compliance reward mechanism. By constructing a two-stage training framework of supervised fine-tuning and reinforcement fine-tuning, and introducing structured legal knowledge, it brings multiple beneficial effects. Specifically: through targeted four-stage chain training (structural understanding, compliance matching, risk identification, and suggestion generation) in the supervised fine-tuning stage, the model acquires deep legal text structure analysis capabilities and professional compliance knowledge, improving the accuracy of risk identification and the professionalism of suggestion generation. In the reinforcement fine-tuning stage, a reward model is used to simulate expert judgment. Through compliance reward signals and clause-level risk feedback, the model is guided not only to learn surface patterns but also to deeply understand legal consequences, thereby possessing stronger identification and generalization capabilities for low-frequency and hidden compliance risks. Finally, a strategy optimization using KL divergence constraints effectively prevents the model from outputting absurd or incoherent content in pursuit of high rewards during reinforcement learning, ensuring the legal logic rigor and linguistic fluency of the generated text, and improving the reliability of the model in practical applications.

[0089] For example, a fintech company needs to automate the compliance review of its massive number of loan contracts. After adopting the above solution:

[0090] In the S1 phase, the system integrates the Contract Law section of the Civil Code, financial regulatory provisions, and relevant court judgments to construct a monitoring and fine-tuning dataset with three layers of labels (such as life cycle: performance; scenario: loan; liability: borrower's obligation to verify the authenticity of information).

[0091] In phase S2, the generalized large language model underwent a four-stage chain-based supervised fine-tuning process. The model successfully learned to identify "acceleration clauses" in contracts and the specific conditions that need to be triggered (structural understanding), and was able to determine whether the stipulation of such clauses exceeded the reasonable scope stipulated by law (compliance matching).

[0092] In stages S3 and S4, experts provided a large number of positive examples (compliant accelerated maturity clauses) and negative examples (such as overly domineering clauses like "Party A has the right to declare the loan due immediately under any circumstances") and trained the reward model.

[0093] In the S5 stage, fine-tuning is performed. When the model generates a low-risk clause revision suggestion, the reward model awards a positive score; if the generated suggestion still contains vague wording (such as "when Party A deems it necessary"), the reward model will award a negative score based on clause-level analysis, and combine KL constraints to ensure that the suggested text is fluent and legal. The final model can accurately locate unreasonable acceleration clauses in loan contracts and generate professional and compliant revision suggestions such as "If the borrower commits any of the material defaults listed in Article X of this contract and fails to rectify the situation after Party A's written reminder, Party A has the right to declare the loan due in advance." This improves review efficiency while significantly reducing the false positive rate.

[0094] Figure 2 This is a flowchart illustrating the overall process of the contract review large language model fine-tuning method described in this invention. The diagram clearly shows the five core steps executed sequentially: supervised fine-tuning dataset construction (S1), supervised fine-tuning (S2), reinforcement learning dataset construction (S3), reward model training (S4), and policy model reinforcement fine-tuning (S5). The diagram clearly indicates the data flow and model parameter flow between each step. The output of supervised fine-tuning dataset construction (S1) serves as the input to supervised fine-tuning (S2). The supervised fine-tuned model output from supervised fine-tuning (S2) and the output from reinforcement learning dataset construction (S3) work together in subsequent steps, ultimately outputting an optimized contract review model through policy model reinforcement fine-tuning (S5). This constitutes a complete training loop that reflects the three-dimensional coupling relationship between "data—task—model."

[0095] In some embodiments, step S1 includes:

[0096] The first training dataset is generated by structuring and integrating multiple sources of laws and regulations, arbitration award elements, and industry contract templates, respectively.

[0097] The multi-source laws and regulations are structured, including: formatting the legal text into a tree structure, with the articles as nodes, and performing fine-grained analysis of the articles based on syntactic structure and role extraction. At the same time, a three-layer nested semantic tagging system is introduced to annotate the articles. The three layers of tags include a first layer of tags for annotating the contract life cycle stage, a second layer of tags for annotating the contract application scenario, and a third layer of tags for annotating the contract roles and responsibilities.

[0098] Knowledge extraction of the arbitration award elements includes: extracting disputed case elements from the judgment documents to construct an arbitration element map, and based on the arbitration element map, performing cluster analysis on the disputed points through a clustering algorithm to form a dispute focus network, wherein each disputed point is assigned a weight label;

[0099] The industry contract template is annotated using the three-layer nested semantic tagging system;

[0100] The first training dataset was generated by fusing labeled laws and regulations, a clustered and weighted network of disputed issues, and annotated industry contract templates.

[0101] In the above embodiments, by processing laws and regulations into a tree structure and introducing a three-layer nested label system, the model learns not isolated text fragments, but structured knowledge with hierarchical relationships and legal logic, thereby improving the interpretability of the model's reasoning and decision-making during review. By constructing a network of dispute focus points and assigning weight labels to dispute points based on actual cases, the training data can focus on high-frequency, high-risk practical issues, guiding the model to prioritize and learn the contractual aspects most likely to cause disputes, significantly improving the model's review targeting and adaptability in different industries and contract types.

[0102] For example, a large e-commerce platform needs to conduct compliance reviews of supplier contracts for tens of thousands of products. The platform uses the method described above to construct a dataset:

[0103] In the legal and regulatory processing stage of S1, the system formats the "E-commerce Law" and "Consumer Rights Protection Law" into a tree structure and labels the relevant clauses on "liability for breach of contract due to discrepancy in product description" (e.g., lifecycle: performance / breach of contract; scenario: sale of goods; liability: seller's warranty liability).

[0104] In the S1 dispute focus network construction stage, the system clusters and analyzes high-frequency dispute points from a massive amount of e-commerce contract dispute judgments, such as "the criteria for determining whether product images do not match the actual product" and "the applicable exceptions to the seven-day no-reason return policy." Based on the number of cases and the consistency of the judgment results, the system assigns a high-weight label to the dispute point of "images do not match the actual product."

[0105] In subsequent model training, the model, with the help of this structured dataset, was able to deeply understand the legal source (corresponding to the regulatory tree) of the obligation of "the seller must ensure that the description of the goods is consistent with the actual product" and its high risk in practice (corresponding to the weight of the disputed points). Therefore, when reviewing a supplier contract, the model can quickly locate the clauses concerning the description of goods and accurately identify vague statements such as "the appearance of the goods shall be subject to the actual product" that attempt to evade responsibility. At the same time, combined with the knowledge of high-weighted disputed points, it generates high-risk warnings and specific compliance modification suggestions. This allows the model not only to discover problems, but also to explain their legal basis and risk level. The review results were highly recognized by the legal team.

[0106] Figure 3 This diagram illustrates the three-layer nested semantic tagging system for structured legal and contractual corpora described in this invention. It specifically demonstrates how multi-source legal and regulatory documents and industry contract templates are structurally processed in the supervised fine-tuning dataset construction step (S1). As shown, legal provisions are organized into a tree structure, with each provision serving as a node to store metadata. Through fine-grained analysis of the syntactic structure and role extraction of the provisions, they are mapped to three layers of semantic tags: the first layer tags indicate the contract lifecycle stage, the second layer tags indicate the contract application scenario, and the third layer tags indicate the contract roles and responsibilities. This tagging system is the core mechanism for achieving deep structural annotation of legal and contractual texts, providing supervisory information rich in legal semantics for subsequent model training.

[0107] In some embodiments, step S2, which dynamically controls the learning rate of model training through task complexity-driven adaptive parameter adjustment, specifically includes:

[0108] Calculate a task complexity coefficient based on the contract type, industry sector, and structural complexity of the samples in the current training batch. ;

[0109] Based on the task complexity coefficient Dynamically calculate the adaptive learning rate used for this batch of training. ;

[0110] Wherein, the task complexity coefficient It is the density of the clause structure. semantic variability Industry span index The weighted sum is calculated using the following formula:

[0111]

[0112] in, , , These are the preset weighting coefficients (i.e., empirically tuned weights, which are defaulted to 0.4, 0.4, and 0.2 in the experiment, respectively).

[0113] Among them, the density of clause structure This is used to measure the density of contractual structural units, where... The number of clauses (or clauses) in a given sample of contract text is usually obtained by segmentation and identification based on rules or models that identify the structural features of legal texts (such as paragraphs, clauses, specific punctuation marks, conjunctions, etc.). This represents the total number of tokens within the same contract text sample. A token is the basic unit of text processing and can be a character, word, or subword. This formula calculates the number of clauses contained within a unit of text length. The higher the value, the more complex the structure of the contract text and the more detailed the division of clauses, and the greater the difficulty for the model to understand its hierarchy and relationships.

[0114] Semantic variability ,in The semantic vector embedding of the i-th sentence in the sample is used to calculate its standard deviation, which represents the strength of the variation.

[0115] Industry Span Index , representing the KL divergence between the current batch of industry corpus and the overall corpus distribution, where, This indicates the number of different industry or field categories involved in the current training batch; This represents the word distribution (or topic distribution) of the i-th industry or field in the current batch of corpus. This can be calculated using word frequency statistics, topic models (such as LDA), etc. This represents the overall word distribution (or topic distribution) across the entire training corpus (or a very large background corpus). Represents the calculation of distribution Relative to distribution The KL divergence measures the degree of difference between the usage habits, terminology distribution, and general context of the i-th industry. This formula calculates the average distribution difference between the industry-specific corpora and the general corpus included in the current batch. The higher the value, the more specialized and dispersed the industry sectors involved in the current batch of contract texts are, or the greater the difference from the general corpus. This requires the model to have broader cross-domain knowledge or stronger domain adaptability.

[0116] The adaptive learning rate The calculation formula is:

[0117]

[0118] in, β is a preset base learning rate, and β is a hyperparameter used to control the sensitivity of the learning rate to changes in task complexity.

[0119] In the above embodiments, a task complexity coefficient is introduced. The learning rate is adaptively adjusted. When faced with complex, semantically variable, or cross-industry challenging samples, the learning step size is automatically reduced, effectively avoiding gradient oscillations or divergence risks during training and ensuring smooth and efficient convergence. This adaptive learning rate mechanism enables the model to learn deeper patterns in complex contract samples in a more refined and robust manner, avoiding overfitting to simple patterns. This improves the model's generalization performance and robustness when facing unfamiliar contract types or complex clause structures.

[0120] For example, the legal department of a multinational corporation needs to train a contract review model capable of handling its global business, encompassing various types such as sales of goods, technology licensing, and engineering contracting. When using the method described above for supervised fine-tuning (S2):

[0121] The system first calculates the complexity of each training batch. For example, a batch of mixed-language template contracts has an industry span index of [missing information]. The density of clauses in a complex engineering contract with numerous nested conditions will be significantly higher than in a single goods purchase and sale contract. and semantic variability It is also relatively high.

[0122] After comprehensive calculation, this complex batch will have a high task complexity coefficient. According to the formula The model will automatically calculate and use a lower learning rate for this batch.

[0123] At this lower learning rate, the model updates parameters more cautiously and meticulously for complex samples, successfully learning the differences between different contract types and the deep logic of complex clauses without experiencing drastic fluctuations in training loss. Ultimately, when processing a novel hybrid contract for a company that combines technology outsourcing and hardware procurement, the model demonstrated excellent review accuracy and stability, accurately identifying the ambiguous intellectual property ownership clauses. In contrast, models trained with a fixed learning rate were prone to misjudgment in such complex contracts.

[0124] In some embodiments, in step S2, training is performed using a task-stage decoupling chain-like fine-tuning strategy. This strategy decouples the contract review task into four sequentially executed specialized stages: structural understanding, compliance matching, risk identification, and recommendation generation. For each stage, a dedicated model structure or loss function is used for chain-like fine-tuning, specifically including:

[0125] In the structural understanding phase, the structural hierarchy of contract clauses is identified through syntactic analysis and role extraction, and an embedding representation is calculated for each clause unit. The calculation formula is as follows:

[0126]

[0127] in For the text of the terms, Number its structural level;

[0128] During the compliance matching phase, a dual-tower model structure is adopted to encode contract terms and corresponding legal provisions separately, and the semantic cosine distance loss is minimized. The training is performed, and the loss function is:

[0129]

[0130] In the risk identification phase, based on triplet samples Training is performed where the triples include risk segments. Risk type and explanation of the reasons The loss function at this stage The weighted sum of classification loss and generation loss:

[0131]

[0132] in Losses are categorized by risk type. For the generation loss of the explanatory sentence, For hyperparameters;

[0133] During the suggestion generation phase, based on the output of the risk identification phase, compliant contract clause revision suggestions are generated in a sequence-to-sequence manner, and industry corpus templates are incorporated to standardize the format and style of the generated text.

[0134] In the above embodiments, through progressive training of "structural understanding -> compliance matching -> risk identification -> suggestion generation," the model, like a legal expert, first understands the contract framework, then compares it with regulations, subsequently identifies risks, and finally provides suggestions. Each step is based on the solid foundation of the previous step, ensuring the professionalism and accuracy of each sub-capability. Decoupled training effectively prevents the model from confusing different task features (such as structural features and compliance features) under a single training objective, effectively solving the "seesaw effect" in multi-task learning (i.e., performance improvement in one task leads to performance decline in another), ensuring that the performance of each sub-task reaches its optimal level. Chain-like fine-tuning ensures that subsequent stages (such as risk identification) can fully utilize and deepen the learning results of previous stages (such as structural understanding), ultimately systematically constructing the model's end-to-end composite review capability, making its generated suggestions not only legal but also consistent with the contract context and logically rigorous.

[0135] For example, a law firm might want to develop a model to assist lawyers in reviewing equity investment agreements, using the methods described above:

[0136] During the structural understanding phase, the model learns from a large number of agreement texts to accurately grasp the standard structure of equity investment agreements (such as the hierarchical relationships of sections like "Whereinafter," "Investment Term Sheet," "Corporate Governance," and "Exit Mechanism"). When an agreement is input, the model can accurately identify that the "Anti-Dilution Clause" is located in the "Shareholder Rights" subsection under the "Investment Term Sheet" chapter.

[0137] During the compliance matching phase, the model learns to compare specific terms in the agreement (such as the "full ratchet anti-dilution clause") with the principles of equal shareholder rights and share issuance pricing in the Company Law and related judicial interpretations.

[0138] In the risk identification phase, based on the previous two phases, the model can determine that the "full ratchet" clause may pose a risk of excessive dilution to the founders' shareholders. Combining the triplet sample, it generates a risk explanation: "If the valuation in this round is high, this clause may lead to excessive dilution of the founders' equity in future low-priced financing."

[0139] During the suggestion generation phase, the model ultimately generates professional suggestion clauses based on the above analysis: "It is recommended to adjust the terms using the 'weighted average method' or to set trigger conditions for the application of the full ratchet clause (such as only triggering it when the valuation of the next round of financing is lower than 80% of the valuation of this round) in order to balance the interests of investors and founders."

[0140] Through this phased, chain-like training, the model provides review opinions that are clearly structured, based on explicit evidence, and offer feasible suggestions, greatly improving the efficiency and quality of lawyers' reviews.

[0141] In some embodiments, step S3, constructing the second training dataset includes:

[0142] Select compliant contract excerpts from real contract texts as positive examples, and attach the corresponding legal provisions to each positive example as the basis for its legality;

[0143] Contract fragments with compliance deficiencies were collected from contract review cases as negative examples. Each negative example was labeled with its risk type and corresponding legal consequences. The risk types included ambiguous clauses, illegal clauses, and missing clauses.

[0144] Based on the positive and negative samples, sample pairs are generated, and each sample pair is assigned a compliance score, with the positive sample having a higher score than the negative sample. This forms structured data containing contract fragments, legal basis, risk labeling, and compliance scores, which serves as the second training dataset.

[0145] In the above embodiments, by systematically collecting positive (compliant) and negative (non-compliant) contract fragments from real-world cases and annotating each sample with clear legal basis and risk labels, high-quality, high-information-density training sample pairs are constructed. This provides clear and accurate learning objectives for the subsequent reward model, avoiding the need to learn vague compliance concepts from scratch, and greatly improving the training efficiency and discrimination accuracy of the reward model. Through clear legal basis (positive examples) and risk types and legal consequences (negative examples), a clear legal compliance boundary is established for the model. This enables the model not only to judge "right or wrong," but also to understand "why it is right and why it is wrong." Thus, when faced with complex or novel contract terms, it can perform reasoning based on legal principles, rather than simple pattern matching. Finally, through preset compliance scoring rules (high score for positive examples, low score for negative examples), a stable and consistent initial reward signal baseline is provided for reinforcement learning. This effectively avoids the problem of unstable or divergent policy model training caused by sparse reward signals or excessive noise in the early stages of training, laying a reliable foundation for subsequent fine-tuning.

[0146] For example, a cloud service provider needs to optimize its contract review model to accurately identify compliance risks in service agreements (SLAs). The reinforcement learning dataset (S3) is constructed using the methods described above:

[0147] In selecting positive examples, experts chose excerpts from the "Data Security and Confidentiality" clauses in standard agreements that had passed legal review, and attached the legal basis for these clauses, such as "The data encryption requirements of this clause comply with Article 21 of the Cybersecurity Law regarding the security protection obligations of network operators, as well as the relevant standards of ISO 27001 certification."

[0148] During the collection of negative examples, experts identified a non-compliant disclaimer clause in an older agreement from a customer complaint case: "The service provider shall not be liable for any data breaches caused by third-party cyberattacks." Experts labeled this clause a "violation of the law" and explained the legal consequences: "This clause, by completely exempting itself from its core security obligations, violates Article 497 of the Civil Code regarding the invalidity of standard clauses and may be deemed invalid."

[0149] The system then generates sample pairs of positive and negative examples, assigning compliance scores based on rules (+1 for positive examples, -1 for negative examples). This structured dataset (the second training dataset) clearly demonstrates which commitments are compliant and which exemptions are illegal and invalid regarding "data security responsibility." When this dataset is used for subsequent reward model training, the model can quickly and accurately learn to determine the compliance of relevant clauses in the SLA.

[0150] In some embodiments, step S4 includes:

[0151] Format the samples in the second training dataset obtained in step S3 into quadruplets. ,in This is a fragment of the original contract. For the candidate outputs to be evaluated, Standard compliance outputs provided to experts The reward value is pre-set based on the compliance of the sample. For positive samples, r is set to +1, and for negative samples, r is set to a negative value according to their risk level.

[0152] Train the reward model using formatted quadruples. The input to the reward model is a contract fragment. With candidate output The concatenated sequence is output as a scalar score representing the compliance of the candidate output. ;

[0153] When training the reward model, preference pairs are constructed based on the quadruplet samples. ,in It is superior to The candidate outputs are selected and a loss function based on pairwise sorting is applied. Optimize:

[0154]

[0155] in and For the reward model, the candidate outputs are respectively and The predicted score.

[0156] In the above embodiments, the reward model does not perform a simple "compliance / violation" binary classification, but learns to rank the relative merits of multiple candidate outputs. This allows the model to capture subtle differences in the degree of compliance (e.g., a clause with slight ambiguity vs. a clause that is seriously illegal), thereby providing more refined scores that are closer to expert thinking and guiding the strategy model to generate higher quality outputs.

[0157] Pairwise Ranking Loss is trained by comparing the relative preferences of sample pairs. Its learning objective (A is better than B) is more robust than directly regressing an absolute score. Even if the absolute reward value (r) labeled by experts has a certain degree of subjectivity, as long as the preference order is correct, the model can still learn effectively, reducing the sensitivity of training to label noise.

[0158] This method allows limited expert attention to be focused on determining "which version is better," which is far more efficient than requiring experts to precisely score every possible output. This formatted preference data can be efficiently learned by reward models, significantly reducing the cost of obtaining high-quality supervision signals.

[0159] Continuing from the previous example, the cloud service provider uses the constructed dataset to train the reward model using the method described above (S4):

[0160] After formatting the training samples, the system constructs a four-tuple consisting of contract fragment x, a candidate revision clause y1 generated by the model, a standard clause y* provided by the expert, and the corresponding reward value r. For example, for a problematic disclaimer clause, the model might generate two candidate revisions:

[0161] y1: "The service provider will take commercially reasonable measures to prevent cyberattacks, but will not be liable for losses caused by force majeure or attacks by third parties." (This still constitutes an improper disclaimer.)

[0162] y2 (provided by experts): "The service provider will take encryption and security measures that comply with industry standards to protect user data. If data leakage occurs due to the service provider's failure to fulfill the above security obligations, the service provider shall bear corresponding compensation liability in accordance with the law." (Compliance)

[0163] During reward model training, the system constructs preference pairs (y2, y1), indicating that y2 is significantly better than y1. Reward Model The learning objective is not to memorize that the score of y2 is +1 and the score of y1 is -0.5, but to learn to judge: for the same contract problem x, the compliance of outputting y2 should be much higher than that of outputting y1.

[0164] By minimizing the Pairwise Ranking Loss across a massive number of such preference pairs, the reward model gradually internalizes the standards for expert judgment on compliance. Ultimately, when the strategy model generates a clause such as "the service provider shall bear responsibility for data breaches caused by obvious deficiencies in its security measures," the reward model can award a high positive score; conversely, if the strategy model generates a clause containing the phrase "not liable," the reward model will award a strong negative score. This refined scoring capability greatly improves the accuracy of guiding the optimization direction of the strategy model.

[0165] In some embodiments, in step S5, a policy optimization algorithm with KL divergence constraints is used to optimize the policy model. Reinforcement learning fine-tuning is performed by updating the parameters of the policy model by minimizing the following loss function, LR:

[0166]

[0167] in, This indicates that the expectation is based on the current policy model. Generate output , The reward model evaluates the input contract fragment. and generate output The compliance score given This is the current strategy model. Compared with the initial policy model The KL divergence between the initial policy models is used to constrain the magnitude of policy updates, and β is a hyperparameter controlling the strength of the KL divergence constraint. The model is obtained after supervised fine-tuning in step S2.

[0168] In the above embodiments, the KL divergence constraint is used as a regularization term to enforce the updated policy model. Compared with the initial policy model The output distribution remains within a certain similarity range, effectively preventing the policy model from going out of control in pursuit of high rewards during reinforcement learning, such as getting stuck in continuously generating seemingly compliant but semantically repetitive or absurd text patterns (pattern collapse). This ensures the rationality and controllability of the model's output behavior; the initial policy model The model (i.e., the model after supervision and fine-tuning) already possesses good language generation capabilities and basic legal knowledge. KL constraints ensure that the reinforcement and fine-tuning is carried out on this high-quality basis as "fine-tuning" rather than "reconstruction", thereby always maintaining the language fluency, grammatical correctness and basic legal logic of the generated text, avoiding the risk of sacrificing the readability and logic of the text in pursuit of compliance.

[0169] This constraint allows the policy model to initialize the policy. Instead of blindly exploring the entire policy space, the system effectively explores the vicinity of the "safe zone" to find optimization directions that yield higher rewards. This significantly improves the efficiency and security of reinforcement learning training, making it more suitable for legal text generation scenarios where output stability and reliability are extremely important.

[0170] For example, a company fine-tunes its contract review model using the method described in claim 1, applying the KL constraint strategy optimization of the above embodiment in step S5:

[0171] Initial state: Initial policy model After monitoring and fine-tuning (S2), it can generate coherent and basically compliant clauses. For example, regarding "acceptance criteria", it may generate: "After completing the project development, Party B shall notify Party A to conduct acceptance. The acceptance criteria shall be based on the technical specifications confirmed by both parties in advance."

[0172] The Risk of Unconstrained Practice: Without KL constraints, reinforcement learning models may discover a "shortcut": by generating extremely conservative clauses containing numerous absolute disclaimers, they can obtain high compliance scores from reward models. For example, it might degenerate into generating: "Party A has the sole and final say on the acceptance results, and Party B has no objection." While this clause seems to avoid all risks, its language is rigid, and the rights and obligations are severely unequal, making it impractical in real-world business scenarios.

[0173] The role of KL constraints: Under KL constraints, the loss function... The second term in the model is a penalty policy. Generation and The output text has excessively large distribution differences; therefore, the model is guided to maintain... Based on sound language and legal knowledge, the model optimizes compliance, ultimately generating the following output: "Acceptance shall be conducted jointly by representatives of both parties, and the acceptance standard shall be based on the technical specifications confirmed by both parties' signatures. If there are minor deviations, Party B shall rectify them within a specified period; if there are significant deviations, Party A has the right to refuse acceptance and pursue Party B's liability for breach of contract as stipulated in the contract." This output is both professional and fluent, achieving higher compliance by clearly defining the rights and obligations of both parties, thus balancing reward enhancement with behavioral stability.

[0174] In some embodiments, the clause-level risk feedback mechanism in step S5 is implemented in the following way:

[0175] Apply the strategy model to the input contract fragment Generated candidate output text Divided into multiple clauses ,in, This indicates the k-th clause;

[0176] Using the reward model For each clause Perform independent compliance scoring to obtain clause scores. ;

[0177] According to each clause The type of contract clause pre-determines its importance weight. Among them, the heavy The impact of the contract's validity and the core rights of the parties can be preset according to the type of clause. For example, the weight of the breach of contract clause can be preset to be higher than that of the notice and service clause.

[0178] By aggregating the weighted scores of all clauses, the total reward value used for policy model updates is calculated. .

[0179] In the above embodiments, the contract generation result is segmented into clauses and scored independently, breaking away from the traditional coarse-grained model of evaluating the entire contract text as a whole. This allows the model to identify situations where long paragraphs may contain "partial compliance but global risk" or "mostly compliant but with a few high-risk clauses," achieving more refined risk positioning and control. By pre-setting importance weights for clauses of different clause types (such as payment, breach of contract, and intellectual property), this mechanism guides the training focus to the parts that have the greatest impact on the contract's validity and the core rights of the parties. The model therefore prioritizes ensuring the compliance and rigor of key clauses (such as breach of contract clauses) rather than applying equal weight, significantly improving the practical value of the review results. Clause-level scoring feedback allows the model to more clearly "understand" which specific expression or agreement led to a decrease in compliance during the training process. This helps the model internalize more accurate risk knowledge, so when generating suggestions, it can not only point out problems but also more specifically optimize specific clauses in the revision suggestions, improving the interpretability and professionalism of the output.

[0180] For example, the "Intellectual Property Clause" in a technology development contract might contain multiple clauses. In reinforcement fine-tuning (S5), the policy model generates a candidate text:

[0181] Clause 1 (Attribution Agreement): "All intellectual property rights generated from this project belong to Party A."

[0182] Clause 2 (Party B's Rights): "However, Party B has the right to mention its participation in this project in its promotional materials."

[0183] Clause 3 (Liability for Infringement): "If the technical solution provided by Party B results in intellectual property infringement, Party B shall bear full responsibility."

[0184] After applying the above clause-level risk feedback mechanism:

[0185] The system first presets weights based on the type of clause: intellectual property ownership (clause 1) and infringement liability (clause 3) are core clauses and are given high weights (e.g., w1=0.5, w3=0.4); publicity rights (clause 2) are secondary clauses and are given lower weights (e.g., w2=0.1).

[0186] The reward model scores each clause independently: Clause 1 is too absolute and may be given a low score (e.g., s1=-0.8); Clause 2 is basically compliant (e.g., s2=0.6); Clause 3’s “full responsibility” may be given a negative score because it is too strict (e.g., s3=-0.7).

[0187] Calculate the total reward: =(0.5*-0.8)+(0.1*0.6)+(0.4*-0.7)=-0.4+0.06-0.28=-0.62. This strong negative signal mainly comes from the negative scores of the high-weight clauses 1 and 3.

[0188] After this signal is fed back to the strategy model, the model will clearly recognize that the problem mainly focuses on the description of intellectual property ownership and infringement liability. In the next iteration, the model may optimize clause 1 to: "The intellectual property rights generated in this project, except for the intellectual property rights of Party B, belong to Party A." And optimize clause 3 to: "If the technical solution provided by Party B infringes on the intellectual property rights of a third party, Party B shall bear the direct losses caused to Party A as a result, except for those caused by the technical requirements provided by Party A."

[0189] Through this fine-grained, weighted feedback, the model is precisely guided to prioritize the correction of the riskiest and most important parts of the contract, thereby efficiently generating high-quality, low-risk contract terms.

[0190] In some embodiments, step S5 further includes a high-risk clause penalty and logical consistency verification mechanism, specifically including:

[0191] A set of high-risk clause patterns are predefined, including but not limited to "unlimited authorization", "asymmetric disclaimers", and "vague performance definitions";

[0192] When it is detected that any of the high-risk clause patterns are present in the candidate output generated by the strategy model, a fixed negative penalty term is introduced when calculating the total reward value;

[0193] At the same time, a set of predefined rules for logical consistency of clauses are provided, including: if a contract clause contains a conditional statement, it must correspond to a clear description of the consequences; if a contract clause contains a negative expression, it cannot exempt the contracting party from key obligations.

[0194] The candidate outputs are verified in real time based on the logical consistency rules through a rule engine.

[0195] The verification result is converted into a binary signal, where a positive value indicates successful verification and a negative value indicates unsuccessful verification. This signal is then added to the total reward value as an additional reward item to form the final reward signal used to update the policy model.

[0196] In the above embodiments, by predefining high-risk modes (such as "indefinite authorization") and logical consistency rules (such as "condition-consequence" correspondence), a hard safety boundary is set for the model output. Even if the reward model makes a mistake in certain complex scenarios, this mechanism can directly intercept or punish outputs that are obviously illegal or logically contradictory, providing double insurance for the reliability of contract review and effectively preventing serious legal errors. Through logical consistency rule verification, it is ensured that the contract terms generated by the model are logically self-consistent and complete (for example, a right declaration must have a corresponding exercise path, and an obligation setting must have a corresponding liability for breach of contract). This significantly improves the professional quality of the generated text, avoiding basic errors such as internal contradictions in clauses or missing key elements, making the contract text not only compliant but also highly operable.

[0197] For example, a company applied the mechanism described above when reviewing its software license agreement:

[0198] High-risk clause penalty scenario: When generating the scope of authorization clause, the initial output of the strategy model might include: "Party A grants Party B a perpetual, irrevocable, worldwide license to the software." The rule engine detects this predefined high-risk pattern of "perpetual and irrevocable" in real time and immediately triggers the penalty mechanism, adding a significant negative value (e.g., -1.0) to the total reward value. This strong negative signal forces the model to adjust the output in the next iteration, changing it to a more reasonable one: "During the term of this agreement, Party A grants Party B a license to use the software within the agreed geographical area."

[0199] Logical consistency verification scenario: When drafting a confidentiality clause, the model generates: "Party B shall be obligated to maintain the confidentiality of Party A's trade secrets upon access. The breaching party shall bear legal responsibility." The rule engine verifies the clause based on rules such as "negative expressions should not cover key obligations," finding that the clause only sets out obligations, but the corresponding description of "liability for breach of contract" is vague ("bear legal responsibility"), failing to meet the requirements of logical completeness. The verification failure generates a negative signal, guiding the model to optimize the output to: "...If Party B breaches this confidentiality obligation, it shall pay Party A [specific amount] as liquidated damages and compensate Party A for all losses incurred as a result." This mechanism ensures that the generated clause is logically rigorous and complete in its elements.

[0200] Figure 4This diagram illustrates the reward-driven mechanism employed in the strategy model reinforcement and fine-tuning (S5) of this invention. The diagram clearly shows the interaction between the strategy model and the reward model: the strategy model generates candidate outputs based on the input contract fragments, while the reward model scores the compliance of these outputs and feeds the scoring results back to the strategy model as reinforcement signals to guide its parameter updates. The illustrated mechanism specifically includes clause-level risk feedback and a high-risk clause penalty strategy: the candidate output text is split into clause units for independent scoring and weighted aggregation, while negative incentives are applied to detected high-risk clause patterns. This mechanism works synergistically with the KL divergence constraint strategy to ensure the stability of the reinforcement and fine-tuning process and the compliance of the model output.

[0201] Among them, formula This is a typical form of policy optimization objective function in reinforcement learning, where... This means using distribution P to approximate the distribution. The information loss caused by the time, where P represents the current strategy model to be optimized. The output probability distribution given the input; P represents the initial policy model (or reference policy). The output probability score under the same input, where λ is the regularization strength coefficient and R is the weighted total reward.

[0202] In some embodiments, when implementing the enhanced fine-tuning in step S5, a risk grading enhancement mechanism is also included:

[0203] Based on the network of disputed issues and weight labels extracted from the elements of arbitration awards, risk levels are preset for different types of contract risks;

[0204] When the reward model scores the compliance of clauses, the risk level is incorporated as a weighting factor into the scoring calculation, so that compliance defects corresponding to high risk levels generate stronger negative reward signals.

[0205] The strategy model is optimized based on reward signals incorporating risk level information to improve the model's sensitivity to identifying high-risk compliance issues.

[0206] In the above embodiments, the risk grading enhancement mechanism enables the reward model not only to determine whether a clause is compliant, but also to distinguish the severity of compliance defects. By using risk levels derived from a large number of judicial cases as weighting factors, the model can assign higher attention weights to risk types with high frequency, high loss rate, or high compensation amount (such as "asymmetric disclaimer clauses"), thereby guiding the strategy model to prioritize avoiding those risk points with the most serious consequences in legal practice. By incorporating the weight labels in the dispute focus network (reflecting the frequency and severity of a certain type of risk in historical disputes) into the training, the model indirectly learns "lessons learned" in legal practice, enabling its risk identification ability to go beyond the surface compliance of the clauses and to have the "foresight" to assess the potential legal consequences of the clauses, greatly enhancing the practical value of the review recommendations.

[0207] In reinforcement learning training, limited exploration resources are more intelligently directed towards optimizing high-risk problems. The model spends more "effort" learning how to properly handle those problems marked as high-risk, thereby improving the model's overall performance on key issues more efficiently and maximizing training utility.

[0208] For example, when fine-tuning a model used for reviewing supplier contracts, the above risk grading enhancement mechanism can be applied:

[0209] Based on the constructed network of dispute points, the system recognizes that "unclear acceptance criteria" is the most likely high-risk point to cause disputes in technical service contracts (it is given a high-weight label, corresponding to a high-risk level).

[0210] In the S5 enhancement and fine-tuning, the strategy model generated a payment clause: "Upon completion of the project, Party A shall make payment upon Party B's payment request." On the surface, this clause does not appear to be obviously illegal. Simultaneously, the model generated an acceptance clause: "Acceptance criteria shall be based on Party A's satisfaction."

[0211] When scoring, the reward model generally assigns a negative score to ambiguity in payment terms ("after a payment request is made"). However, for acceptance terms, since "unclear acceptance criteria" is pre-defined as a high-risk level, the reward model applies a high-weight factor to this defect, thus generating a much stronger negative reward signal.

[0212] Upon receiving feedback, the strategy model will prioritize optimizing high-risk acceptance clauses. It might revise the acceptance clauses to read: "Acceptance standards shall be based on the functional indicators explicitly listed in the Technical Requirements Specification signed by both parties. Party A shall organize acceptance within 15 working days of receiving Party B's acceptance notice. Failure to organize acceptance within this period without raising any written objections shall be deemed as acceptance passing." The payment clauses might be slightly optimized to read: "Upon final project acceptance, Party A shall make payment within 30 days of receiving a compliant invoice from Party B."

[0213] In this way, the model is effectively trained to focus on and address the risks that are most likely to cause serious problems in real legal settings, making its review recommendations more targeted and practical.

[0214] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement a fine-tuning method for any of the contract review large language models described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0215] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0216] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0217] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0218] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in any of the fine-tuning methods for the large language model of contract review described in the above embodiments. The computer-readable storage medium may be volatile or non-volatile.

[0219] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described asynchronous tracking method for a vehicle remote control link.

[0220] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0221] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0222] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0223] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0224] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0225] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0226] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0227] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0228] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0229] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for fine-tuning a large language model for contract review, characterized in that, The steps are executed in the following order: S1. By integrating multiple sources of laws and regulations, arbitration award elements, and industry contract templates, a first training dataset for monitoring and fine-tuning is generated through structured annotation and semantic nesting mechanisms. S2. The initial large language model is trained using the first training dataset. A supervised fine-tuning model is obtained through a chain fine-tuning strategy that decouples task stages from adaptive parameter adjustment driven by task complexity. The chain fine-tuning strategy includes four specialized training stages in sequence: contract structure understanding, compliance clause matching, risk point identification, and review suggestion generation. S3. Based on expert knowledge, collect and label sample pairs containing positive and negative contract fragments, wherein risk points are marked for the negative contract fragments and compliance adjustment examples are provided, forming a second training dataset containing legal basis and compliance score; S4. Based on the second training dataset, train a reward model, which is configured to score the compliance of the input contract text fragment and its candidate revision text, wherein the reward model is trained using a loss function based on the ranking of paired samples. S5. Using the supervised fine-tuning model as the strategy model to be optimized, and the reward model as the reward signal, the strategy model is fine-tuned by reinforcement learning using a strategy optimization algorithm with KL divergence constraints. Clause-level risk feedback and high-risk clause penalty mechanisms are introduced during the fine-tuning process, and finally the optimized contract review model is output.

2. The method according to claim 1, characterized in that, Step S1 includes: The first training dataset is generated by structuring and integrating multiple sources of laws and regulations, arbitration award elements, and industry contract templates, respectively. The multi-source laws and regulations are structured, including: formatting the legal text into a tree structure, with the articles as nodes, and performing fine-grained analysis of the articles based on syntactic structure and role extraction. At the same time, a three-layer nested semantic tagging system is introduced to annotate the articles. The three layers of tags include a first layer of tags for annotating the contract life cycle stage, a second layer of tags for annotating the contract application scenario, and a third layer of tags for annotating the contract roles and responsibilities. Knowledge extraction of the arbitration award elements includes: extracting disputed case elements from the judgment documents to construct an arbitration element map, and based on the arbitration element map, performing cluster analysis on the disputed points through a clustering algorithm to form a dispute focus network, wherein each disputed point is assigned a weight label; The industry contract template is annotated using the three-layer nested semantic tagging system; The first training dataset was generated by fusing labeled laws and regulations, a clustered and weighted network of disputed issues, and annotated industry contract templates.

3. The method according to claim 1, characterized in that, In step S2, the learning rate of model training is dynamically controlled through task complexity-driven adaptive parameter adjustment, specifically including: Calculate a task complexity coefficient based on the contract type, industry sector, and structural complexity of the samples in the current training batch. ; Based on the task complexity coefficient Dynamically calculate the adaptive learning rate used for this batch of training. ; Wherein, the task complexity coefficient It is the density of the clause structure. semantic variability Industry span index The weighted sum is calculated using the following formula: ； Where α1, α2, and α3 are preset weight coefficients; The adaptive learning rate The calculation formula is: ； in, β is a preset base learning rate, and β is a hyperparameter used to control the sensitivity of the learning rate to changes in task complexity.

4. The method according to claim 1, characterized in that, In step S2, training is performed using a task-stage decoupling chain-like fine-tuning strategy. This strategy decouples the contract review task into four sequentially executed specialized stages: structural understanding, compliance matching, risk identification, and recommendation generation. For each stage, a dedicated model structure or loss function is used for chain-like fine-tuning, specifically including: In the structural understanding phase, the structural hierarchy of contract clauses is identified through syntactic analysis and role extraction, and an embedding representation is calculated for each clause unit. The calculation formula is as follows: ； in For the text of the terms, Number its structural level; During the compliance matching phase, a dual-tower model structure is adopted to encode contract terms and corresponding legal provisions separately, and the semantic cosine distance loss is minimized. The training is performed, and the loss function is: ； In the risk identification phase, based on triplet samples The training process involves triplets comprising risk segments and risk types. and explanation of the reasons The loss function at this stage The weighted sum of classification loss and generation loss: ； in Losses are categorized by risk type. The loss for generating the explanatory sentence is λ, which is a hyperparameter. During the suggestion generation phase, based on the output of the risk identification phase, compliant contract clause revision suggestions are generated in a sequence-to-sequence manner, and industry corpus templates are incorporated to standardize the format and style of the generated text.

5. The method according to claim 1, characterized in that, In step S3, constructing the second training dataset includes: Select compliant contract excerpts from real contract texts as positive examples, and attach the corresponding legal provisions to each positive example as the basis for its legality; Contract fragments with compliance deficiencies were collected from contract review cases as negative examples. Each negative example was labeled with its risk type and corresponding legal consequences. The risk types included ambiguous clauses, illegal clauses, and missing clauses. Based on the positive and negative samples, sample pairs are generated, and each sample pair is assigned a compliance score, with the positive sample having a higher score than the negative sample. This forms structured data containing contract fragments, legal basis, risk labeling, and compliance scores, which serves as the second training dataset.

6. The method according to claim 1, characterized in that, Step S4 includes: Format the samples in the second training dataset obtained in step S3 into quadruplets. ,in This is a fragment of the original contract. For the candidate outputs to be evaluated, Standard compliance outputs provided to experts The reward value is pre-set based on the compliance of the sample. For positive samples, r is set to +1, and for negative samples, r is set to a negative value according to their risk level. Train the reward model using formatted quadruples. The input to the reward model is a contract fragment. With candidate output The concatenated sequence is output as a scalar score representing the compliance of the candidate output. ; When training the reward model, preference pairs are constructed based on the quadruplet samples. ,in It is superior to The candidate outputs are selected and a loss function based on pairwise sorting is applied. Optimize: ； in and For the reward model, the candidate outputs are respectively and The predicted score.

7. The method according to claim 1, characterized in that, In step S5, a policy optimization algorithm with KL divergence constraints is used to optimize the policy model. Reinforcement learning fine-tuning is performed by updating the parameters of the policy model by minimizing the following loss function, LR: ； in, This indicates that the expectation is based on the current policy model. Generate output , The reward model evaluates the input contract fragment. and generate output The compliance score given This is the current strategy model. Compared with the initial policy model The KL divergence between the initial policy models is used to constrain the magnitude of policy updates, and β is a hyperparameter controlling the strength of the KL divergence constraint. The model is obtained after supervised fine-tuning in step S2.

8. The method according to claim 1, characterized in that, In step S5, the clause-level risk feedback mechanism is implemented in the following way: Apply the strategy model to the input contract fragment Generated candidate output text Divided into multiple clauses ; Using the reward model For each clause Perform independent compliance scoring to obtain clause scores. ; According to each clause The type of contract clause pre-determines its importance weight. ; By aggregating the weighted scores of all clauses, the total reward value used for policy model updates is calculated. .

9. The method according to claim 1, characterized in that, Step S5 also includes a high-risk clause penalty and logical consistency verification mechanism, specifically including: A set of high-risk clause patterns are predefined, including but not limited to indefinite authorization, asymmetric disclaimers, and ambiguous performance definitions; When it is detected that any of the high-risk clause patterns are present in the candidate output generated by the strategy model, a fixed negative penalty term is introduced when calculating the total reward value; At the same time, a set of predefined rules for logical consistency of clauses are provided, including: if a contract clause contains a conditional statement, it must correspond to a clear description of the consequences; if a contract clause contains a negative expression, it cannot exempt the contracting party from key obligations. The candidate outputs are verified in real time based on the logical consistency rules through a rule engine. The verification result is converted into a binary signal, where a positive value indicates successful verification and a negative value indicates unsuccessful verification. This signal is then added to the total reward value as an additional reward item to form the final reward signal used to update the policy model.

10. The method according to claim 1, characterized in that, In step S5, when implementing enhanced fine-tuning, a risk grading enhancement mechanism is also included: Based on the network of disputed issues and weight labels extracted from the elements of arbitration awards, risk levels are preset for different types of contract risks; When the reward model scores the compliance of clauses, the risk level is incorporated as a weighting factor into the scoring calculation, so that compliance defects corresponding to high risk levels generate stronger negative reward signals. The strategy model is optimized based on reward signals incorporating risk level information to improve the model's sensitivity to identifying high-risk compliance issues.

Images