A contract review report generation method based on large model integration optimization

Abstract

The present invention relates to the field of intelligent recognition model application technology, and in particular to a contract review report generation method based on large model integration optimization, which solves the technical problem in the prior art that there is a lack of methods to improve the efficiency and accuracy of contract review through automated processes to conduct in-depth analysis and review of contract texts, contract classification process, intent recognition process, and the obtained contract type and contract intent, traverse each of the audit questions in the risk point review library, and pass them as input to multiple pre-trained large language models, integrate the outputs of the multiple pre-trained large language models, use a voter to summarize and evaluate the results, integrate the audit results of all risk points, and form a comprehensive review report, which can greatly improve the model's ability to understand specific parts of legal documents, identify potential risks and obligations, etc. This method can improve the accuracy and pertinence of the model and ensure higher review quality.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition model application technology, and in particular to a contract review report generation method based on large model integrated optimization. Background Art

[0002] Contract review refers to the process of carefully analyzing and evaluating contract documents to ensure the legality, fairness and effectiveness of the contract terms. In contract review, professionals usually review the contract terms one by one to assess whether they comply with laws and regulations, whether they are reasonable and fair, and whether there are any terms that may lead to legal risks or disputes.

[0003] The purpose of contract review is to ensure that the rights and interests of all parties to the contract are protected, to avoid potential legal risks and disputes, and to maximize the execution and performance of the contract; contract review is an important measure to ensure the legality and fairness of the contract;

[0004] Relevant legal departments or institutions are responsible for maintaining market order, protecting the legitimate rights and interests of the parties, and promoting economic development and social stability. Contract review focuses on the legality, fairness and effectiveness of contract terms to ensure that they do not violate laws and regulations and do not harm the interests of the parties. During the review process, attention will also be paid to contract risk control to avoid possible legal risks and disputes. Through contract review, the security and stability of contract transactions can be guaranteed, providing a good legal environment for economic and social development.

[0005] In the internal management of an enterprise, the generation of contract review reports is a crucial task, which is usually undertaken by the company's legal team or contract management team; this process includes collecting relevant contract documents, such as sales, purchase and lease contracts, and then conducting an in-depth analysis of the contract terms to evaluate their legality, fairness and effectiveness.

[0006] Subsequently, the legal team provides legal opinions and suggestions, and prepares a review report based on the analysis results. The report usually includes a detailed description of the contract terms, legal assessment, risk warnings, and possible revision suggestions; the generation of a contract review report helps companies avoid legal risks, protect their legitimate rights and interests, and improve the efficiency and quality of contract management. Through the review report, companies can better formulate contract strategies, reduce the occurrence of contract disputes, and thus maintain the company's reputation and interests.

[0007] In the current method, contract review is still mostly done manually, which requires professionals to carefully read the contract content based on their knowledge and experience, which is time-consuming and labor-intensive. In order to improve the efficiency of review, automated tools are often used;

[0008] Most current automated tools are rule-based contract review tools that analyze and evaluate contract terms using preset rules that usually involve legal requirements, industry standards, company policies, and specific agreements between the parties to the contract.

[0009] During the rule-based review process, the contract content is compared with these preset rules to determine whether the contract content meets all relevant requirements and whether there are any deviations or inconsistencies. The advantage of this approach is that it is structured and consistent, and can quickly identify non-compliant clauses in the contract.

[0010] However, due to the extremely insufficient flexibility of the rules and the over-reliance on clearly defined knowledge, which cannot accurately capture the semantics of the text, its limitations are very obvious when faced with complex and non-standardized contracts, mainly reflected in the following aspects:

[0011] 1. Rigidity and limited flexibility: Limited by preset rules, they cannot adapt well to contract terms that do not fully match the rules, especially when dealing with complex or non-standardized contract content;

[0012] 2. Failure to capture the semantics of the text and lack of ability to understand the deeper meaning or context of the text may lead to misjudgment when evaluating ambiguous or ambiguous clauses, affecting the accuracy of the review;

[0013] 3. Unable to handle non-standardized contracts. For non-standardized contracts or industry-specific clauses, rule-driven review tools may perform poorly because they lack sufficient flexibility to adapt to these situations;

[0014] 4. Manual maintenance of rules is required, and the rule set needs to be updated and maintained regularly, which increases costs and time, especially when faced with frequently changing legal environments or industry standards, which may cause the tool's rules to be no longer accurate or outdated. Summary of the invention

[0015] The present invention aims to solve the technical problem in the prior art of lacking a method for improving the efficiency and accuracy of contract review through an automated process to conduct in-depth analysis and review of contract texts, and provides a contract review report generation method based on large model integrated optimization.

[0016] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0017] A method for generating a contract review report based on large model integration optimization, comprising:

[0018] Step 1, contract classification process;

[0019] Use external contract classification tools to automatically identify the type of contract itself, so as to quickly determine the basic framework and clause types of the contract;

[0020] Wherein, by accurately classifying the type of the contract itself, the input of the external contract classification tool is the contract content and the output is the contract type;

[0021] Step 2: Intent recognition process;

[0022] Analyze the contract text through the intention recognition model to determine the commercial intent and main purpose of the contract;

[0023] Among them, intention judgment provides important clues for the accurate selection of risk points. The input of the intention recognition model is the contract content, and the output is the contract intention.

[0024] Step 3: Based on the contract types and contract intent obtained in steps 1 and 2, develop a specific risk point review library;

[0025] The risk point review library contains multiple series of risk point audit questions, and each audit question can be presented in the form of an inquiry;

[0026] The audit questions are used as model input to systematically cover various possible risk points in the contract and construct a risk point set, which is expressed as formula (10):

[0027] (10),

[0028] in, Indicates the built Prompt library, Indicates the corresponding contract type, Indicates the corresponding intent category;

[0029] When the input is and , traverse the risk point set s and output the corresponding ;

[0030] Then the Prompt library is the risk point review library, which can be expressed as formula (20):

[0031] (20)

[0032] Step 4: When conducting a specific contract review, traverse each of the review question prompts in the risk point review library and pass them as input to multiple pre-trained large language models, with parameter T being the specific class of the contract category and parameter C being the specific class of the intent category;

[0033] By integrating the outputs of the plurality of pre-trained large language models, a voter is used to summarize and evaluate the results, and the option with the most votes will be selected as the final result; or

[0034] Each of the audit questions and contract content in the Prompt library is used as the input of multiple large language models, and these outputs are weighted and voted to obtain another final result;

[0035] Among them, when there are N basic large language models preset, use represents the prediction result of the i-th model for the input x. After the weighted voting ωi is executed, the final integrated prediction result can be obtained by weighted average calculation and expressed as formula (30):

[0036] (30),

[0037] Ultimately, the audit results of all risk points will be integrated into a comprehensive review report (8).

[0038] Specifically, in step 1 and step 2, the pre-trained language model BERT is used to encode the contract content;

[0039] Based on the encoding operation, training a classifier;

[0040] The pre-trained language model BERT is pre-trained on a large-scale text dataset to capture language knowledge and contextual information;

[0041] The pre-trained language model BERT learns to extract features from the contract text and map them to predefined contract intent and contract type;

[0042] Assume that the contract text T is a sequence of words or tokens, with a total of n words or tokens. The pre-trained language model BERT encodes the contract text T into a sequence and expresses it as:

[0043] (40),

[0044] The classifier maps the pre-trained language model BERT representation to the space of predefined intent categories and contract types to obtain a prediction result.

[0045] Specifically, the training process of the classifier model is:

[0046] First, use a labeled contract dataset to train the classifier model;

[0047] The contract data set used for training includes:

[0048] N samples, expressed as ;

[0049] in, is the i-th contract text;

[0050] in, is the corresponding true label, represented by the predefined intent category and contract type (4);

[0051] Next, the cross entropy loss function is used to measure the difference between the predicted result and the true label. The loss function is expressed as formula (50):

[0052] (50),

[0053] Afterwards, by minimizing the loss function, optimization algorithms such as gradient descent are used to update the parameters of the classifier model so that the classifier model can perform accurate classification predictions on the contract text.

[0054] Specifically, in step three, the audit questions provide a clear task objective by directly asking whether there are specific problems with the contract;

[0055] The answer format for the audit questions is:

[0056] First, yes or no;

[0057] Second, locate the fragment or 0;

[0058] The output of the answer format provides a structured framework.

[0059] Specifically, the audit result includes a prompt; the prompt includes:

[0060] Definition and description of the risk points to be reviewed;

[0061] The definitions and descriptions of the review risk points are used to provide additional contextual information to help the model understand the specific meaning behind the questions asked.

[0062] Specifically, in step 4, an ensemble learning method is used to integrate the outputs of multiple large language models to improve the accuracy and reliability of the review;

[0063] Each language model independently processes the same input review questions, and then aggregates the results through a voting mechanism;

[0064] The voting mechanism is: majority voting or weighted voting;

[0065] The voting mechanism weight of the language model is allocated according to its performance on the training set or the validation set, so as to utilize the relative advantages of different language models in processing specific types of texts and reduce the deviation that may be introduced by any single model through integration.

[0066] The present invention has the following beneficial effects:

[0067] The advantage of this approach is that a specific prompt library designed for contract review can improve the prompt engineering of task-specific performance; the core of prompt engineering is to "guide" the model to more accurately understand the task requirements through carefully designed prompts; this is based on observations in cognition and machine learning that although pre-trained models have extensive knowledge, they may need appropriate "cues" to effectively apply this knowledge to specific tasks;

[0068] In the specific application scenario of contract review, by designing specialized prompt libraries for different types of contracts, clauses, and review requirements, the model's ability to understand specific parts of legal documents can be greatly improved, especially in parsing complex clauses, identifying potential risks and obligations, etc. This approach can improve the accuracy and pertinence of the model, thereby ensuring higher review quality.

[0069] On the other hand, model integration optimization enhances robustness and accuracy by combining the prediction results of multiple independent models to improve the overall performance. It can be understood that different models may have different "insights" on different types of data, and integrating them can reduce the single-point bias or overfitting phenomenon that may exist in these models to a certain extent.

[0070] In the contract review task, this ensemble optimization approach is able to combine the strengths of multiple models, especially when they are designed to focus on different aspects of the contract;

[0071] This not only improves the accuracy of the overall review process, but also enhances the ability to robustly handle complex contract content;

[0072] By optimizing the integration of these models, we can ensure that even if errors occur in a single model, the performance of the overall system will not be greatly affected, thereby ensuring the reliability and accuracy of the review results.

[0073] This method, which combines prompt engineering and model integration optimization, not only significantly improves processing efficiency and quality for the specific task of contract review, but also shows obvious advantages when processing other highly professional and information-dense documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0075] Figure 1 A schematic diagram of a contract review model of the present invention;

[0076] Figure 2 It is a schematic diagram of the contract review flow chart of the present invention.

[0077] Reference numerals

[0078] 1. Contract content; 2. Classifier; 4. Contract type; 3. Intent recognition; 5. Contract intent; 6. Prompt library; 7. Voting machine; 8. Review report. DETAILED DESCRIPTION

[0079] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention; it should be noted that in this application, for the convenience of description, the "left side" in the current view is the "first end", the "right side" is the "second end", the "upper side" is the "first end", and the "lower side" is the "second end". The purpose of such description is to clearly express the technical solution, and it should not be understood as an improper limitation on the technical solution of this application.

[0080] The present invention provides a contract review method based on large model integration optimization, which makes full use of the excellent language understanding ability and rich implicit knowledge of the large model, improves the quality of the prompt text through the prompt engineering method, and combines the output of multiple models as the final result through the integrated optimization method, so as to increase the reasoning ability of the large model and ensure the accuracy and reliability of the contract review content. This method makes full use of the excellent language understanding ability and rich implicit knowledge of the large model to make up for the shortcomings of the rule-driven method. Specifically, please refer to Figure 1-2 As shown, the contract review report generation method based on large model integrated optimization includes:

[0081] Step 1, contract classification process;

[0082] Use external contract classification tools to automatically identify the type of contract itself, so as to quickly determine the basic framework and clause types of the contract;

[0083] Among them, through accurate classification of the contract type itself, the external contract classification tool inputs the contract content 1 and outputs the contract type 4; step one can quickly determine the basic framework of the contract and the types of terms usually included, such as sales contracts, lease agreements, service contracts, etc. Through accurate classification of contract types, it can lay the foundation for subsequent in-depth analysis and ensure the pertinence and effectiveness of the review work.

[0084] Step 2: Intent recognition process;

[0085] Analyze the contract text through the model of intent recognition 3 to determine the commercial intent and main purpose of the contract;

[0086] Among them, intention judgment provides important clues for the precise selection of risk points. The input of the model of intention identification 3 is contract content 1, and the output is contract intention 5. Step 2 deeply explores the business background of the contract and the main concerns of both parties to help understand the business logic of the contract terms and potential risk points. Intent judgment provides important clues for the precise selection of risk points.

[0087] Step 3: Based on the contract type 4 and contract intent 5 obtained in Steps 1 and 2, a specific risk point review library is developed;

[0088] The risk point review library contains multiple series of risk point audit question prompts, and each audit question can be presented in the form of an inquiry;

[0089] The audit question prompt is used as the model input to systematically cover various risk points that may exist in the contract and construct a risk point set s. The risk point set s is expressed as formula (10):

[0090] (10),

[0091] in, Indicates the built Prompt library (6), Indicates the corresponding contract type, Indicates the corresponding intent category;

[0092] When the input is and , traverse the risk point set s and output the corresponding ;

[0093] Then the Prompt library 6 is the risk point review library, which can be expressed as formula (20):

[0094] (20)

[0095] Each question is designed in a specific form of inquiry. For example, for the missing contract subject: Is there a missing contract subject problem in this contract? The answer is returned in the form of [yes / no, location fragment / 0] Tip: The missing contract subject means that the specific identity information of one or both parties is not clearly marked in the contract document, including but not limited to key information such as the name, address and legal status of an individual, company or other organization. These questions will be used as model inputs to systematically cover various risk points that may exist in the contract.

[0096] Step 4: When conducting a specific contract review, traverse each of the review question prompts in the risk point review library and pass them as input to multiple pre-trained large language models, as shown in the attached figure. Figure 1 As shown in the figure, the pre-trained model integrated in the experiment represented by LLM1 is Chatgpt-3.5, the pre-trained model integrated in the experiment represented by LLM2 is Tongyi Qianwen-7B, and the pre-trained model integrated in the experiment represented by LLM3 is Wenxin Yiyan; more experiments integrate pre-trained models, that is, pre-trained large language models, as shown in the experimental comparison results in Table 1.

[0097] By integrating the outputs of the plurality of pre-trained large language models, a voter 7 is used to summarize and evaluate the results, and the option with the most votes will be selected as the final result; or

[0098] Each of the audit question prompts in the prompt library 6 and the contract content 1 are used as inputs of multiple large language models, and these outputs are weightedly voted to obtain another final result;

[0099] Among them, when there are N basic large language models preset, use represents the prediction result of the i-th model for the input x. After the weighted voting is performed, the final integrated prediction result can be obtained by weighted average calculation and expressed as formula (30):

[0100] (30),

[0101] Ultimately, the audit results of all risk points will be integrated to form a comprehensive review report8, which not only points out the potential risk points in the contract, but also provides corresponding tips and improvement suggestions, providing strong support for decision makers.

[0102] In a specific embodiment, in step 1 and step 2, the contract content 1 is encoded using the pre-trained language model BERT;

[0103] Based on the encoding operation, training a classifier 2;

[0104] The pre-trained language model BERT is pre-trained on a large-scale text dataset to capture language knowledge and contextual information;

[0105] The pre-trained language model BERT learns to extract features from the contract text and map them to predefined contract intent 5 and contract type 4;

[0106] Assume that the contract text T is a sequence of words or tokens, with a total of n words or tokens. The pre-trained language model BERT encodes the contract text T into a sequence and expresses it as formula (40):

[0107] (40),

[0108] The classifier 2 maps the pre-trained language model BERT representation to the space of predefined intent categories and contract types 4 to obtain a prediction result.

[0109] In a specific embodiment, the training process of the model of the classifier 2 is:

[0110] First, use a labeled contract dataset to train the model of classifier 2;

[0111] The contract data set used for training includes:

[0112] N samples, expressed as ;

[0113] in, is the i-th contract text;

[0114] in, is the corresponding true label, which is represented by the predefined intent category and contract type4;

[0115] Next, the cross entropy loss function is used to measure the difference between the predicted result and the true label. The loss function is expressed as formula (50):

[0116] (50),

[0117] Afterwards, by minimizing the loss function, optimization algorithms such as gradient descent are used to update the parameters of the model of classifier 2 so that the model of classifier 2 can perform accurate classification prediction on the contract text.

[0118] In a specific embodiment, in step 3, the audit question prompt provides a clear task objective by directly asking whether there are specific problems with the contract;

[0119] The answer format of the audit question prompt is:

[0120] First, yes or no;

[0121] Second, locate the fragment or 0;

[0122] The output in the answer format provides a structured framework so that the results are easy to parse and further process. This structured output is particularly useful for automating the contract review process, allowing issues to be quickly identified, categorized and summarized.

[0123] In a specific embodiment, the audit result includes a prompt; the prompt includes:

[0124] Definition and description of the risk points to be reviewed;

[0125] The definitions and descriptions of the review risk points are used to provide additional contextual information to help the model understand the specific meaning behind the questions asked.

[0126] In step 4, an ensemble learning method is used to integrate the outputs of multiple large language models to improve the accuracy and reliability of the review;

[0127] Each language model independently processes the same input audit question prompt, and then aggregates the results through a voting mechanism;

[0128] The voting mechanism is: majority voting or weighted voting;

[0129] The voting mechanism weight of the language model is allocated according to its performance on the training set or the validation set, so as to utilize the relative advantages of different language models in processing specific types of texts and reduce the deviation that may be introduced by any single model through integration.

[0130] The advantage of this approach is that a specific prompt library designed for contract review can improve the prompt engineering of task-specific performance; the core of prompt engineering is to "guide" the model to more accurately understand the task requirements through carefully designed prompts; this is based on observations in cognition and machine learning that although pre-trained models have extensive knowledge, they may need appropriate "cues" to effectively apply this knowledge to specific tasks;

[0131] In the specific application scenario of contract review, by designing specialized prompt libraries for different types of contracts, clauses, and review requirements, the model's ability to understand specific parts of legal documents can be greatly improved, especially in parsing complex clauses, identifying potential risks and obligations, etc. This approach can improve the accuracy and pertinence of the model, thereby ensuring higher review quality.

[0132] On the other hand, model integration optimization enhances robustness and accuracy by combining the prediction results of multiple independent models to improve the overall performance. It can be understood that different models may have different "insights" on different types of data, and integrating them can reduce the single-point bias or overfitting phenomenon that may exist in these models to a certain extent.

[0133] In the contract review task, this integrated optimization method can combine the advantages of multiple models, especially when they are designed to focus on different aspects of the contract (for example, some models may focus more on language understanding, while others may be stronger in logical reasoning). This not only improves the accuracy of the overall review process, but also enhances the robustness of complex contract content. By optimizing the integration of these models, it can be ensured that even if a single model has errors, the performance of the overall system will not be greatly affected, thereby ensuring the reliability and accuracy of the review results.

[0134] This method, which combines prompt engineering and model integration optimization, not only significantly improves processing efficiency and quality for the specific task of contract review, but also shows obvious advantages when processing other highly professional and information-dense documents.

[0135] Table 1 Experimental comparison results

[0136]

[0137] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for generating a contract review report based on large model integration optimization, characterized in that: include: Step 1, contract classification process; Use external contract classification tools to automatically identify the type of contract itself, so as to quickly determine the basic framework and clause types of the contract; Wherein, by accurately classifying the type of the contract itself, the input of the external contract classification tool is the contract content (1) and the output is the contract type (4); Step 2: Intent recognition process; Analyze the contract text through the model of intent recognition (3) to determine the commercial intent and main purpose of the contract; Among them, intention judgment provides important clues for the accurate selection of risk points. The input of the model of intention identification (3) is the contract content (1), and the output is the contract intention (5); Step 3: Based on the contract type (4) and contract intent (5) obtained in steps 1 and 2, develop a specific risk point review library; The risk point review library contains multiple series of risk point audit question prompts, and each audit question can be presented in the form of an inquiry; The audit question prompt is used as the model input to systematically cover various risk points that may exist in the contract and construct a risk point set s. The risk point set s is expressed as formula (10): （10)， in, Indicates the built Prompt library (6), Indicates the corresponding contract type, Indicates the corresponding intent category; When the input is and , traverse the risk point set s and output the corresponding ; Then the Prompt library (6) is the risk point review library, which can be expressed as formula (20): (20)， Step 4: When conducting a specific contract review, each of the review question prompts in the risk point review library is traversed and passed as input to multiple pre-trained large language models; By integrating the outputs of the plurality of pre-trained large language models, a voter (7) is used to summarize and evaluate the results, and the option with the most votes is selected as the final result; or Each of the audit question prompts in the prompt library (6) and the contract content (1) are used as inputs of multiple large language models, and these outputs are weighted and voted to obtain another final result; Among them, when there are N basic large language models preset, use represents the prediction result of the i-th model for the input x. After the weighted voting is performed, the final integrated prediction result can be obtained by weighted average calculation and expressed as formula (30): （30）， Finally, the audit results of all risk points will be integrated into a comprehensive review report (8); In steps 1 and 2, the contract content (1) is encoded using the pre-trained language model BERT; Based on the encoding operation, training a classifier (2); The pre-trained language model BERT is pre-trained on a large-scale text dataset to capture language knowledge and contextual information; The pre-trained language model BERT learns to extract features from the contract text and map them to predefined contract intent (5) and contract type (4); Assume that the contract text T is a sequence of words or tokens, with a total of n words or tokens. The pre-trained language model BERT encodes the contract text T into a sequence and expresses it as formula (40): (40)， The classifier (2) maps the pre-trained language model BERT representation to a space of predefined intent categories and contract types (4) to obtain a prediction result; The training process of the model of the classifier (2) is as follows: First, use a labeled contract dataset to train the classifier (2) model; The contract data set used for training includes: N samples, expressed as ; in, is the i-th contract text; in, is the corresponding true label, represented by the predefined intent category and contract type (4); Next, the cross entropy loss function is used to measure the difference between the predicted result and the true label. The loss function is expressed as formula (50): (50)， Afterwards, by minimizing the loss function, an optimization algorithm such as gradient descent is used to update the parameters of the model of the classifier (2) so that the model of the classifier (2) can perform accurate classification prediction on the contract text.

2. The method for generating a contract review report based on large model integrated optimization according to claim 1, characterized in that: In step 3, the audit question prompt provides a clear task objective by directly asking whether there are specific problems with the contract; The answer format of the audit question prompt is: First, yes or no; Second, locate the fragment or 0; The output of the answer format provides a structured framework.

3. The method for generating a contract review report based on large model integrated optimization as claimed in claim 2, characterized in that: The audit result includes a prompt; the prompt includes: Definition and description of the review risk points; The definitions and descriptions of the review risk points are used to provide additional contextual information to help the model understand the specific meaning behind the questions asked.

4. The method for generating a contract review report based on large model integrated optimization as claimed in claim 3 is characterized in that: In step 4, an ensemble learning method is used to integrate the outputs of multiple large language models to improve the accuracy and reliability of the review; Each language model independently processes the same input audit question prompt, and then aggregates the results through a voting mechanism; The voting mechanism is: majority voting or weighted voting; The voting mechanism weight of the language model is allocated according to its performance on the training set or the validation set, so as to utilize the relative advantages of different language models in processing specific types of texts and reduce the deviation that may be introduced by any single model through integration.

Images