Intelligent contract review method and system based on large language model and knowledge graph, and computer equipment

Abstract

The invention relates to an intelligent contract review method and system based on a large language model and a knowledge graph, and computer equipment. The method comprises the following steps: extracting contract terms in a contract, and identifying keywords in the contract terms; acquiring information associated with the keyword from a knowledge graph according to the keyword; filling a preset prompt template with the contract terms and the associated information to generate prompt information; and inputting the prompt information into a large language model to generate an examination report of the contract. By adopting the method, efficient and accurate intelligent contract review can be realized.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent contract review method, system, and computer device based on large language models and knowledge graphs. Background Technology

[0002] With the development of artificial intelligence technology, large language models have been widely used. A large language model is a deep learning model trained with massive amounts of text data. This model can generate natural language text or understand the meaning of language text.

[0003] Currently, contract review relies heavily on manual work by legal professionals. This approach is not only time-consuming and labor-intensive, but also prone to overlooking key risks, logical conflicts between clauses, or potential clashes with the latest laws and regulations. Although attempts have been made to use large language models to assist in the review process and improve efficiency, the results generated often lack a basis and are prone to producing seemingly reasonable but actually erroneous legal advice or contractual wording.

[0004] There is currently no effective solution for achieving efficient and accurate intelligent contract review. Summary of the Invention

[0005] Therefore, it is necessary to provide an intelligent contract review method, system, and computer equipment based on a large language model and knowledge graph that can achieve efficient and accurate intelligent contract review, addressing the aforementioned technical issues.

[0006] Firstly, this application provides an intelligent contract review method based on large language models and knowledge graphs, including:

[0007] Extract contract clauses from the contract and identify keywords within those clauses;

[0008] Obtain information associated with the keywords from the knowledge graph;

[0009] The contract terms and associated information are filled into a preset prompt template to generate a prompt message;

[0010] The prompt information is input into a large language model to generate a review report for the contract.

[0011] In one embodiment, the knowledge graph includes: keywords and legal basis, as well as the association between the keywords and the legal basis;

[0012] The step of obtaining information associated with the keyword from the knowledge graph includes obtaining the legal basis associated with the keyword from the knowledge graph.

[0013] In one embodiment, the generated prompt information includes:

[0014] A vector database is constructed based on a question-and-answer pair library used to improve the quality of the contract review opinions;

[0015] The keywords are converted into vectors, and relevant question-answer pairs are matched in the vector database based on the vectors.

[0016] The text corresponding to the relevant question-and-answer pairs, the contract terms, and the associated information are filled into a preset prompt template to generate prompt information.

[0017] In one embodiment, the extraction of contract terms from the contract includes:

[0018] When the contract is in the form of a scanned image or contains text that cannot be directly edited, the contract terms are extracted using image text extraction technology.

[0019] In one embodiment, the intelligent contract review method based on large language models and knowledge graphs further includes:

[0020] If the legal basis associated with the keyword is not currently valid, then the legal basis is updated, and an association relationship is established between the keyword and the updated legal basis.

[0021] In one embodiment, the intelligent contract review method based on large language models and knowledge graphs further includes:

[0022] If no relevant question-answer pair is matched based on the keyword, the question-answer pair library is expanded, and a vector database is constructed based on the expanded question-answer pair library.

[0023] In one embodiment, the intelligent contract review method based on large language models and knowledge graphs further includes:

[0024] When the content of the review report contains errors, the correct output is obtained through manual annotation, and the correct output and the corresponding prompt information are used to fine-tune the large language model.

[0025] Secondly, this application also provides an intelligent contract review system based on a large language model and knowledge graph, the system comprising: an extraction module, a prompt information generation module, and a large language model invocation module; wherein,

[0026] The extraction module is used to extract contract clauses from the contract, identify keywords in the contract clauses, and obtain information associated with the keywords from the knowledge graph;

[0027] The prompt information generation module is used to fill the contract terms and the associated information into a preset prompt template to generate prompt information;

[0028] The large language invocation module is used to input the prompt information into the large language model to generate a review report of the contract.

[0029] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0030] Extract contract clauses from the contract and identify keywords within those clauses;

[0031] Obtain information associated with the keywords from the knowledge graph;

[0032] The contract terms and associated information are filled into a preset prompt template to generate a prompt message;

[0033] The prompt information is input into a large language model to generate a review report for the contract.

[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0035] Extract contract clauses from the contract and identify keywords within those clauses;

[0036] Obtain information associated with the keywords from the knowledge graph;

[0037] The contract terms and associated information are filled into a preset prompt template to generate a prompt message;

[0038] The prompt information is input into a large language model to generate a review report for the contract.

[0039] The aforementioned intelligent contract review method, system, and computer equipment based on a large language model and knowledge graph employ the following steps: extracting contract clauses from the contract and identifying keywords within those clauses; obtaining information associated with the keywords from the knowledge graph; filling the contract clauses and associated information into a preset prompt template to generate prompt information; and inputting the prompt information into the large language model to generate a review report for the contract. Obtaining associated information from the knowledge graph and inputting it into the large language model makes intelligent contract review more accurate, while utilizing contract keywords to obtain associated information from the knowledge graph makes intelligent contract review more efficient, thus achieving efficient and accurate intelligent contract review. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is an application environment diagram of an intelligent contract review method based on a large language model and knowledge graph in one embodiment;

[0042] Figure 2 This is a flowchart illustrating an intelligent contract review method based on a large language model and knowledge graph in one embodiment.

[0043] Figure 3 This is a flowchart illustrating an intelligent contract review method based on a large language model and knowledge graph in an example embodiment.

[0044] Figure 4 This is a structural block diagram of an intelligent contract review system based on a large language model and knowledge graph in one embodiment;

[0045] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] The intelligent contract review method based on large language models and knowledge graphs provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 inputs a contract into server 104. Server 104 extracts contract terms from the contract, identifies keywords in the contract terms, obtains information associated with the keywords from a knowledge graph, fills the contract terms and associated information into a preset prompt template to generate prompt information, and inputs the prompt information into a large language model to generate a review report of the contract. Terminal 102 receives the review report generated by server 104. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0048] In one embodiment, such as Figure 2 As shown, a process for an intelligent contract review method based on a large language model and knowledge graph is presented, including the following steps:

[0049] Step 202: Extract the contract terms from the contract and identify the keywords in the contract terms.

[0050] The key words in the contract terms are words or phrases that can highly summarize and represent the core legal obligations, rights, or risks of the contract terms.

[0051] Optionally, a keyword database for contract terms can be established to determine whether there are keywords in the contract terms that match the keyword database. If so, the keywords in the contract terms can be identified.

[0052] Optionally, a named entity recognition model can be used to identify keywords in the contract terms.

[0053] By identifying keywords in contract terms, information associated with those keywords can be retrieved from a knowledge graph, making intelligent contract review more efficient.

[0054] Step 204: Obtain information associated with the keyword from the knowledge graph.

[0055] Optionally, the knowledge graph query interface can be called, and the JSON format data returned by the interface can be parsed to obtain structured information associated with the keywords.

[0056] Step 206: Fill the contract terms and related information into the preset prompt template to generate prompt information.

[0057] Using prompt templates ensures that all generated prompts have a consistent style and structure.

[0058] Step 208: Input the prompt information into the large language model to generate a review report of the contract.

[0059] By filling the contract terms and related information into the preset prompt template, the large language model can be guided to perform more accurate analysis and response.

[0060] The aforementioned intelligent contract review method based on a large language model and knowledge graph involves extracting contract clauses from the contract and identifying keywords within those clauses; retrieving information associated with the keywords from the knowledge graph; filling the contract clauses and associated information into a preset prompt template to generate prompt information; and inputting the prompt information into the large language model to generate a contract review report. Retrieving associated information from the knowledge graph and inputting it into the large language model makes intelligent contract review more accurate, while utilizing contract keywords to retrieve associated information from the knowledge graph makes intelligent contract review more efficient, thus achieving efficient and accurate intelligent contract review.

[0061] In one embodiment, the knowledge graph includes: keywords and legal basis, and the association between the keywords and the legal basis; obtaining information associated with the keywords from the knowledge graph includes: obtaining the legal basis associated with the keywords from the knowledge graph.

[0062] Knowledge graphs are structured semantic knowledge bases used to represent entities and their relationships in the form of graphs, typically using triples (entity-relationship-entity).

[0063] Optionally, the knowledge graph may also include similar cases, as well as the relationships between the keywords, the legal basis, and the similar cases.

[0064] Optionally, the attributes of the knowledge graph can be enhanced by adding attributes such as the level of validity, statute of limitations, jurisdiction, and applicable scenarios to the keywords and legal bases in the knowledge graph.

[0065] In one embodiment, generating the prompt information includes: constructing a vector database based on a question-and-answer pair library for improving the quality of the contract review opinions; converting the keywords into vectors and matching relevant question-and-answer pairs in the vector database according to the vectors; and filling the text corresponding to the relevant question-and-answer pairs, the contract terms, and the associated information into a preset prompt template to generate the prompt information.

[0066] Optionally, the question-and-answer database is an industry standard and internal risk control rule question-and-answer database.

[0067] Optionally, the prompt template may also include character settings and information about both parties to the contract.

[0068] By constructing a vector database based on the question-answer pair library and matching relevant question-answer pairs according to the keywords, the large language model can be further enabled to perform accurate analysis and responses.

[0069] In one example implementation, assuming the contract is a cloud service procurement contract with a contract amount of 5 million yuan per year, the extracted contract terms are "the supplier must guarantee monthly service availability ≥ 99.9999%, and if this is not achieved, the supplier must refund double the amount for the duration of non-compliance." The extracted keywords are "service availability" and "compensation standard." Then, "service availability" and "compensation standard" are converted into vectors, and relevant question-answer pairs are matched against the vector database based on these vectors. Assuming the relevant question and answer pair corresponds to the text "Industry standard: CDN service availability industry benchmark is 99.5%; Internal risk control rules: Compensation amount shall not exceed 20% of the total contract amount", and the related information obtained from the knowledge graph is "Legal basis: Article 584 of the Civil Code (Scope of compensation for losses); Similar case: A cloud service contract dispute case (2023) Hu 0105 Min Chu 1234 (compensation standard was deemed too high); Industry standard: The Ministry of Industry and Information Technology's "Cloud Computing Service SLA Specification" has an availability compensation cap of 30% of the monthly fee", then this information will be filled into the preset prompt template to generate prompt information.

[0070] In one embodiment, extracting contract terms from a contract includes: when the contract is in the form of a scanned image or contains text content that cannot be directly edited, extracting the contract terms from the contract using image text extraction technology.

[0071] Optionally, if the contract is a scanned PDF or a Word document containing tables, the contract terms are extracted using image text extraction technology. Otherwise, the contract terms are extracted directly.

[0072] Optionally, the text content can be identified and extracted using an Optical Character Recognition (OCR) model;

[0073] By extracting contract terms from the contract using image text extraction technology, a foundation was laid for identifying keywords in the contract terms.

[0074] In one embodiment, the intelligent contract review method based on a large language model and knowledge graph further includes: when the legal basis associated with the keyword is not currently valid, updating the legal basis and constructing an association between the keyword and the updated legal basis.

[0075] "Non-currently valid" refers to existing legal grounds that have lost their legal effect, typically including those that have been amended, repealed, replaced by new law, or whose validity period has expired. By updating non-currently valid legal grounds and reconstructing the relationship between the aforementioned keywords and the updated legal grounds, the reliability of the generated review report is ensured.

[0076] In one embodiment, the intelligent contract review method based on a large language model and knowledge graph further includes: when no relevant question-answer pair is matched according to the keyword, expanding the question-answer pair library and constructing a vector database based on the expanded question-answer pair library.

[0077] By expanding the question-answer pair database, large language models can perform more accurate analysis and responses.

[0078] In one embodiment, the intelligent contract review method based on a large language model and knowledge graph further includes: when there is an error in the content of the review report, obtaining the correct output result through manual annotation, and using the correct output result and the corresponding prompt information to fine-tune the large language model.

[0079] Optionally, the review report includes review results, root cause analysis, and modification suggestions. If any of the review results, root cause analysis, or modification suggestions are found to be incorrect after manual review, legal personnel will annotate the data to obtain the correct output. The prompt information and the correct output are then used as training data pairs to train the large language model.

[0080] By fine-tuning the large language model, its professionalism and reliability can be improved.

[0081] In one example embodiment, an intelligent contract review method based on a large language model and knowledge graph is provided. Figure 3 The flowchart of this method includes the following steps:

[0082] S301, Obtaining a Contract.

[0083] S302, determine whether the contract file format is a scanned image or contains text content that cannot be directly edited. If yes, proceed to step S304; otherwise, proceed to step S303.

[0084] S303, directly extract the contract terms from the contract.

[0085] S304, Extract contract terms from the contract using image text extraction technology.

[0086] S305, Identify keywords in contract terms.

[0087] S306, Obtain information associated with the keyword from the knowledge graph.

[0088] S307, Construct a vector database based on a question-and-answer pair library used to improve the quality of the contract review comments.

[0089] S308, convert the keywords obtained in step S305 into vectors, and match relevant question-answer pairs in the vector database based on the vectors.

[0090] S309, fill the text corresponding to the relevant question-and-answer pair, the contract terms, and the related information into the preset prompt template to generate prompt information.

[0091] S310, The prompt information is input into the large language model to generate a review report of the contract.

[0092] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0093] In one embodiment, such as Figure 4 As shown, an intelligent contract review system based on a large language model and knowledge graph is provided. The system includes: an extraction module 41, a prompt information generation module 42, and a large language model invocation module 43; wherein,

[0094] The extraction module 41 is used to extract contract clauses from the contract, identify keywords in the contract clauses, and obtain information associated with the keywords from the knowledge graph;

[0095] The prompt information generation module 42 is used to fill the contract terms and the associated information into a preset prompt template to generate prompt information;

[0096] The large language invocation module 43 is used to input the prompt information into the large language model to generate a review report of the contract.

[0097] Furthermore, the knowledge graph includes: keywords and legal basis, as well as the association between the keywords and the legal basis; the extraction module 41 is used to obtain the legal basis associated with the keywords from the knowledge graph.

[0098] Furthermore, the prompt information generation module 42 is also used to construct a vector database based on a question-and-answer pair library for improving the quality of the contract review opinions; convert the keywords into vectors, and match relevant question-and-answer pairs in the vector database according to the vectors; fill the text corresponding to the relevant question-and-answer pairs, the contract terms, and the related information into a preset prompt template to generate prompt information.

[0099] Furthermore, the extraction module 41 is also used to extract contract terms from the contract using image text extraction technology when the contract file format is a scanned image or contains text content that cannot be directly edited.

[0100] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an intelligent contract review method based on a large language model and knowledge graph.

[0101] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0102] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0103] Extract contract clauses from the contract and identify keywords within those clauses;

[0104] Information associated with the keywords is retrieved from the knowledge graph based on the keywords.

[0105] The contract terms and associated information are filled into a preset prompt template to generate a prompt message;

[0106] The prompt information is input into a large language model to generate a review report for the contract.

[0107] Furthermore, the knowledge graph includes: keywords and legal basis, as well as the association between the keywords and the legal basis; when the processor executes the computer program, it also implements the step of: obtaining the legal basis associated with the keywords from the knowledge graph.

[0108] Furthermore, when the processor executes the computer program, it also performs the following steps: constructing a vector database based on a question-and-answer pair library for improving the quality of the contract review opinions; converting the keywords into vectors and matching relevant question-and-answer pairs in the vector database according to the vectors; and filling the text corresponding to the relevant question-and-answer pairs, the contract terms, and the associated information into a preset prompt template to generate prompt information.

[0109] Furthermore, when the processor executes the computer program, it also performs the following steps: when the file format of the contract is a scanned image or contains text content that cannot be directly edited, the contract terms in the contract are extracted using image text extraction technology.

[0110] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0111] Extract contract clauses from the contract and identify keywords within those clauses;

[0112] Obtain information associated with the keywords from the knowledge graph;

[0113] The contract terms and associated information are filled into a preset prompt template to generate a prompt message;

[0114] The prompt information is input into a large language model to generate a review report for the contract.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An intelligent contract review method based on large language models and knowledge graphs, characterized in that, include: Extract contract clauses from the contract and identify keywords within those clauses; Obtain information associated with the keywords from the knowledge graph; The contract terms and associated information are filled into a preset prompt template to generate a prompt message; The prompt information is input into a large language model to generate a review report for the contract.

2. The method according to claim 1, characterized in that, The knowledge graph includes: keywords and legal basis, as well as the relationship between the keywords and the legal basis; The step of obtaining information associated with the keyword from the knowledge graph includes obtaining the legal basis associated with the keyword from the knowledge graph.

3. The method according to claim 2, characterized in that, The generated prompt information includes: A vector database is constructed based on a question-and-answer pair library used to improve the quality of the contract review opinions; The keywords are converted into vectors, and relevant question-answer pairs are matched in the vector database based on the vectors. The text corresponding to the relevant question-and-answer pairs, the contract terms, and the associated information are filled into a preset prompt template to generate prompt information.

4. The method according to claim 1, characterized in that, The contract terms in the extraction contract include: When the contract is in the form of a scanned image or contains text that cannot be directly edited, the contract terms are extracted using image text extraction technology.

5. The method according to claim 2, characterized in that, Also includes: If the legal basis associated with the keyword is not currently valid, then the legal basis is updated, and an association relationship is established between the keyword and the updated legal basis.

6. The method according to claim 3, characterized in that, Also includes: If no relevant question-answer pair is matched based on the keyword, the question-answer pair library is expanded, and a vector database is constructed based on the expanded question-answer pair library.

7. The method according to claim 1 or claim 3, characterized in that, Also includes: When the content of the review report contains errors, the correct output is obtained through manual annotation, and the correct output and the corresponding prompt information are used to fine-tune the large language model.

8. An intelligent contract review system based on a large language model and knowledge graph, characterized in that, The system includes: an extraction module, a prompt message generation module, and a large language model invocation module; wherein... The extraction module is used to extract contract clauses from the contract, identify keywords in the contract clauses, and obtain information associated with the keywords from the knowledge graph; The prompt information generation module is used to fill the contract terms and the associated information into a preset prompt template to generate prompt information; The large language invocation module is used to input the prompt information into the large language model to generate a review report of the contract.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images