Artificial intelligence-based contract review method, device, equipment, and storage medium

Abstract

This application belongs to the field of artificial intelligence technology and provides a contract review method, device, equipment and storage medium based on artificial intelligence. The method includes: obtaining a sample contract, performing syntactic analysis on the terms of the sample contract, and obtaining a syntactic analysis binary tree corresponding to the terms of the sample contract; training an improved Transformer model with enhanced attention mechanism according to the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; obtaining a contract to be reviewed, and using the trained improved Transformer model to predict the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed; using a pre-trained classification model to classify the target sentence embedding representation matrix, and obtaining a review result of whether the contract to be reviewed is missing a clause based on the classification result. This application can improve the training speed and effect of the model and improve the accuracy of contract review.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based contract review method, apparatus, device, and storage medium. Background Art

[0002] Currently, the review of missing essential contract clauses is an essential part of legal review. As laws gradually improve and clauses become more diverse and numerous, the traditional manual review process undoubtedly consumes enormous manpower.

[0003] Related technologies use Transformer-based language models, such as BERT and XLNet, to review contracts. When training these language models, the Transformer's attention mechanism considers the weight distribution of individual characters across the entire sentence. Contract clauses are typically long texts, and the attention of individual characters across long texts is too sparse and dispersed, increasing the difficulty of model training and resulting in low model accuracy. This sparse attention also inevitably generates excessive super-matrix operations, slowing model training. This results in long training times and low review accuracy in contract review applications. Summary of the Invention

[0004] The main purpose of this application is to provide an artificial intelligence-based contract review method, device, equipment and storage medium, aiming to solve the technical problems in related technologies of using a Transformer-based language model to review contracts, which results in long model training time and low review accuracy.

[0005] In a first aspect, the present application provides a contract review method based on artificial intelligence, the method comprising:

[0006] Obtaining a sample contract, performing syntactic analysis on clauses of the sample contract, and obtaining a syntactic analysis binary tree corresponding to the clauses of the sample contract;

[0007] Training the improved Transformer model enhanced with the attention mechanism according to the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model;

[0008] Obtaining a contract to be reviewed, and using the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to a clause of the contract to be reviewed;

[0009] The target sentence embedding representation matrix is ​​classified using a pre-trained classification model, and a review result of whether the contract to be reviewed is missing clauses is obtained based on the classification result.

[0010] In a second aspect, the present application further provides a contract review device based on artificial intelligence, the device comprising:

[0011] An analysis module, configured to obtain a sample contract, perform syntactic analysis on the clauses of the sample contract, and obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;

[0012] A training module is used to train the improved Transformer model enhanced with the attention mechanism according to the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model;

[0013] A prediction module is used to obtain a contract to be reviewed and use the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed;

[0014] A classification module classifies the target sentence embedding representation matrix using a pre-trained classification model, and obtains a review result of whether the contract to be reviewed is missing clauses based on the classification result. In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned artificial intelligence-based contract review method are implemented.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the above-mentioned artificial intelligence-based contract review method is implemented.

[0016] The present application discloses an artificial intelligence-based contract review method, apparatus, device and storage medium. The artificial intelligence-based contract review method obtains a sample contract, performs syntactic analysis on the terms of the sample contract, and obtains a syntactic analysis binary tree corresponding to the terms of the sample contract; trains an improved Transformer model with enhanced attention mechanism according to the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; obtains a contract to be reviewed, and uses the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed; uses a pre-trained classification model to classify the target sentence embedding representation matrix, and obtains a review result of whether the contract to be reviewed is missing a clause based on the classification result. Since the improved Transformer model enhances the original attention mechanism of the Transformer model, the attention distribution of the improved Transformer model is no longer dispersed, which can effectively improve the training speed and model accuracy during training. The trained improved Transformer model can more quickly obtain a sentence embedding representation matrix that can accurately represent the semantics of the terms of the contract to be reviewed. Then, based on the sentence embedding representation matrix of the terms of the contract to be reviewed, the contract review is realized through the classification model, which greatly improves the review efficiency and review accuracy.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a flowchart of an embodiment of the artificial intelligence-based contract review method of this application;

[0020] Figure 2 This is a computational flow chart of the attention mechanism enhanced by the improved Transformer model involved in an embodiment of the artificial intelligence-based contract review method of this application;

[0021] Figure 3 A schematic block diagram of an artificial intelligence-based contract review device provided in one embodiment of the present application;

[0022] Figure 4This is a schematic block diagram of the structure of a computer device involved in one embodiment of the present application.

[0023] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0026] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0027] It should be further understood that the term “and / or” used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] The embodiments of the present application provide an artificial intelligence-based contract review method, apparatus, device, and storage medium. The artificial intelligence-based contract review method is mainly applied to an artificial intelligence-based contract review device, which can be a terminal device with data processing capabilities, such as a server.

[0029] Among them, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0030] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0031] Please refer to Figure 1 , Figure 1 A flowchart of an artificial intelligence-based contract review method provided in an embodiment of the present application.

[0032] like Figure 1 As shown, the artificial intelligence-based contract review method includes steps S101 to S104.

[0033] Step S101: Obtain a sample contract, perform syntactic analysis on the clauses of the sample contract, and obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract.

[0034] Among them, the artificial intelligence-based contract review method mainly includes two processes. The first is to train the improved transformer model and classification model to obtain the trained improved transformer model and classification model, and the classification model can be a softmax classifier; the second is to use the trained improved transformer model and classification model to realize the review of the contract to be reviewed.

[0035] In order to improve the review accuracy, unlike the transformer model in related technologies, this application creatively enhances the original attention mechanism of the transformer model. The enhanced attention mechanism can effectively focus the attention of text characters and target training, greatly reducing the computational complexity of the transformer model, and improving the model training speed and prediction accuracy.

[0036] The Transformer model is a natural language processing (NLP) model that relies entirely on the attention mechanism to compute representations of its input and output, abandoning the recurrent neural networks and convolutional neural networks used in traditional NLP models. This significantly alleviates the vanishing and exploding gradient problems. The Transformer model primarily consists of two parts: an encoder and a decoder. Both the encoder and decoder include functions such as attention generation and forward propagation.

[0037] First, let's introduce the original attention mechanism of the Transformer model. The calculation process of the original attention mechanism of the Transformer model mainly includes two steps:

[0038] 1) When a sentence is input into the Transformer model, a linear transformation is performed on the word embedding matrix X of the input sentence to generate a query (Q, queries) matrix, a key (K, keys) matrix, and a value (V, values) matrix;

[0039] 2) Multiply the Q and K matrices to obtain the attention matrix, and then multiply the attention matrix with the V matrix to obtain the sentence embedding representation matrix.

[0040] This shows that the original attention mechanism of the Transformer model considers the weight distribution of a single character across the entire sentence, which has the following two shortcomings: 1) For long sentences, the attention of a single character is too sparse and dispersed across the sentence, increasing the difficulty of model training and resulting in low model training accuracy; 2) The sparse dispersion of attention inevitably results in excessive super-matrix operations, slowing down model training.

[0041] In view of the shortcomings of the original attention mechanism of the Transformer model above, the original attention mechanism of the Transformer model is improved to obtain an improved Transformer model with enhanced attention mechanism. Please refer to Figure 2 , Figure 2 To improve the computational flow of the enhanced attention mechanism of the Transformer model, the following steps are involved:

[0042] 1) Constructing a syntax analysis masking matrix:

[0043] a. First, use the Stanford Parser syntactic analysis tool to perform syntactic analysis on the sentence. The syntactic analysis results are presented in the form of a binary tree. The leaf nodes correspond to each character in the sentence (one-to-one correspondence). The distance dis(i, j) between any two characters is defined as the distance between the two nodes in the corresponding binary tree.

[0044] b. If the length of the sentence is l, then construct a syntactic parsing masking matrix M of size (l, l) and define a threshold m. When dis(i, j)>m, M[i, j] is equal to negative infinity, and when dis(i, j)<=m, M[i, j]=0.

[0045] 2) Generate a new attention matrix:

[0046] a. Multiply the Transformer's Q and K matrices to obtain the first attention matrix A, of size (l,l). This first attention matrix A is added to the syntactic parsing mask matrix M to obtain the second attention matrix G, of size (l,l). It can be understood that when M[i, j] equals negative infinity, then G[i, j] equals negative infinity, indicating that i and j are sufficiently far apart that no attention is required. During model training backpropagation, the gradient at negative infinity is 0, eliminating the need for training and significantly speeding up model training. When M[i, j] equals 0, G[i, j] equals A[i, j], maintaining the original attention weights.

[0047] b. Set a gate mechanism alpha to reconcile the two attention matrices A and G. The final enhanced new attention matrix is ​​alpha*A+(1-alpha)*G.

[0048] 3) Generate a new sentence embedding representation matrix:

[0049] Finally, the new sentence embedding matrix is ​​equal to the product of alpha*A+(1-alpha)*G and V matrix.

[0050] In order to apply the improved transformer model enhanced with the attention mechanism to contract review, the improved transformer model enhanced with the attention mechanism is trained according to sample contracts to obtain the trained improved transformer model. The training process is as follows:

[0051] First, we obtain a sample contract containing several clauses. Before using the contract sample to train the improved Transformer model, we first segment the sample contract into paragraphs to obtain the clauses. Then, we use the Stanford parsing tool to perform syntactic analysis on the sample contract, generating a binary syntactic analysis tree corresponding to the clauses in the sample contract. The leaf nodes of this binary syntactic analysis tree correspond to each character in the clauses in the sample contract.

[0052] Step S102: Training the improved Transformer model enhanced with the attention mechanism according to the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model.

[0053] Afterwards, the improved Transformer model is trained according to the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model.

[0054] In some embodiments, the improved Transformer model enhanced with the attention mechanism is trained according to the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model, specifically: the terms of the sample contract and the corresponding syntactic analysis binary tree are input into the improved Transformer model, so that the attention mechanism enhanced by the improved Transformer model generates a query matrix, a key matrix and a value matrix according to the terms of the sample contract, and constructs a syntactic analysis mask matrix according to the syntactic analysis binary tree; the improved Transformer model is trained according to the query matrix, the key matrix, the value matrix and the syntactic analysis mask matrix to obtain a trained improved Transformer model.

[0055] That is, the terms of the sample contract and their corresponding syntactic analysis binary tree are used as the input of the improved Transformer model, and the attention mechanism of the improved Transformer model is enhanced. First, a query matrix, a key matrix and a value matrix are generated based on the terms of the sample contract, and a syntactic analysis masking matrix is ​​constructed based on the syntactic analysis binary tree corresponding to the terms of the sample contract. Then, according to the query matrix, key matrix and value matrix corresponding to the terms of the sample contract and the syntactic analysis masking matrix, the improved Transformer model with the enhanced attention mechanism is trained to obtain a trained improved Transformer model.

[0056] In some embodiments, the improved Transformer model is trained based on the query matrix, the key matrix, the value matrix and the syntactic analysis mask matrix to obtain a trained improved Transformer model, specifically: multiplying the query matrix and the key matrix to obtain a first attention matrix; adding the first attention matrix and the syntactic analysis mask matrix to obtain a second attention matrix; reconciling the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix; multiplying the enhanced new attention matrix with the value matrix to obtain a sentence embedding representation matrix corresponding to the terms of the sample contract.

[0057] That is, the query matrix and key matrix corresponding to the terms of the sample contract are multiplied to obtain a first attention matrix, and then the first attention matrix is ​​added to the syntactic analysis masking matrix corresponding to the terms of the sample contract to obtain a second attention matrix. The first attention matrix and the second attention matrix are then reconciled to obtain an enhanced new attention matrix. The enhanced new attention matrix is ​​then multiplied by the value matrix corresponding to the terms of the sample contract to obtain a sentence embedding representation matrix corresponding to the terms of the sample contract. The parameters of the improved Transformer model are then adjusted according to the sentence embedding representation matrix corresponding to the terms of the sample contract until the Transformer model converges to obtain a trained improved Transformer model.

[0058] In some embodiments, the first attention matrix and the second attention matrix are harmonized to obtain an enhanced new attention matrix, specifically: obtaining a preset harmonization formula alpha*A+(1-alpha)*G, where alpha represents a preset gate mechanism, A represents the first attention matrix, and G represents the second attention matrix; substituting the first attention matrix and the second attention matrix into the preset harmonization formula for calculation to obtain an enhanced new attention matrix.

[0059] That is, the first attention matrix and the second attention matrix corresponding to the clauses of the sample contract are substituted into the preset reconciliation formula shown below to calculate and obtain the enhanced new attention matrix corresponding to the clauses of the sample contract:

[0060] alpha*A+(1-alpha)*G

[0061] Among them, alpha represents the preset gate mechanism, A represents the first attention matrix, and G represents the second attention matrix.

[0062] The improved Transformer model is trained using sample contracts. Since the enhanced attention mechanism of the improved Transformer model can effectively focus the attention of the characters of the sample contract clauses and target them for training, it can improve the training speed and training accuracy, making the trained improved Transformer model have higher accuracy in contract review applications.

[0063] Furthermore, after completing the training of the improved Transformer model, it is necessary to embed the sentences corresponding to the clauses of the sample contract into a representation matrix and the true categories of the clauses of the sample contract as samples for training the softmax classifier, and train the softmax classifier. The true categories of the clauses of the sample contract can be pre-labeled.

[0064] The softmax classifier is a common linear classifier that is suitable for multi-classification prediction problems. Specifically, the sentence embedding representation matrix corresponding to the terms of the sample contract and the true category of the terms of the sample contract are used as the input of the softmax classifier to obtain the probability prediction value of the sentence embedding representation matrix corresponding to the terms of the sample contract belonging to each preset category. By comparing the probability prediction value of the sentence embedding representation matrix corresponding to the terms of the sample contract belonging to each preset standard term category with the true category of the terms of the sample contract, the cross-entropy loss is established to obtain the loss function of the softmax classifier. The gradient descent method is used to optimize the loss function so that the loss function becomes smaller and smaller until the loss function converges, and a trained softmax classifier can be obtained. The input of the trained softmax classifier is the sentence embedding representation matrix, and the output is the classification probability value between [0, 1]. The trained softmax classifier can achieve accurate classification of contract terms.

[0065] Step S103: Obtain the contract to be reviewed, and use the trained improved Transformer model to predict the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed.

[0066] The following is the process of reviewing the contract using the trained improved Transformer model and softmax classifier.

[0067] First, obtain the contract to be reviewed, and use the trained improved Transformer model to predict the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed.

[0068] In some embodiments, the trained improved Transformer model is used to predict the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed, specifically: performing syntactic analysis on the terms of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the terms of the contract to be reviewed; inputting the terms of the contract to be reviewed and their corresponding target syntactic analysis binary tree into the trained improved Transformer model for prediction to obtain the target sentence embedding representation matrix.

[0069] Among them, the syntactic analysis of the terms of the contract to be reviewed to obtain the target syntactic analysis binary tree corresponding to the terms of the contract to be reviewed is specifically: the paragraph splitting of the contract to be reviewed to obtain the terms of the contract to be reviewed; and the use of the Stanford syntactic analysis tool to perform syntactic analysis on the terms of the contract to be reviewed to obtain the target syntactic analysis binary tree.

[0070] Similarly, the contract to be reviewed is first split into paragraphs to obtain the terms of the contract to be reviewed, and then the Stanford syntactic analysis tool is used to perform syntactic analysis on the terms of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the terms of the contract to be reviewed, wherein the leaf nodes of the target syntactic analysis binary tree correspond to each character of the terms of the contract to be reviewed.

[0071] The terms of the contract to be reviewed and their corresponding target syntactic analysis binary tree are input into the trained improved Transformer model. The trained improved Transformer model uses the enhanced attention mechanism to first generate a query matrix, a key matrix and a value matrix based on the terms of the contract to be reviewed, and construct a target syntactic analysis masking matrix based on the target syntactic analysis binary tree corresponding to the terms of the contract to be reviewed. The query matrix and the key matrix corresponding to the terms of the reviewed contract are multiplied to obtain a first target attention matrix, and then the first target attention matrix is ​​added to the target syntactic analysis masking matrix to obtain a second target attention matrix. The first target attention matrix and the second target attention matrix are then reconciled to obtain an enhanced new target attention matrix. The enhanced new target attention matrix is ​​then multiplied with the value matrix corresponding to the terms of the contract to be reviewed to obtain the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed and output it.

[0072] The trained improved Transformer model can more quickly obtain a sentence embedding representation matrix that can accurately represent the semantics of the terms of the contract to be reviewed.

[0073] Step S104: using a pre-trained classification model to classify the target sentence embedding representation matrix, and obtaining a review result of whether the contract to be reviewed is missing clauses based on the classification result.

[0074] Furthermore, the trained softmax classifier is used to classify the target sentence embedding representation matrix corresponding to the clauses of the contract to be reviewed, so as to obtain the review result of whether the contract to be reviewed is missing clauses based on the classification result.

[0075] In some embodiments, the target sentence embedding representation matrix is ​​classified using a pre-trained classification model, and a review result of whether the contract to be reviewed is missing clauses is obtained based on the classification result. Specifically, the target sentence embedding vector is input into a pre-trained softmax classifier for classification prediction to obtain a probability value that the clauses of the contract to be reviewed belong to a preset standard clause category; and whether the clauses of the contract to be reviewed are missing is determined based on the probability value.

[0076] That is, the target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed is input into the trained softmax classifier for classification prediction, and the predicted classification probability value output by the trained softmax classifier is obtained. The predicted classification probability value represents the probability value of the terms of the contract to be reviewed belonging to the preset standard terms category. The closer the probability value is to 1, the more it indicates that there are no missing terms in the contract to be reviewed. The closer the predicted classification probability value output by the trained softmax classifier is to 0, the more it indicates that there are missing terms in the contract to be reviewed.

[0077] Exemplarily, the predicted classification probability value output by the trained softmax classifier can be compared with a preset threshold. When the predicted classification probability value output by the trained softmax classifier exceeds the preset threshold, the review result is that there are no missing clauses in the contract to be reviewed. When the predicted classification probability value output by the trained softmax classifier does not exceed the preset threshold, the review result is that there are missing clauses in the contract to be reviewed. The preset threshold can be 0.5.

[0078] Using the trained softmax classifier to perform classification prediction on the target sentence embedding representation matrix corresponding to the clauses of the contract to be reviewed will result in smaller errors and more accurate classification.

[0079] The artificial intelligence-based contract review method provided in the above embodiment obtains a sample contract, performs syntactic analysis on the terms of the sample contract, and obtains a syntactic analysis binary tree corresponding to the terms of the sample contract; trains an improved Transformer model with an enhanced attention mechanism based on the terms of the sample contract and their corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; obtains a contract to be reviewed, and uses the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed; uses a pre-trained classification model to classify the target sentence embedding representation matrix, and obtains a review result of whether the contract to be reviewed is missing a clause based on the classification result. Because the improved Transformer model enhances the original self-attention mechanism of the Transformer model, the attention distribution of the improved Transformer model is no longer dispersed, which can effectively improve the training speed and model accuracy during training. This allows the trained improved Transformer model to more quickly obtain a sentence embedding representation matrix that can accurately represent the semantics of the terms of the contract to be reviewed. Then, based on the sentence embedding representation matrix of the terms of the contract to be reviewed, contract review is implemented through a classification model, greatly improving review efficiency and review accuracy.

[0080] Please refer to Figure 3 , Figure 3A schematic block diagram of an artificial intelligence-based contract review device provided in an embodiment of the present application.

[0081] like Figure 3 As shown, the device 300 includes: an analysis module 301 , a training module 302 , a prediction module 303 and a classification module 304 .

[0082] The analysis module 301 is used to obtain a sample contract, perform syntactic analysis on the clauses of the sample contract, and obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract;

[0083] A training module 302 is configured to train the improved Transformer model enhanced with the attention mechanism based on the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model;

[0084] Prediction module 303, for obtaining a contract to be reviewed, and using the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to a clause of the contract to be reviewed;

[0085] The classification module 304 classifies the target sentence embedding representation matrix using a pre-trained classification model, and obtains a review result of whether the contract to be reviewed is missing any clauses based on the classification result.

[0086] It should be noted that those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned embodiment of the artificial intelligence-based contract review method, and will not be repeated here.

[0087] The apparatus provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 4 Runs on the computer equipment shown.

[0088] See also Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a personal computer (PC), a server, or other device with data processing capabilities.

[0089] like Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0090] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause a processor to execute any one of the artificial intelligence-based contract review methods.

[0091] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0092] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any artificial intelligence-based contract review method.

[0093] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0094] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0095] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0096] A sample contract is obtained, and the terms of the sample contract are syntactically analyzed to obtain a syntactic analysis binary tree corresponding to the terms of the sample contract; an improved Transformer model enhanced with an attention mechanism is trained based on the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; a contract to be reviewed is obtained, and the trained improved Transformer model is used to predict a target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed; the target sentence embedding representation matrix is ​​classified using a pre-trained classification model, and a review result of whether the contract to be reviewed is missing a clause is obtained based on the classification result.

[0097] In some embodiments, the processor implements the training of the improved Transformer model enhanced with the attention mechanism based on the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain the trained improved Transformer model, and is used to implement:

[0098] Inputting the terms of the sample contract and their corresponding syntactic analysis binary trees into the improved Transformer model, so that the enhanced attention mechanism of the improved Transformer model generates a query matrix, a key matrix, and a value matrix according to the terms of the sample contract, and constructs a syntactic analysis masking matrix according to the syntactic analysis binary trees;

[0099] An improved Transformer model is trained according to the query matrix, the key matrix, the value matrix, and the syntactic analysis mask matrix to obtain a trained improved Transformer model.

[0100] In some embodiments, when the processor implements the training of the improved Transformer model according to the query matrix, the key matrix, the value matrix, and the syntactic analysis mask matrix to obtain the trained improved Transformer model, the processor is configured to implement:

[0101] Multiplying the query matrix and the key matrix to obtain a first attention matrix;

[0102] Adding the first attention matrix to the syntactic analysis masking matrix to obtain a second attention matrix;

[0103] Reconcile the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix;

[0104] Multiplying the enhanced new attention matrix by the value matrix to obtain a sentence embedding representation matrix corresponding to the clauses of the sample contract;

[0105] The parameters of the improved Transformer model are updated according to the sentence embedding representation matrix until the improved Transformer model converges to obtain a trained improved Transformer model.

[0106] In some embodiments, when the processor implements the step of reconciling the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix, the processor is configured to implement:

[0107] Obtain a preset harmonic formula alpha*A+(1-alpha)*G, where alpha represents a preset gate mechanism, A represents the first attention matrix, and G represents the second attention matrix;

[0108] Substitute the first attention matrix and the second attention matrix into the preset reconciliation formula for calculation to obtain an enhanced new attention matrix.

[0109] In some embodiments, when the processor implements the step of using the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to a clause of the contract to be reviewed, the processor is configured to implement:

[0110] Performing syntactic analysis on the clauses of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed;

[0111] The clauses of the contract to be reviewed and their corresponding target syntactic analysis binary trees are input into the trained improved Transformer model for prediction to obtain the target sentence embedding representation matrix.

[0112] In some embodiments, when the processor implements the syntactic analysis of the clauses of the contract to be reviewed and obtains the target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed, it is configured to implement:

[0113] Splitting the contract to be reviewed into paragraphs to obtain clauses of the contract to be reviewed;

[0114] The Stanford syntax analysis tool is used to perform syntax analysis on the clauses of the contract to be reviewed to obtain the target syntax analysis binary tree.

[0115] In some embodiments, when the processor implements the classification of the target sentence embedding representation matrix using a pre-trained classification model and obtains the review result of whether the contract to be reviewed is missing a clause based on the classification result, it is configured to implement:

[0116] Inputting the target sentence embedding vector into a pre-trained softmax classifier for classification prediction, and obtaining a probability value of whether the clause of the contract to be reviewed belongs to the preset standard clause category;

[0117] Determine whether a clause of the contract to be reviewed is missing based on the probability value.

[0118] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the artificial intelligence-based contract review method of the present application.

[0119] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., equipped on the computer device.

[0120] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0121] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0122] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0123] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A contract review method based on artificial intelligence, characterized in that: The method comprises the following steps: Obtaining a sample contract, performing syntactic analysis on clauses of the sample contract, and obtaining a syntactic analysis binary tree corresponding to the clauses of the sample contract; Training the improved Transformer model enhanced with the attention mechanism according to the terms of the sample contract and its corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; Obtaining a contract to be reviewed, and using the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to a clause of the contract to be reviewed; The target sentence embedding representation matrix is ​​classified using a pre-trained classification model, and a review result of whether the contract to be reviewed is missing clauses is obtained based on the classification result.

2. The artificial intelligence-based contract review method according to claim 1, characterized in that: The improved Transformer model enhanced with the attention mechanism is trained according to the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model, including: Inputting the terms of the sample contract and their corresponding syntactic analysis binary trees into the improved Transformer model, so that the enhanced attention mechanism of the improved Transformer model generates a query matrix, a key matrix, and a value matrix according to the terms of the sample contract, and constructs a syntactic analysis masking matrix according to the syntactic analysis binary trees; An improved Transformer model is trained according to the query matrix, the key matrix, the value matrix, and the syntactic analysis mask matrix to obtain a trained improved Transformer model.

3. The artificial intelligence-based contract review method according to claim 2, characterized in that: The step of training the improved Transformer model according to the query matrix, the key matrix, the value matrix, and the syntactic analysis mask matrix to obtain a trained improved Transformer model includes: Multiplying the query matrix and the key matrix to obtain a first attention matrix; Adding the first attention matrix to the syntactic analysis masking matrix to obtain a second attention matrix; Reconcile the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix; Multiplying the enhanced new attention matrix by the value matrix to obtain a sentence embedding representation matrix corresponding to the clauses of the sample contract; The parameters of the improved Transformer model are updated according to the sentence embedding representation matrix until the improved Transformer model converges to obtain a trained improved Transformer model.

4. The artificial intelligence-based contract review method according to claim 3, characterized in that: The step of reconciling the first attention matrix and the second attention matrix to obtain an enhanced new attention matrix includes: Obtain a preset harmonic formula alpha*A+(1-alpha)*G, where alpha represents a preset gate mechanism, A represents the first attention matrix, and G represents the second attention matrix; Substitute the first attention matrix and the second attention matrix into the preset reconciliation formula for calculation to obtain an enhanced new attention matrix.

5. The artificial intelligence-based contract review method according to claim 1, characterized in that: The method of using the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to a clause of the contract to be reviewed includes: Performing syntactic analysis on the clauses of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed; The clauses of the contract to be reviewed and their corresponding target syntactic analysis binary trees are input into the trained improved Transformer model for prediction to obtain the target sentence embedding representation matrix.

6. The artificial intelligence-based contract review method according to claim 5, characterized in that: The step of performing syntactic analysis on the clauses of the contract to be reviewed to obtain a target syntactic analysis binary tree corresponding to the clauses of the contract to be reviewed includes: Splitting the contract to be reviewed into paragraphs to obtain clauses of the contract to be reviewed; The Stanford syntax analysis tool is used to perform syntax analysis on the clauses of the contract to be reviewed to obtain the target syntax analysis binary tree.

7. The artificial intelligence-based contract review method according to claim 1, characterized in that: The method of using a pre-trained classification model to classify the target sentence embedding representation matrix and obtaining a review result of whether the contract to be reviewed is missing clauses based on the classification result includes: Inputting the target sentence embedding vector into a pre-trained softmax classifier for classification prediction, and obtaining a probability value of whether the clause of the contract to be reviewed belongs to the preset standard clause category; Determine whether a clause of the contract to be reviewed is missing based on the probability value.

8. A contract review device based on artificial intelligence, characterized in that: The artificial intelligence-based contract review device includes: An analysis module, configured to obtain a sample contract, perform syntactic analysis on the clauses of the sample contract, and obtain a syntactic analysis binary tree corresponding to the clauses of the sample contract; A training module is used to train the improved Transformer model enhanced with the attention mechanism according to the terms of the sample contract and the corresponding syntactic analysis binary tree to obtain a trained improved Transformer model; A prediction module is used to obtain a contract to be reviewed and use the trained improved Transformer model to predict a target sentence embedding representation matrix corresponding to the terms of the contract to be reviewed; The classification module uses a pre-trained classification model to classify the target sentence embedding representation matrix, and obtains a review result of whether the contract to be reviewed is missing any clauses based on the classification result.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the artificial intelligence-based contract review method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the artificial intelligence-based contract review method as described in any one of claims 1 to 7 are implemented.

Images