Contract and law consistency review method based on fine-grained semantic multi-task learning

Abstract

The invention discloses a contract and law consistency review method based on fine-grained semantic multi-task learning. The contract and law consistency review method can be used for legal contract review. According to the method, a legal provision matching task serves as a main task, a BERT model is used for learning coarse-grained representation of a legal text, deeper fine-grained semantics contained in the legal text are extracted by constructing a fine-grained legal semantic enhancement module, and cosine similarity is used for matching legal provisions with high relevancy with input contract terms. According to the method, a legal text classification task is used as an auxiliary task, and the learning ability of legal text features is improved by sharing rich representations learned by legal texts in different tasks. Experiments on a CLSAD data set prove the effectiveness of the method.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and further relates to a contract and legal consistency review method based on fine-grained semantic multi-task learning in the field of electronic digital data processing technology. The present invention mines the fine-grained semantic features of legal contract texts, matches the semantic consistency between contract terms and relevant legal provisions, and thus realizes intelligent review of the consistency between contracts and relevant laws. Background Art

[0002] The technologies related to the intelligent review of the consistency between contracts and relevant laws can be divided into four categories: The first category is contract review tools based on automated analysis technology. This type of method usually outputs analysis results based on keyword categories to complete risk detection. For example, Fenech et al. developed a tool for automatically analyzing contract conflict clauses written in contract language for the case of airline boarding stations. The second category is contract review tools based on entity recognition. This type of method usually performs entity recognition on contracts and outputs different auxiliary information according to different entity categories. For example, Leivaditi et al. developed a language model called ALeaseBERT for lease contracts to extract the contract entities and red flags involved. Xu Yaqiong et al. constructed a contract named entity recognition dataset and designed a deep learning model for contract text element extraction based on the dataset.

[0003] Chongqing Binbo Network Technology Co., Ltd. provides a method and system for contract review warning using artificial intelligence in its patent document "A method and system for contract review warning using AI" (application number: CN 202410405960.X, application publication number: CN 118229224 A). The core steps of the method are as follows: First, train the first contract classification model to identify the general category of the input contract. Then, train the second contract clause classification model to determine the specific category of each clause. For two consecutive clauses identified as the same category in the second model, remove the intersection of their segmentations to generate training samples. Use the samples to train the third model. The output of the third model contains a set of classification labels for a set of basic contract clauses. When reviewing a new contract, the model can be used to determine whether the contract to be reviewed contains all the basic clauses. If missing, the system will automatically output a prompt for missing clauses. The shortcomings of this invention are that although the automatic inspection of the completeness of the contract clauses can help the reviewer quickly identify the contract elements that may be missing, there is a lack of review solutions for the specific semantic content of the elements, and the reviewer cannot obtain the legal basis for the content of the elements through this invention.

[0004] The third category is contract review tools based on key value consistency. This type of method extracts key transaction values, determines contract risks by analyzing whether the previous and subsequent logics are consistent, and provides assistance for automated contract review. For example, Zhang et al. designed an end-to-end framework for paired blank resolution, which completes intelligent contract review by detecting the consistency of blanks reserved in the precompiled format of the contract. The fourth category is the understanding of contract text. This type of method helps lawyers capture the key points of the contract and complete the contract review by judging the key level of the contract text. For example, Hendrycks et al. constructed the contract understanding dataset CUAD (The Contract Understanding Atticus Dataset) to help deep learning models automatically extract and identify key terms from contracts. Wang et al. annotated a large contract reading comprehension dataset MAUD (the MergerAgreement Understanding Dataset). For given transaction key points questions and transaction key points text, the model can use this dataset to predict the correct answer from a list of possible answers.

[0005] Wuyouwulu (Wuxi) Technology Co., Ltd. provides an intelligent self-service contract review method based on a large language model in its patent application "An intelligent self-service contract review method based on a large language model" (application number: CN 202410480404.9, application publication number: CN 118379165A). The implementation steps of this method can be summarized as follows: 1) configure the review point; 2) generate a reference review step based on the review logic of the configured review point through the large model question and answer; 3) contract text segmentation and vectorization. Divide the contract according to common Chinese sentence delimiters or according to the content of the contract chapters, and cut it according to the title and format of the contract; 4) locate the contract clauses. Recall the clauses in the contract related to the review point according to the contract and the review logic; 5) Generate the review results based on the clauses and review logic, design the prompt related to the risk judgment in the review logic according to the recalled clauses and the risk judgment method in the review logic, judge whether there is a risk in the clause through the large model, and generate risk prompts and modification suggestion information according to the needs; 6) Position the review results. This method has two shortcomings: first, the method of generating risk warnings and modification suggestions through natural language description cannot accurately provide users with the legal basis of various contract terms. Second, directly generating sentence-granular encoding of contract terms through vector models will lead to insufficient deep semantic learning of contract terms. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies in the above-mentioned prior art and propose a contract and legal consistency review method based on fine-grained semantic multi-task learning, aiming to solve the problem that the existing contract review methods are insufficient in deep semantic analysis of legal texts and do not fully consider the semantic consistency between contract terms and legal provisions.

[0007] The technical idea to achieve the purpose of the present invention is to build a subnetwork for learning fine-grained semantic features of legal texts by connecting the BERT module and the FLSE module designed by the present invention in series. The coarse-grained semantic features and deeper fine-grained semantic features of legal texts are learned simultaneously through a multi-level semantic extraction structure, so as to improve the learning ability of complex legal texts and improve the shortcomings of existing methods in understanding complex legal texts. Starting from the semantic consistency between contract terms and related legal provisions, the present invention designs a multi-task learning framework, takes legal text semantic matching as the main task, and takes legal text classification as the auxiliary task. By using the correlation between the main task and the auxiliary task, the legal text features are learned from different angles, and the semantics contained in the legal text is deeply understood, thereby improving the model's ability to understand the semantics of the legal text, thereby solving the problem that the existing contract review method does not fully consider and learn the semantic consistency between contract terms and legal provisions. The present invention is aimed at the multi-task loss dynamic fusion strategy of intelligent review of legal contracts. Through the strategy, the weights of different tasks in the model training process are balanced to ensure that the auxiliary task will not cause too much interference to the main task while providing additional semantic information, thereby achieving optimal coordination between tasks and improving the effect of legal text matching.

[0008] To achieve the above object, the present invention comprises the following steps:

[0009] Step 1: Generate training set:

[0010] At least 1,200 contract clauses and their corresponding legal provisions are combined into a sample set; all characters contained in the sample set are extracted to construct a vocabulary mapping table; the legal categories in the sample set are annotated, and the contract clauses, corresponding legal provisions and their legal category labels are combined into a training set; according to the vocabulary mapping table, the legal texts contained in the sample set are sequentially segmented and tokenized to obtain a token sequence of the legal text;

[0011] Step 2: construct a legal text fine-grained semantic feature learning sub-network consisting of a bidirectional encoder representation method (Bidirectional Encoder Representations from Transformers, BERT) module and a fine-grained legal semantics enhancement module (Fine-Grained Legal Semantics Enhancement, FLSE) in series;

[0012] Step 3: construct a contract legal consistency review network consisting of a legal text fine-grained semantic feature learning sub-network, a legal provision matching sub-network, and a legal provision classification sub-network;

[0013] Step 4: Training the contract legal consistency review network:

[0014] The training set is input into the contract legal consistency review network in batches, and the Adam loss function is used to optimize the model parameters. The weight values ​​of the main task and the auxiliary task are dynamically updated until the weighted sum of the main task loss function and the auxiliary task loss function converges, thus obtaining a trained contract legal consistency review network.

[0015] Step 5: Review the contract terms for legal consistency:

[0016] In the same way as step 1, the legal text in the test set is preprocessed to obtain a token sequence; the token sequence is input into the trained contract legal consistency review network to output the matching result of the contract legal consistency.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] First, the present invention realizes the technical idea of ​​constructing a sub-network for learning fine-grained semantic features of legal texts by connecting the BERT module and the FLSE module designed by the present invention in series. The sub-network can simultaneously extract multi-level coarse-grained semantic features and fine-grained semantic features from legal texts, avoiding the problem that existing intelligent contract review methods do not fully understand the deep semantics of legal texts, making the present invention have the advantage of more comprehensive and accurate understanding of the legal meaning of contract terms.

[0019] Second, the present invention constructs a contract legal consistency review network, takes the semantic consistency between contract terms and related legal provisions as the starting point, and uses the feature sharing mechanism between tasks to enhance the model's deep semantic understanding of legal texts, thus realizing the technical idea of ​​designing a multi-task learning framework. This overcomes the problem that the existing intelligent contract review methods do not fully consider the semantic consistency between contract terms and legal provisions, so that the present invention can accurately match the relevant legal provisions according to the content of the contract terms, thereby ensuring the legality and accuracy of the contract.

[0020] Third, when training the contract legal consistency review network, the present invention realizes the technical idea of ​​designing a multi-task loss dynamic fusion strategy for the intelligent review of legal contracts by dynamically balancing the loss weights of the main task and the auxiliary task. The present invention effectively improves the auxiliary role of the auxiliary task to the main task, optimizes the semantic matching performance of the main task, and enhances the model's ability to learn complex legal texts, thereby ensuring the effectiveness and reliability of contract review. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the present invention;

[0022] Figure 2 It is a structural diagram of the fine-grained legal semantics enhancement module of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of the legal text classification module of the present invention;

[0024] Figure 4 This is a visualization diagram of the multi-task loss fusion strategy of the present invention;

[0025] Figure 5 It is a simulation diagram of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments. The invention content is illustrated by way of example and should not be construed as limiting the present invention.

[0027] Reference Figure 1 , further describing the implementation steps of the embodiment of the present invention.

[0028] Step 1: Generate a training set.

[0029] The data set in the embodiments of the present invention adopts the Contract-Legal Semantic Alignment Dataset (CLSAD) collected by the research group of the present invention. The data set is in the data format of text pairs, and each contract clause involves three legal provisions. The contract clauses and legal provisions are pre-processed with word segmentation to obtain the legal text word segmentation vector. Taking the legal text word segmentation vector as a sample, each contract clause and three related legal provisions are a set. See the data examples listed in Table 1. For each legal provision, it is composed of legal text, code category, code category number and the serial number of the legal provision in the code. There are at least six types of contract terms: sales contract, loan contract, gift contract, carriage contract, technology contract and lease contract.

[0030] Table 1 Data example table in the sample set

[0031]

[0032] In the embodiment of the present invention, the steps of pre-processing the legal provisions and contract terms by segmentation and generating the corresponding token sequence are as follows:

[0033] The first step is to segment the text contained in the sample set at the character level to obtain a segmented legal text set and complete word segmentation.

[0034] The second step is to query the mapping position of each character in the segmented legal text set in the vocabulary mapping table one by one, use the position token to replace the original character, and obtain a token sequence.

[0035] Step 2: Construct a sub-network for learning fine-grained semantic features of legal texts, which is composed of a BERT module and a fine-grained legal semantic enhancement module connected in series.

[0036] The legal text fine-grained semantic feature learning sub-network is used to simultaneously learn the coarse-grained features and fine-grained semantic features of the legal text. By integrating the coarse-grained and fine-grained features, the ability of the contract legal consistency review network to understand the legal text is enhanced.

[0037] Step 2.1, the bidirectional encoder representation method BERT (Bidirectional Encoder Representations from Transformers) module is a model using the 'bert-base-chinese' pre-trained parameters to learn the legal provisions L emb and Contract Clause C emb The coarse-grained features of are expressed as follows:

[0038]

[0039] Among them, BERT(·) represents the BERT model, C bert represents the coarse-grained features of the contract terms, L bert Represents the coarse-grained features of legal provisions.

[0040] Step 2.2, build a fine-grained legal semantics enhancement module.

[0041] Figure 2 FIG. 1 is a schematic diagram of the structure of the fine-grained legal semantics enhancement module of the present invention. Figure 2 , the structure of the fine-grained legal semantics enhancement module is further described.

[0042] The fine-grained legal semantic enhancement module is composed of a linear mapping layer, a maximum pooling layer, a multi-head attention submodule, an average pooling layer, a normalization layer and a reshape layer, and is used to extract fine-grained semantic features of legal texts. The input of this module is the coarse-grained features output by the BERT module. In the first part, the coarse-grained features are respectively sent to the maximum pooling layer, the series structure of the multi-head attention submodule and the average pooling layer, and the linear mapping layer. In the second part, the outputs of the series structure of the maximum pooling layer, the multi-head attention submodule and the average pooling layer are added, and the addition result is sent to the normalization layer, the reshape layer and the linear mapping layer. Finally, the output of the second part is added to the output of the linear mapping layer of the first part to obtain the fine-grained semantic features of the legal text as the output of the fine-grained legal semantic enhancement module. The output dimension of the linear mapping layer is set to 384, and the number of multi-head attention submodules is set to 8; the normalization layer is implemented using the function l2norm.

[0043] Based on the coarse-grained features of contract terms C bert Taking the learning process of as an example, the process of fine-grained semantic feature extraction is explained.

[0044] The first step is to bert Perform the maximum pooling operation and extract C bert The most notable parts:

[0045] C max =MaxPool(C bert )

[0046] Among them, MaxPool(·) represents the maximum pooling layer, C max represents the features extracted by the maximum pooling layer. This method can enhance the ability of the legal text fine-grained semantic feature learning sub-network to capture key information. Then, the Multi-Head Attention module used by Vaswani et al. in the Transformer model is used to enhance C bert The contextual relationship contained in the model can improve the model's ability to represent features.

[0047] C′=MultiAtten(C bert )

[0048] Among them, C′ represents the features output by the multi-head attention module.

[0049] In the second step, the average pooling layer is used to further extract the feature information contained in C′:

[0050] C avg =AvgPool(C′)

[0051] Among them, AvgPool(·) represents the average pooling layer, Cavg Represents the features extracted by the average pooling layer.

[0052] The third step is to convert C avg and C max The information contained in is integrated:

[0053] C pool =(1-α)×C avg +α×C max

[0054] Among them, C pool Represents C avg and C max The fusion result. The comparative experiment shows that the best effect is achieved when the value of α is 0.8.

[0055] Step 4: C pool Normalization and reshaping are performed to ensure that the extracted features have the same scale in the vector space for subsequent text similarity measurement:

[0056] C 1 =Reshape(Norm(C pool ))

[0057] Among them, Reshape(·) represents the shape reshaping layer, and Norm(·) represents the normalization layer.

[0058] The fifth step is to use two linear connection layers to process C separately. 1 and C bert , complete the linear addition:

[0059] C=θC 1 +θ(C bert )

[0060] Among them, Θ represents the linear connection layer, and C represents the contract terms feature that combines both coarse-grained and fine-grained features.

[0061] Through the above steps, the deep and fine-grained semantics extraction of legal provisions and contract terms is achieved.

[0062] Step 3: Construct a contract legal consistency review network consisting of a legal text fine-grained semantic feature learning subnetwork, a legal provision matching subnetwork, and a legal provision classification subnetwork.

[0063] In step 3.1, the legal text matching subnetwork consists of the dot product attention calculation module and the cosine similarity calculation module improved by Lee et al.

[0064] The improved dot product attention calculation module is used to calculate the i-th contract clause feature C to be matched iand the jth legal provision feature L j Before the cosine similarity, capture the association between them. By calculating C i With L j The attention weights between the nth regions in are used to capture the association, which is specifically implemented by the following formula:

[0065]

[0066] Among them, Attn(·) represents the improved dot product attention calculation module, λ 1 represents the reciprocal temperature parameter of the Softmax function, and the setting value is 9. Sim(·) represents the cosine similarity function. Indicates the feature L of the jth legal provision j The nth region of j exp(·) represents the exponential function with the natural constant e as the base.

[0067] The cosine similarity calculation module is used to calculate the i-th contract clause C to be matched i and Article j of the law L j The cosine similarity between is implemented by the following formula:

[0068]

[0069] Among them, the superscript T represents the transpose operation, and ||·|| represents the L2 norm operation.

[0070] According to the terms of the contract to be matched C i The similarity values ​​of all legal text features L are sorted to obtain the matching results, which is specifically implemented by the following formula:

[0071] result=Rank(Sim(C i ,L))

[0072] Among them, Rank(·) represents the feature C of the contract terms to be matched. i The cosine similarity between all legal clause features L is performed in descending order, and the result is the sum of all legal clause features L and the contract clause features C to be matched. i The consistency sorting results are obtained, and the three relevant legal provisions with the highest matching degree with the current contract terms are obtained according to the sorting results.

[0073] Step 3.2: construct the legal text classification subnetwork corresponding to the auxiliary task to complete the semantic classification of legal texts.

[0074] Figure 3 This is the structure diagram of the legal text classification module. Figure 3, further describe the structure of the legal text classification sub-network.

[0075] The input of the legal text classification subnetwork is the legal text features output by the fine-grained legal semantics enhancement module. The input is sent to the averaging layer and the linear mapping layer respectively, and the results are multiplied. The multiplication results are then sent to the Dropout layer, the linear mapping layer, and the Softmax layer in turn to obtain the final classification result.

[0076] In the first step, the input legal provisions features are processed using the averaging layer. By averaging the input legal provisions features, the scale differences between different features are eliminated:

[0077] L mean =Mean(L)

[0078] Among them, Mean(·) represents the averaging layer, L mean Represents the averaged characteristics of legal provisions.

[0079] In the second step, the input legal text feature L is linearly transformed according to the following formula and mapped to a new space for enhancement:

[0080] L' = Θ(L).

[0081] Calculate L and L according to the following formula: mean The product operation result is:

[0082]

[0083] Dropout is applied to randomly discard some neurons to reduce the risk of overfitting of the model.

[0084] In the third step, according to the following formula, the product result is mapped to the new feature space through linear mapping, and the activation function Softmax is used to complete the nonlinear transformation, further enhancing the model's ability to capture the complex semantics of the legal text. Finally, the classification results of the generated legal text are as follows:

[0085] L class = Dropout(Θ(Softmax(L″)))

[0086] Among them, L class represents the final classification result, Dropout(·) represents the Dropout layer, and Softmax(·) represents the Softmax activation function layer.

[0087] Step 4: Train the contract legal consistency review network.

[0088] The training set is input into the contract legal consistency review network in batches, and the Adam loss function is used to optimize the model parameters. The weight values ​​of the main task and the auxiliary task are dynamically updated until the weighted sum of the main task loss function and the auxiliary task loss function converges to obtain a trained contract legal consistency review network.

[0089] The main task loss function is as follows:

[0090]

[0091] Among them, Loss(·) represents the main task loss value obtained by the triple loss function, β represents the margin hyperparameter, which is set to 0.2. The characteristics of negative samples of legal provisions, which refer to legal provisions that do not match the contract terms to be reviewed. Characterizes the contractual terms that are not related to L.

[0092] The auxiliary task loss function is as follows:

[0093]

[0094] Among them, Loss law (·) represents the auxiliary task loss value obtained by the cross entropy loss function, y represents the true category label of the legal text, It represents the predicted label of the legal text output by the legal text classification subnetwork, and n represents the number of samples in the current training batch.

[0095] Figure 4 This is a visualization diagram of the multi-task loss fusion strategy of the present invention. Figure 4 , further describe the multi-task loss fusion strategy:

[0096] Start the multi-task parallel framework when the model is trained to the 20th round, and initialize the loss to 1: weight 1 , loss 2: weight 2 The loss value weight is updated every 10 rounds, and the update method is as follows:

[0097] weight 1 =min(10,1.5×weight 1 )

[0098] weight 2 =max(0.05,0.5×weight 2 )

[0099] Among them, weight 1 Indicates the weight of the main task loss value 1, weight2 Indicates the weight of the auxiliary task loss value 2.

[0100] According to the following formula, calculate the multi-task loss Loss total .

[0101] Loss total =weight 1 ×Loss(·)+weight 2 ×Loss law (·)

[0102] Step 5: Review the contract terms for legal consistency.

[0103] In the same way as step 1, the test set of at least 100 samples is preprocessed to obtain a token sequence. The token sequence is input into the trained contract legal consistency review network, and the matching result of the contract legal consistency is output. For the legal text in the test set, the contract legal consistency review network will perform consistency matching based on the semantics of the contract clauses to be matched, and return 3 semantically related legal clauses as matching results, as shown in Table 2.

[0104] Table 2 Contract legal consistency matching example

[0105]

[0106] The effect of the present invention can be further illustrated by the following simulation.

[0107] 1. Simulation conditions.

[0108] The software platform of the simulation experiment of the present invention is: Ubuntu20.04 operating system, Python3.10 programming language, PyTorch2.0.0 training framework, and parallel computing architecture CUDA11.7.

[0109] The hardware platform of the simulation experiment of the present invention is: graphics card NVIDIA GeForce RTX 3060GPU 12GB.

[0110] The sample set used in the simulation experiment of the present invention is the Contract-Legal Semantic Alignment Dataset (CLSAD) data set collected and produced by the research group of the present invention in 2024.

[0111] 2. Analysis of simulation content and simulation results:

[0112] The simulation experiment of the present invention uses the present invention and an existing technology (BERT model) to match the input contract terms with relevant legal provisions to obtain matching results.

[0113] The prior art BERT model refers to the bidirectional encoder representation method proposed by Devlin et al. in “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.”, referred to as the BERT model.

[0114] Combine the following Figure 5 The simulation diagram of the present invention is further described.

[0115] Figure 5 In order to adopt the BERT model of the prior art, the fine-grained semantic enhancement module and multi-task learning framework designed by the present invention are added one by one on top of the BERT model, and the comparison charts of the recall rates R@1, R@5, and R@10 of these three technologies in the legal text matching task are tested respectively. Specifically, R@1, R@5, and R@10 represent the probability of finding at least one relevant legal text in the first, top 5, and top 10 matching results, respectively.

[0116] Depend on Figure 5 It can be seen that on the basis of the BERT model, adding the fine-grained legal semantics enhancement module and multi-task learning framework designed by the present invention can improve the matching recall rate of the contract legal consistency review network, proving that the present invention can improve the effect of legal text matching.

[0117] The comparison results of the matching recall rate of the present invention on the sample set test set are plotted in Table 3.

[0118] Table 3 Comparison of matching recall rates of the present invention in simulation experiments

[0119]

[0120] As can be seen from Table 3, when the BERT model used in the present invention is embedded in a fine-grained semantic enhancement module under a multi-task framework, the probability of finding at least one relevant legal provision in the first, top 5, and top 10 matching results returned is the highest, which indicates that the contract legal consistency review network has the best performance under the above combination, which can reach: R@1:90.28; R@5:97.22; R@10:97.92.

[0121] The above simulation experiments show that the contract legal consistency review method designed by the present invention based on fine-grained semantic multi-task learning can learn the deep fine-grained semantic features contained in the legal text. By using the legal text classification task as an auxiliary task and sharing the rich representations learned by the legal text in different tasks, the learning ability of the legal text features can be improved, and the legal provisions with a high degree of relevance to the input contract terms can be matched. This solves the problem that the existing contract review methods are insufficient in deep semantic analysis of legal texts and do not fully consider the semantic consistency between contract terms and legal provisions. It is a very practical legal provision matching method.

Claims

1. A contract and legal consistency review method based on fine-grained semantic multi-task learning, characterized in that: Construct a sub-network for learning fine-grained semantic features of legal texts, use the triplet loss function to calculate the main task loss, use the cross entropy loss function to calculate the auxiliary task loss, and realize dynamic fusion of multi-task losses; the steps of the review method include the following: Step 1: Generate training set: At least 1,200 contract clauses and their corresponding legal provisions are combined into a sample set; all characters contained in the sample set are extracted to construct a vocabulary mapping table; the legal categories in the sample set are annotated, and the contract clauses, corresponding legal provisions and their legal category labels are combined into a training set; according to the vocabulary mapping table, the legal texts contained in the sample set are sequentially segmented and tokenized to obtain a token sequence of the legal text; Step 2: construct a sub-network for learning fine-grained semantic features of legal texts, which consists of a BERT module and a fine-grained legal semantic enhancement module connected in series; Step 3: construct a contract legal consistency review network consisting of a legal text fine-grained semantic feature learning sub-network, a legal provision matching sub-network, and a legal provision classification sub-network; Step 4: Training the contract legal consistency review network: The training set is input into the contract legal consistency review network in batches, and the Adam loss function is used to optimize the model parameters. The weight values ​​of the main task and the auxiliary task are dynamically updated until the weighted sum of the main task loss function and the auxiliary task loss function converges, thus obtaining a trained contract legal consistency review network. Step 5: Review the contract terms for legal consistency: In the same way as step 1, the legal text in the test set is preprocessed to obtain a token sequence; the token sequence is input into the trained contract legal consistency review network to output the matching result of the contract legal consistency.

2. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The contract terms in the sample set described in Step 1 are derived from at least six categories of legal contracts.

3. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: Extracting all characters contained in the sample set in step 1 and constructing a vocabulary mapping table refers to assigning a unique index position to each character appearing in the sample set to form a vocabulary mapping table.

4. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The steps of word segmentation and tokenization preprocessing described in step 1 are as follows: The first step is to segment the text contained in the sample set at the character level to obtain a segmented legal text set and complete word segmentation; The second step is to query the mapping position of each character in the segmented legal text set in the vocabulary mapping table one by one, use the position token to replace the original character, and obtain a token sequence.

5. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The BERT module described in step 2 is a model using the 'bert-base-chinese' pre-training parameters.

6. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The fine-grained legal semantic enhancement module described in step 2 consists of a linear mapping layer, a maximum pooling layer, a multi-head attention submodule, an average pooling layer, and a normalization layer; the output dimension of the linear mapping layer is set to 384, and the number of multi-head attention submodules is set to 8; the normalization layer is implemented using the function l2norm.

7. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The legal text matching subnetwork described in step 3 is composed of an improved dot product attention calculation module and a cosine similarity calculation module; wherein the cosine similarity calculation module is implemented by the following formula: Among them, Sim(·) represents the cosine similarity function, C i represents the characteristics of the i-th contract clause, L j Indicates the characteristics of the jth legal provision, the superscript · T represents the transpose operation, and ||·|| represents the L2 norm operation.

8. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The legal text classification subnetwork described in step 3 is composed of a feature mapping layer, a Dropout layer, a linear mapping layer, and a Softmax layer connected in series in sequence; the feature mapping layer includes an averaging layer and a linear mapping layer.

9. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The main task loss function described in step 4 is as follows: Among them, Loss(·) represents the main task loss value obtained by the triple loss function, β represents the margin hyperparameter, ranging from 0 to 1, Represents the characteristics of negative samples of legal provisions, Characterizes the contractual terms that are not related to L.

10. The contract and legal consistency review method based on fine-grained semantic multi-task learning according to claim 1 is characterized in that: The auxiliary task loss function described in step 4 is as follows: Among them, Loss law (·) represents the auxiliary task loss value obtained by the cross entropy loss function, y represents the true category label of the legal text, It represents the predicted label of the legal text output by the legal text classification subnetwork, and n represents the number of samples in the current training batch.

Images