A contract risk detection method and device based on a large language model

Abstract

The application discloses a contract risk detection method and device based on a large language model, and the method comprises the following steps: 1) classifying an input contract, including word embedding, capturing context features, learning features through an attention mechanism, and classification; each word in the contract text after word segmentation is converted into a vector representation with specific features, then a BiLSTM and a CNN neural network model are combined to capture features, then relevant features are learned through an attention mechanism, finally, the vector output by the attention mechanism layer is input into a full connection layer, dimensional change is carried out in the full connection layer, and the feature vector dimension is mapped to the contract text label dimension; 2) retrieval enhancement, including a hybrid retrieval composed of sparse retrieval and dense retrieval, the sparse retrieval calculates the relevance of all words q 1 ,q 2 ,...,q n in the input query and documents, and the dense retrieval calculates the cosine similarity of the input query and document embedding vectors. Finally, the results recalled by the two retrieval methods are combined, and a reciprocal ranking fusion algorithm is used to calculate the score of the result returned by the hybrid retrieval. 3) reordering: through a trained reordering model, complex relationships and context information in legal regulations documents and the query are captured, and the ability is utilized to enhance the relevance of information retrieval recall. The application improves the accuracy of contract risk detection.

Description

Technical Field

[0001] This invention relates to the field of contract risk detection, and specifically to a contract risk detection method and apparatus based on a large language model. Background Technology

[0002] Research on contract risk detection based on large language models mainly faces the following difficulties:

[0003] (1) Low accuracy. Large language models have been extensively trained on a large amount of text data, making them very useful in various tasks such as language translation, question answering, and text generation. However, their performance in specific vertical fields is unsatisfactory. Directly applying large language models to the field of contract risk detection may lead to unsatisfactory accuracy in contract risk assessment, optimization schemes, and other modification suggestions, thereby causing losses to users;

[0004] (2) Information cannot be updated in a timely manner. When external information is updated, since this data did not exist when the large language model was trained, the large language model has not learned the features of the relevant information, which may cause the large model to return the wrong answer.

[0005] In order to solve these problems, there is an urgent need to provide an effective method for contract risk detection in the current complex environment. Summary of the Invention

[0006] To overcome the shortcomings of existing contract risk assessment methods, this invention proposes a contract risk detection method and apparatus based on a large language model.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] The first aspect of this invention relates to a language model-based contract risk detection method, comprising the following steps:

[0009] 1) Contract classification: A combination of BiLSTM and CNN neural network models is used; the BiLSTM model is used to capture the contextual features of the contract text, while the CNN model is used to capture the local salient features of the sequence.

[0010] 2) Retrieval Enhancement: First, sparse retrieval is used to calculate the relevance between the input query and the document. At the same time, dense retrieval is used to calculate the cosine similarity between the embedded query and the document to evaluate the similarity between the query and the document. The relevance of the information retrieval system is enhanced by reordering the results returned by dense and sparse retrieval.

[0011] 3) Reordering: By capturing complex relationships and contextual information in legal and regulatory documents and queries, we can leverage their capabilities to enhance the relevance of information retrieval recall.

[0012] Preferably, the contract classification in step 1) specifically includes:

[0013] Step 101, word embedding, converts each word in the segmented contract text into a vector representation with specific features. In the word vector representation stage of the contract text, the original text sequence is transformed into a vector sequence of words using the vector matrix of the searched words, thereby performing a vector representation of the contract text; assuming the model input... ,in This indicates the first sentence in the contract text. 10 words, find the word vector matrix to obtain Corresponding word vectors ;vector The sentence vector obtained after the word embedding layer is calculated using the following formula:

[0014] (1-1)

[0015] Step 102: Capture contextual features by combining two neural network models, BiLSTM and CNN. The BiLSTM model is used to capture the contextual features of the contract text, while the CNN model is used to capture the local salient features of the sequence.

[0016] Step 103: Attention mechanism learns features. The attention mechanism allows the neural network to focus on important parts during the processing of the input sequence, thereby improving the performance and efficiency of the model. In the contract text classification task, the attention mechanism can learn the weights of different words in the input contract text, thereby adaptively focusing on important words and improving the accuracy and interpretability of contract classification.

[0017] Step 104, Classification: Classify the vector output by the attention mechanism layer. The input is fed into a fully connected layer, where dimensionality transformation is performed to map the feature vector dimension to the contract text label dimension. Finally, a softmax classifier is used to calculate a score for each category, which is then converted into a probability value. The category with the highest score is then selected as the output category, as shown in the following formula:

[0018] (1-12)

[0019] Preferably, the search enhancement in step 2) specifically includes:

[0020] Step 201, sparse retrieval, mainly calculates all words in a query. The scores are then summed based on the relevance of the documents to the text; the final text similarity result is obtained.

[0021] (2-1)

[0022] Step 202, Dense retrieval, constructs vector representations of the query and legal documents, and for each input query, its embedding is computed at search time, and then cosine similarity is used to evaluate the similarity between the query and the legal documents;

[0023] Step 203, Hybrid Retrieval: Combine the results of the two retrieval methods using a hybrid retrieval method. Calculate the score of the legal and regulatory text block returned by the hybrid search using the Reverse Rank Fusion (RRF) algorithm. The formula for calculating the RRF score is as follows:

[0024] (2-5)

[0025] Preferably, the reordering in step 3) specifically includes:

[0026] The reordering model is used to reorder the text blocks returned by the hybrid search; its ability to enhance the relevance of the information retrieval system is enhanced by capturing complex relationships and contextual information in legal and regulatory documents and queries.

[0027] The Bge-reranker-base model is used as the reranking model, based on conditional probability. Sort the text, where It's a query. This represents the retrieved legal and regulatory documents; the model processes both queries and legal and regulatory documents simultaneously; on the other hand, bge-reranker-base is a sequence-to-sequence reorderer that uses the T5 model to generate relevance scores between queries and legal and regulatory documents; the T5 model treats all tasks as text-to-text, requiring some adjustments for the query-legal and regulatory document similarity task; during training, the format "query: {query}, document: {document};" is used, with a label of yes if the legal and regulatory document is relevant to the query, and no otherwise; during inference, the query and legal and regulatory documents are formatted using the same format as the training data; then, the score is obtained by calculating the softmax value by considering only the no and yes labels and selecting the value corresponding to the yes label.

[0028] A second aspect of the present invention relates to a contract risk detection device based on a large language model, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a contract risk detection method based on a large language model according to the present invention.

[0029] A third aspect of the present invention relates to a computer-readable storage medium, characterized in that it stores a program thereon, which, when executed by a processor, implements a contract risk detection method based on a large language model according to the present invention.

[0030] The technical concept of this invention is as follows: With the continuous expansion and deepening of commercial activities, the number of various contracts signed is increasing year by year, making contract review increasingly important. This invention proposes a contract risk detection method based on a large language model. Text features are extracted and classified using BiLSTM and CNN, and then legal and regulatory texts are recalled and ranked through hybrid retrieval and re-ranking to accurately return laws and regulations related to contract risks. This invention aims to improve the accuracy of contract risk detection and return relevant reference laws and regulations.

[0031] The advantages of this invention are: 1) it improves the accuracy of contract risk detection and returns relevant legal and regulatory basis; 2) it improves the efficiency of contract risk detection; and 3) it can update external information in real time. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method of the present invention.

[0033] Figure 2 This is a diagram of the LSTM model of the present invention.

[0034] Figure 3 This is a diagram of the BiLSTM model of the present invention.

[0035] Figure 4 This is a diagram of the hybrid search mode of the present invention.

[0036] Figure 5 This is a flowchart of the reordering model reasoning process of the present invention. Detailed Implementation

[0037] The invention will now be further described with reference to the accompanying drawings.

[0038] Example 1

[0039] Reference Figure 1 This embodiment relates to a contract risk detection method based on a large language model, the steps of which are as follows;

[0040] 1) Contract classification;

[0041] 1.1) Word embedding: This process converts each word in the segmented contract text into a vector representation with specific features. In the word vector representation stage, the original text sequence is transformed into a word vector sequence using a lookup word vector matrix, thus achieving a vector representation of the contract text. Assume the model input... ,in This indicates the first sentence in the contract text. 10 words, find the word vector matrix to obtain Corresponding word vectors .vector The sentence vector obtained after the word embedding layer is calculated using the following formula:

[0042] (1-1)

[0043] 1.2) Capturing Contextual Features: Combining BiLSTM and CNN Neural Network Models. The BiLSTM model is used to capture the contextual features of the contract text, while the CNN model is used to capture the local salient features of the sequence. 1.2.1) The LSTM transmission direction is unidirectional from front to back. The LSTM recurrent unit structure mainly includes three gates and two states: the input gate, the forget gate, and the output gate. Its structure is as follows: Figure 2 As shown.

[0044] Given the input at time t Hidden state The calculation formula is as follows:

[0045] (1-2)

[0046] (1-3)

[0047] (1-4)

[0048] (1-5)

[0049] (1-6)

[0050] (1-7)

[0051] , , These represent the states of the forget gate, input gate, and output gate at time t. Let be the candidate state calculated by the tanh function at time t. It represents the state of memory cells at time t. It's the sigmod function. This is the corresponding weight matrix. This is the corresponding bias.

[0052] Because contract texts have strong internal information relationships, the contextual information of the contract text has a significant impact on the classification results. Therefore, a BiLSTM model is constructed using two directional LSTM layers to model the contract text context. The advantage of the BiLSTM model is that it can bidirectionally encode the contract text, simultaneously extracting forward and backward semantic features of the contract context, thus obtaining more comprehensive information. Furthermore, BiLSTM effectively solves the problems of vanishing and exploding gradients in computation through its unique gating mechanism, thus better handling the semantic dependencies of long texts. The BiLSTM is formed by concatenating the forward and backward layers of an LSTM to extract and preserve the contract contextual information. The BiLSTM network structure is as follows: Figure 3 Hidden layer outputs of forward and backward LSTM units. and The results are obtained from equations (1-2) to (1-7), and then... and By concatenating the layers, we obtain the hidden layer output of the BiLSTM at time t. Finally, the output vector after the BiLSTM layer is .

[0053] 1.2.2) The CNN model further extracts contract text features based on the bidirectional semantic features captured by the BiLSTM layer. The CNN contains convolutional layers and pooling layers to capture key thematic features in the contract text for contract text classification. In the CNN model, the convolutional layer is one of the most important components, containing multiple convolutional kernels for convolution calculations. By performing convolution calculations on a local window and convolutional kernels of the input contract text data, significant contract text classification features are extracted. The formula for the convolution operation is shown in Equation (1-8).

[0054] (1-8)

[0055] In equation (1-8): is the ReLU activation function; Represents the convolution kernel ,in The height of the convolution kernel. The width of the convolution kernel is denoted by , and its size is equal to the output feature dimension of the BiLSTM unit. These are word vectors input from the BiLSTM layer; For bias terms; This is the result of the convolution calculation. The convolution window, with a stride of 1, slides downwards to obtain the local feature vector. .

[0056] Pooling layers are a crucial component in convolutional neural networks. They reduce the dimensionality of features through downsampling and prevent overfitting while reducing the number of model parameters. There are two main pooling operations: max pooling and average pooling. These help the network better capture the importance of features, further improving model performance. Here, we choose max pooling, which typically preserves key features better in contract text classification. The calculation formula is: Finally, all sampled feature values ​​are merged into... As the output of the CNN layer.

[0057] 1.3) Attention Mechanism for Feature Learning: The attention mechanism allows the neural network to focus on important parts during the processing of the input sequence, thereby improving the model's performance and efficiency. In contract text classification tasks, the attention mechanism can learn the weights of different words in the input contract text, thus adaptively focusing on important words and improving the accuracy and interpretability of contract classification. In other words, the attention mechanism can help the classifier better understand the key information in the input contract text, thereby improving the effectiveness and interpretability of contract text classification. This approach can effectively increase the impact of keyword information on the classification of contract clauses, further improving the accuracy of contract classification. The output obtained through the hybrid neural network layer is sent to the attention mechanism layer, and the processing of the attention mechanism is shown in equations (1-9) to (1-11).

[0058] (1-9)

[0059] (1-10)

[0060] (1-11)

[0061] After using the CNN layer, a vector is obtained. This vector consists of the output of the CNN. Input vector The output vector is obtained after processing by equations (1-9) to (1-11). , and These are all parameters that change during model training. In this process, the attention mechanism enhances the neural network model's focus on contract text keywords by calculating the weights of different components, thereby increasing the contribution of keywords to the classification of contract clauses and improving the accuracy of classification.

[0062] 1.4) Classification: The vector output by the attention mechanism layer... The input is fed into a fully connected layer, where dimensionality transformation is performed to map the feature vector dimension to the contract text label dimension. Finally, a softmax classifier is used to calculate a score for each category, converting it into a probability value, and then selecting the category with the highest score as the output category. The calculation formula is shown below:

[0063] (1-12)

[0064] 2) Enhanced search:

[0065] 2.1) Sparse retrieval, which mainly calculates all words in a query. The relevance of the text to the document is calculated, and then the scores are summed. The final text similarity result is obtained.

[0066] (2-1)

[0067] in This represents the weight of the i-th word, calculated using the TF-IDF algorithm. The second term of the formula... This indicates the relevance between each word in the query and the article segment d. The calculation formula is:

[0068] (2-2)

[0069] (2-3)

[0070] (2-4)

[0071] N represents the total number of documents d in all documents D. This represents the number of articles containing this word. To avoid the denominator in the logarithm being equal to 0, 0.5 is added to both the numerator and denominator. This 0.5 is called the tuning coefficient. The smaller the value, the larger the IDF value, indicating a greater weight for the word. In equations (2-3) and (2-4)... They are all regulatory factors. express Frequency of occurrence in the query;

[0072] 2.2) Dense retrieval: The ADA-002 model was used in dense retrieval. This model mainly adopts a dual encoder architecture to construct vector representations for all available legal and regulatory document blocks. For each input query, its embedding is calculated at search time. Subsequently, cosine similarity is used to evaluate the similarity between the query and the legal and regulatory documents.

[0073] 2.3) Hybrid retrieval: The results of two retrieval methods are combined using a hybrid retrieval method. The Reverse Rank Fusion (RRF) algorithm is used to calculate the score of the legal and regulatory text block returned by the hybrid search. The formula for calculating the RRF score is as follows:

[0074] (2-5)

[0075] in This indicates the position of document d retrieved by the search mechanism r. The term k is introduced to help control the outlier system.

[0076] Figure 4 This shows how to calculate document blocks. In the diagram, there are four blocks that were retrieved in different orders using two search methods (sparse search and dense search). The reciprocal ranking score for each returned text block is calculated, and these values ​​are then summed to create a new score. The final mixed list is the sorted list of blocks using this new score.

[0077] 3) Reordering.

[0078] 3.1) The text returned by the hybrid search is relevant to the user's query, but the degree of relevance is unclear. Although Reverse Rank Fusion (RRF) can obtain a relevance score for the returned documents, the score obtained in this way is relatively coarse, and may be inaccurate when two legal documents are similar. Here, we use a reordering model to reorder the text blocks returned by the hybrid search.

[0079] By capturing complex relationships and contextual information within legal and regulatory documents and queries, we leverage their capabilities to enhance the relevance of information retrieval systems. Here, we use the Bge-reranker-base model as the reranking model. It works by using conditional probability... This is achieved by sorting the text, where It's a query. This indicates the legal and regulatory documents returned by the retrieval. This model processes both the query and the legal and regulatory documents simultaneously. This simultaneous processing leads to richer interactions between them, typically improving efficiency.

[0080] On the other hand, bge-reranker-base is a sequence-to-sequence reorderer, such as Figure 5The example uses the T5 model to generate relevance scores between queries and legal documents. The T5 model treats all tasks as text-to-text, requiring some adjustments for the query-legal document similarity task. During training, the format "query: {query}, document: {document};" is used, with a label of "yes" if the legal document is relevant to the query, and "no" otherwise. During inference, the query and legal documents are formatted using the same format as the training data. Scores are then obtained by calculating the softmax value and selecting the value corresponding to the "yes" label, considering only the "no" and "yes" labels.

[0081] Example 2

[0082] This embodiment relates to a contract risk detection device based on a large language model, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a contract risk detection method based on a large language model according to the present invention.

[0083] Example 3

[0084] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements a contract risk detection method based on a large language model according to the present invention.

[0085] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of this invention also includes equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A contract risk detection method based on a large language model, characterized in that: The specific steps are as follows: 1) Contract classification: A combination of BiLSTM and CNN neural network models is used; the BiLSTM model captures the contextual features of the contract text, and the CNN model captures the local salient features of the sequence. 2) Retrieval Enhancement: First, sparse retrieval is used to calculate the relevance between the input query and the document. At the same time, dense retrieval is used to calculate the cosine similarity between the embedded query and the document to evaluate the similarity between the query and the document. The relevance of the information retrieval system is enhanced by reordering the results returned by dense and sparse retrieval. 3) Reordering: By capturing complex relationships and contextual information in legal and regulatory documents and queries, its capabilities are leveraged to enhance the relevance of information retrieval recall; The contract classification in step 1) specifically includes: Step 101, word embedding, converts each word in the segmented contract text into a vector representation with specific features. In the word vector representation stage of the contract text, the original text sequence is transformed into a vector sequence of words using the vector matrix of the searched words, thereby performing a vector representation of the contract text; assuming the model input... ,in This indicates the first sentence in the contract text. 10 words, find the word vector matrix to obtain Corresponding word vectors ;vector The sentence vector obtained after the word embedding layer is calculated using the following formula: （1-1） Step 102: Capture contextual features by combining the BiLSTM and CNN neural network models; use the BiLSTM model to capture the contextual features of the contract text and the CNN model to capture the local salient features of the sequence. Step 103: Attention mechanism learns features. The attention mechanism enables the neural network to focus on important parts during the processing of the input sequence. In the contract text classification task, the attention mechanism can learn the weights of different words in the input contract text. Step 104, Classification: Classify the vector output by the attention mechanism layer. The input is fed into a fully connected layer, where dimensionality transformation is performed to map the feature vector dimension to the contract text label dimension. Finally, a softmax classifier is used to calculate a score for each category, which is then converted into a probability value. The category with the highest score is then selected as the output category, as shown in the following formula: （1-12） The search enhancement in step 2) specifically includes: Step 201, Sparse Search: Calculate all words in a query. The scores are then summed based on the relevance of the documents to the text; the final text similarity result is obtained. （2-1） in This represents the weight of the i-th word. This indicates the relevance of each word in the query to document d; Step 202, Dense retrieval, constructs vector representations of the query and legal documents, and for each input query, its embedding is computed at search time, and then cosine similarity is used to evaluate the similarity between the query and the legal documents; Step 203, Hybrid Retrieval: Combine the results of the two retrieval methods using a hybrid retrieval method. Calculate the score of the legal and regulatory text block returned by the hybrid search using the Reverse Rank Fusion (RRF) algorithm. The formula for calculating the RRF score is as follows: （2-5） in This indicates the position of document d retrieved by the search mechanism r; The reordering in step 3) specifically includes: The reordering model is used to reorder the text blocks returned by the hybrid search; its ability to enhance the relevance of the information retrieval system is enhanced by capturing complex relationships and contextual information in legal and regulatory documents and queries. The Bge-reranker-base model is used as the reranking model, based on conditional probability. Sort the text, where It's a query. This represents the retrieved legal and regulatory documents; the model processes both queries and legal and regulatory documents simultaneously; on the other hand, bge-reranker-base is a sequence-to-sequence reorderer that uses the T5 model to generate relevance scores between queries and legal and regulatory documents; the T5 model treats all tasks as text-to-text, requiring some adjustments for the query-legal and regulatory document similarity task; during training, the format "query: {query}, document: {document};" is used, where the label is yes if the legal and regulatory document is relevant to the query, and no otherwise; during inference, the query and legal and regulatory documents are formatted using the same format as the training data; then, the score is obtained by calculating the softmax value and selecting the value corresponding to the yes label, considering only the no and yes labels.

2. A contract risk detection device based on a large language model, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the contract risk detection method based on a large language model as described in claim 1.

3. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the contract risk detection method based on a large language model as described in claim 1.

Images