A deep learning-based enterprise contract compliance risk identification method

Abstract

This invention discloses a deep learning-based method for identifying enterprise contract compliance risks, comprising the following steps: S1, acquiring and preprocessing the enterprise contract text to generate a sequence of contract clauses; S2, semantically encoding the contract clause sequence to generate a clause semantic sequence and constructing a contract relationship graph; S3, performing embedding calculations and feature aggregation on the contract relationship graph to generate a risk representation sequence; S4, generating a rule embedding sequence based on a contract knowledge graph; S5, calculating the matching relationship between the risk representation sequence and the rule embedding sequence to generate a clause risk sequence; S6, combining manual review information to perform error calculations, and performing outlier detection and deviation correction to generate an error sequence; S7, updating the contract knowledge graph based on the error sequence. This invention, through deep learning semantic modeling, possesses advantages such as high accuracy in risk identification, high efficiency in contract review, and high stability of identification results.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for identifying enterprise contract compliance risks based on deep learning. Background Technology

[0002] In the daily operations of enterprises, the number of various procurement contracts, sales contracts, service contracts, and technology cooperation contracts continues to increase. These contracts typically contain clauses outlining multiple rights and obligations, liabilities for breach of contract, and performance conditions. Different contract clauses not only have semantic connections but also relationships of reference and dependence on each other. If there are unclear divisions of responsibility, unequal liabilities for breach of contract, missing key clauses, or conflicts between clauses, enterprises may face legal disputes and compliance risks during contract performance.

[0003] Current corporate contract review methods primarily rely on legal personnel to read and assess the risks of each clause in the contract text, identifying potentially risky clauses and proposing revisions based on their experience. While this method can ensure a certain level of accuracy when the number of contracts is small, as the business expands and the number of contracts increases, manual review becomes extremely time-consuming and resource-intensive. Furthermore, differences in experience and judgment standards among reviewers can lead to inconsistent review results, impacting the efficiency and stability of contract risk identification.

[0004] To improve contract review efficiency, some technical solutions have begun to employ natural language processing (NLP) to automatically analyze contract texts, identifying risk clauses through keyword matching, rule matching, or traditional machine learning methods. However, these methods typically rely on pre-defined rules or limited features for identification, and are less effective at recognizing risk clauses with complex expressions or implicit meanings. Furthermore, contract clauses often have referencing and logical dependencies, making it difficult to accurately assess the overall risk level by analyzing a single clause, while existing methods lack the ability to model the overall structural relationships of contract clauses.

[0005] Therefore, how to provide a deep learning-based method for identifying enterprise contract compliance risks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based method for identifying enterprise contract compliance risks. This invention utilizes deep learning semantic modeling, contract relationship graph construction, and contract knowledge graph reasoning techniques to perform semantic encoding and structural relationship analysis on the clauses in enterprise contract texts. It then identifies contract clause risks through risk representation calculation and rule embedding matching. Simultaneously, it combines manual review information to correct errors in the identification results and updates the contract knowledge graph, thereby achieving automatic identification and continuous optimization of enterprise contract compliance risks. This method boasts advantages such as high accuracy in risk identification, high efficiency in contract review, and high stability in risk identification results.

[0007] A method for identifying enterprise contract compliance risks based on deep learning according to an embodiment of the present invention includes the following steps: S1. Obtain the enterprise contract text to be identified and preprocess it to generate a sequence of contract clauses with a clause index; S2. The contract clause sequence is bidirectionally context-encoded using a pre-trained BERT model. The TextCNN network is then used to perform one-dimensional convolution, max pooling, and linear mapping operations on the encoded result to generate a clause semantic sequence. Finally, a contract relationship graph is constructed by combining the clause index. S3. The Node2Vec algorithm is used to perform random path sampling and Skip-Gram embedding training in the contract relationship graph. The embedding training results are then processed by a graph attention network to perform attention calculation and weighted aggregation operations to obtain the risk representation sequence. S4. In the pre-built contract knowledge graph, the RotatE algorithm is used to model rotation relationships. Based on the modeling results, a hypergraph structure is constructed, and feature propagation and node update operations are performed through the hypergraph neural network to generate a rule embedding sequence. S5. Perform tensor interaction computation on the risk representation sequence and rule embedding sequence through a neural tensor network to generate a risk matching sequence, and perform a search operation using the MAP-Elites algorithm. Calculate the clause risk score based on the search results to form a clause risk sequence. S6. Collect information from manual review, perform clause index alignment and error calculation on the clause risk sequence and manual review information, and perform outlier detection and deviation correction to generate an error sequence; S7. Perform gradient update operation on the contract knowledge graph according to the error sequence, and use the updated contract knowledge graph in the next round of enterprise contract compliance risk identification.

[0008] Optionally, the enterprise contract text refers to an electronic text data set containing multiple contract terms signed or to be signed by the enterprise. The preprocessing includes unified character encoding, format standardization, punctuation standardization, clause boundary identification, and clause index allocation operations. The clause index represents a unique identifier assigned to each contract clause in the contract clause sequence, used to identify the positional relationship of the contract clause in the enterprise contract text. The manual review information represents the risk assessment result generated by manual reviewers after conducting a compliance review of the contract clauses.

[0009] Optionally, S2 specifically includes: S21. Read the contract clause sequence according to the clause index, divide each contract clause into multiple clause fragments according to the preset size, and map them into a word vector sequence according to the preset vocabulary; S22. In the pre-trained BERT model, the word vector sequence is processed by bidirectional context modeling through multi-layer self-attention calculation and feedforward calculation to obtain the context encoding results of the corresponding contract terms, and the encoding sequence is formed by arranging them according to the order of the terms index. S23. In the TextCNN network, a fixed-length convolution kernel is used to perform one-dimensional convolution calculation on the encoding sequence. Weighted summation and activation calculation operations are performed on the encoding results in each convolution window to extract the convolution feature sequence. Max pooling operation is performed on the convolution feature sequence to generate the clause feature sequence. S24. Perform a linear mapping operation on the clause feature sequence to numerically rearrange the clause feature sequence under a unified feature dimension, generate a clause semantic sequence, and establish connection relationships between the semantic vectors of each clause according to the clause index to form a contract relationship diagram.

[0010] Optionally, the pre-training process of the BERT model specifically includes: Collect historical contract texts, perform unified character encoding, format standardization, punctuation standardization, and clause boundary recognition on the historical contract texts, and sequentially segment the text content according to preset lengths to generate a sequence of text fragments; The text segment sequence is segmented into words, and the segmentation results are mapped into a word vector sequence according to a preset vocabulary. Position encoding information is superimposed on the word vector sequence, and the sequence is arranged according to the order of the text segment sequence to generate a training sequence. In the training sequence, multiple word vector positions are selected according to a preset ratio for masking. The word vectors at the selected positions are replaced with word vectors corresponding to the preset mask markers to generate a mask sequence. The real word identifier sequence of the replaced positions in the preset vocabulary is recorded. The mask sequence is input into the BERT model. In each layer of self-attention calculation, based on the query matrix, key matrix and value matrix in the BERT model, three linear mapping operations are performed on each position of the mask sequence to generate query vector, key vector and value vector. Calculate the similarity between each query vector and all key vectors, and then perform exponential normalization to obtain a weight sequence; A weighted summation operation is performed on all value vectors using a weight sequence to obtain an attention sequence. Feedforward computation is then performed on the attention sequence, and the feedforward computation result is passed layer by layer to generate an encoding sequence. Read the encoding vector corresponding to the position to be replaced in the encoding sequence and perform a linear mapping operation to obtain a score sequence with a preset vocabulary dimension; Perform exponential normalization on the score sequence to generate a word probability sequence. Based on the real word identifier sequence, locate the word probability value corresponding to the real word identifier in the word probability sequence. Perform logarithmic and negative operations on the word probability values ​​and accumulate the negative values ​​to obtain the mask training error. Construct a sequence of text fragment pairs according to the order of the text fragment sequence, and construct a sequence of sequence tags corresponding to the text fragment pairs sequence; The text fragments are input into the BERT model to obtain the encoding results of the text fragments. The encoding vector corresponding to the preset starting position is read from the encoding results, and a linear mapping operation is performed to obtain the sequential score sequence. Perform exponential normalization on the sequential score sequence to obtain the sequential probability sequence; Based on the sequence of sequence labels, locate the sequence probability value corresponding to the sequence label in the sequence probability sequence, and perform logarithmic and negative operations. The negative values ​​are accumulated to obtain the sequence training error. The mask training error is added to the sequential training error to obtain the training error. The parameters of the BERT model are then updated by backpropagation based on the training error to complete the pre-training of the BERT model.

[0011] Optionally, S24 specifically includes: S241. According to the clause index order, based on the preset weight matrix and bias vector, perform matrix multiplication and bias addition calculations on each clause feature vector in the clause feature sequence, so that the clause feature sequence is numerically mapped under a unified feature dimension, and a mapped feature sequence is generated. S242. Perform feature dimension rearrangement operation on each mapping feature vector in the mapping feature sequence, and arrange all mapping feature vectors in the order of the clause index to generate the clause semantic sequence; S243. Based on the clause index, read the semantic vectors of adjacent clauses in the semantic sequence in sequence, establish sequential connection relationships between adjacent clause semantic vectors, and generate a sequential connection set; S244. Perform a reference relationship identification operation on each contract clause in the contract clause sequence, detect the index identifier representing the reference in the contract clause text, read the clause index corresponding to the referenced clause, and establish a one-way connection relationship between the semantic vector of the current clause and the semantic vector of the referenced clause to generate a dependency connection set. S245. Merge the sequential connection set and the dependent connection set to construct an edge set. Use all contract terms as the node set, write the semantic vector of each term in the semantic sequence of terms as the node feature into the node set, and generate a contract relationship graph based on the edge set.

[0012] Optionally, S3 specifically includes: S31. Extract the node set and edge set from the contract relationship graph; S32. Select each node in the node set as the starting node in turn. Under the constraint of the edge set, use the Node2Vec algorithm to perform random path sampling operation. From the starting node, visit the adjacent nodes step by step along the connection relationship and generate the node visit sequence according to the visit order. Repeat the random path sampling operation on all nodes to generate a random path set. S33. Read the node access sequence in the random path set, read the current node in the node access sequence, take the current node as the center node, read the adjacent nodes within the preset window length range before and after the center node as the context node set, and build a node pair for the center node and each node in the context node set to form a node pairing sequence. S34. Perform Skip-Gram embedding training on the node pairing sequence, map the center node and context node in the node pairing sequence to node vector representations respectively, calculate the similarity between node vector representations, construct training error based on similarity and accumulate it, iteratively update node vector representations based on accumulation results until the preset convergence condition is reached, and generate node embedding sequence. S35. Input the node embedding sequence into the graph attention network according to the clause index order. In the graph attention network, read the neighboring node embedding vector of each node according to the edge set. Perform attention weight calculation on the current node embedding vector and the neighboring node embedding vector to generate the adjacent attention set. S36. Perform weighted aggregation calculation on the embedding vectors of adjacent nodes based on the adjacency attention set to obtain the node aggregation vector, and arrange all the node aggregation vectors in the order of the clause index to form a risk representation sequence.

[0013] Optionally, S32 specifically includes: S321. Read the nodes in the node set in the order of the clause index and determine the starting node. Search the set of adjacent nodes that are connected to the starting node in the edge set. S322. Based on the Node2Vec algorithm, establish path transition relationships between the starting node and each adjacent node, calculate the transition probability set based on the path transition relationships, perform cosine similarity calculation on the semantic vectors of the clauses corresponding to each node in the set of the starting node and the adjacent nodes, write the cosine similarity into the transition probability set and perform normalization processing to generate the constraint transition probability set. S323. Select access nodes from the set of adjacent nodes based on the constraint transition probability set, and connect the starting node and access nodes in the access order to generate an access path. S324. Read the current access node in the access path, retrieve the set of adjacent nodes that are connected to the current access node in the edge set, perform cosine similarity calculation on the semantic vector of the clauses of the current access node and each node in the corresponding set of adjacent nodes, and generate a semantic similarity set. S325. The Node2Vec algorithm is used to calculate the node transition probability between the current access node and each node in the corresponding set of neighboring nodes. The node transition probability is then weighted and updated based on the semantic similarity set to generate an updated transition probability set. Based on the updated transition probability set, an access node is selected from the corresponding set of neighboring nodes, and the current access node is connected to the access node to generate an extended access path. S326. Repeatedly execute the access node selection and access path expansion operation. After each access node selection, update the current access node and continue to select access nodes according to the updated transition probability set, so that the access path gradually increases according to the node access order until the number of nodes in the access path reaches the preset path length, and generate a node access sequence. S327. Repeat the node access sequence generation operation on all nodes in the node set, and record all node access sequences in the generation order to form a random path set.

[0014] Optionally, S4 specifically includes: S41. Extract the set of rule nodes and the set of relations from the pre-built contract knowledge graph, and perform node vector initialization operation on each rule node to generate rule node vectors and construct a rule vector sequence; S42. Read each relation type sequentially from the relation set, perform relation vector initialization for each relation type, and use the RotatE algorithm to perform relation rotation mapping calculation on the relation vector and the corresponding rule node vector to generate a rotated relation sequence containing relation rotation distances. The relation rotation mapping calculation specifically includes: Read the rule node vectors sequentially from the rule vector sequence, and read the relation vector corresponding to the current rule node from the relation set. Perform a phase mapping operation on the relation vectors to generate relation rotation vectors. Map the current rule node vector to a preset complex vector space, perform complex multiplication on the rule node vector and the relation rotation vector in the complex vector space, rotate the rule node vector according to the phase angle corresponding to the relation rotation vector, and generate a rotated node vector; Read the target rule nodes that are connected to the current rule node from the set of rule nodes, and perform vector distance calculation on the rotated node vector and the target rule node vector in the complex vector space to generate node relationship distance values; Arrange all node relationship distance values ​​according to the rule node identifier order to generate a rotation relationship sequence; S43. Based on the rotation relationship sequence, identify rule combinations with multi-node associations in the rule node set, read the corresponding rule node vector in each rule combination and establish a multi-node association structure, and summarize all multi-node association structures to form a hyperedge set. S44. Construct a hypergraph structure based on the set of rule nodes and the set of hyperedges. Read each rule node vector in the hypergraph structure in sequence, and read the adjacent rule node vectors that are related to the current rule node vector under the constraints of the hyperedge set. S45. Perform feature propagation calculation on the current rule node vector and the adjacent rule node vectors, calculate the association weight of the adjacent rule nodes based on the relation rotation distance, perform weighted aggregation operation on the adjacent rule node vectors, write the aggregation result into the current rule node vector, and generate the updated rule vector sequence. S46. Arrange the updated rule vector sequence according to the rule node identifier order, and perform a unified feature dimension rearrangement operation to generate a rule embedding sequence.

[0015] Optionally, S5 specifically includes: S51. Read each risk representation vector in the risk representation sequence in the order of the clause index, and read each rule embedding vector in the rule embedding sequence in the order of the rule node identifier. Establish a pairing relationship between each risk representation vector and the rule embedding vector to generate a vector pairing sequence. S52. Read each vector pair in the vector pairing sequence sequentially in the neural tensor network, perform tensor mapping processing on the risk representation vector and the rule embedding vector in the vector pair, perform multidimensional interaction calculation on the risk representation vector and the rule embedding vector through multiple sets of tensor mapping weights in the neural tensor network, and perform dimension-by-dimensional accumulation and activation operations on the multidimensional interaction results to generate an interaction feature sequence. S53. Perform a linear mapping operation on each interactive feature vector in the interactive feature sequence to rearrange the interactive feature vectors numerically under a unified feature dimension, and arrange all mapping results in the order of the clause index to generate a risk matching sequence. S54. Using the MAP-Elites algorithm, each risk matching result in the risk matching sequence is written into a preset search space. Multiple feature units are constructed in the search space based on the numerical range of the risk matching results. The risk matching results are then assigned to the corresponding feature units according to the clause index order, generating a candidate risk set, specifically including: Read each risk matching result in the risk matching sequence in the order of the clause index, read all feature units in the search space, and establish the record position of the risk matching result in each feature unit; The search space is divided into intervals based on the numerical range of the risk matching results, forming multiple feature units in the search space, and a corresponding numerical interval is set for each feature unit. Perform a numerical range location operation on each risk matching result, retrieve the numerical range corresponding to the risk matching result in the search space, and determine the corresponding feature unit; Write the risk matching result into the corresponding feature unit, and read the risk matching result already recorded in the current feature unit. Perform a numerical comparison operation between the written risk matching result and the recorded risk matching result. The risk matching results in the corresponding feature units are updated based on the numerical comparison results, so that the risk matching result with the largest recorded value is recorded in each feature unit. The risk matching results in all feature units are read in the order of the clause index to form a candidate risk set. S55. In the candidate risk set, score calculation is performed based on the risk matching results and the corresponding rule node identifiers. The score values ​​of each risk matching result are sorted and filtered, and a clause risk score set is generated according to the clause index order. S56. Write the risk scores of each clause into the corresponding clause index position according to the order of the clause index to form a clause risk sequence.

[0016] Optionally, S6 specifically includes: S61. Collect manual review information, and perform clause boundary positioning and clause index extraction operations on the manual review information to generate a sequence of review clauses containing risk assessment results; S62. Read the clause risk scores in the clause risk sequence and the corresponding risk assessment results in the audit clause sequence according to the clause index order, and perform the clause index alignment operation to generate the alignment result sequence; S63. Read the clause risk score and risk assessment result corresponding to each clause index in the alignment result sequence in sequence, perform difference calculation on the clause risk score and risk assessment result to generate clause error value, and arrange all clause error values ​​in the order of clause index to form an initial error sequence; S64. Read the clause risk score and risk assessment result of each clause in the alignment result sequence according to the clause index order, perform consistency assessment operation on the clause risk score and risk assessment result, generate a consistency mark at each clause index position, arrange all consistency marks according to the clause index order to form a consistency mark sequence; S65. Perform a sliding window expansion operation on the initial error sequence, calculate the median error and the absolute deviation of the median in each sliding window, and construct an anomaly detection threshold based on the median error and the absolute deviation of the median. S66. Use a Hampel filter to perform outlier detection on the initial error sequence, mark the outlier error positions that exceed the outlier detection threshold in the initial error sequence, and generate an outlier marking sequence. S67. Perform deviation correction operation on the marked positions in the abnormal marking sequence, read the clause error value corresponding to the unmarked position in the corresponding sliding window, perform weighted aggregation calculation on the unmarked clause error value according to the preset weight set, generate the correction error value, write the correction error value into the abnormal error position, and obtain the correction error sequence. S68. Merge the corrected error sequence and the consistency mark sequence in the order of the clause index to generate an error sequence.

[0017] The beneficial effects of this invention are: First, this invention performs deep learning semantic modeling and clause structure parsing on enterprise contract texts. The contract clause sequence is input into a pre-trained language model for contextual semantic encoding, and a convolutional neural network is used to extract the semantic features of the clauses to generate a clause semantic sequence. Based on this, a contract relationship graph is constructed according to the clause index, so that the reference and dependency relationships between contract clauses are expressed in a structured way. This achieves joint modeling of the semantic information and structural relationships of contract clauses, effectively improving the ability to identify hidden risk clauses in complex contract texts and improving the accuracy of contract risk identification.

[0018] Secondly, this invention constructs a contract knowledge graph and uses a relational rotation modeling method to represent the rule nodes and relations in the knowledge graph. Based on this, a hypergraph structure is constructed and feature propagation calculation is performed to generate a rule embedding sequence. Then, a neural tensor network is used to perform tensor interaction calculation on the contract risk representation sequence and the rule embedding sequence. Combined with a search algorithm, risk matching and scoring calculation are performed, enabling the semantic information of contract terms to establish a structured relationship with compliance rules. This achieves accurate matching and quantitative assessment of contract term risks, and improves the automation level of enterprise contract compliance risk identification.

[0019] Finally, this invention aligns the clause index with the results of manual review and clause risk identification, and corrects the identification errors using error calculation and anomaly detection methods. At the same time, it updates the contract knowledge graph based on the corrected error results, so that the contract compliance rule system can be continuously adjusted and optimized with the actual review results. This forms a risk identification mechanism that is continuously learning and dynamically updated, improving the stability and adaptability of the enterprise's contract compliance risk identification method in the long-term application process. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a deep learning-based enterprise contract compliance risk identification method proposed in this invention; Figure 2 This is a flowchart of the semantic modeling and relation graph construction process for an enterprise contract compliance risk identification method based on deep learning proposed in this invention. Figure 3 This is a flowchart illustrating the risk correction and knowledge graph update process of a deep learning-based enterprise contract compliance risk identification method proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A deep learning-based method for identifying enterprise contract compliance risks includes the following steps: S1. Obtain the enterprise contract text to be identified and preprocess it to generate a sequence of contract clauses with a clause index; S2. The contract clause sequence is bidirectionally context-encoded using a pre-trained BERT model. The TextCNN network is then used to perform one-dimensional convolution, max pooling, and linear mapping operations on the encoded result to generate a clause semantic sequence. Finally, a contract relationship graph is constructed by combining the clause index. S3. The Node2Vec algorithm is used to perform random path sampling and Skip-Gram embedding training in the contract relationship graph. The embedding training results are then processed by a graph attention network to perform attention calculation and weighted aggregation operations to obtain the risk representation sequence. S4. In the pre-built contract knowledge graph, the RotatE algorithm is used to model rotation relationships. Based on the modeling results, a hypergraph structure is constructed, and feature propagation and node update operations are performed through the hypergraph neural network to generate a rule embedding sequence. S5. Perform tensor interaction computation on the risk representation sequence and rule embedding sequence through a neural tensor network to generate a risk matching sequence, and perform a search operation using the MAP-Elites algorithm. Calculate the clause risk score based on the search results to form a clause risk sequence. S6. Collect manual review information, perform clause index alignment and error calculation operations on the clause risk sequence and manual review information, and use Hampel filter to detect outliers and correct deviations to generate an error sequence; S7. Perform gradient update operation on the contract knowledge graph according to the error sequence, and use the updated contract knowledge graph in the next round of enterprise contract compliance risk identification.

[0023] In this embodiment, the enterprise contract text refers to an electronic text data set containing multiple contract terms signed or to be signed by the enterprise. The preprocessing includes unified character encoding, format standardization, punctuation standardization, clause boundary identification, and clause index allocation operations. The clause index represents a unique identifier assigned to each contract clause in the contract clause sequence, used to identify the positional relationship of the contract clause in the enterprise contract text. The manual review information represents the risk assessment result generated by manual reviewers after conducting compliance review of the contract clauses.

[0024] In this embodiment, S2 specifically includes: S21. Read the contract clause sequence according to the clause index, divide each contract clause into multiple clause fragments according to the preset size, and map them into a word vector sequence according to the preset vocabulary; S22. In the pre-trained BERT model, the word vector sequence is processed by bidirectional context modeling through multi-layer self-attention calculation and feedforward calculation to obtain the context encoding results of the corresponding contract terms, and the encoding sequence is formed by arranging them according to the order of the terms index. S23. In the TextCNN network, a fixed-length convolution kernel is used to perform one-dimensional convolution calculation on the encoding sequence. Weighted summation and activation calculation operations are performed on the encoding results in each convolution window to extract the convolution feature sequence. Max pooling operation is performed on the convolution feature sequence to generate the clause feature sequence. S24. Perform a linear mapping operation on the clause feature sequence to numerically rearrange the clause feature sequence under a unified feature dimension, generate a clause semantic sequence, and establish connection relationships between the semantic vectors of each clause according to the clause index to form a contract relationship diagram.

[0025] In this embodiment, the pre-training process of the BERT model specifically includes: Collect historical contract texts, perform unified character encoding, format standardization, punctuation standardization, and clause boundary recognition on the historical contract texts, and sequentially segment the text content according to preset lengths to generate a sequence of text fragments; The text segment sequence is segmented into words, and the segmentation results are mapped into a word vector sequence according to a preset vocabulary. Position encoding information is superimposed on the word vector sequence, and the sequence is arranged according to the order of the text segment sequence to generate a training sequence. In the training sequence, multiple word vector positions are selected according to a preset ratio for masking. The word vectors at the selected positions are replaced with word vectors corresponding to the preset mask markers to generate a mask sequence. The real word identifier sequence of the replaced positions in the preset vocabulary is recorded. The mask sequence is input into the BERT model. In each layer of self-attention calculation, based on the query matrix, key matrix and value matrix in the BERT model, three linear mapping operations are performed on each position of the mask sequence to generate query vector, key vector and value vector. Calculate the similarity between each query vector and all key vectors, and then perform exponential normalization to obtain a weight sequence; A weighted summation operation is performed on all value vectors using a weight sequence to obtain an attention sequence. Feedforward computation is then performed on the attention sequence, and the feedforward computation result is passed layer by layer to generate an encoding sequence. Read the encoding vector corresponding to the position to be replaced in the encoding sequence and perform a linear mapping operation to obtain a score sequence with a preset vocabulary dimension; Perform exponential normalization on the score sequence to generate a word probability sequence. Based on the real word identifier sequence, locate the word probability value corresponding to the real word identifier in the word probability sequence. Perform logarithmic and negative operations on the word probability values ​​and accumulate the negative values ​​to obtain the mask training error. Construct a sequence of text fragment pairs according to the order of the text fragment sequence, and construct a sequence of sequence tags corresponding to the text fragment pairs sequence; The text fragments are input into the BERT model to obtain the encoding results of the text fragments. The encoding vector corresponding to the preset starting position is read from the encoding results, and a linear mapping operation is performed to obtain the sequential score sequence. Perform exponential normalization on the sequential score sequence to obtain the sequential probability sequence; Based on the sequence of sequence labels, locate the sequence probability value corresponding to the sequence label in the sequence probability sequence, and perform logarithmic and negative operations. The negative values ​​are accumulated to obtain the sequence training error. The mask training error is added to the sequential training error to obtain the training error. The parameters of the BERT model are then updated by backpropagation based on the training error to complete the pre-training of the BERT model.

[0026] In this embodiment, S23 specifically includes: S231. Input the encoding sequence into the TextCNN network in the order of the clause index, slide the convolution kernels sequentially on the encoding sequence according to the preset convolution window length, and perform dimension-wise multiplication and addition accumulation calculations on the encoding vectors within the coverage of the convolution window to obtain the convolution result sequence. S232. Perform ReLU activation operation on each value in the convolution result sequence to generate an activation sequence, and arrange all activation sequences according to the clause index order corresponding to the convolution window to form a convolution feature sequence. S233. Perform max pooling on the convolutional feature sequence, select the convolutional feature with the largest value within the coverage of each convolutional window as the feature result corresponding to that window, and arrange them in the order of the clause index to form the clause feature sequence.

[0027] In this embodiment, S24 specifically includes: S241. According to the clause index order, based on the preset weight matrix and bias vector, perform matrix multiplication and bias addition calculations on each clause feature vector in the clause feature sequence, so that the clause feature sequence is numerically mapped under a unified feature dimension, and a mapped feature sequence is generated. S242. Perform feature dimension rearrangement operation on each mapping feature vector in the mapping feature sequence, and arrange all mapping feature vectors in the order of the clause index to generate the clause semantic sequence; S243. Based on the clause index, read the semantic vectors of adjacent clauses in the semantic sequence in sequence, establish sequential connection relationships between adjacent clause semantic vectors, and generate a sequential connection set; S244. Perform a reference relationship identification operation on each contract clause in the contract clause sequence, detect the index identifier representing the reference in the contract clause text, read the clause index corresponding to the referenced clause, and establish a one-way connection relationship between the semantic vector of the current clause and the semantic vector of the referenced clause to generate a dependency connection set. S245. Merge the sequential connection set and the dependent connection set to construct an edge set. Use all contract terms as the node set, write the semantic vector of each term in the semantic sequence of terms as the node feature into the node set, and generate a contract relationship graph based on the edge set.

[0028] In this embodiment, S3 specifically includes: S31. Extract the node set and edge set from the contract relationship graph; S32. Select each node in the node set as the starting node in turn. Under the constraint of the edge set, use the Node2Vec algorithm to perform random path sampling operation. From the starting node, visit the adjacent nodes step by step along the connection relationship and generate the node visit sequence according to the visit order. Repeat the random path sampling operation on all nodes to generate a random path set. S33. Read the node access sequence in the random path set, read the current node in the node access sequence, take the current node as the center node, read the adjacent nodes within the preset window length range before and after the center node as the context node set, and build a node pair for the center node and each node in the context node set to form a node pairing sequence. S34. Perform Skip-Gram embedding training on the node pairing sequence, map the center node and context node in the node pairing sequence to node vector representations respectively, calculate the similarity between node vector representations, construct training error based on similarity and accumulate it, iteratively update node vector representations based on accumulation results until the preset convergence condition is reached, and generate node embedding sequence. S35. Input the node embedding sequence into the graph attention network according to the clause index order. In the graph attention network, read the neighboring node embedding vector of each node according to the edge set. Perform attention weight calculation on the current node embedding vector and the neighboring node embedding vector to generate the adjacent attention set. S36. Perform weighted aggregation calculation on the embedding vectors of adjacent nodes based on the adjacency attention set to obtain the node aggregation vector, and arrange all the node aggregation vectors in the order of the clause index to form a risk representation sequence.

[0029] In this embodiment, S32 specifically includes: S321. Read the nodes in the node set in the order of the clause index and determine the starting node. Search the set of adjacent nodes that are connected to the starting node in the edge set. S322. Based on the Node2Vec algorithm, establish path transition relationships between the starting node and each adjacent node, calculate the transition probability set based on the path transition relationships, perform cosine similarity calculation on the semantic vectors of the clauses corresponding to each node in the set of the starting node and the adjacent nodes, write the cosine similarity into the transition probability set and perform normalization processing to generate the constraint transition probability set. S323. Select access nodes from the set of adjacent nodes based on the constraint transition probability set, and connect the starting node and access nodes in the access order to generate an access path. S324. Read the current access node in the access path, retrieve the set of adjacent nodes that are connected to the current access node in the edge set, perform cosine similarity calculation on the semantic vector of the clauses of the current access node and each node in the corresponding set of adjacent nodes, and generate a semantic similarity set. S325. The Node2Vec algorithm is used to calculate the node transition probability between the current access node and each node in the corresponding set of neighboring nodes. The node transition probability is then weighted and updated based on the semantic similarity set to generate an updated transition probability set. Based on the updated transition probability set, an access node is selected from the corresponding set of neighboring nodes, and the current access node is connected to the access node to generate an extended access path. S326. Repeatedly execute the access node selection and access path expansion operation. After each access node selection, update the current access node and continue to select access nodes according to the updated transition probability set, so that the access path gradually increases according to the node access order until the number of nodes in the access path reaches the preset path length, and generate a node access sequence. S327. Repeat the node access sequence generation operation on all nodes in the node set, and record all node access sequences in the generation order to form a random path set.

[0030] In this embodiment, S4 specifically includes: S41. Extract the set of rule nodes and the set of relations from the pre-built contract knowledge graph, and perform node vector initialization operation on each rule node to generate rule node vectors and construct a rule vector sequence; S42. Read each relation type sequentially from the relation set, perform relation vector initialization for each relation type, and use the RotatE algorithm to perform relation rotation mapping calculation on the relation vector and the corresponding rule node vector to generate a rotated relation sequence containing relation rotation distances. Specifically, the relation rotation mapping calculation includes: Read the rule node vectors sequentially from the rule vector sequence, and read the relation vector corresponding to the current rule node from the relation set. Perform a phase mapping operation on the relation vectors to generate relation rotation vectors. Map the current rule node vector to a preset complex vector space, perform complex multiplication on the rule node vector and the relation rotation vector in the complex vector space, rotate the rule node vector according to the phase angle corresponding to the relation rotation vector, and generate a rotated node vector; Read the target rule nodes that are connected to the current rule node from the set of rule nodes, and perform vector distance calculation on the rotated node vector and the target rule node vector in the complex vector space to generate node relationship distance values; Arrange all node relationship distance values ​​according to the rule node identifier order to generate a rotation relationship sequence; S43. Based on the rotation relationship sequence, identify rule combinations with multi-node associations in the rule node set, read the corresponding rule node vector in each rule combination and establish a multi-node association structure, and summarize all multi-node association structures to form a hyperedge set. S44. Construct a hypergraph structure based on the set of rule nodes and the set of hyperedges. Read each rule node vector in the hypergraph structure in sequence, and read the adjacent rule node vectors that are related to the current rule node vector under the constraints of the hyperedge set. S45. Perform feature propagation calculation on the current rule node vector and the adjacent rule node vectors, calculate the association weight of the adjacent rule nodes based on the relation rotation distance, perform weighted aggregation operation on the adjacent rule node vectors, write the aggregation result into the current rule node vector, and generate the updated rule vector sequence. S46. Arrange the updated rule vector sequence according to the rule node identifier order, and perform a unified feature dimension rearrangement operation to generate a rule embedding sequence.

[0031] In this embodiment, the process of constructing the contract knowledge graph specifically includes: Acquire historical contract text data, and perform unified character encoding, format standardization, punctuation standardization, and clause boundary recognition operations on the historical contract text data to generate a sequence of historical contract clauses; Read each historical contract clause sequentially from the historical contract clause sequence, perform word segmentation and word vector mapping operations on the historical contract clauses, and generate a semantic sequence of historical clauses. Based on the semantic sequence of historical clauses, rule entities and risk entities in historical contract clauses are identified, and rule entities and risk entities are treated as nodes according to entity type to generate a set of rule nodes; Detect the semantic expressions representing rule triggering relationships, liability constraint relationships, and risk association relationships in historical contract terms, and establish relationship connections between corresponding rule nodes based on the detection results to generate a relationship set; Based on the set of rule nodes, a knowledge graph structure is generated by establishing connections between each rule node according to the set of relationships. Perform node attribute writing operations on the rule nodes in the knowledge graph structure to generate a contract knowledge graph.

[0032] In this embodiment, S5 specifically includes: S51. Read each risk representation vector in the risk representation sequence in the order of the clause index, and read each rule embedding vector in the rule embedding sequence in the order of the rule node identifier. Establish a pairing relationship between each risk representation vector and the rule embedding vector to generate a vector pairing sequence. S52. Read each vector pair in the vector pairing sequence sequentially in the neural tensor network, perform tensor mapping processing on the risk representation vector and the rule embedding vector in the vector pair, perform multidimensional interaction calculation on the risk representation vector and the rule embedding vector through multiple sets of tensor mapping weights in the neural tensor network, and perform dimension-by-dimensional accumulation and activation operations on the multidimensional interaction results to generate an interaction feature sequence. S53. Perform a linear mapping operation on each interactive feature vector in the interactive feature sequence to rearrange the interactive feature vectors numerically under a unified feature dimension, and arrange all mapping results in the order of the clause index to generate a risk matching sequence. S54. Using the MAP-Elites algorithm, each risk matching result in the risk matching sequence is written into a preset search space. Multiple feature units are constructed in the search space based on the numerical range of the risk matching results. The risk matching results are then assigned to the corresponding feature units according to the clause index order, generating a candidate risk set, specifically including: Read each risk matching result in the risk matching sequence in the order of the clause index, read all feature units in the search space, and establish the record position of the risk matching result in each feature unit; The search space is divided into intervals based on the numerical range of the risk matching results, forming multiple feature units in the search space, and a corresponding numerical interval is set for each feature unit. Perform a numerical range location operation on each risk matching result, retrieve the numerical range corresponding to the risk matching result in the search space, and determine the corresponding feature unit; Write the risk matching result into the corresponding feature unit, and read the risk matching result already recorded in the current feature unit. Perform a numerical comparison operation between the written risk matching result and the recorded risk matching result. The risk matching results in the corresponding feature units are updated based on the numerical comparison results, so that the risk matching result with the largest recorded value is recorded in each feature unit. The risk matching results in all feature units are read in the order of the clause index to form a candidate risk set. S55. In the candidate risk set, score calculation is performed based on the risk matching results and the corresponding rule node identifiers. The score values ​​of each risk matching result are sorted and filtered, and a clause risk score set is generated according to the clause index order. S56. Write the risk scores of each clause into the corresponding clause index position according to the order of the clause index to form a clause risk sequence.

[0033] In this embodiment, multidimensional interactive computing specifically includes: Each vector pair is read sequentially according to the vector pairing sequence. Based on the corresponding tensor mapping weights, matrix multiplication is performed on the risk representation vector and the rule embedding vector in the vector pair, so that the risk representation vector and the rule embedding vector generate multiple interactive mapping results in multiple feature dimension spaces. In the interaction mapping results, the interaction results of each dimension are read sequentially according to the feature dimension order, and the interaction results of each dimension are accumulated dimension by dimension to generate a multi-dimensional interaction vector. Perform a nonlinear activation operation on the multidimensional interaction vector to generate interaction feature vectors, and arrange all interaction feature vectors in the order of the clause index to form an interaction feature sequence.

[0034] In this embodiment, S6 specifically includes: S61. Collect manual review information, and perform clause boundary positioning and clause index extraction operations on the manual review information to generate a sequence of review clauses containing risk assessment results; S62. Read the clause risk scores in the clause risk sequence and the corresponding risk assessment results in the audit clause sequence according to the clause index order, and perform the clause index alignment operation to generate the alignment result sequence; S63. Read the clause risk score and risk assessment result corresponding to each clause index in the alignment result sequence in sequence, perform difference calculation on the clause risk score and risk assessment result to generate clause error value, and arrange all clause error values ​​in the order of clause index to form an initial error sequence; S64. Read the clause risk score and risk assessment result of each clause in the alignment result sequence according to the clause index order, perform consistency assessment operation on the clause risk score and risk assessment result, generate a consistency mark at each clause index position, arrange all consistency marks according to the clause index order to form a consistency mark sequence; S65. Perform a sliding window expansion operation on the initial error sequence, calculate the median error and the absolute deviation of the median in each sliding window, and construct an anomaly detection threshold based on the median error and the absolute deviation of the median. S66. Use a Hampel filter to perform outlier detection on the initial error sequence, mark the outlier error positions that exceed the outlier detection threshold in the initial error sequence, and generate an outlier marking sequence. S67. Perform deviation correction operation on the marked positions in the abnormal marking sequence, read the clause error value corresponding to the unmarked position in the corresponding sliding window, perform weighted aggregation calculation on the unmarked clause error value according to the preset weight set, generate the correction error value, write the correction error value into the abnormal error position, and obtain the correction error sequence. S68. Merge the corrected error sequence and the consistency mark sequence in the order of the clause index to generate an error sequence.

[0035] In this embodiment, S7 specifically includes: S71. Read the error sequence and locate the corresponding rule node in the contract knowledge graph based on the clause index; S72. Write the corrected error values ​​in the error sequence into the corresponding rule node vector according to the order of the clause index to generate rule error records; S73. Perform weighted cumulative calculation on the rule error records to generate a node error sequence; S74. Calculate the update gradient of the rule node vector based on the node error sequence, and generate the node gradient sequence; S75. Perform parameter update operations on the rule node vectors in the contract knowledge graph based on the node gradient sequence; S76. Write the updated rule node vector into the corresponding rule node position in the contract knowledge graph to obtain the updated contract knowledge graph, and use the updated contract knowledge graph in the next round of enterprise contract compliance risk identification.

[0036] Example 1: To verify the feasibility of this invention in practice, it was applied to a company's contract management environment. In this environment, the company needs to process a large number of contract texts, including procurement contracts, sales contracts, technical service contracts, and cooperation agreements, during its daily operations. Due to the complexity of the contract texts and the existence of referencing relationships, liability dependencies, and performance constraints between different clauses, manual review requires reading and assessing the risks of each contract clause by clause. With the increasing number of contracts, manual review becomes inefficient. Furthermore, different reviewers have different standards for judging risk clauses, which can easily lead to inconsistent contract risk identification results, thus affecting the company's contract review efficiency and compliance management capabilities. To solve these problems, this example introduces the deep learning-based enterprise contract compliance risk identification method proposed in this invention to automate the analysis and risk identification of enterprise contract texts.

[0037] In practical applications, the process begins by collecting and preprocessing historical contract texts from enterprises. This involves applying unified character encoding, format standardization, and clause boundary recognition to the contract texts. Complete sequences of contract clauses are extracted from the texts, and each clause is assigned a unique clause index. Based on this, the contract clause sequence is input into a pre-trained language model for bidirectional contextual encoding. A convolutional neural network extracts semantic features of the clauses, generating a semantic sequence. Sequential connections and dependency relationships are then established between the semantic vectors of each clause based on their respective clause indices, thus constructing a contract relationship graph. This approach simultaneously expresses the semantic information and structural relationships of the clauses in the contract text, transforming the traditional linear text structure into structured graph data.

[0038] After constructing the contract relationship graph, node embedding computation is performed within it. Potential association features between contract terms are extracted using random path sampling and embedding training methods. Adjacency information aggregation is then performed on the node embedding vectors using a graph attention network to generate a risk representation sequence reflecting the risk characteristics of the contract terms. Simultaneously, a contract knowledge graph is pre-constructed within the system. This knowledge graph contains various compliance rule nodes and their interrelationships. Relationship rotation modeling is used to vectorize the relationships between rule nodes, and a hypergraph neural network propagates and updates the rule node features, generating a rule embedding sequence. Subsequently, a neural tensor network is used to interactively compute the contract term risk representation sequence and the rule embedding sequence, enabling an effective matching relationship between the semantic information of the contract terms and compliance rules. A search algorithm then filters and scores the matching results, generating a term risk sequence, thereby completing the automatic risk identification of contract terms.

[0039] During system operation, the risk assessment results of human reviewers on contract terms are simultaneously collected. The human review information is aligned with the system-generated clause risk sequence using clause indexing, and the discrepancies between the identified results and the human review results are analyzed through error calculation. To avoid anomalies affecting model updates, Hampel filtering is used during error analysis to perform outlier detection and deviation correction on the error sequence, thereby generating a stable error sequence. The contract knowledge graph is then updated based on this error sequence, enabling the system to continuously optimize its risk identification capabilities based on human review experience, achieving continuous learning of the contract risk identification model.

[0040] In actual testing, contract text data from multiple business scenarios were selected for verification, and the traditional manual review method was compared and analyzed with the method of this invention. Through statistical analysis of indicators such as contract risk identification accuracy, contract review time, and risk miss rate, the following experimental results were obtained: Table 1. Comparison of Contract Compliance Risk Identification Effectiveness

[0041] As can be seen from Table 1, when the number of contract clauses is between 48 and 92, it takes an average of more than 60 minutes to complete a risk review of a contract using the traditional manual review method. When the number of contract clauses is large, the manual review time can even reach more than 80 minutes.

[0042] Under the same conditions, the method of this invention uses deep learning semantic modeling, contract relationship graph analysis and contract knowledge graph reasoning mechanism to automatically analyze and identify risks in contract terms, with an average identification time of 4 to 7 minutes, which is more than 10 times more efficient than manual review.

[0043] Regarding accuracy, manual review achieves an accuracy rate of 92% to 100% in most contract samples, while the accuracy rate of the method of this invention remains stable between 85% and 92%, with an overall average accuracy rate of approximately 90%. Although manual identification has a slightly higher accuracy rate in individual contracts, manual review requires significant time and manpower costs, while the method of this invention can complete contract risk identification in a shorter time and maintain a high and stable accuracy rate.

[0044] As can be seen from the data in the table, the method of the present invention maintains a high recognition accuracy while significantly shortening the time for contract risk identification, enabling enterprises to quickly identify potential risk clauses when processing a large number of contract texts, thereby significantly improving contract review efficiency and reducing the burden of manual review, and has high practical application value.

[0045] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying enterprise contract compliance risks based on deep learning, characterized in that, Includes the following steps: S1. Obtain the enterprise contract text to be identified and preprocess it to generate a sequence of contract clauses with a clause index; S2. The contract clause sequence is bidirectionally context-encoded using a pre-trained BERT model. The TextCNN network is then used to perform one-dimensional convolution, max pooling, and linear mapping operations on the encoded result to generate a clause semantic sequence. Finally, a contract relationship graph is constructed by combining the clause index. S3. The Node2Vec algorithm is used to perform random path sampling and Skip-Gram embedding training in the contract relationship graph. The embedding training results are then processed by a graph attention network to perform attention calculation and weighted aggregation operations to obtain the risk representation sequence. S4. In the pre-built contract knowledge graph, the RotatE algorithm is used to model rotation relationships. Based on the modeling results, a hypergraph structure is constructed, and feature propagation and node update operations are performed through the hypergraph neural network to generate a rule embedding sequence. S5. Perform tensor interaction computation on the risk representation sequence and rule embedding sequence through a neural tensor network to generate a risk matching sequence, and perform a search operation using the MAP-Elites algorithm. Calculate the clause risk score based on the search results to form a clause risk sequence. S6. Collect manual review information, perform clause index alignment and error calculation operations on the clause risk sequence and manual review information, and use Hampel filter to detect outliers and correct deviations to generate an error sequence; S7. Perform gradient update operation on the contract knowledge graph according to the error sequence, and use the updated contract knowledge graph in the next round of enterprise contract compliance risk identification.

2. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, The enterprise contract text refers to an electronic text data set containing multiple contract terms signed or to be signed by the enterprise. The preprocessing includes unified character encoding, format standardization, punctuation standardization, clause boundary identification, and clause index allocation operations. The clause index represents a unique identifier assigned to each contract clause in the contract clause sequence, used to identify the positional relationship of the contract clause in the enterprise contract text. The manual review information represents the risk assessment result generated by manual reviewers after conducting compliance review of the contract clauses.

3. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Read the contract clause sequence according to the clause index, divide each contract clause into multiple clause fragments according to the preset size, and map them into a word vector sequence according to the preset vocabulary; S22. In the pre-trained BERT model, the word vector sequence is processed by bidirectional context modeling through multi-layer self-attention calculation and feedforward calculation to obtain the context encoding results of the corresponding contract terms, and the encoding sequence is formed by arranging them according to the order of the terms index. S23. In the TextCNN network, a fixed-length convolution kernel is used to perform one-dimensional convolution calculation on the encoding sequence. Weighted summation and activation calculation operations are performed on the encoding results in each convolution window to extract the convolution feature sequence. Max pooling operation is performed on the convolution feature sequence to generate the clause feature sequence. S24. Perform a linear mapping operation on the clause feature sequence to numerically rearrange the clause feature sequence under a unified feature dimension, generate a clause semantic sequence, and establish connection relationships between the semantic vectors of each clause according to the clause index to form a contract relationship diagram.

4. The method for identifying enterprise contract compliance risks based on deep learning according to claim 3, characterized in that, The pre-training process of the BERT model specifically includes: Collect historical contract texts, perform unified character encoding, format standardization, punctuation standardization, and clause boundary recognition on the historical contract texts, and sequentially segment the text content according to preset lengths to generate a sequence of text fragments; The text segment sequence is segmented into words, and the segmentation results are mapped into a word vector sequence according to a preset vocabulary. Position encoding information is superimposed on the word vector sequence, and the sequence is arranged according to the order of the text segment sequence to generate a training sequence. In the training sequence, multiple word vector positions are selected according to a preset ratio for masking. The word vectors at the selected positions are replaced with word vectors corresponding to the preset mask markers to generate a mask sequence. The real word identifier sequence of the replaced positions in the preset vocabulary is recorded. The mask sequence is input into the BERT model. In each layer of self-attention calculation, based on the query matrix, key matrix and value matrix in the BERT model, three linear mapping operations are performed on each position of the mask sequence to generate query vector, key vector and value vector. Calculate the similarity between each query vector and all key vectors, and then perform exponential normalization to obtain a weight sequence; A weighted summation operation is performed on all value vectors using a weight sequence to obtain an attention sequence. Feedforward computation is then performed on the attention sequence, and the feedforward computation result is passed layer by layer to generate an encoding sequence. Read the encoding vector corresponding to the position to be replaced in the encoding sequence and perform a linear mapping operation to obtain a score sequence with a preset vocabulary dimension; Perform exponential normalization on the score sequence to generate a word probability sequence. Based on the real word identifier sequence, locate the word probability value corresponding to the real word identifier in the word probability sequence. Perform logarithmic and negative operations on the word probability values ​​and accumulate the negative values ​​to obtain the mask training error. Construct a sequence of text fragment pairs according to the order of the text fragment sequence, and construct a sequence of sequence tags corresponding to the text fragment pairs sequence; The text fragments are input into the BERT model to obtain the encoding results of the text fragments. The encoding vector corresponding to the preset starting position is read from the encoding results, and a linear mapping operation is performed to obtain the sequential score sequence. Perform exponential normalization on the sequential score sequence to obtain the sequential probability sequence; Based on the sequence of sequence labels, locate the sequence probability value corresponding to the sequence label in the sequence probability sequence, and perform logarithmic and negative operations. The negative values ​​are accumulated to obtain the sequence training error. The mask training error is added to the sequential training error to obtain the training error. The parameters of the BERT model are then updated by backpropagation based on the training error to complete the pre-training of the BERT model.

5. The method for identifying enterprise contract compliance risks based on deep learning according to claim 3, characterized in that, S24 specifically includes: S241. According to the clause index order, based on the preset weight matrix and bias vector, perform matrix multiplication and bias addition calculations on each clause feature vector in the clause feature sequence, so that the clause feature sequence is numerically mapped under a unified feature dimension, and a mapped feature sequence is generated. S242. Perform feature dimension rearrangement operation on each mapping feature vector in the mapping feature sequence, and arrange all mapping feature vectors in the order of the clause index to generate the clause semantic sequence; S243. Based on the clause index, read the semantic vectors of adjacent clauses in the semantic sequence in sequence, establish sequential connection relationships between adjacent clause semantic vectors, and generate a sequential connection set; S244. Perform a reference relationship identification operation on each contract clause in the contract clause sequence, detect the index identifier representing the reference in the contract clause text, read the clause index corresponding to the referenced clause, and establish a one-way connection relationship between the semantic vector of the current clause and the semantic vector of the referenced clause to generate a dependency connection set. S245. Merge the sequential connection set and the dependent connection set to construct an edge set. Use all contract terms as the node set, write the semantic vector of each term in the semantic sequence of terms as the node feature into the node set, and generate a contract relationship graph based on the edge set.

6. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, S3 specifically includes: S31. Extract the node set and edge set from the contract relationship graph; S32. Select each node in the node set as the starting node in turn. Under the constraint of the edge set, use the Node2Vec algorithm to perform random path sampling operation. From the starting node, visit the adjacent nodes step by step along the connection relationship and generate the node visit sequence according to the visit order. Repeat the random path sampling operation on all nodes to generate a random path set. S33. Read the node access sequence in the random path set, read the current node in the node access sequence, take the current node as the center node, read the adjacent nodes within the preset window length range before and after the center node as the context node set, and build a node pair for the center node and each node in the context node set to form a node pairing sequence. S34. Perform Skip-Gram embedding training on the node pairing sequence, map the center node and context node in the node pairing sequence to node vector representations respectively, calculate the similarity between node vector representations, construct training error based on similarity and accumulate it, iteratively update node vector representations based on accumulation results until the preset convergence condition is reached, and generate node embedding sequence. S35. Input the node embedding sequence into the graph attention network according to the clause index order. In the graph attention network, read the neighboring node embedding vector of each node according to the edge set. Perform attention weight calculation on the current node embedding vector and the neighboring node embedding vector to generate the adjacent attention set. S36. Perform weighted aggregation calculation on the embedding vectors of adjacent nodes based on the adjacency attention set to obtain the node aggregation vector, and arrange all the node aggregation vectors in the order of the clause index to form a risk representation sequence.

7. The method for identifying enterprise contract compliance risks based on deep learning according to claim 6, characterized in that, Specifically, S32 includes: S321. Read the nodes in the node set in the order of the clause index and determine the starting node. Search the set of adjacent nodes that are connected to the starting node in the edge set. S322. Based on the Node2Vec algorithm, establish path transition relationships between the starting node and each adjacent node, calculate the transition probability set based on the path transition relationships, perform cosine similarity calculation on the semantic vectors of the clauses corresponding to each node in the set of the starting node and the adjacent nodes, write the cosine similarity into the transition probability set and perform normalization processing to generate the constraint transition probability set. S323. Select access nodes from the set of adjacent nodes based on the constraint transition probability set, and connect the starting node and access nodes in the access order to generate an access path. S324. Read the current access node in the access path, retrieve the set of adjacent nodes that are connected to the current access node in the edge set, perform cosine similarity calculation on the semantic vector of the clauses of the current access node and each node in the corresponding set of adjacent nodes, and generate a semantic similarity set. S325. The Node2Vec algorithm is used to calculate the node transition probability between the current access node and each node in the corresponding set of neighboring nodes. The node transition probability is then weighted and updated based on the semantic similarity set to generate an updated transition probability set. Based on the updated transition probability set, an access node is selected from the corresponding set of neighboring nodes, and the current access node is connected to the access node to generate an extended access path. S326. Repeatedly execute the access node selection and access path expansion operation. After each access node selection, update the current access node and continue to select access nodes according to the updated transition probability set, so that the access path gradually increases according to the node access order until the number of nodes in the access path reaches the preset path length, and generate a node access sequence. S327. Repeat the node access sequence generation operation on all nodes in the node set, and record all node access sequences in the generation order to form a random path set.

8. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Extract the set of rule nodes and the set of relations from the pre-built contract knowledge graph, and perform node vector initialization operation on each rule node to generate rule node vectors and construct a rule vector sequence; S42. Read each relation type sequentially from the relation set, perform relation vector initialization for each relation type, and use the RotatE algorithm to perform relation rotation mapping calculation on the relation vector and the corresponding rule node vector to generate a rotated relation sequence containing relation rotation distances. The relation rotation mapping calculation specifically includes: Read the rule node vectors sequentially from the rule vector sequence, and read the relation vector corresponding to the current rule node from the relation set. Perform a phase mapping operation on the relation vectors to generate relation rotation vectors. Map the current rule node vector to a preset complex vector space, perform complex multiplication on the rule node vector and the relation rotation vector in the complex vector space, rotate the rule node vector according to the phase angle corresponding to the relation rotation vector, and generate a rotated node vector; Read the target rule nodes that are connected to the current rule node from the set of rule nodes, and perform vector distance calculation on the rotated node vector and the target rule node vector in the complex vector space to generate node relationship distance values; Arrange all node relationship distance values ​​according to the rule node identifier order to generate a rotation relationship sequence; S43. Based on the rotation relationship sequence, identify rule combinations with multi-node associations in the rule node set, read the corresponding rule node vector in each rule combination and establish a multi-node association structure, and summarize all multi-node association structures to form a hyperedge set. S44. Construct a hypergraph structure based on the set of rule nodes and the set of hyperedges. Read each rule node vector in the hypergraph structure in sequence, and read the adjacent rule node vectors that are related to the current rule node vector under the constraints of the hyperedge set. S45. Perform feature propagation calculation on the current rule node vector and the adjacent rule node vectors, calculate the association weight of the adjacent rule nodes based on the relation rotation distance, perform weighted aggregation operation on the adjacent rule node vectors, write the aggregation result into the current rule node vector, and generate the updated rule vector sequence. S46. Arrange the updated rule vector sequence according to the rule node identifier order, and perform a unified feature dimension rearrangement operation to generate a rule embedding sequence.

9. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Read each risk representation vector in the risk representation sequence in the order of the clause index, and read each rule embedding vector in the rule embedding sequence in the order of the rule node identifier. Establish a pairing relationship between each risk representation vector and the rule embedding vector to generate a vector pairing sequence. S52. Read each vector pair in the vector pairing sequence sequentially in the neural tensor network, perform tensor mapping processing on the risk representation vector and the rule embedding vector in the vector pair, perform multidimensional interaction calculation on the risk representation vector and the rule embedding vector through multiple sets of tensor mapping weights in the neural tensor network, and perform dimension-by-dimensional accumulation and activation operations on the multidimensional interaction results to generate an interaction feature sequence. S53. Perform a linear mapping operation on each interactive feature vector in the interactive feature sequence to rearrange the interactive feature vectors numerically under a unified feature dimension, and arrange all mapping results in the order of the clause index to generate a risk matching sequence. S54. Using the MAP-Elites algorithm, each risk matching result in the risk matching sequence is written into a preset search space. Multiple feature units are constructed in the search space based on the numerical range of the risk matching results. The risk matching results are then assigned to the corresponding feature units according to the clause index order, generating a candidate risk set, specifically including: Read each risk matching result in the risk matching sequence in the order of the clause index, read all feature units in the search space, and establish the record position of the risk matching result in each feature unit; The search space is divided into intervals based on the numerical range of the risk matching results, forming multiple feature units in the search space, and a corresponding numerical interval is set for each feature unit. Perform a numerical range location operation on each risk matching result, retrieve the numerical range corresponding to the risk matching result in the search space, and determine the corresponding feature unit; Write the risk matching result into the corresponding feature unit, and read the risk matching result already recorded in the current feature unit. Perform a numerical comparison operation between the written risk matching result and the recorded risk matching result. The risk matching results in the corresponding feature units are updated based on the numerical comparison results, so that the risk matching result with the largest recorded value is recorded in each feature unit. The risk matching results in all feature units are read in the order of the clause index to form a candidate risk set. S55. In the candidate risk set, score calculation is performed based on the risk matching results and the corresponding rule node identifiers. The score values ​​of each risk matching result are sorted and filtered, and a clause risk score set is generated according to the clause index order. S56. Write the risk scores of each clause into the corresponding clause index position according to the order of the clause index to form a clause risk sequence.

10. The method for identifying enterprise contract compliance risks based on deep learning according to claim 1, characterized in that, S6 specifically includes: S61. Collect manual review information, and perform clause boundary positioning and clause index extraction operations on the manual review information to generate a sequence of review clauses containing risk assessment results; S62. Read the clause risk scores in the clause risk sequence and the corresponding risk assessment results in the audit clause sequence according to the clause index order, and perform the clause index alignment operation to generate the alignment result sequence; S63. Read the clause risk score and risk assessment result corresponding to each clause index in the alignment result sequence in sequence, perform difference calculation on the clause risk score and risk assessment result to generate clause error value, and arrange all clause error values ​​in the order of clause index to form an initial error sequence; S64. Read the clause risk score and risk assessment result of each clause in the alignment result sequence according to the clause index order, perform consistency assessment operation on the clause risk score and risk assessment result, generate a consistency mark at each clause index position, arrange all consistency marks according to the clause index order to form a consistency mark sequence; S65. Perform a sliding window expansion operation on the initial error sequence, calculate the median error and the absolute deviation of the median in each sliding window, and construct an anomaly detection threshold based on the median error and the absolute deviation of the median. S66. Use a Hampel filter to perform outlier detection on the initial error sequence, mark the outlier error positions that exceed the outlier detection threshold in the initial error sequence, and generate an outlier marking sequence. S67. Perform deviation correction operation on the marked positions in the abnormal marking sequence, read the clause error value corresponding to the unmarked position in the corresponding sliding window, perform weighted aggregation calculation on the unmarked clause error value according to the preset weight set, generate the correction error value, write the correction error value into the abnormal error position, and obtain the correction error sequence. S68. Merge the corrected error sequence and the consistency mark sequence in the order of the clause index to generate an error sequence.

Images