Contract classification method and system based on deep learning
By employing a multimodal collaborative contract classification method that combines SBERT, CNN, and GAT networks and integrates text, metadata, and relational features, this approach addresses the issue of unsatisfactory contract classification results in existing technologies, achieving more efficient automatic contract classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies struggle to effectively understand and classify complex contract texts, especially since they neglect metadata, relational information, and tag hierarchy, resulting in unsatisfactory classification results.
A multimodal collaborative contract classification method is adopted, which integrates features of contract text, metadata, relationship information and label hierarchy through cross-modal attention mechanism, and uses SBERT model, CNN and GAT network for feature extraction and fusion.
It improves the accuracy and generalization ability of contract classification, reduces the dependence on the amount of labeled data, and enhances the ability to model semantic associations.
Smart Images
Figure CN122285901A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of text analysis, feature extraction, and graph structure learning, and in particular to a contract classification method and system based on deep learning. Background Technology
[0002] With the acceleration of digital transformation, enterprises and institutions need to process massive amounts of contract documents in their daily operations, including procurement contracts, lease agreements, and terms of service. These contracts often have complex legal semantics and diverse text structures, making manual classification and management inefficient and costly. Therefore, automatic contract classification technology has become an important research direction in the field of Natural Language Processing (NLP). Its main purpose is to reduce manual review time and management costs through automated classification, avoid the subjectivity and errors of manual classification, and ensure the standardization of contract archiving. Despite the significant value of contract classification, it still faces many challenges in practical application. First, contract texts often contain professional legal terminology, long sentence structures, and implicit logic, making it difficult to accurately understand their semantics using traditional methods. Second, contracts may contain structured clauses (such as amounts and dates) and unstructured text (such as disclaimers), requiring comprehensive analysis combining multiple features.
[0003] Traditional rule-based or machine learning-based methods rely on keyword matching or manually defined features for classification. These methods depend heavily on expert experience and struggle to understand the semantic knowledge embedded in contract texts, resulting in poor generalization ability. In recent years, with the rapid development of deep learning, deep learning-based contract classification methods have been widely applied, primarily achieving patent classification through semantic analysis of text content. However, these methods focus on contract text content, neglecting the role of metadata, relational information, and tag hierarchy, leading to less than ideal results. To address these issues, this invention proposes a multimodal collaborative contract classification method. Leveraging multimodal features including metadata, relational information, textual semantics, and tag hierarchy, and employing a cross-modal attention mechanism, it adaptively fuses representations from multiple modalities to improve classification performance. Summary of the Invention
[0004] To address the aforementioned issues and effectively classify contracts, this paper proposes a deep learning-based contract classification method and system.
[0005] The specific technical solution adopted by the contract classification method based on deep learning in this invention is as follows:
[0006] Step (1) Contract Data Acquisition: For each sample in the contract dataset, extract the contract text, metadata (such as contract number, contract type, company name, signing date, amount, contract form type), and classification labels; represent each contract sample as a triple. ,in This indicates the contract text. Represents metadata, Indicates category label.
[0007] Step (2) Data Preprocessing: Noise removal, terminology standardization, and sentence segmentation are performed on the contract text to ensure a consistent expression format; the format of numerical fields such as dates and amounts is standardized, and missing or abnormal data is supplemented and corrected; the preprocessed contract sample is represented as... ,in The preprocessed text, For standardized metadata.
[0008] Step (3) Feature vector representation of the text: First, the preprocessed feature vector of each contract sample is represented as a feature vector. text Inputting the pre-trained SBERT model yields various sentences. First, generate a semantic vector; then, perform average pooling on all sentence vectors to output the text feature vector of the contract. .
[0009] Step (4) Feature vector representation of metadata: Based on the preprocessed contract data, a metadata matrix is generated, and the matrix is split and concatenated; a convolutional neural network (CNN) is used to capture the local patterns of the metadata and output the metadata feature vector of the contract. .
[0010] Step (5) Feature vector representation of the relationship: Construct a contract relationship graph, where nodes represent contracts and companies, and edges represent the signing relationship between contracts and companies; use round-trip random walks and a graph attention network (GAT) to learn node representations and aggregate neighbor information; finally, output the relationship representation vector of the contract. .
[0011] Step (6) Multimodal feature fusion: Fusion of text feature vectors Metadata feature vector , correlation feature vector Dynamic weighted fusion is performed, and cross-modal interaction weights are calculated through a multi-head attention mechanism to generate... The final representation vector.
[0012] Step (7) Classifier design: Input the fused features into the fully connected layer, use Softmax to output the classification probability, use the cross-entropy function as the classification loss function, and perform training and testing.
[0013] In another aspect, the present invention also provides a contract classification system based on deep learning, comprising the following modules:
[0014] Data module: Used to acquire contract data and perform preprocessing.
[0015] Feature acquisition module: It is used to acquire text feature vectors, metadata feature vectors, and relationship feature vectors from preprocessed contract data using a large language model, a convolutional neural network, and a graph attention network that uses round-trip random walks.
[0016] Classification module: Used to dynamically weight and fuse text feature vectors, metadata feature vectors, and relationship feature vectors, and output the classification result through a classifier.
[0017] Compared with traditional classification methods, the beneficial effects of this invention are: (1) It integrates multimodal data such as text data, metadata, and graph structure data to solve the problem of insufficient information from a single data source; (2) The proposed round-trip random walk strategy introduces a random revisit mechanism for the path history, allowing non-current nodes on the starting path to continue walking; it effectively enhances the structural diversity and semantic coverage of the walk sequence and improves the modeling ability of semantic associations in the graph; (3) It uses a multi-head attention mechanism to calculate the weights of each modality data and adaptively distinguishes the importance of different modalities; (4) It uses small samples to fine-tune the pre-trained model (SBERT) to reduce the dependence on the amount of labeled data. Attached Figure Description
[0018] Figure 1 This is a flowchart of the deep learning-based contract classification method of the present invention;
[0019] Figure 2 This is a flowchart illustrating the execution of the multi-head attention mechanism. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 and attached Figure 2 This invention patent provides a detailed description of the deep learning-based contract classification method, including the following steps:
[0021] Step (1) Contract Data Acquisition: For each sample in the contract dataset, extract the contract text, metadata (such as contract number, contract type, company name, signing date, amount, contract form type), and classification labels; represent each contract sample as a triple. ,in This indicates the contract text. Represents metadata, Indicates category label.
[0022] Step (2) Data Preprocessing: Noise removal, terminology standardization, and sentence segmentation are performed on the contract text to ensure a consistent expression format; the format of numerical fields such as dates and amounts is standardized, and missing or abnormal data is supplemented and corrected; the preprocessed contract sample is represented as... ,in The preprocessed text, For standardized metadata.
[0023] Step (3) Feature vector representation of the text: First, each contract sample... text Input the pre-trained SBERT model to generate sentences. First, generate a semantic vector; then, perform average pooling on all sentence vectors to output the text feature vector of the contract. .
[0024] Step (4) Feature vector representation of metadata:
[0025] 4-1. Initial Matrix Generation: For each contract sample ,Will Features are categorized into two types: numerical features and discrete features. For numerical features such as amounts and dates, Min-Max normalization is first used for scaling, and then a linear mapping layer is used to convert them into different dimensions. For character-based features such as contract number, contract type, and company name, their strings are first mapped to unique integer codes, and then each integer code is converted into a vector with dimension [missing information]. The vector; finally, the two dimensions are... The vectors are concatenated to form a metadata matrix. ,in This represents the number of metadata entries.
[0026] 4-2. Matrix Splitting and Joining: [The text abruptly ends here, likely due to an incomplete The row vectors in the array are randomly shuffled to form a sequence containing Matrix combination of matrices ,in The row vectors in the data all come from ,and The row vectors of any two matrices in the array are arranged in a different order; The matrices in the first dimension are concatenated to obtain the matrix. ,in .
[0027] The matrix splitting and concatenation steps proposed in this invention are equivalent to adding a sequential perturbation-based data augmentation mechanism to the metadata input layer, enabling CNNs to learn more robust and comprehensive feature representations under different permutations and combinations. This improves both the model's generalization ability and the diversity of feature extraction.
[0028] 4-3. CNN Feature Extraction: Construct a CNN network, with the input layer receiving the metadata matrix. Captured by a convolutional layer with a kernel size of 3 The local features in the data are extracted and their dimensions are compressed using max pooling; the final output is the metadata feature vector of the contract. .
[0029] Step (5) Feature vector representation of the association relationship:
[0030] 5-1. Contract Relationship Graph Construction: Treat contracts as one type of node and companies as another type of node; for each contract sample in the dataset... From its metadata Obtain the name of the signing party company in [the process]. An undirected edge is established between the corresponding node and the corresponding node of the signing company to represent their signing relationship.
[0031] 5-2. Round-trip random walk: Starting from each contract node, use the following random walk method: If currently at the starting point, randomly select any neighboring node as the next step; otherwise, proceed with the next step based on probability. Randomly select any neighboring node of the current node as the next node, with probability. Return to any node on the path from the starting point to the current node (excluding the current node), and continue this random walk process; repeat the above operation until the random walk sequence contains... There are 10 nodes.
[0032] The proposed round-trip random walk strategy introduces a random revisit mechanism for the path history, allowing the walk to continue from a non-current node on the starting path. This "forward-backward" combined strategy effectively enhances the structural diversity and semantic coverage of the walk sequence, improves the modeling ability of semantic associations in the graph, and is particularly suitable for practical application scenarios where there are complex multi-hop relationships between contracts and companies. Compared with traditional random walk methods, this strategy is better able to capture the balance between long-term dependencies and local structures, and improves the expressiveness and discriminativeness of contract representations in downstream tasks.
[0033] 5-3. Node Vector Initialization: Input the generated set of fixed-length random walk sequences into the SkipGram model for training to obtain the initial vector representation of all nodes in the graph.
[0034] 5-4. GAT Representation Learning: A multi-layer GAT network is used to process graph-structured data. Each layer includes three steps: linear transformation of node features, calculation of attention coefficients based on the similarity of neighboring node features, and weighted feature aggregation. By stacking three layers of GAT, the receptive field is gradually expanded to capture the vector representation of multi-hop neighbors. Finally, the association feature vector of the contract is output. .
[0035] Step (6) Multimodal feature fusion:
[0036] 6-1. Dynamic Weighting of Features: Applying weights to text feature vectors Metadata feature vector , correlation feature vector Perform normalization and calculate the weighted sum. The formula is as follows:
[0037]
[0038] Among them, weight parameters Learned from the softmax function, satisfying ;
[0039] 6-2. Cross-modal interaction modeling: A 3-head attention mechanism is adopted, where the third... Size The calculation formula is:
[0040]
[0041] in , , For linear projection, , , For learnable weight matrix, For the feature dimension; the outputs of multiple heads are concatenated to obtain the output features after the attention mechanism. The calculation formula is as follows:
[0042]
[0043] in This is a learnable weight matrix.
[0044] 6-3. Residual Normalization: The final features are obtained through residual connections and layer normalization. The calculation formula is as follows:
[0045]
[0046] Step (7) Classifier design and loss calculation:
[0047] 7-1. Classifier Design: Construct a fully connected network with two hidden layers, where the input layer receives fused features. Number of output layer nodes and number of labels Same; each hidden layer is followed by a ReLU function and a Dropout function.
[0048] 7-2. Use the cross-entropy function as the classification loss function. The formula for calculating the cross-entropy function is as follows:
[0049]
[0050] in For the total number of samples, It is the first Each sample in category The actual label (0 or 1) on it. It is the first Each sample in category The predicted probability value. It is the cross-entropy loss function.
[0051] Step (8) Model training: Using the AdamW optimizer with a learning rate of 0.001, update the parameters on the training set based on the cross-entropy function;
[0052] Step (9) Model prediction:
[0053] 9-1. Regarding the target contract sample The classifier outputs a dimensional vector ,in Indicates the classifier for the first... The original scores for each category.
[0054] 9-2. The output score is normalized using the Softmax function to obtain... dimensional vector ,in This indicates that the current contract sample belongs to the first... The predicted probabilities of each category;
[0055] 9-3. Based on the probability vector The index corresponding to the element with the highest probability value is selected as the predicted category of the target contract sample.
[0056] This invention provides a contract classification system based on deep learning, comprising the following modules:
[0057] Data module: Used to acquire contract data and perform preprocessing.
[0058] Feature acquisition module: It is used to acquire text feature vectors, metadata feature vectors, and relationship feature vectors from preprocessed contract data using a large language model, a convolutional neural network, and a graph attention network that uses round-trip random walks.
[0059] Classification module: Used to dynamically weight and fuse text feature vectors, metadata feature vectors, and relationship feature vectors, and output the classification result through a classifier.
Claims
1. A contract classification method based on deep learning, characterized in that, Includes the following steps: Step 1: Obtain contract data and perform preprocessing; Step 2: Based on the preprocessed contract data, text feature vectors, metadata feature vectors, and association feature vectors are obtained through a large language model, a convolutional neural network, and a graph attention network using round-trip random walks, respectively. Step 3: Dynamically weight and fuse the text feature vector, metadata feature vector, and association feature vector, output the classification result through the classifier, and then conduct training and testing.
2. The contract classification method based on deep learning according to claim 1, characterized in that, The specific implementation process of step 1 is as follows: Step 1.1: For each sample in the contract dataset, extract the contract text, metadata, and classification labels; represent each contract sample as a triple. ,in This indicates the contract text. Represents metadata, Indicates category tags; Step 1.2: Perform noise cleanup, terminology standardization, and sentence segmentation on the contract text; standardize the format of numerical fields; and complete and correct missing or abnormal data; represent the preprocessed contract sample as follows: ,in For the preprocessed text, For standardized metadata.
3. The contract classification method based on deep learning according to claim 2, characterized in that, The metadata includes contract number, contract type, company name, signing date, amount, and contract form type.
4. The contract classification method based on deep learning according to claim 3, characterized in that, The specific implementation process of step 2 is as follows: Step 2.1: First, process each preprocessed contract sample... text Inputting the pre-trained SBERT model yields various sentences. First, generate a semantic vector; then, perform average pooling on all sentence vectors to output the text feature vector of the contract. ; Step 2.2: Based on the preprocessed contract data, generate a metadata matrix, split and concatenate the matrix, and then use a convolutional neural network (CNN) to capture local patterns in the metadata, outputting the metadata feature vector of the contract. ; Step 2.3: Construct a contract relationship graph, where nodes represent contracts and companies, and edges represent the signing relationship between contracts and companies; use round-trip random walks and a graph attention network (GAT) to learn node representations and aggregate neighbor information; finally, output the relationship representation vector of the contract. .
5. The contract classification method based on deep learning according to claim 4, characterized in that, The specific implementation process of step 2.2 is as follows: Initial matrix generation: for each contract sample ,Will Features are categorized into two types: numerical features and discrete features. For numerical features, Min-Max normalization is first used for scaling, and then a linear mapping layer is used to convert them to different dimensions. For character features, first map their strings to unique integer codes, then convert each integer code into a vector of dimension 1. The vector; finally, the two dimensions are... The vectors are concatenated to form a metadata matrix. ,in The number of metadata items; Will The row vectors in the array are randomly shuffled to form a sequence containing Matrix combination of matrices ,in The row vectors in the data all come from ,and The row vectors of any two matrices in the array are arranged in a different order; The matrices in the first dimension are concatenated to obtain the matrix. ,in ; Construct a CNN network, with the input layer receiving a metadata matrix. Captured through convolutional layers The local features in the data are extracted and their dimensions are compressed using max pooling; the final output is the metadata feature vector of the contract. .
6. The contract classification method based on deep learning according to claim 4, characterized in that, The construction of the contract relationship graph specifically involves: treating contracts as one type of node and companies as another type of node; for each contract sample in the dataset... From its metadata Obtain the name of the signing party company in [the process]. An undirected edge is established between the corresponding node and the corresponding node of the signing company to represent their signing relationship.
7. The contract classification method based on deep learning according to claim 4, characterized in that, The round-trip random walk is specifically implemented as follows: starting from each contract node, the following random walk method is used: if the current node is at the starting point, then any neighboring node is randomly selected as the next step; otherwise, the next step is selected based on probability. Randomly select any neighboring node of the current node as the next node, with probability. , and If the sum of the elements is 1, return any node on the path from the starting point to the current node that does not contain the current node, and continue this random walk process; repeat the above random walk operation until the random walk sequence contains the element that is not present in the current node. There are 10 nodes.
8. The contract classification method based on deep learning according to claim 4, characterized in that, The specific implementation process of GAT representation learning is as follows: The generated set of fixed-length random walk sequences is input into the SkipGram model for training, and the initial vector representations of all nodes in the graph are obtained. For the initial vector representation, a multi-layer GAT network is used to process the graph-structured data. Each layer includes three steps: linear transformation of node features, calculation of attention coefficients based on the similarity of neighboring node features, and weighted feature aggregation. By stacking GATs, the receptive field is gradually expanded to capture the vector representation of multi-hop neighbors. Finally, the association feature vector of the contract is output. .
9. The contract classification method based on deep learning according to claim 7, characterized in that, The specific implementation process of step 3 is as follows: Step 3.1: Process the text feature vector Metadata feature vector Feature vectors of association Dynamic weighted fusion is performed, cross-modal interaction weights are calculated through a multi-head attention mechanism, and generated through residual connections and layer normalization. The final representation vector; Step 3.2: Input the fused features into the fully connected layer, and use Softmax to output the classification probability for the target contract sample. The classifier outputs a raw score vector, which is then normalized using the Softmax function to obtain a probability vector, from which the probability vector is determined. The category to which it belongs; Step 3.3: Use the cross-entropy function as the classification loss function and use the AdamW optimizer to update the parameters based on the cross-entropy function.
10. A deep learning-based contract classification system for implementing the contract classification method according to any one of claims 1 to 9, characterized in that, Includes the following modules: Data module: Used to acquire contract data and perform preprocessing; Feature acquisition module: Used to acquire text feature vectors, metadata feature vectors, and association feature vectors from preprocessed contract data using a large language model, a convolutional neural network, and a graph attention network employing round-trip random walks, respectively. Classification module: Used to dynamically weight and fuse text feature vectors, metadata feature vectors, and relationship feature vectors, and output the classification result through a classifier.