A Context-Aware Imbalanced Smart Legal Contract Representation System and Method

Abstract

This invention discloses a context-aware imbalanced intelligent legal contract representation system and method, primarily addressing the problem of inaccurate element extraction caused by data imbalance and complex contextual information in existing technologies. The solution combines a context-aware deep learning model with imbalanced data processing techniques, optimizing the efficiency of training data utilization through progressive filtering and weight adjustment. This includes: segmenting and tagging legal contract texts; utilizing dynamic context filling methods to enhance the model's understanding of context; and progressively filtering featureless samples to improve the extraction capability of key elements. Ultimately, the system can accurately and efficiently represent intelligent elements in legal texts, effectively reducing legal risks arising from the ambiguity of natural language. This invention employs innovative imbalanced data processing strategies, context-aware mechanisms, and edge-weighted loss functions, effectively improving the representation accuracy and generalization ability of intelligent contract legal texts.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing based on deep learning, and relates to a method and system for representing smart contract legal text based on data rebalancing and context awareness. It can be used to extract predefined intelligent elements from legal contract text and realize legal text representation. Background Technology

[0002] With the development of blockchain technology, smart contracts have become an important tool for automating contract execution. Smart contracts have shown broad application prospects in finance, law, insurance, and other fields. However, due to the automated and self-executing characteristics of smart contracts, accurately and comprehensively representing the legal provisions within them becomes particularly important. In the context of the legalization of smart contracts, the legal risks arising from the ambiguity and vagueness of natural language urgently need to be addressed, making the generation of high-quality, efficient, and highly generalizable legal provision representation models a key research focus.

[0003] Representing legal provisions in smart contracts involves several complex steps, including data preprocessing, handling imbalanced data, feature identification and annotation, and building a context-aware representation model. Current technologies face challenges in handling these steps, particularly in addressing data imbalance and improving feature-level representation performance. While existing solutions have made some progress in handling imbalanced data and context-aware technologies, many problems remain to be solved for legal scenarios. The following introduces several existing technical solutions and their advantages and disadvantages:

[0004] Guilin University of Technology proposed an energy-efficient federated learning method for imbalanced data distribution in its patent application, "An Energy-Saving Federated Learning Method for Imbalanced Data Distribution" (Application No.: CN202410235759.1, Publication No.: CN118133984A). This method first constructs a system model for an enterprise credit assessment scenario. It ensures a smooth decrease in global loss by constructing a local proxy function for each client, and uses the maximum latency of a single round of federated learning as a unified latency constraint in synchronous communication. Finally, by jointly allocating client CPU frequency and bandwidth, an energy-efficient federated learning method is proposed. This method reduces system energy consumption while mitigating the impact of data imbalance on model training. However, this method is mainly aimed at federated learning scenarios and fails to address the context awareness problem in the representation of legal provisions.

[0005] Xi'an Jiaotong University, in its patent application "A Data Imbalanced Target Recognition Method, System, Device and Storage Medium" (Application No.: CN202210143095.7, Publication No.: CN114528933A), proposed a data imbalanced target recognition method based on Bayesian theory and a multi-classifier system. This method improves the prediction confidence of minority class samples by dynamically selecting and combining base classifiers, and provides the confidence level for each prediction result. However, while this method improves the prediction accuracy of minority class samples, it is mainly applied to the field of target recognition and fails to address the context-aware problem of legal text representation.

[0006] In its patent application "A Multi-Level Optimization Oversampling Method for Imbalanced Data" (Application No.: CN202310837190.1, Publication No.: CN117056716A), Nanjing University of Aeronautics and Astronautics proposed a method to enhance the sensitivity of minority class sample feature recognition by improving the binary cross-entropy loss function and weighted SMOTE oversampling, and performing multi-level optimization. This method maintains the advantages of oversampling ensemble methods while utilizing collaborative optimization to reduce the possibility of inappropriate sampling strategies, thus improving the recognition rate of minority class samples. However, this method mainly optimizes from the data preprocessing and feature levels, failing to address how to utilize contextual information for deep learning in the representation of legal provisions.

[0007] In its patent application "A Feature Selection System and Method Based on Imbalanced Data Environment" (Application No.: CN202311847399.2, Publication No.: CN117556233A), Chaohu University proposed a method for feature selection by generating and evaluating feature subsets through a feature processing system. This method effectively removes irrelevant features from the dataset, improving classification efficiency and simplicity. However, this method mainly focuses on feature selection and classification efficiency, failing to address the context-awareness issue in the representation of legal provisions in smart contracts.

[0008] In summary, while existing technologies have made breakthroughs in addressing the problem of data imbalance, a comprehensive solution for context-aware representation of legal provisions in smart contracts is still lacking. There is an urgent need to research effective solutions that can improve the quality and efficiency of legal provision representation and meet the practical needs of the legalization process of smart contracts. Summary of the Invention

[0009] This invention aims to address the problem of inaccurate element extraction in smart contract legal text processing due to data imbalance and complex context. By introducing a context-aware deep learning model and combining it with imbalanced data processing techniques, the extraction accuracy of predefined elements in legal text is effectively improved.

[0010] The technical idea of ​​this invention is to design an intelligent model that includes preheating training, screening training, and dynamic context filling methods to classify and represent the character sequences in legal contracts, and further optimize the training process of the model by deleting invalid data, so as to achieve efficient representation of legal text of smart contracts.

[0011] To achieve the above objectives, the technical solution proposed by this invention is as follows:

[0012] A context-aware, imbalanced intelligent legal contract representation system includes a contract text preprocessing module, a context-aware model, and an imbalanced data processing module.

[0013] The contract text preprocessing module is used to preprocess the input smart contract legal text, including text cleaning, structure extraction and sentence segmentation operations, to generate contract text data;

[0014] The context-aware model includes a deep learning module and a smart contract representation model generation module. It is a multi-layer neural network model based on deep learning technology. It fills the contract text data generated by the contract text preprocessing module with context information through the context dynamic filling CDPM method and obtains the element recognition result.

[0015] The imbalanced data processing module is used to handle the imbalance of legal text data. It improves the recognition accuracy of minority class samples through screening training and dynamic weight adjustment methods, thereby optimizing the performance of the context-aware model.

[0016] Furthermore, the aforementioned contract text preprocessing module splits the input text into independent sentences using predefined rules, and synchronously divides the sentences with their corresponding element classification label sequences, thus completing a one-to-one correspondence between each character and label in the contract text.

[0017] Furthermore, in the aforementioned context-aware model, the deep learning module takes the contract text data generated by the contract text preprocessing module as input and outputs element prediction results at the word level of the contract text. Specifically, it uses the BERT model to generate word embedding representations of the contract text, then uses a bidirectional GRU network to extract contextual information from the word embedding sequence, and finally uses a Tri-Attention network layer to capture jumpy contextual information. The smart contract representation generation module takes the mapping between the contract text input and the element prediction results output of the deep learning module as input, aggregates, labels, and outputs the contract text labeled as elements.

[0018] Furthermore, the aforementioned imbalanced data processing module provides a marginal weighted loss function and an AUC-weighted loss index for deep learning model training, dynamically calculating the loss values ​​of various types of samples during training. This dynamic calculation refers to gradually increasing the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various elements, and employing a negative sample data screening method to enhance the element extraction capability of the deep learning model. The negative sample data screening method involves sorting the batches of data with a label of 0 in the contract text after each round of training in the imbalanced data processing module, and deleting the batches of no-feature data with the lowest loss.

[0019] A method for representing unbalanced intelligent legal contracts based on a context-aware system is characterized by the following steps:

[0020] (1) Obtain multiple legal contract texts and assign corresponding element classification labels to each character to construct a dataset;

[0021] (2) Input the constructed dataset into the contract text preprocessing module of the system. The module processes the dataset through a word segmenter to obtain contract text converted into a token sequence, and performs sentence filling processing according to the context dynamic filling method to obtain preprocessed text data, which is then randomly divided into training set, validation set and test set.

[0022] (3) Use the training set to perform multiple rounds of warm-up training on the context-aware model. In each round, the data is divided into multiple batches according to the batch size for training. For each training batch in each round, BERT is used to generate contract text word embedding representations. Bidirectional GRU and Tri-Attention network are used to extract context information and obtain the prediction results of the data in the training set.

[0023] (4) The imbalanced data processing module filters and trains the context-aware model according to the preset interval number of rounds. That is, every certain number of rounds, the loss of each batch in the current round is calculated, and low-loss no-feature data batches are filtered and deleted. At the same time, the loss weight is dynamically adjusted according to the loss of minority class samples to improve the model's recognition accuracy of minority class elements.

[0024] (5) Perform backpropagation of the model network based on the loss, update the model weights, and obtain the trained model;

[0025] (6) Input the legal contract to be represented into the system and use the trained model to complete the contract representation.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] First, this invention introduces a mechanism based on dynamic adjustment of sample importance. By progressively calculating the weight of each training sample, samples with little contribution to the model, such as those without features or of low value, are filtered out. This not only improves the training efficiency of the model but also significantly increases the recognition rate of minority class samples, thereby effectively solving the limitations of traditional methods in dealing with data imbalance. Compared with the existing AUC weighted loss method, the solution of this invention is more efficient and can more effectively address the problem of sample imbalance.

[0028] Secondly, the invention employs a dynamic weighting mechanism during model training, which significantly improves training efficiency by progressively filtering samples with no features or low value. This dynamic adjustment method accelerates the training speed of the model and improves the impact of samples on model performance during training, thereby enhancing the overall stability and accuracy of the model.

[0029] Third, considering the special characteristics of smart contracts, this invention proposes an edge-weighted loss method to effectively handle the boundary problem between key elements and non-key elements in the contract. This ensures that the model can accurately identify and represent key elements when processing legal provisions of the contract, and significantly improves the application effect of the model in real legal scenarios.

[0030] Fourth, this invention proposes a novel deep learning annotation model that combines a context-aware mechanism. This model can deeply extract feature elements from legal texts and generate rich semantic element representations. When dealing with complex legal texts in smart contracts, this model can more accurately capture key content and effectively improve the quality and efficiency of text representation. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the overall implementation of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating the implementation process of processing unbalanced data in this invention;

[0033] Figure 3 This is a schematic diagram of the structure of the smart contract representation module in this invention;

[0034] Figure 4 This is a comparison chart of the model performance indicators before and after unbalanced data processing in this invention. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] Example 1: Refer to Figure 1 The context-aware, imbalanced intelligent legal contract representation system proposed in this invention includes:

[0037] The system comprises a contract text preprocessing module, a context-aware model, and an imbalanced data processing module. The contract text preprocessing module preprocesses the input smart contract legal text, including text cleaning, structured extraction, and sentence segmentation, generating contract text data. The context-aware model, consisting of a deep learning module and a smart contract representation model generation module, is a multi-layer neural network model based on deep learning technology. It uses the context-filling dynamic method (CDPM) to fill in the context information of the contract text data generated by the contract text preprocessing module and obtains element recognition results. The imbalanced data processing module addresses the imbalance of the legal text data, improving the recognition accuracy of minority class samples through selective training and dynamic weight adjustment methods, thereby optimizing the performance of the context-aware model.

[0038] The contract text preprocessing module splits the input text into independent sentences according to predefined rules, and synchronously divides the sentences with their corresponding element classification label sequences, thus completing the one-to-one correspondence between each character and label in the contract text.

[0039] In the context-aware model, the deep learning module takes the contract text data generated by the contract text preprocessing module as input and outputs element prediction results at the word level of the contract text. Specifically, it uses the BERT model to generate word embedding representations of the contract text, then uses a bidirectional GRU network to extract contextual information from the word embedding sequence, and finally uses a Tri-Attention network layer to capture jumpy contextual information. The smart contract representation generation module takes the mapping between the contract text input and the element prediction results output of the deep learning module as input, aggregates, labels, and outputs the contract text labeled as elements.

[0040] The imbalanced data processing module provides a marginal weighted loss function and an AUC-weighted loss index for deep learning model training, dynamically calculating the loss values ​​of various types of samples during training. This dynamic calculation refers to gradually increasing the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various elements, and employing a negative sample data screening method to enhance the element extraction capability of the deep learning model. The negative sample data screening method involves sorting the batches of data with a label of 0 in the contract text after each round of training in the imbalanced data processing module, and deleting the batches of no-feature data with the lowest loss.

[0041] Example 2: Refer to Figure 1 The method for characterizing the system based on Embodiment 1 proposed in this invention includes the following steps:

[0042] Step 1. Obtain multiple legal contract texts and assign corresponding element classification labels to each character to construct a dataset;

[0043] Step 2. Input the constructed dataset into the system's contract text preprocessing module. This module processes the dataset using a word segmenter to obtain contract text converted into token sequences, and performs sentence completion processing using a context-based dynamic completion method to obtain preprocessed text data. This preprocessed data is then randomly divided into training, validation, and test sets. In this embodiment, the context-based dynamic completion method dynamically fills in the target sentence based on its context to ensure the target sentence reaches the standard input length.

[0044] Step 3. Use the training set to perform multiple rounds of warm-up training on the context-aware model. In each round, the data is divided into multiple batches according to the batch size for training. For each training batch in each round, BERT is used to generate word embeddings of the contract text. Bidirectional GRU and Tri-Attention network are used to extract context information and obtain the prediction results of the data in the training set.

[0045] Step 4. The imbalanced data processing module performs selective training on the context-aware model according to a preset number of training rounds. Specifically, every certain number of rounds, the loss of each batch in the current round is calculated, and batches of data without features and low loss are filtered and deleted. Simultaneously, the loss weights are dynamically adjusted based on the loss of minority class samples to improve the model's accuracy in recognizing minority class features. In this embodiment, after each round of selective training, the training batches with the smallest loss are deleted, and the next round of training continues until the preset number of training rounds is reached or the loss function converges.

[0046] Step 5. Perform backpropagation of the model network based on the loss, update the model weights, and obtain the trained model;

[0047] Step Six. Input the legal contract to be represented into the system and use the trained model to complete the contract representation. In this embodiment, the deep learning module first uses BERT to generate word embeddings for each batch of contract text data, and then uses a bidirectional GRU and Tri-Attention network to extract contextual information, outputting word-level element prediction results for the contract text. The smart contract representation model generation module maps the prediction results to the contract text, outputting the element text contained in the contract text and its element category one by one, thus obtaining the contract representation result.

[0048] Example 3: The overall implementation steps of the representation method proposed in this example are the same as in Example 2. The training / testing process of the model is described in further detail below:

[0049] Step A. Generation of training / inference datasets:

[0050] The input consists of multiple legal contract texts. During training, a sequence of element classification labels corresponding to each character is also required. To ensure a balanced approach to model training, validation, and testing, the contract data is randomly divided into training, validation, and test sets in a 7:1.5:1.5 ratio. Simultaneously, based on sentence-ending punctuation and maximum character length limits, the legal contract texts are split into sentences, and the label sequences are also split synchronously according to sentences to ensure the correspondence between characters and labels.

[0051] Step B. Model Data Preprocessing:

[0052] First, the tokenizer of the base model is used to convert the text character sequence into a token sequence, and the tag sequence is also converted into a format that corresponds one-to-one with the token. Next, the Context-sensitive Dynamic Padding Method (CDPM) is called to pad the target sentence with the context sentences to reach the standard input length, and then the sentences are organized into batches for model training.

[0053] Step C. Warm-up Training:

[0054] This invention employs Bidirectional Encoder Representations from Transformers (BERT) to embed words into token sequences, generating tensor sequences. Subsequently, a bidirectional Gated Recurrent Unit (GRU) network is used to extract continuous contextual information from the tensor sequences. To further enhance the model's ability to capture complex contexts, a Tri-Attention network layer is designed to extract jumpy contextual information. During training, an edge-weighted loss function and the Area Under Curve (AUC) metric are used to calculate the training loss, and the model weights are dynamically updated through backpropagation.

[0055] Step D. Screening and Training:

[0056] In each round of training, the steps from the warm-up training are repeated. For each batch with all label sequences being 0 (i.e., data containing no significant features), its total loss is recorded. After each round of training, the losses of the batches are sorted, and the batch with the smallest loss is removed to improve training efficiency and the model's feature recognition ability.

[0057] Step E. Model Training:

[0058] Model training involves alternating between warm-up training and selective training until a predetermined number of training rounds or convergence of the loss function is achieved. Through this process, the model can be progressively optimized, enhancing its ability to extract intelligent elements from legal texts.

[0059] Step F. Model Testing / Inference:

[0060] During the testing phase, the model's performance is evaluated using a test set, and metrics such as F1 score and accuracy are calculated. In the inference phase, after inputting the contract text, the model can output corresponding element recognition results, thereby achieving intelligent representation of legal texts.

[0061] Example 4: Refer to Figure 1-3 The system and method proposed in this embodiment are implemented in the same way as in Embodiments 1 and 2. Now, according to the training process of Embodiment 3, the specific dataset and related parameter settings are given, and the invention is further described in detail through specific examples:

[0062] In this embodiment, 1,000 contracts were collected from government procurement websites in various regions in a legal and compliant manner and manually annotated, defining 8 major categories and 22 subcategories of sales contract elements.

[0063] Step 1. Generation of Training / Inference Datasets

[0064] Step 1.1 Input multiple legal contracts. In the training case, you also need to input the label sequence of the element category corresponding to each character in the contract.

[0065] Step 1.2 In the training case, divide the contracts into training set, test set and validation set contracts in a ratio of 7:1.5:1.5.

[0066] Step 1.3 breaks down the legal contract into a series of sentences based on sentence-ending punctuation and maximum length. Specifically, the contract is broken down into sentences according to ending punctuation marks (including "?", "!", etc.) and line breaks. If a sentence is too long, other punctuation marks, such as ",", are also used as ending punctuation. During this process, only consecutive spaces, tabs, or line breaks are reduced to one, as consecutive spaces or line breaks are meaningless. The label sequence is also broken down into cases where each label corresponds one-to-one with a word in the sentence. The resulting dataset after these steps is shown in the table below:

[0067] Table 1. Number of samples in the dataset

[0068]

[0069]

[0070] Step 2. Model data preprocessing.

[0071] Step 2.1 The tokenizer of the base model converts the text sequence into a token sequence, and the label sequence is also changed to correspond to the tokens one by one. Suppose the length of the input sentence is m, and the maximum length of the model input is set to N. Then the i-th sentence s i can be represented as s i =(t i,1 , t i,2 , t i,3 , …, t i,m ).

[0072] Step 2.2 Invoke the Context-sensitive Dynamic Padding Method (CDPM) to fill each target sentence to the standard input length using the context sentences and divide them into batches. Specifically, suppose the length of the input sentence is m, and the maximum length of the model input is set to N. Then the sentence s i can be represented as s i =(t i,1 , t i,2 , t i,3 , …, t i,m ). For each sentence s i , we first evaluate its length m i , and m i±1 of s i±1 . If m i +m i±1 <N, then the tokens of s i-1 will be filled in front of s i , and the tokens of s i+1 will be filled behind s i . The newly formed input containing three sentences can be represented as:

[0073]

[0074] Many times, the total length of the three sentences is still less than N. Then s i±2 ]>, s i±3 will also be evaluated and concatenated to the input in the above way until the newly added sentence makes the input size greater than N.

[0075] Most of the time, newly formed input cannot perfectly fit the maximum input length, and in this case, traditional padding mechanisms will still be activated and fill in the remaining characters. However, contract-context-based padding mechanisms can often fill in more content, making our input contain more information that helps with feature extraction. Thus, our input can be divided into four parts: preceding text, target text, subsequent text, and meaningless padding. During processing, we will treat meaningless padding as part of the subsequent text, therefore the input becomes three parts: preceding, target, and subsequent. For simplicity, let's use... This represents the output of this module. At this point, s i The token can be represented as Here z+1 represents s i exist The starting position in the middle.

[0076] Step 3. Warm-up Training Steps

[0077] Reference Figure 2 The implementation steps of the embodiments of the present invention will be further described below.

[0078] Step 3.1 Use models such as Bidirectional Encoder Representations from Transformers (BERT) to perform word embeddings, converting the token sequence into a tensor sequence, for example:

[0079]

[0080] Step 3.2 uses a bidirectional GRU network to further extract continuous contextual information from the tensor sequence. GRU is a common recurrent neural network that can effectively extract temporal information. In contract text, temporal information helps infer element types. For example, in clause type elements, conditional elements appear before outcome elements, while postconditions appear after outcome elements. Using a bidirectional GRU layer to enhance temporal information extraction can effectively improve the performance of contract element extraction. Each After being computed by a bidirectional GRU, it is represented as Right now:

[0081]

[0082] Step 3.3 uses a Tri-Attention network layer to extract the jump context information of the tensor sequence. The Tri-Attention layer consists of three attention layers concatenated: predecessor attention, self-attention, and successor attention. They process different parts of the input. If the input sequence is the output sequence of a GRU, i.e. Then the self-attention layer will... That is, the tensor corresponding to the target sentence serves as the value and key input to the attention layer. The preceding sentence is then used... The following sentence uses At the same time, all three attention layers will As the query input for the attention layer.

[0083] To explain how a single attention layer works, let's take the computation of forward attention as an example.

[0084] We input and mask Get the unweighted query input Q = All of these positions are not in (z, z+m) i Values ​​within the range will be set to 0, as will the key input. The value 0 will not participate in the subsequent calculations. Next, we flip the first z elements of the key input to obtain the actual key input. The portion of the tensor closest to the target is moved to a fixed position in the weight matrix at the front. This is because, compared to the "first character in the sentence," the "character closest to the target" is more likely to help infer the character type.

[0085] Next, the weight matrix makes the three-headed attention layer trainable. The first matrix W q Used for query input. The second matrix W v pre The value input is used for forward attention. The three attention layers share W. q But their W v It's different. Therefore, in Luong-style, the attention weight of the preceding text can be calculated as follows:

[0086] σ(Q,K)=Q·W q ·K T

[0087] Att(Q,K,V)=softmax(σ(Q,K))·(V·W v pre )

[0088] Taking forward attention as an example, we flip a portion of the result (restoring the previously flipped portion) to obtain Att. pre As a result, through a process similar to that described above, we can obtain the attention result Att for all parts. pre Att self Att nextSince the masked portion is set to 0 and does not participate in the calculation, we can simply add the three attention results together to obtain Att. all =Att pre +Att self +Att next Therefore, the attention-weighted output of the three-head attention layer can be calculated as follows:

[0089]

[0090] Step 3.4 During training, the edge-weighted cross-entropy loss and the AUC function are used together to calculate the training loss based on the tensor sequence output and the label sequence. During backpropagation, the weights of each network node are updated based on the loss.

[0091] Specifically, if there are 23 categories Indicates t i,j-z The probability of belonging to one of the 23 categories, using Indicates t i,j-z The one-hot encoding of the true classification value. Generally, cross-entropy loss is used to calculate the loss.

[0092]

[0093] The cross-entropy loss function aims to correctly identify as many tokens as possible. However, when using sequence tags for element extraction, the position of correctly predicted tokens may be more important than their quantity. Specifically, the edges of ground-value elements are difficult to label accurately, while the accuracy of predicted edges also significantly impacts the results. Therefore, we propose an edge-weighted loss, which assigns higher weights to edge tokens of both ground-value and predicted elements during training when calculating the loss. The edge-weighted cross-entropy loss is calculated as follows:

[0094]

[0095] Since the AUC function is for binary classification, we have two possible ways to use it: 1. Treat tokens that do not belong to any element as negative samples, and all others as positive samples; this is the general way to use AUC. 2. Treat tokens belonging to a certain class as positive samples, and all others as negative samples, calculate the AUC for each class and take the average. Experiments have shown that the latter is more effective when the imbalance between positive and negative samples is not severe (the number of positive and negative samples is roughly equal during training); the former is more effective when the imbalance is severe. Given that contracts in reality generally have severe imbalance, we adopted the former as the main method. For a batch, assume that the number of positive samples and the number of negative samples are n respectively. pos and nneg The AUC is calculated as follows:

[0096]

[0097] Since a lower AUC indicates a worse result and requires more attention during training, the weighted loss of all sentences in this batch can be calculated as follows:

[0098]

[0099] Step 4. Selection and Training Steps

[0100] Step 4.1 is the same as steps 3.1 to 3.4.

[0101] Step 4.2 For each batch with a label sequence of all zeros, calculate its total loss and record the batch and its loss as a key-value pair.

[0102] Step 4.3 Repeat steps 4.1 to 4.2 for each batch in each round. At the end of each round, sort the batches in the training set according to the loss and remove the batches with the smallest δ% loss, which will not participate in the next training.

[0103] Step 5. Model Training:

[0104] Steps 3 and 4 are executed alternately according to a certain pattern, repeating a certain number of rounds. (Refer to...) Figure 3 At the end of each round, inference is performed on the development set. After each execution of step 3 or 4, the inference output and the label sequence of the development set are used to calculate the development set loss. The number of rounds is estimated based on the convergence of the development set loss. The optimal number of rounds is: execute step 4 once after every 4 executions of step 3, repeating for 3 rounds, for a total of 15 times; finally, execute step 3 5 times to complete model training.

[0105] Step 6. Model Testing / Inference:

[0106] In the test case, F1 and Accuracy are calculated based on the output and the true value. In the inference case, the output is the identification and representation result of the element.

[0107] The effects of this invention can be further illustrated by the following simulation.

[0108] 1. Simulation conditions

[0109] The simulation experiments of this invention were conducted in the following hardware and software environment:

[0110] Software platform: Ubuntu 20.04 operating system, Python 3.10 programming language, PyTorch 2.0.0 training framework, CUDA 11.7 parallel computing architecture.

[0111] Hardware platform: Graphics card NVIDIA GeForce RTX 3060 GPU 12GB.

[0112] Dataset: The Chinese Purchase Contracts Dataset (CPCD) used in this invention was collected and created in 2023 and is a specific dataset used for legal text representation and smart contract analysis.

[0113] 2. Simulation Content and Result Analysis

[0114] This simulation experiment mainly compares the performance of the method of this invention with that of the prior art (BERT model) in terms of imbalanced data processing and context awareness.

[0115] exist Figure 4 The paper presents a performance comparison between models without and without data rebalancing (i.e., without applying a specific loss function and data filtering mechanism). Without data rebalancing, the model's average F1 score was 0.635, the weighted average F1 score was 0.721, and the training time was 356 minutes. After using data rebalancing, the model's average F1 score increased to 0.639, the weighted average F1 score increased to 0.735, and the training time decreased to 272 minutes. This demonstrates that the dynamic sample weight adjustment mechanism for progressively removing featureless data proposed in this invention effectively improves model performance and significantly shortens training time.

[0116] also, Figure 4 The invention also demonstrates the further enhancement of model performance by the context-aware module designed in this invention. Compared with the traditional BERT model that does not use a rebalancing method, the model using the method of this invention not only performs better in handling imbalanced data, but also improves the ability to understand the implicit semantics in complex legal clauses through the context-aware mechanism, further enhancing the model's generalization ability and robustness.

[0117] 3. Conclusion

[0118] The simulation experiments demonstrated that the proposed method for representing smart contract legal text based on imbalanced data processing and context awareness significantly improves the model's classification and matching performance when handling imbalanced legal text. Specifically, by dynamically adjusting sample weights and progressively eliminating data lacking essential features, this invention effectively addresses the negative impact of imbalanced data on model training and improves the accuracy of identifying key legal elements. Furthermore, the introduced context-aware module further enhances the model's understanding of complex legal clauses, strengthening its robustness and generalization ability. This method not only helps reduce the risks associated with the ambiguity of natural language in smart contracts but also improves the legality and consistency of contract terms, showing broad promise in smart contract applications.

[0119] The above simulation analysis proves the correctness and effectiveness of the method proposed in this invention.

[0120] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0121] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A context-aware, imbalanced intelligent legal contract representation system, characterized in that, include: Contract text preprocessing module, context-aware model, and imbalanced data processing module; The contract text preprocessing module is used to preprocess the input smart contract legal text, including text cleaning, structure extraction and sentence segmentation operations, to generate contract text data; The context-aware model includes a deep learning module and a smart contract representation model generation module. It is a multi-layer neural network model based on deep learning technology. It fills the contract text data generated by the contract text preprocessing module with context information through the context dynamic filling CDPM method and obtains the element recognition result. The imbalanced data processing module is used to handle the imbalance of legal text data, and improves the recognition accuracy of minority class samples through screening training and dynamic weight adjustment methods, thereby optimizing the performance of the context-aware model. The deep learning module takes the contract text data generated by the contract text preprocessing module as input and outputs the element prediction results at the word level of the contract text. Specifically, the BERT model is used to generate word embeddings of the contract text, then a bidirectional GRU network is used to extract contextual information from the word embedding sequence, and finally a Tri-Attention network layer is used to capture jump contextual information. The imbalanced data processing module provides edge-weighted loss function and AUC-weighted loss index for deep learning model training, and dynamically calculates the loss value of each type of sample during training. The dynamic calculation refers to gradually increasing the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various elements, and using negative sample data screening method to improve the element extraction capability of deep learning model.

2. The system according to claim 1, characterized in that: The contract text preprocessing module splits the input text into independent sentences according to predefined rules, and synchronously divides the sentences with their corresponding element classification label sequences, thus completing the one-to-one correspondence between each character and label in the contract text.

3. The system according to claim 1, characterized in that: In the context-aware model, the smart contract representation generation module takes the mapping between the contract text input and the element prediction result output of the deep learning module as input, aggregates, marks and outputs the contract text marked as elements.

4. The system according to claim 1, characterized in that: The negative sample data screening method refers to sorting the data batches with a label of 0 in the contract text after each round of screening training in the imbalanced data processing module, and deleting the data batches with the least loss that have no features.

5. A method for characterization using the system of claim 1, characterized in that, Includes the following steps: (1) Obtain multiple legal contract texts and assign corresponding element classification labels to each character to construct a dataset; (2) Input the constructed dataset into the contract text preprocessing module of the system. This module processes the dataset through a word segmenter to obtain contract text converted into a token sequence, and performs sentence filling processing according to the context dynamic filling method to obtain preprocessed text data, which is then randomly divided into training set, validation set and test set. (3) Use the training set to perform multiple rounds of warm-up training on the context-aware model. In each round, the data is divided into multiple batches according to the batch size for training. For each training batch in each round, BERT is used to generate the word embedding representation of the contract text. The bidirectional GRU and Tri-Attention network are used to extract context information and obtain the prediction results of the data in the training set. (4) The imbalanced data processing module filters and trains the context-aware model according to the preset interval number of rounds. Every certain number of rounds, the loss of each batch in the current round is calculated, and low-loss no-feature data batches are filtered and deleted. At the same time, the loss weight is dynamically adjusted according to the loss of minority class samples to improve the model's recognition accuracy of minority class elements. (5) Perform backpropagation of the model network based on the loss, update the model weights, and obtain the trained model; (6) Input the legal contract to be represented into the system and use the trained model to complete the contract representation.

6. The method according to claim 5, characterized in that: The contextual dynamic filling method in step (2) refers to dynamically filling the target sentence according to the context of the target sentence so that the target sentence reaches the standard input length.

7. The method according to claim 5, characterized in that: In step (4), after each round of training is completed, the training batch with the smallest loss is deleted, and the next round of training continues until the preset number of training rounds or the loss function converges.

8. The method according to claim 5, characterized in that: In step (6), the contract representation is completed using the trained model, specifically as follows: (6.1) The deep learning module uses BERT to generate word embeddings of the contract text for each batch of contract text data, and uses bidirectional GRU and Tri-Attention network to extract context information and output the element prediction results of the contract text at the word level. (6.2) The smart contract representation model generation module maps the prediction results to the contract text, outputs the element text contained in the contract text and its element category one by one, and obtains the contract representation results.

Images