Unbalanced intelligent legal contract characterization system and method based on context awareness

Abstract

The invention discloses an unbalanced intelligent legal contract characterization system and method based on context awareness, and mainly solves the problem of inaccurate element extraction caused by data imbalance and complex context information in the prior art. According to the scheme, a context-aware deep learning model and an unbalanced data processing technology are combined, and the use efficiency of training data is optimized through step-by-step screening and weight adjustment. Comprising the steps of performing word segmentation and tagging processing on a legal contract text, enhancing the understanding of a model on a context by utilizing a context dynamic filling method, and finally enabling a system to accurately and efficiently represent intelligent elements in the legal text by gradually screening element-free samples and improving the extraction capability of key elements. And legal risks caused by fuzziness of natural languages are effectively reduced. According to the method, an innovative unbalanced data processing strategy, a context sensing mechanism and an edge weighting loss function are adopted, and the representation precision and generalization ability of the intelligent contract legal text are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing based on deep learning, and relates to a smart contract legal text representation method and system based on data rebalancing and context awareness, which can be used to extract predefined intelligent elements in legal contract texts and realize legal text representation. Background Art

[0002] With the development of blockchain technology, smart contracts have become an important tool for realizing automated contract execution. Smart contracts have shown broad application prospects in the fields of finance, law, insurance, etc. However, due to the automation and self-executing characteristics of smart contracts, it is particularly important to accurately and comprehensively represent the legal provisions in smart contracts. In the context of the legalization of smart contracts, the legal risks caused by the ambiguity and vagueness of natural language need to be urgently resolved, and generating high-quality, efficient, and generalized legal provision representation models has become a research focus.

[0003] The representation of legal provisions in smart contracts involves multiple complex steps, including data preprocessing, unbalanced data processing, element identification and annotation, and the construction of context-aware representation models. Current technologies face certain challenges in handling these steps, especially in dealing with data imbalance and improving the representation performance at the element level. In the context of dealing with unbalanced data and context-aware technology, existing technical solutions have made some progress, but there are still many problems that need to be solved for legal scenarios. The following introduces several existing technical solutions and their advantages and disadvantages:

[0004] Guilin University of Technology proposed an energy-saving federated learning method for data imbalance distribution in its patent application "An Energy-Saving Federated Learning Method for Data Imbalance Distribution" (application number: CN202410235759.1, publication number: CN118133984A). This method first builds a system model for the enterprise credit assessment scenario, constructs a local proxy function for each client to ensure the steady decline of the global loss, and uses the maximum delay of a single round of federated learning as a unified delay constraint in synchronous communication. Finally, an energy-saving federated learning method is proposed by combining the client CPU frequency and bandwidth allocation. This method reduces system energy consumption while alleviating the impact of data imbalance on model training. However, this method is mainly aimed at federated learning scenarios and fails to solve the problem of context perception in the representation of legal texts.

[0005] Xi'an Jiaotong University proposed a data imbalance target recognition method based on Bayesian theory and a multi-classifier system in its patent application "A data imbalance target recognition method, system, device and storage medium" (application number: CN202210143095.7, publication number: CN114528933A). This method improves the prediction credibility of minority class samples by dynamically selecting and combining base classifiers, and gives the confidence of each prediction result. However, although this method improves the prediction accuracy of minority class samples, it is mainly used in the field of target recognition and fails to involve the context perception problem of legal text representation.

[0006] In the patent document applied for by Nanjing University of Aeronautics and Astronautics, "A Multi-level Optimization Oversampling Method for Data Imbalance" (application number: CN202310837190.1, publication number: CN117056716A), a method of improving the binary cross entropy loss function and weighted SMOTE oversampling was proposed to enhance the sensitivity of feature recognition of minority samples and perform multi-level optimization. While maintaining the advantages of the oversampling ensemble method, this method uses collaborative optimization to reduce the possibility of improper sampling strategies and improve the recognition rate of minority samples. However, this method mainly optimizes from the data preprocessing and feature levels, and fails to involve how to use contextual information for deep learning in the process of legal text representation.

[0007] In its patent application, "A feature selection system and method based on an unbalanced data environment" (application number: CN202311847399.2, publication number: CN117556233A), Chaohu University proposed a method for feature selection by generating and evaluating feature subsets through a feature processing system. This method effectively removes irrelevant features in the data set and improves classification efficiency and simplicity. However, this method mainly focuses on feature selection and classification efficiency, and fails to solve the problem of context perception in the representation of legal provisions of smart contracts.

[0008] In summary, although existing technical solutions have made breakthroughs in dealing with data imbalance, there is no comprehensive solution for context awareness in the representation of legal provisions in smart contracts. It is urgent to develop an effective solution that can improve the quality and efficiency of legal provision representation and meet the actual needs of the legalization process of smart contracts. Summary of the invention

[0009] This invention aims to solve the problem of inaccurate element extraction caused by data imbalance and complex context in the processing of smart contract legal texts. By introducing a deep learning model that combines context awareness and unbalanced data processing technology, the extraction accuracy of predefined elements in legal texts can be effectively improved.

[0010] The technical idea of ​​the present invention is to design an intelligent model that includes preheating training, screening training and context dynamic filling methods, classify and characterize the character sequences in legal contracts, and further optimize the training process of the model by deleting invalid data, ultimately achieving efficient characterization of the legal text of the smart contract.

[0011] To achieve the above purpose, the technical solution proposed by the present invention is as follows:

[0012] A context-aware unbalanced smart legal contract representation system, including a contract text preprocessing module, a context-aware model and an unbalanced data processing module;

[0013] The contract text preprocessing module is used to preprocess the input smart contract legal text, including text cleaning, structure extraction and sentence segmentation operations, to generate contract text data;

[0014] The context-aware model includes a deep learning module and a smart contract representation model generation module. It is a multi-layer neural network model based on deep learning technology. It uses the context dynamic filling CDPM method to fill the contract text data generated by the contract text preprocessing module with context information and obtains the element recognition result.

[0015] The imbalanced data processing module is used to process the imbalance of legal text data, improve the recognition accuracy of minority class samples through screening training and dynamic weight adjustment methods, and achieve performance optimization of the context-aware model.

[0016] Furthermore, the contract text preprocessing module splits the input text into independent sentences according to predefined rules, and simultaneously divides the sentences and their corresponding element classification label sequences, thereby completing a one-to-one correspondence between each character in the contract text and the label.

[0017] Furthermore, in the above context-aware model, the deep learning module takes the contract text data generated by the contract text preprocessing module as input, and outputs the element prediction results of the contract text at the word level granularity; specifically, the BERT model is used to generate the word embedding representation of the contract text, and then the bidirectional GRU network is used to extract the context information of the word embedding sequence, and finally the Tri-Attention network layer is used to capture the jumping context information. The smart contract representation generation module takes the mapping between the contract text input and the element prediction result output of the deep learning module as input, aggregates, marks and outputs the contract text marked as elements.

[0018] Furthermore, the above-mentioned unbalanced data processing module provides edge weighted loss function and AUC weighted loss index for deep learning model training, and dynamically calculates the loss value of each type of sample during the training process; the dynamic calculation refers to gradually improving the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various elements, and using negative sample data screening method to improve the element extraction ability of the deep learning model. The negative sample data screening method refers to sorting the data batches with a label of 0 in the contract text after each round of screening training of the unbalanced data processing module, and deleting the data batches without elements with the smallest loss.

[0019] A method for characterizing a context-aware unbalanced smart legal contract characterization system, characterized by comprising the following steps:

[0020] (1) Obtain multiple legal contract texts and assign corresponding element classification labels to each character to construct a dataset;

[0021] (2) Input the constructed data set into the system's contract text preprocessing module, which processes the data set through a word segmenter to obtain a contract text converted into a token sequence, and performs sentence completion processing based on the context dynamic completion method to obtain preprocessed text data, which is randomly divided into a training set, a validation set, and a test set;

[0022] (3) The context-aware model was pre-trained for multiple rounds using the training set. In each round, the data was divided into multiple batches according to the batch size for training. For each training batch in each round, BERT was used to generate word embedding representations of the contract text, and bidirectional GRU and Tri-Attention networks were used to extract context information to obtain prediction results for the data in the training set.

[0023] (4) The imbalanced data processing module screens and trains the context-aware model according to a preset number of interval rounds. That is, at certain intervals, the loss of each batch in the current round is calculated, and low-loss batches of feature-free data are screened and deleted. At the same time, the loss weight is dynamically adjusted according to the loss of minority class samples to improve the model's recognition accuracy for minority class features.

[0024] (5) Perform back propagation of the model network based on the loss, update the model weights, and obtain a trained model;

[0025] (6) Input the legal contract to be represented into the system and use the trained model to complete the contract representation.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] First, the present invention introduces a mechanism based on dynamic adjustment of sample importance. By gradually calculating the weight of each training sample, the samples without elements or low value that contribute little to the model are screened out, which not only improves the training efficiency of the model, but also significantly improves the recognition rate of minority class samples, thereby effectively solving the limitations of traditional methods in dealing with data imbalance. Compared with the existing AUC weighted loss method, the solution of the present invention is more efficient and can deal with the sample imbalance problem more specifically.

[0028] Second, the invention adopts a dynamic weight calculation mechanism in the model training process, which significantly improves the training efficiency by gradually screening out samples without elements or low-value samples; this dynamic adjustment method speeds up the training of the model, while improving the impact of samples on model performance during training, thereby improving the stability and accuracy of the overall model.

[0029] Third, the present invention takes into account the particularity of smart contracts and proposes an edge weighted loss method for it, which effectively deals with the boundary problem between key elements and non-elements in the contract, and can ensure that the model accurately identifies and characterizes key elements when processing the legal provisions of the contract, significantly improving the application effect of the model in actual legal scenarios.

[0030] Fourth, the present invention combines the context-aware mechanism and proposes a new deep learning annotation model, which can deeply extract the characteristic elements in the legal text and generate rich semantic element representations; this model can more accurately capture key content when processing complex legal texts in smart contracts, and effectively improve the quality and efficiency of text representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of the overall implementation of the present invention;

[0032] Figure 2 A schematic diagram of the implementation process of processing unbalanced data in the present invention;

[0033] Figure 3 This is a schematic diagram of the structure of the smart contract representation module in the present invention;

[0034] Figure 4 This is a comparison chart of model performance indicators before and after unbalanced data processing of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0036] Example 1: Reference Figure 1 The context-aware unbalanced smart legal contract representation system proposed in the present invention includes:

[0037] Contract text preprocessing module, context-aware model and unbalanced data processing module; the contract text preprocessing module is used to preprocess the input smart contract legal text, including text cleaning, structured extraction and sentence segmentation operations, to generate contract text data; the context-aware model includes a deep learning module and a smart contract representation model generation module, which is a multi-layer neural network model based on deep learning technology. It uses the context dynamic filling CDPM method to fill the context information of the contract text data generated by the contract text preprocessing module and obtain the element recognition result; the unbalanced data processing module is used to deal with the imbalance of legal text data, improve the recognition accuracy of minority samples through screening training and dynamic weight adjustment methods, and achieve performance optimization of the context-aware model.

[0038] The contract text preprocessing module splits the input text into independent sentences through predefined rules, and simultaneously divides the sentences and their corresponding element classification label sequences to complete the one-to-one correspondence between each character in the contract text and the label.

[0039] In the context-aware model, the deep learning module takes the contract text data generated by the contract text preprocessing module as input and outputs the element prediction results of the contract text at the word level granularity; specifically, the BERT model is used to generate the word embedding representation of the contract text, and then the bidirectional GRU network is used to extract context information from the word embedding sequence, and finally the Tri-Attention network layer is used to capture the jumping context information. The smart contract representation generation module takes the mapping between the contract text input and the element prediction result output of the deep learning module as input, aggregates, marks and outputs the contract text marked as elements.

[0040] The imbalanced data processing module provides edge weighted loss function and AUC weighted loss index for deep learning model training, and dynamically calculates the loss value of each type of sample during the training process; the dynamic calculation refers to gradually improving the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various elements, and using negative sample data screening method to improve the element extraction ability of the deep learning model. The negative sample data screening method refers to sorting the data batches with a label of 0 in the contract text after each round of screening training of the imbalanced data processing module, and deleting the data batches without elements with the smallest loss.

[0041] Example 2: Reference Figure 1 The method for characterizing based on the system described in Example 1 proposed by the present invention comprises the following steps:

[0042] Step 1. Obtain multiple legal contract texts, assign corresponding element classification labels to each character, and construct a dataset;

[0043] Step 2. Input the constructed data set into the contract text preprocessing module of the system. The module processes the data set through a word segmenter to obtain a contract text converted into a token sequence, and performs sentence filling processing according to the context dynamic filling method to obtain preprocessed text data, which is randomly divided into a training set, a verification set, and a test set. In this embodiment, the context dynamic filling method is to dynamically fill in the context sentences of the target sentence so that the target sentence reaches the standard input length.

[0044] Step 3. Use the training set to perform multiple rounds of warm-up training on the context-aware model. In each round, divide the data into multiple batches according to the batch size for training. For each training batch in each round, use BERT to generate word embedding representations of the contract text, use bidirectional GRU and Tri-Attention networks to extract context information, and obtain the prediction results of the data in the training set.

[0045] Step 4. The unbalanced data processing module performs screening training on the context-aware model according to a preset number of interval rounds, that is, at regular intervals, the loss of each batch in the current round is calculated, and low-loss batches of feature-free data are screened and deleted; at the same time, the loss weight is dynamically adjusted according to the loss of minority class samples to improve the model's recognition accuracy for minority class features. This embodiment deletes a portion of the training batches with the smallest loss after each round of screening training, and continues with the next round of training until the preset training round is reached or the loss function converges.

[0046] Step 5. Perform back propagation of the model network according to the loss, update the model weights, and obtain the trained model;

[0047] Step 6. Input the legal contract to be represented into the system and use the trained model to complete the contract representation. This embodiment first uses BERT to generate a contract text word embedding representation for each batch of contract text data through the deep learning module, uses the bidirectional GRU and Tri-Attention network to extract context information, and outputs the element prediction results of the contract text at the word level; the intelligent contract representation model generation module maps the prediction results with the contract text, outputs the element texts contained in the contract text and their element categories one by one, and obtains the contract representation results.

[0048] Embodiment 3: The overall implementation steps of the characterization method proposed in this embodiment are the same as those in Embodiment 2. The training / testing process of the model is now further described in detail:

[0049] Step A. Training / inference dataset generation:

[0050] Input multiple legal contract texts. In the case of training, you also need to input the element classification label sequence corresponding to each character. In order to ensure that the training, verification and testing of the model are balanced, the contract data is randomly divided into training set, verification set and test set in a ratio of 7:1.5:1.5. At the same time, according to the sentence-end punctuation and maximum character length limit, the legal contract text is split into sentences, and the label sequence is also split synchronously by sentence to ensure the correspondence between characters and labels.

[0051] Step B. Model data preprocessing:

[0052] First, the tokenizer of the base model is used to convert the text character sequence into a token sequence, and the label sequence is also converted into a format that corresponds to the token one by one. Then, the context-sensitive Dynamic Padding Method (CDPM) is called to pad the target sentence with the context sentence to reach the standard input length, and organize it into batches for model training.

[0053] Step C. Warm-up training:

[0054] The present invention uses Bidirectional Encoder Representations from Transformers (BERT) to embed the token sequence and generate a tensor sequence. Subsequently, a bidirectional GRU (Gated Recurrent Unit) network is used to extract continuous context information in the tensor sequence. In order to further enhance the model's ability to capture complex contexts, a Tri-Attention network layer is designed to extract jump context information. During the training process, the edge weighted loss function and the AUC (Area Under Curve) indicator are used to calculate the training loss, and the weight of the model is dynamically updated through back propagation.

[0055] Step D. Screening Training:

[0056] In each round of screening training, the steps in warm-up training are repeated. For each batch whose label sequence is all 0, that is, data without significant features, its total loss is recorded. After each round of training, the batch losses are sorted and the batch with the smallest loss is deleted to improve the training efficiency and the model's feature recognition ability.

[0057] Step E. Model training:

[0058] Model training is performed by alternating warm-up training and screening training steps until the set training rounds are reached or the loss function converges. Through this process, the model can be gradually optimized and the ability to extract intelligent elements from legal texts can be enhanced.

[0059] Step F. Model testing / inference:

[0060] In the testing phase, the performance of the model is evaluated using the test set, and indicators such as F1 value and accuracy are calculated. In the reasoning phase, after the contract text is input, the model can output the corresponding element recognition results, thereby realizing the intelligent representation of the legal text.

[0061] Example 4: Reference Figure 1-3 The overall implementation of the system and method proposed in this embodiment is the same as that of Embodiments 1 and 2. Now, a specific data set and related parameter settings are given according to the training process of Embodiment 3, and the present invention is further described in detail through specific examples:

[0062] In this embodiment, 1,000 contracts were collected and manually annotated in a legal and compliant manner from local government procurement websites, and 8 major categories and 22 minor categories of sales contract elements were defined.

[0063] Step 1. Training / inference dataset generation

[0064] Step 1.1 Input multiple legal contracts. In the training case, you also need to input the label sequence of the element classification corresponding to each character in the contract.

[0065] Step 1.2 In the training case, the contracts are divided into training set, test set and validation set contracts in a ratio of 7:1.5:1.5.

[0066] Step 1.3 splits the legal contract into a series of sentences according to the end punctuation and maximum length. Specifically, the contract is broken down into sentences according to the ending punctuation (including "?", "!", etc.) and line breaks. If a sentence is too long, some other punctuation marks, such as "," will also be used as the end punctuation. In this process, only consecutive spaces, tabs, or line breaks will be reduced to one, because these consecutive symbols are meaningless. The label sequence is also split into a one-to-one correspondence with the words in the sentence. After the above steps, the formed data set is shown in the following table:

[0067] Table 1. Number of samples in the dataset

[0068]

[0069]

[0070] Step 2. Model data preprocessing.

[0071] Step 2.1 The tokenizer of the base model converts the text sequence into a token sequence, and the label sequence is also changed to correspond one-to-one with the tokens. Suppose the length of the input sentence is m, and the maximum length of the model input is set to N. Then the i-th sentence s i can be represented as s i =(t i,1 , t i,2 , t i,3 , …, t i,m ).

[0072] Step 2.2 Call the Context-sensitive Dynamic Padding Method (CDPM) to fill each target sentence to the standard input length with context sentences and divide them into batches. Specifically, suppose the length of the input sentence is m, and the maximum length of the model input is set to N. Then the sentence s i can be represented as s i =(t i,1 , t i,2 , t i,3 , …, t i,m ). For each sentence s i , we first evaluate its length m i , and m i±1 of s i±1 . If m i +m i±1 <N, then the tokens of s i-1 will be filled in front of s i , and the tokens of s i+1 will be filled behind s i . The newly formed input containing three sentences can be represented as:

[0073]

[0074] Many times, the total length of the three sentences is still less than N. Then s i±2 , s i±3 will also be evaluated and concatenated to the input in the above way until the newly added sentence makes the input size greater than N.

[0075] Most of the time, the newly composed input cannot completely fit the maximum input length. At this time, the traditional filling mechanism will still be enabled and fill in the remaining characters. However, the filling mechanism based on the contract context can often fill in more content, so that our input contains more information that is helpful for feature extraction. In this way, our input can be divided into four parts: predecessor text, target text, subsequent text, and meaningless filling. When processing, we will treat the meaningless filling as part of the subsequent text, so the input becomes three parts: predecessor, target, and subsequent. For simplicity, let's use represents the output of this module. At this time, s i The token can be represented as Here z+1 represents s i exist The starting position in .

[0076] Step 3. Warm-up training steps

[0077] Reference Figure 2 , further describing the implementation steps of the embodiment of the present invention.

[0078] Step 3.1 Use a model such as Bidirectional Encoder Representations from Transformers (BERT) for word embedding and convert the token sequence into a tensor sequence, for example:

[0079]

[0080] Step 3.2 uses a bidirectional GRU network to further extract continuous context information of the tensor sequence. GRU is a common recurrent neural network that can effectively extract timing information. In the contract text, timing information helps to infer the element type. For example, in the clause type element, the condition element appears before the result element, and the postcondition appears after the result element. Using a bidirectional GRU layer to enhance the extraction of timing information can effectively improve the performance of contract element extraction. After being calculated by the bidirectional GRU, it is expressed as Right now:

[0081]

[0082] Step 3.3 uses the Tri-Attention network layer to extract the jump context information of the tensor sequence. The Tri-Attention layer is composed of three attention layers: the previous attention layer, the self-attention layer, and the subsequent attention layer, which process different parts of the input. If the input sequence is the output sequence of GRU, that is, Then the self-attention layer will That is, the tensor corresponding to the target sentence is used as the value and key input of the attention layer. The predecessor sentence uses The subsequent sentence uses At the same time, all three attention layers will As the query input of the attention layer.

[0083] To explain how a single attention layer works, let’s take the calculation of forward attention as an example.

[0084] We enter and mask Get the unweighted query input Q = All positions are not in (z,z+m i ) will be set to 0, as well as the key input The 0 value will not be used in the following calculations. Next, we flip the first z elements of the key input to get the real key input. The part closest to the target tensor position is moved to the front fixed position of the weight matrix. This is because, compared with the "first word in the sentence", the "word closest to the target" is more likely to help infer the character type.

[0085] Next, the weight matrix makes the three-head attention layer trainable. The first matrix W q Used for query input. The second matrix W v pre The value input for the forward attention. The three attention layers share W q , but their W v Different. In this way, in Luong-style, the attention weight of the previous text can be calculated as follows:

[0086] σ(Q,K)=Q·W q ·K T

[0087] Att(Q,K,V)=softmax(σ(Q,K))·(V·W v pre )

[0088] Still taking the previous attention as an example, we flip part of the result (restore the previously flipped part) and get Att pre As a result, through a process similar to that described above, we can get the attention results of all parts Att pre ,Att self ,Att nextSince the masked part is set to 0 and does not participate in the calculation, we can simply add the three attention results and get Att all =Att pre +Att self +Att next Then the attention-weighted result of the three-head attention layer output can be calculated as:

[0089]

[0090] Step 3.4 During training, the edge weighted cross entropy loss is combined with the AUC function to calculate the training loss based on the tensor sequence output and the label sequence. During back propagation, the weights of the nodes of each network are updated according to the loss.

[0091] Specifically, if there are 23 categories, Indicates t i,j-z The possibility of belonging to 23 categories is expressed as Indicates t i,j-z One-hot encoding of the true classification value. In general, cross entropy loss is used to calculate the loss:

[0092]

[0093] The cross entropy loss function aims to correctly identify as many tokens as possible. However, when using sequence labeling for element extraction, the position of correctly predicted tokens may be more important than their number. Among them, the edges of true value elements are more difficult to mark accurately when labeling, and the edge accuracy of predicted values ​​will also have an important impact on the results. Therefore, we propose a weighted loss based on feature edges, giving higher weights to edge tokens of true value elements and predicted value elements when calculating losses during training. The edge weighted cross entropy loss is calculated as follows:

[0094]

[0095] Since the AUC function is a function for binary classification, we have two possible ways of using it: 1. Treat the tokens that do not belong to any element as negative samples, and the others as positive samples. This is the general way to use AUC; 2. Treat the tokens that belong to a certain category of elements as positive samples, and the rest as negative samples. Calculate the AUC for each category of elements and take the average. Experimental verification shows that the latter is more effective when the imbalance between positive and negative samples is not serious (the number of positive and negative samples in training is basically the same); the former is more effective when the imbalance between positive and negative samples is serious. In view of the fact that there is basically a serious imbalance in real contracts, we adopted the former as the main method. For a batch, assuming that the number of positive samples and negative samples is n respectively. pos and nneg , then the AUC is calculated as follows:

[0096]

[0097] Since lower AUC means worse results and more attention should be paid in training, the weighted Loss of all sentences in the batch can be calculated as follows:

[0098]

[0099] Step 4. Screening training steps

[0100] Step 4.1 is the same as steps 3.1 to 3.4.

[0101] Step 4.2: For each batch whose label sequence is all 0, calculate its total loss and record the batch and its loss as a key-value pair.

[0102] Step 4.3 Repeat steps 4.1 to 4.2 for each batch in each round. At the end of each round, sort the batches in the training set according to the loss, and delete the δ% batches with the smallest loss. They will not participate in the next training.

[0103] Step 5. Model training:

[0104] Follow a certain pattern and perform step 3 or 4 alternately, repeating a certain number of times. Figure 3 At the end of each round, the development set is inferred. After each execution of step 3 or 4, the inference output and the label sequence of the development set are used to calculate the development set loss. The number of rounds is estimated based on the convergence of the development set loss. The best rounds are: every 4 times step 3 is executed, step 4 is executed once, 3 rounds back and forth, a total of 15 times; finally, step 3 is executed 5 times to complete the model training.

[0105] Step 6. Model testing / inference:

[0106] In the test case, F1 and Accuracy are calculated based on the output and the true value. In the reasoning case, the output is the recognition and representation result of the element.

[0107] The effect of the present invention can be further illustrated by the following simulation.

[0108] 1. Simulation conditions

[0109] The simulation experiment of the present invention is carried out under the following hardware and software environment:

[0110] Software platform: Ubuntu 20.04 operating system, Python 3.10 programming language, PyTorch 2.0.0 training framework, parallel computing architecture CUDA 11.7.

[0111] Hardware platform: Graphics card NVIDIA GeForce RTX 3060 GPU 12GB.

[0112] Dataset: The Chinese Purchase Contracts Dataset (CPCD) used in this invention is a specific dataset collected and produced in 2023 for legal text representation and smart contract analysis.

[0113] 2. Analysis of simulation content and simulation results

[0114] This simulation experiment mainly compares the performance of the method of the present invention with the prior art (BERT model) in terms of unbalanced data processing and context awareness.

[0115] exist Figure 4 In the data rebalancing method, the performance comparison between not using the data rebalancing method (i.e., not applying a specific loss function and data screening mechanism) and using the rebalancing method is shown. When the data rebalancing method is not used, the average F1 value of the model is 0.635, the weighted average F1 value is 0.721, and the training time is 356 minutes. After using the data rebalancing method, the average F1 value of the model is increased to 0.639, the weighted average F1 value is increased to 0.735, and the training time is reduced to 272 minutes. This shows that the dynamic sample weight adjustment mechanism for gradually screening out data without elements proposed in the present invention effectively improves the model performance and significantly shortens the training time.

[0116] also, Figure 4 It also demonstrates the further enhancement of the model performance by the context-aware module designed by the present invention. Compared with the traditional BERT model that does not use the rebalancing method, the model using the method of the present invention not only performs better in processing unbalanced data, but also improves the ability to understand the implicit semantics in complex legal clauses through the context-aware mechanism, further enhancing the generalization ability and robustness of the model.

[0117] 3. Conclusion

[0118] Through the above simulation experiments, it can be concluded that the smart contract legal text representation method based on unbalanced data processing and context awareness proposed in the present invention can significantly improve the classification and matching performance of the model in processing unbalanced legal texts. Specifically, by dynamically adjusting the sample weights and gradually screening out data without elements, the present invention effectively solves the negative impact of unbalanced data on model training and improves the recognition accuracy of key legal elements. At the same time, the introduced context-aware module further improves the model's ability to understand complex legal terms and enhances the model's robustness and generalization capabilities. The method of the present invention not only helps to reduce the risks of smart contracts due to the ambiguity of natural language, but also improves the legality and consistency of contract terms, and has broad prospects in the application of smart contracts.

[0119] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0120] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A context-aware unbalanced smart legal contract representation system, characterized in that: include: Contract text preprocessing module, context-aware model and imbalanced data processing module; The contract text preprocessing module is used to preprocess the input smart contract legal text, including text cleaning, structure extraction and sentence segmentation operations, to generate contract text data; The context-aware model includes a deep learning module and a smart contract representation model generation module. It is a multi-layer neural network model based on deep learning technology. It uses the context dynamic filling CDPM method to fill the contract text data generated by the contract text preprocessing module with context information and obtains the element recognition result. The imbalanced data processing module is used to process the imbalance of legal text data, improve the recognition accuracy of minority class samples through screening training and dynamic weight adjustment methods, and achieve performance optimization of the context-aware model.

2. The system according to claim 1, characterized in that: The contract text preprocessing module splits the input text into independent sentences according to predefined rules, and simultaneously divides the sentences and their corresponding element classification label sequences, thereby completing a one-to-one correspondence between each character in the contract text and the label.

3. The system according to claim 1, characterized in that: In the context-aware model, the deep learning module takes the contract text data generated by the contract text preprocessing module as input and outputs the element prediction results of the word-level granularity of the contract text; Specifically, the BERT model is used to generate word embedding representations of the contract text, and then the bidirectional GRU network is used to extract contextual information from the word embedding sequence. Finally, the Tri-Attention network layer is used to capture the jump context information.

4. The system according to claim 3, characterized in that: In the context-aware model, the smart contract representation generation module takes the mapping between the contract text input and the element prediction result output of the deep learning module as input, aggregates, marks and outputs the contract text marked as elements.

5. The system according to claim 1, characterized in that: The unbalanced data processing module provides an edge weighted loss function and an AUC weighted loss indicator for deep learning model training, and dynamically calculates the loss value of each type of samples during the training process; the dynamic calculation refers to gradually improving the recognition weight of minority class samples through a dynamic weight adjustment mechanism to achieve balanced recognition of various factors, and use negative sample data screening method to improve the factor extraction capability of the deep learning model.

6. The system according to claim 5, characterized in that: The negative sample data screening method refers to sorting the data batches with the label of 0 in the contract text after each round of screening training of the imbalanced data processing module, and deleting the data batches without elements with the smallest loss.

7. A method for characterization using the system of claim 1, characterized in that: The following steps are involved: (1) Obtain multiple legal contract texts and assign corresponding element classification labels to each character to construct a dataset; (2) Input the constructed data set into the system's contract text preprocessing module, which processes the data set through a word segmenter to obtain a contract text converted into a token sequence, and performs sentence completion processing based on the context dynamic completion method to obtain preprocessed text data, which is randomly divided into a training set, a validation set, and a test set; (3) The context-aware model was pre-trained for multiple rounds using the training set. In each round, the data was divided into multiple batches according to the batch size for training. For each training batch in each round, BERT was used to generate word embedding representations of the contract text, and bidirectional GRU and Tri-Attention networks were used to extract context information to obtain prediction results for the data in the training set. (4) The imbalanced data processing module screens and trains the context-aware model according to a preset number of interval rounds. That is, at certain intervals, the loss of each batch in the current round is calculated, and low-loss batches of feature-free data are screened and deleted. At the same time, the loss weight is dynamically adjusted according to the loss of minority class samples to improve the model's recognition accuracy for minority class features. (5) Perform back propagation of the model network based on the loss, update the model weights, and obtain a trained model; (6) Input the legal contract to be represented into the system and use the trained model to complete the contract representation.

8. The method according to claim 7, characterized in that: The context dynamic filling method in step (2) refers to dynamically filling the context sentences of the target sentence so that the target sentence reaches the standard input length.

9. The method according to claim 7, characterized in that: In the step (4), after each round of screening training, a portion of the training batches with the smallest loss is deleted, and the next round of training is continued until a preset number of training rounds is reached or the loss function converges.

10. The method according to claim 7, characterized in that: In step (6), the trained model is used to complete the contract representation, which is specifically implemented as follows: (6.1) The deep learning module uses BERT to generate word embedding representations of the contract text for each batch of contract text data, uses the bidirectional GRU and Tri-Attention network to extract context information, and outputs the element prediction results of the contract text at the word level; (6.2) The smart contract representation model generation module maps the prediction results with the contract text, outputs the element texts and their element categories contained in the contract text one by one, and obtains the contract representation results.

Images