Method, device and equipment for converting natural language contract into intelligent legal contract

Abstract

The invention relates to the technical field of intelligent legal contracts, in particular to a method, device and equipment for converting a natural language contract into an intelligent legal contract, and the method comprises the steps: generating a training corpus of a target large model according to natural language contract data and business process data; an input sequence of a fine tuning framework is generated according to the training corpus and the intelligent legal contract code, a target large model under the fine tuning framework is iteratively trained through the input sequence, the fine tuning framework comprises a decoder, an encoder and a comprehensive loss function, low-rank matrix layers are added to multiple parameter weight matrixes of the encoder respectively, and the low-rank matrix layers and the comprehensive loss function are combined; freezing part of model parameters of the target large model, and adjusting the model parameters of the target large model based on the low-rank matrix layer and the comprehensive loss function; and realizing an intelligent legal contract modeling function by using the trained target large model. Therefore, the problems of equivalent modeling limitation, low modeling efficiency, high modeling cost and the like in the field of intelligent legal contract codes in related technologies are solved.

Description

Technical Field

[0001] The present invention relates to the field of smart legal contract technology, and in particular to a method for converting a natural language contract into a smart legal contract. Background Art

[0002] Smart legal contracts are built between smart contracts and natural language contracts. They have both the legal characteristics and understandability of contracts and the standardization of computer program codes. Modeling natural language contracts as smart legal contracts not only helps legal personnel monitor contract performance risks, but also helps computer personnel understand legal logic and behavior, and promotes cross-domain collaboration between technical personnel and legal personnel. However, the professional terms, legal rigor, and scenario complexity of natural language contracts themselves make modeling them as smart legal contracts face challenges such as difficult rule conversion, complex system construction, and high maintenance costs.

[0003] Related technologies Rule-based modeling methods and symbolic modeling and formal methods achieve equivalent modeling in the field of natural language contracts. Rule-based modeling methods define a set of rules and conventions to automatically process the terms and relationships between contract terms, and use decision tables, state machines and other technologies to represent the logic and conditions of contract terms. Symbolic modeling and formal methods convert the terms and logical relationships of contracts into formalized languages ​​or symbolic representations for accurate verification and execution.

[0004] However, rule-based modeling methods have poor flexibility and adaptability. When the business involved in the natural language contract changes, the original rules often need to be modified and redefined in large quantities to adapt; symbolic modeling and formal methods require professional technical knowledge, have a high learning threshold, and have high costs and low efficiency in processing highly complex natural language contracts. Summary of the invention

[0005] The present invention provides a method, device and equipment for converting a natural language contract into an intelligent legal contract, so as to solve the problems of limitations in equivalent modeling in the field of intelligent legal contract code, low modeling efficiency and high modeling cost in the related technology.

[0006] The first aspect of the present invention provides a method for converting a natural language contract into a smart legal contract, comprising the following steps: obtaining natural language contract data and business process data of a business process model expressing the natural language contract; generating training corpus for a target large model based on the natural language contract data and the business process data; generating an input sequence for a fine-tuning framework based on the training corpus and the smart legal contract code, and iteratively training the target large model under the fine-tuning framework using the input sequence, wherein the fine-tuning framework includes a decoder, an encoder, and a comprehensive loss function, and low-rank matrix layers are respectively added to multiple parameter weight matrices of the encoder to freeze some model parameters of the target large model, and the model parameters of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function; and the trained target large model is used to realize the smart legal contract modeling function, wherein the smart legal contract modeling function is to model the natural language contract into a smart legal contract.

[0007] Optionally, training corpus for a target large model is generated based on natural language contract data and business process data, including: format conversion and data cleaning of natural language contract data and business process data; identifying various clauses and elements in natural language contract data, establishing a mapping relationship between various clauses and elements and various elements in business process data, and forming training samples based on natural language contract data, business process data and the mapping relationship; dividing the training samples into a training set, a validation set and a test set, and generating training corpus for a target large model based on the training set, the validation set and the test set.

[0008] Optionally, an input sequence of a fine-tuning framework is generated based on the training corpus and the smart legal contract code, including: converting the natural language contract data and the training corpus into the input format of the fine-tuning framework; performing word segmentation processing on the smart legal contract code and the training corpus corresponding to the natural language contract data; loading the target large model into the fine-tuning framework, marking the word units after the word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word units; encoding the marked word units into a vector matrix through the embedding layer of the fine-tuning framework, and using the vector matrix as the input sequence of the fine-tuning framework.

[0009] Optionally, the encoder's attention mechanism module and fully connected feedforward module, multiple parameter weight matrices include first to third parameter weight matrices, wherein the attention weight of the attention mechanism module is determined based on the interaction of the first parameter weight matrix and the second parameter weight matrix, the third attention weight is the integrated information weight of the attention mechanism module, and the three parameter weight matrices are used to assist in learning the characteristics of natural language contract data and business process data.

[0010] Optionally, the comprehensive loss function includes semantic consistency loss, structural loss, and semantic and structural alignment loss, wherein the formula of the comprehensive loss function is:

[0011] L=λ1L token +λ2L s +λ3L align

[0012] Among them, L represents the loss function value, λ1, λ2 and λ3 represent weight parameters, and L token represents the semantic consistency loss, L s represents the structural loss, L align Represents semantic and structural alignment loss.

[0013] Optionally, the target large model under the fine-tuning framework is iteratively trained using the input sequence, including: setting hyperparameters of the fine-tuning framework to start fine-tuning the target large model; using the input sequence to iteratively train the target large model to calculate the loss function value in the training process according to the comprehensive loss function, and optimizing the model parameters of the target large model according to the loss value; after the iteration stop condition is met, the iterative training of the target large model is completed.

[0014] Optionally, the encoder includes a self-attention module and a fully connected feed-forward module, wherein a low-rank matrix layer is added after the self-attention module.

[0015] Optionally, the fine-tuning framework includes a Seq2Seq framework, and the decoder and encoder are based on the Transformer architecture.

[0016] According to a second aspect of the present invention, an embodiment provides a device for converting a natural language contract into a smart legal contract, comprising: an acquisition module for acquiring natural language contract data and business process data of a business process model expressing a natural language contract; a processing module for generating training corpus of a target large model based on the natural language contract data and the business process data; a fine-tuning module for generating an input sequence of a fine-tuning framework based on the training corpus and the smart legal contract code, and iteratively training the target large model under the fine-tuning framework using the input sequence, wherein the fine-tuning framework includes a decoder, an encoder, and a comprehensive loss function, and low-rank matrix layers are respectively added to multiple parameter weight matrices of the encoder to freeze some model parameters of the target large model, and the model parameters of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function; a deployment module for realizing the smart legal contract modeling function using the trained target large model, wherein the smart legal contract modeling function is to model a natural language contract into a smart legal contract.

[0017] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for converting a natural language contract into an intelligent legal contract as in the above embodiment.

[0018] Thus, the present invention includes the following beneficial effects:

[0019] The embodiment of the present invention constructs a corpus of a specific field (natural language contract-smart legal contract), inserts a LoRA layer near the three weight matrices in the self-attention module of the basic pre-trained model encoder, and designs a loss function based on the semantic loss and structural loss between the natural language contract and the smart natural language contract to achieve fine-tuning of the large model parameters, so that the large model can better understand the legal terms and regulatory logic in the natural language contract, and then optimize the smart legal contract code modeling process, so as to utilize the technical advantages of the large model such as the ability to understand and capture complex structures, dynamic adaptation and generalization across domains, and efficient modeling and processing capabilities, aiming to solve the limitations of equivalent modeling in the field of smart legal contract code, improve modeling efficiency, and reduce modeling costs.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a method for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention;

[0023] Figure 2 The validity period and renewal terms according to the embodiment of the present invention

[0024] Figure 3 An example diagram of distributor liability clauses according to an embodiment of the present invention;

[0025] Figure 4 An example diagram of service confirmation terms according to an embodiment of the present invention;

[0026] Figure 5 A flowchart of a method for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention;

[0027] Figure 6 A block diagram of a device for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention;

[0028] Figure 7 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0030] The technical terms used in the following embodiments are explained as follows:

[0031] Smart legal contract: a computer program that contains the elements of a contract and covers the performance agreements reached by the parties to the contract based on offers and promises. It is a transitional means between real natural language contracts and smart contracts.

[0032] Rule-based modeling: is a modeling method that describes system behaviors, constraints, and logical relationships by defining a series of rules. These rules are usually presented in the form of "if...then..." and are used to automate decision-making, reasoning, or behavior selection. Rule-based modeling methods are widely used in many fields, including expert systems, natural language contract modeling, decision support systems, etc.

[0033] Symbolic modeling and formal methods: It is a modeling method that describes complex systems, concepts or problems in a symbolic way and uses formal languages ​​to accurately represent and analyze them. This method relies on precise syntax, semantics and reasoning rules, and can help solve problems such as system verification, optimization, reasoning and proof.

[0034] Large model fine-tuning: Fine-tuning usually involves adjusting the model's parameters through a small amount of task-specific data based on the general knowledge that the pre-trained model has learned, so that it can better adapt to specific task requirements.

[0035] Modeling natural language contracts as smart legal contracts helps legal personnel monitor contract performance risks, helps computer personnel understand legal logic and behavior, promotes cross-domain collaboration between technical and legal personnel, and makes the contract processing process more efficient, transparent and intelligent.

[0036] Since the big model has better understanding and capture capabilities of complex structures, cross-domain dynamic adaptation and generalization capabilities, efficient modeling and processing capabilities, etc., the present invention proposes a method for converting natural language contracts into smart legal contracts, constructs a corpus in a specific field to achieve fine-tuning of the big model parameters, so that the big model can better understand the legal terms and regulatory logic in natural language contracts, and thus optimize the smart legal contract code modeling process.

[0037] The following describes the method, device and electronic device for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention with reference to the accompanying drawings. Specifically, Figure 1The present invention is a flowchart of a method for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention.

[0038] like Figure 1 As shown, the method for converting a natural language contract to an intelligent legal contract includes the following steps:

[0039] In step S101 , natural language contract data and business process data of a business process model expressing the natural language contract are acquired.

[0040] It is understandable that the embodiments of the present invention can obtain various natural language contracts and BPMN business process data expressing natural language contracts through various channels such as legal databases, open source projects, GitHub, etc., wherein the natural language contract data includes multiple types of natural language contracts.

[0041] In step S102, a training corpus of a target large model is generated based on natural language contract data and business process data.

[0042] In an embodiment of the invention, training corpus for a target large model is generated based on natural language contract data and business process data, including: format conversion and data cleaning of natural language contract data and business process data; identifying various clauses and elements in the natural language contract data, establishing a mapping relationship between various clauses and elements and various elements in the business process data, and forming training samples based on the natural language contract data, business process data and the mapping relationship; dividing the training samples into a training set, a validation set and a test set, and generating training corpus for the target large model based on the training set, the validation set and the test set.

[0043] It can be understood that after collecting a large number of natural language contracts of various types, the embodiment of the present invention determines the various clauses and elements in the contract and maps them to various elements of BPMN, and obtains high-quality input data through data cleaning, format conversion, alignment, and segmentation.

[0044] Specifically, S11 preprocesses and cleans natural language contracts and BPMN business process data to remove redundant data.

[0045] S12 data alignment establishes a one-to-one correspondence between natural language contract terms and their corresponding BPMN models to form standardized training samples.

[0046] In the process of identifying and determining the key terms of the contract, we analyze natural language contracts of different types and covering a wide range of business areas, and accurately extract the typical terms that are common to these contracts, are highly representative, and can reflect the core essence of the contract. For the classification of core terms, you can refer to the classification of key terms summarized by the CUAD dataset for multiple contracts. The corpus of mapping a single key term to a BPMN element and a combination of multiple key terms to BPMN includes: Figure 2 The validity period and renewal terms shown, such as Figure 3 The distributor liability clauses shown and Figure 4 Terms of Service Confirmation as shown.

[0047] Therefore, during the data processing stage, the embodiment of the present invention can, based on the precise analysis of the content of the contract text and combining the theories of law and data science, mark the original text document as elements such as the contract subject, validity period, breach of contract clause, applicable law, etc.

[0048] In step S103, an input sequence of a fine-tuning framework is generated according to the training corpus and the smart legal contract code, and the target large model under the fine-tuning framework is iteratively trained using the input sequence, wherein the fine-tuning framework includes a decoder, an encoder, and a comprehensive loss function, and low-rank matrix layers are respectively added to multiple parameter weight matrices of the encoder to freeze some model parameters of the target large model, and the model parameters of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function.

[0049] Among them, the fine-tuning framework may include a Seq2Seq framework, and the decoder and encoder are based on the Transformer architecture.

[0050] It can be understood that the embodiments of the present invention collect a large amount of multi-type natural language contracts and smart legal contract element annotated corpus, and according to the structural differences between natural language contracts and smart natural language contracts, design corresponding loss functions and insert LoRA layers at appropriate locations, and continuously experiment and optimize through efficient parameter fine-tuning.

[0051] Therefore, based on the principle of low-rank approximation, LoRA can learn only a small number of additional parameters without changing the structure of the original model too much. LoRA can use the language knowledge and semantic representation that have been learned in the pre-trained model, and only fine-tune a small number of parameters related to the specific vocabulary and clause logic of the natural language contract, thereby improving the model's understanding and processing capabilities of the natural language contract content; a loss function is designed for the semantic and structural differences between natural language contracts and smart natural language contracts, so that the large model can better match natural language contracts with smart legal contracts; the solution of fine-tuning the large language model for modeling smart legal contracts avoids the complex grammar and strict rules of rule-based modeling and symbolic modeling language and formal methods, and improves the efficiency of complex legal contract conversion.

[0052] In the embodiment of the present invention, Figure 5 As shown, the encoder includes a self-attention module and a fully connected feed-forward module. When computing resources permit, in addition to the W in the encoder’s self-attention module, q , W k , W v By adding a LoRA layer near the three weight matrices, you can consider adding a LoRA layer near the cross-attention mechanism of the decoder to further learn the complex logic and structural differences between legal contracts and smart legal contracts.

[0053] In an embodiment of the present invention, an input sequence of a fine-tuning framework is generated according to a training corpus and a smart legal contract code, including: converting natural language contract data and the training corpus into an input format of the fine-tuning framework; performing word segmentation processing on the smart legal contract code and the training corpus corresponding to the natural language contract data; loading the target large model into the fine-tuning framework, marking the word units after the word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word units; encoding the marked word units into a vector matrix through the embedding layer of the fine-tuning framework, and using the vector matrix as the input sequence of the fine-tuning framework.

[0054] It can be understood that the target large model can be a large language model, such as LegalBert. In the process of fine-tuning the large language model, for example, it can be fine-tuned on the basis of LegalBert to better understand the legal language and optimize the legal logic. Compared with the related art where the basic language model is fine-tuned and may not correctly understand the legal language, and errors may occur in the process of extracting elements, the large language model fine-tuned in the embodiment of the present invention can accurately extract the key terms in the natural language contract, perform intelligent legal contract modeling, and prevent potential legal risks.

[0055] Specifically, the corpus is converted into a format suitable for input under the Seq2Seq framework to construct a good input sequence, as follows:

[0056] S2.1: For natural language contracts and BPMN corpus, first convert them into a format suitable for input into the Seq2Seq framework.

[0057] S2.2 loads the basic pre-trained model into the fine-tuning framework and tokenizes the Solidity and BPMN corpora after tokenization using a tagger compatible with LegalBERT. This process converts tokens into corresponding tags and assigns a unique index to each tag.

[0058] Table 1 compares the two target big models, RoBERTa and LegalBERT. On the whole, choosing LegalBERT as the target big model can well understand the logical relationship between legal terms and related complex contracts, because the model is pre-trained with legal literature and legal-specific corpus.

[0059] Table 1

[0060]

[0061] S2.3: These tokens are encoded into vector representations through the embedding layer. These vectors will serve as the basic unit of input for subsequent models, containing the semantics and position information of the word unit.

[0062] S2.4: Add a decoder based on the Tansformer structure under the seq2seq framework.

[0063] In an embodiment of the present invention, a low-rank matrix is ​​added at an appropriate position, and a loss function is designed. The comprehensive loss function includes semantic consistency loss, structural loss, and semantic and structural alignment loss. The attention mechanism module and the fully connected feedforward module of the encoder, the multiple parameter weight matrices include the first to third parameter weight matrices, wherein the attention weight of the attention mechanism module is determined based on the interaction of the first parameter weight matrix and the second parameter weight matrix, and the third attention weight is the integrated information weight of the attention mechanism module. The three parameter weight matrices are used to assist in learning the characteristics of natural language contract data and business process data.

[0064] Specifically, add a low-rank matrix at the appropriate location, design a loss function, and prepare for fine-tuning, as follows:

[0065] S31: Taking LegalBERT as an example, W in the self-attention module of LegalBERT q , W k , W v Applying the LoRA layer, compared with only one parameter weight matrix in the related art, W q , W k , W vAdd LoRA layers. When processing natural language contracts and BPMN data, this can perform more fine-grained adjustments to the self-attention mechanism. The three parameter weight matrices of the self-attention mechanism are beneficial for better learning of the characteristics of natural language contract data and business process data. W q , W k interactions determine the attention weights and affect the degree of attention to information at different positions. W v is used to integrate information. Adding LoRA layers separately allows the model to perform independent fine-tuning on these different functional dimensions for the complex semantic relationships in natural language contracts and the process structures of BPMN, in order to better learn the characteristics of the data.

[0066] S32: Based on the differences between natural language contracts and BPMN data, comprehensively consider from three aspects: semantic consistency loss, structural loss, and semantic-structure alignment loss, and design a comprehensive loss function.

[0067] Calculating the semantic consistency loss is mainly to ensure that the generated BPMN representation can correctly reflect the semantics of the input natural language contract. Using the cross-entropy loss function as the basic function to measure the difference between each token generated by the model and the tokens of the target BPMN, then the cross-entropy loss function is where y t is the t-th token in the target BPMN sequence, and P(y t |y<t,x) represents the probability of the token y t appearing at time step t given the input natural language contract.

[0068] When considering the structural loss, it is necessary to ensure that the generated BPMN's json data has the correct hierarchy and structure. This solution measures the difference by comparing whether the two BPMN.JSON files are consistent from the root node to the key elements. Then the structural loss function is: where match(p1,p2) represents the consistency between p1 and p2, and L s =0 means that the paths of all activity elements match and the structural similarity is high. The closer L s is to 1, the greater the path difference and the lower the structural similarity. Among them, p a1 is the set of paths of all activity elements from the middle to J1, and p a2 is the set of paths of all activity elements from the middle to J2.

[0069] When considering the semantic-structure loss, use the embedding representation of the generated BPMN (the vector of the last layer of the model) and the embedding representation of the input contract (the output of the encoder) to calculate the contrast loss: L align =1 - cos(Z contract ,z BPMN ), where zcontract The vector representation of the contract embedding, z BPMN The vector representation of BPMN embedding, cos(z contract ,Z BPMN ) uses the cosine similarity measure z contract ,z BPMN The similarity between .

[0070] Therefore, the combined loss function is:

[0071] L=λ1L token +λ2L s +λ3L align

[0072] Among them, L represents the loss function value, λ1, λ2 and λ3 represent weight parameters, and L token represents the semantic consistency loss, L s represents the structural loss, L align Represents semantic and structural alignment loss.

[0073] S33: Freeze most of the parameters of the base model (LegalBERT), initialize the parameters of the LoRA layer, and combine the initialized LoRA layer (A and B matrices) with the corresponding parameters of LegalBERT. The LoRA layer is a technique used in fine-tuning pre-trained models. It introduces a low-rank matrix to fine-tune the weights of the model so that it can better adapt to specific tasks without changing the main structure of the pre-trained model. A matrix: is a matrix with a shape of (hidden layer dimension, r), where r is the rank of the low-rank decomposition, which is much smaller than the hidden layer dimension. It maps the input features to a low-dimensional space. B matrix: is a matrix with a shape of (r, hidden layer dimension), which maps the features in the low-dimensional space back to the original high-dimensional space.

[0074] It should be noted that the word embedding layer is mainly frozen here. When fine-tuning the large model of smart legal contracts, these common semantic and lexical information still play an important basic role for specific tasks in the legal field, and are not prone to drastic changes during the fine-tuning process, so consider freezing the parameters of the word embedding layer. At the same time, the underlying Transformer layer mainly learns relatively common language knowledge such as the basic grammatical structure of the language and the basic semantic combination of sentences. This knowledge has a certain universality in different legal tasks, and the selected model has been pre-trained on a large-scale corpus, and the underlying parameters have been relatively stable and optimized. Therefore, the parameters of several underlying Transformer layers can be frozen to preserve the common language knowledge and reduce the amount of training calculations and time. At this time, the only unfrozen parameters are the LoRA layer, which is set according to step S31.

[0075] In an embodiment of the present invention, the target large model under the fine-tuning framework is iteratively trained using an input sequence, including: setting hyperparameters of the fine-tuning framework to start fine-tuning the target large model; using the input sequence to iteratively train the target large model to calculate the loss function value in the training process according to the comprehensive loss function, and optimizing the model parameters of the target large model according to the loss value; after the iteration stop condition is met, the iterative training of the target large model is completed.

[0076] Among them, the iteration stopping condition may include that the performance of the target large model meets the target requirements, or the number of iterative training times reaches the target number, etc.

[0077] It is understandable that the embodiment of the present invention can set the corresponding hyperparameters (such as learning rate, training length and training rounds, etc.), start model fine-tuning, through multiple iterative training, and adjust the model according to the results of the validation set, and save the optimal model parameters.

[0078] Specifically, if Figure 5 As shown, after setting the corresponding hyper parameters (learning rate, training length, training rounds), the embodiment of the present invention starts model fine-tuning, which is as follows:

[0079] S41: Input the pre-processed natural language contract-BPMN data into the target large model.

[0080] S42: Calculate the loss according to the determined comprehensive loss function, and record the change of the loss value for subsequent analysis of the training progress of the model.

[0081] S43: According to the obtained loss function value, the gradient is calculated using the back propagation algorithm, and the parameters are updated using the optimization algorithm.

[0082] S44: Perform multiple iterations of training, adjust the model based on the results of the validation set, and save the optimal model parameters. Specifically, repeat the above forward propagation, calculate the loss, perform backpropagation and parameter update, and perform multiple rounds of training. Use relevant evaluation indicators (F1 value) to regularly evaluate the performance of the model on the validation set. If the model is found to be overfitting, take some regularization measures. If the model performance is found to be improving slowly, try to adjust the loss function weights, learning rate or other hyperparameters. Among them, the F1 value is an indicator used to evaluate the balance between precision and recall in the model. It is the harmonic mean of precision and recall, providing a single score that takes both into account. The F1 value ranges from 0.0 to 1.0. The closer the value is to 1.0, the better the model.

[0083] In step S104, the trained target big model is used to implement the smart legal contract modeling function, wherein the smart legal contract modeling function is to model the natural language contract into a smart legal contract.

[0084] It is understandable that after the training is completed, the embodiment of the present invention saves and encapsulates the fine-tuning model parameters and embeds the smart legal contract modeling function into the smart contract design platform. Figure 5 As shown: The embodiment of the present invention can use API (Application Programming Interface, Programming Interface Framework) (such as FastAPI) to build smart contract modeling services. The smart legal contract modeling function is embedded in the smart contract design platform to improve usability.

[0085] In summary, the embodiment of the present invention collects multiple types of natural language contracts, determines various clauses and elements in the contract, and aligns the legal contract elements with the smart legal contract elements, so that high-quality input data can be obtained, and the semantic association difficulties and information loss caused by different semantic fields and structures between natural language contracts and smart legal contracts can be alleviated; the pre-trained large language model in the legal field is fine-tuned with the obtained corpus, so that the fine-tuned model can be aligned with the smart legal contract in the form of BPMN on the basis of analyzing the natural language contract elements, so that the fine-tuned model can better understand the concepts and logical relationships of legal terms; the fine-tuned large language model can be integrated with other platforms, for example, the embedding of the smart contract generation platform can improve the generation efficiency of natural language contracts to smart legal contract codes. Therefore, the embodiment of the present invention can construct a specific field corpus of natural language contracts and smart legal contracts based on the large model fine-tuning technology, so that the large model can better understand the legal terms and regulatory logic in the natural language contract, and then optimize the smart legal contract code modeling process, and complete the smart legal contract modeling of the natural language contract at the same time, which is not only conducive to the user's analysis of contract terms and legal risk review, but also conducive to the technical personnel to transform legal contracts into smart contracts and promote the modernization of the legal society.

[0086] The following will further explain the smart contract modeling method based on large model fine-tuning through a specific implementation. Figure 5 As shown in the figure, it mainly includes three stages, S1 data processing stage, S2 model fine-tuning stage and S3 model evaluation and deployment stage, which are as follows: (1) Collect natural language contracts of various types and languages, determine the various terms and elements of the contract, perform structured annotation, and align the data with the smart legal contract in the form of BPMN business process modeling to train the model; (2) Input the pre-processed data set into the basic pre-training of LegalBert, and the big model designs the loss function according to the difference between natural language contracts and smart legal contracts. Then, insert the LoRA layer at the appropriate position of the basic big model for fine-tuning; (3) Evaluate and encapsulate the fine-tuned big model, and embed the function into the smart contract generation platform.

[0087] In the S1 data processing phase, natural language contracts of various types and languages ​​are collected, and the data is aligned with the BPMN business process data to improve the stability of the large language model and the accuracy of extracting complex natural language contract logic. In the S2 model fine-tuning phase, the annotated corpus is pre-processed and input into the LegalBert basic pre-trained large model for parameter fine-tuning. In the S3 model evaluation and deployment phase, the trained model is experimentally tuned and packaged, deployed on the local server and embedded into the smart contract generation platform, while feedback is collected to update the corpus and model.

[0088] Therefore, the embodiment of the present invention proposes a smart legal contract modeling method based on large model fine-tuning, constructs a corpus of natural language contracts-smart legal contracts in specific fields, inserts a LoRA layer near the three weight matrices in the self-attention module of the basic pre-trained model encoder, and designs a loss function based on the semantic loss and structural loss between natural language contracts and smart legal contracts to achieve fine-tuning of large model parameters, so that the large model can better understand the legal terms and regulatory logic in natural language contracts, and thus optimize the smart legal contract code modeling process. The present invention utilizes the technical advantages of large models such as complex structure understanding and capture capabilities, cross-domain dynamic adaptation and generalization capabilities, and efficient modeling and processing capabilities, aiming to solve the limitations of equivalent modeling in the existing smart legal contract code field, improve modeling efficiency, and reduce modeling costs.

[0089] Next, the device for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0090] Figure 6 It is a block diagram of a device for converting a natural language contract into an intelligent legal contract according to an embodiment of the present invention.

[0091] like Figure 6 As shown, the natural language contract to smart legal contract conversion device 10 includes: an acquisition module 100, a processing module 200, a fine-tuning module 300 and a deployment module 400.

[0092] Among them, the acquisition module 100 is used to acquire natural language contract data and business process data of the business process model expressing the natural language contract; the processing module 200 is used to generate training corpus of the target large model based on the natural language contract data and the business process data; the fine-tuning module 300 is used to generate an input sequence of the fine-tuning framework based on the training corpus and the smart legal contract code, and use the input sequence to iteratively train the target large model under the fine-tuning framework, wherein the fine-tuning framework includes a decoder, an encoder and a comprehensive loss function, and low-rank matrix layers are respectively added to multiple parameter weight matrices of the encoder to freeze some model parameters of the target large model, and the model parameters of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function; the deployment module 400 is used to use the trained target large model to realize the smart legal contract modeling function, wherein the smart legal contract modeling function is to model the natural language contract as a smart legal contract.

[0093] It should be noted that the aforementioned explanation of the embodiment of converting a natural language contract into an intelligent legal contract also applies to the device for converting a natural language contract into an intelligent legal contract in this embodiment and will not be repeated here.

[0094] According to the device for converting a natural language contract into an intelligent legal contract proposed in an embodiment of the present invention, a corpus of natural language contracts and intelligent legal contracts in a specific field is constructed, a LoRA layer is inserted near the three weight matrices in the self-attention module of the basic pre-trained model encoder, and a loss function is designed based on the semantic loss and structural loss between the natural language contract and the intelligent legal contract to achieve fine-tuning of the parameters of the large model, so that the large model can better understand the legal terms and regulatory logic in the natural language contract, and then optimize the intelligent legal contract code modeling process, thereby utilizing the technical advantages of the large model such as the ability to understand and capture complex structures, dynamic adaptation and generalization across domains, and efficient modeling and processing capabilities, aiming to solve the limitations of equivalent modeling in the existing intelligent legal contract code field, improve modeling efficiency, and reduce modeling costs.

[0095] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0096] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0097] When the processor 702 executes the program, the method for converting a natural language contract into an intelligent legal contract provided in the above embodiment is implemented.

[0098] Furthermore, the electronic device further comprises:

[0099] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0100] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0101] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0102] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0104] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0106] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0107] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0108] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0109] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0110] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for converting a natural language contract into an intelligent legal contract, characterized in that: The following steps are involved: Acquire natural language contract data and business process data of a business process model expressing the natural language contract; Generate training corpus of a target large model according to the natural language contract data and the business process data; Generate an input sequence of a fine-tuning framework according to the training corpus and the smart legal contract code, and iteratively train the target large model under the fine-tuning framework using the input sequence, wherein the fine-tuning framework includes a decoder, an encoder, and a comprehensive loss function, and add low-rank matrix layers to multiple parameter weight matrices of the encoder respectively, freeze part of the model parameters of the target large model, and adjust the model parameters of the target large model based on the low-rank matrix layer and the comprehensive loss function; The trained target big model is used to realize the smart legal contract modeling function, wherein the smart legal contract modeling function is to model the natural language contract into a smart legal contract.

2. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The generating of the training corpus of the target large model according to the natural language contract data and the business process data comprises: Performing format conversion and data cleaning on the natural language contract data and the business process data; Identify various clauses and elements in the natural language contract data, establish mapping relationships between the various clauses and elements and various elements in the business process data, and form training samples according to the natural language contract data, the business process data, and the mapping relationships; The training samples are divided into a training set, a validation set and a test set, and the training corpus of the target large model is generated according to the training set, the validation set and the test set.

3. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The step of generating an input sequence of a fine-tuning framework according to the training corpus and the smart legal contract code comprises: Converting the natural language contract data and training corpus into an input format for a fine-tuning framework; Performing word segmentation processing on the smart legal contract code corresponding to the natural language contract data and the training corpus; Loading the target large model into the fine-tuning framework, marking the word-units after word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word-units; The marked word-units are encoded into vector matrices through the embedding layer of the fine-tuning framework, and the vector matrix is ​​used as an input sequence of the fine-tuning framework.

4. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The encoder's attention mechanism module and fully connected feedforward module, the multiple parameter weight matrices include first to third parameter weight matrices, wherein the attention weight of the attention mechanism module is determined based on the interaction of the first parameter weight matrix and the second parameter weight matrix, the third attention weight is the integrated information weight of the attention mechanism module, and the three parameter weight matrices are used to assist in learning the natural language contract data and business process data features.

5. The method for converting a natural language contract to an intelligent legal contract according to claim 1, characterized in that: The comprehensive loss function includes semantic consistency loss, structural loss and semantic and structural alignment loss, wherein the formula of the comprehensive loss function is: L=λ1L token +λ2L s +λ3L align Among them, L represents the loss function value, λ1, λ2 and λ3 represent weight parameters, and L token represents the semantic consistency loss, L s represents the structural loss, L align Represents semantic and structural alignment loss.

6. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The iterative training of the target large model under the fine-tuning framework using the input sequence includes: Setting the hyperparameters of the fine-tuning framework and starting fine-tuning of the target large model; Iteratively train the target large model using the input sequence, calculate the loss function value in the training process according to the comprehensive loss function, and optimize the model parameters of the target large model according to the loss value; After the iteration stop condition is met, the iterative training of the target large model is completed.

7. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The encoder comprises a self-attention module and a fully connected feed-forward module, wherein the low-rank matrix layer is added after the self-attention module.

8. The method for converting a natural language contract into an intelligent legal contract according to claim 1, characterized in that: The fine-tuning framework includes a Seq2Seq framework, and the decoder and the encoder are based on a Transformer architecture.

9. A device for converting a natural language contract into an intelligent legal contract, characterized in that: include: An acquisition module, used to acquire natural language contract data and business process data of a business process model expressing the natural language contract; A processing module, used to generate training corpus of a target large model according to the natural language contract data and the business process data; A fine-tuning module, used to generate an input sequence of a fine-tuning framework according to the training corpus and the smart legal contract code, and iteratively train the target large model under the fine-tuning framework using the input sequence, wherein the fine-tuning framework includes a decoder, an encoder, and a comprehensive loss function, and low-rank matrix layers are respectively added to multiple parameter weight matrices of the encoder to freeze some model parameters of the target large model, and the model parameters of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function; A deployment module is used to use the trained target large model to implement the smart legal contract modeling function, wherein the smart legal contract modeling function is to model a natural language contract into a smart legal contract.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for converting a natural language contract into an intelligent legal contract as described in any one of claims 1 to 8.

Images