Intelligent Contract Modeling Method, Device and Electronic Device Based on Fine-Tuning of Large Model

Abstract

The present invention relates to the technical field of smart contracts, and particularly relates to a smart contract modeling method, device, and electronic device based on large model fine-tuning. The method includes: converting smart contract code and business process data into an input sequence of a fine-tuning framework, where the fine-tuning framework includes a decoder and an encoder, adding low-rank matrix layers respectively after the attention mechanism module and the fully connected feed-forward module of the encoder, and after the cross-attention module of the decoder, and designing a comprehensive loss function of the target large model after the fully connected feed-forward module of the decoder; iteratively training the target large model under the fine-tuning framework using the input sequence, encapsulating the model parameters after training the target large model, deploying the target large model on a smart contract design platform, and implementing the smart contract modeling function based on the target large model. Thereby, the problems of limitations in equivalent modeling in the field of smart contract code, low modeling efficiency, and high modeling costs in related technologies are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly to an intelligent contract modeling method, device and electronic device based on large model fine-tuning. Background Art

[0002] Smart contract is one of the core innovations of blockchain technology. Its emergence has changed the traditional transaction method, bringing efficiency improvement and cost reduction to many fields. Modeling smart contract code as domain descriptions helps non-technical personnel understand code behavior and logical structure, facilitates legal experts to verify the legal compatibility of smart contracts, and improves cross-domain collaborative test verification participation and transparency. However, the logical complexity, element integrity, and domain diversity of smart contract code pose challenges such as poor accuracy, low efficiency, and high costs in the equivalent modeling of smart contract code.

[0003] Related technologies mainly use software engineering methods to achieve domain-equivalent modeling of smart contract code. The first type of research uses structured graphical modeling methods, using BPMN (Business Process Model and Notation, graphical representation method) or UML (Unified Modeling Language, unified modeling language) standardized modeling tools and symbols to describe smart contract code behavior and logic. The second type of research focuses on designing DSL (Domain-Specific Language), which is customized according to the specific needs and domain characteristics of smart contracts, aiming to achieve personalized descriptions that are easy to understand and maintain. The third type of research is formal modeling methods, which abstract smart contract code into conditional rules and use first-order logic or linear temporal logic to describe smart contract rule logic.

[0004] However, although the structured graphical modeling method has a high degree of standardization and is easy to perform cross-domain collaborative modeling, its modeling ability is limited by icon elements, and the granularity is relatively coarse, making it difficult to fully capture the logical details of complex smart contract code and resulting in poor expression accuracy. Although DSL and formal modeling methods can achieve fine-grained modeling and effectively describe smart contract behavior and logic, the modeling process requires professional technical knowledge, has a high learning threshold, and has high costs and low efficiency in dealing with complex smart contract code. Summary of the Invention

[0005] The present invention provides an intelligent contract modeling method, device and electronic device based on large model fine-tuning to solve the problems of limitations, low modeling efficiency and high modeling costs in the domain-equivalent modeling of smart contract code in related technologies.

[0006] An embodiment of the first aspect of the present invention provides an intelligent contract modeling method based on large model fine-tuning, including the following steps: obtaining business process data of intelligent contract code and business process models; converting the intelligent contract code and business process data into input sequences of a fine-tuning framework, where the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feed-forward module, and the decoder includes a cross-attention module and a fully connected feed-forward module; adding low-rank matrix layers after the attention mechanism module and the fully connected feed-forward module of the encoder, and after the cross-attention module of the decoder respectively, and designing a comprehensive loss function of the target large model after the fully connected feed-forward module of the decoder; iteratively training the target large model under the fine-tuning framework using the input sequences, and during the training process, adjusting the model parameters during the training process of the target large model based on the low-rank matrix layer and the comprehensive loss function until the iterative training ends; encapsulating the model parameters after training the target large model, deploying the target large model on an intelligent contract design platform, and implementing the intelligent contract modeling function based on the target large model.

[0007] Optionally, the comprehensive loss function includes a semantic similarity loss, a structure matching loss, and a generation loss. The comprehensive loss function constructed based on the semantic similarity loss, the structure matching loss, and the generation loss guides the target large model to learn the corresponding relationship between the intelligent contract and the business process of the business process model.

[0008] Optionally, the formula of the comprehensive loss function is:

[0009] L = αL ss + βL sm + γL g

[0010] where L represents the loss function value, α, β, and γ represent weights, L ss represents the semantic similarity loss, L sm represents the structure matching loss, and L g represents the generation loss.

[0011] Optionally, before converting the intelligent contract code and business process data into input sequences of the fine-tuning framework, it further includes: preprocessing the intelligent contract and business process data, where the preprocessing includes one or more of data cleaning, data format conversion, data alignment, and data segmentation processing.

[0012] Optionally, in the data alignment stage, at least one of functions, logics, and processes in the intelligent contract is mapped to at least one of tasks, gateways, and events in the business process data according to the language mapping principle.

[0013] Optionally, convert the smart contract code and business process data into the input sequence of the fine-tuning framework, including: identifying the input format of the fine-tuning framework; performing word segmentation on the smart contract code and business process data according to the input format; loading the target large model into the fine-tuning framework, marking the word tokens after word segmentation according to the tokenizer of the target large model, and assigning unique indexes to the marked word tokens; converting the marked word tokens into an input sequence through the embedding layer of the fine-tuning framework.

[0014] Optionally, iteratively train the target large model under the fine-tuning framework using the input sequence, including: setting the hyperparameters of the fine-tuning framework and starting the fine-tuning of the target large model; inputting the input sequence into the target large model and iteratively training the target large model using the input sequence. During the training process, the low-rank matrix layer participates in the parameter calculation at the corresponding positions; optimizing the model parameters of the target large model according to the loss function value calculated by the comprehensive loss function, and completing the iterative training of the target large model after meeting the iterative stop condition.

[0015] Optionally, the fine-tuning framework includes a Seq2Seq framework, and the decoder and encoder are based on the Transformer architecture.

[0016] An embodiment of the second aspect of the present invention provides a smart contract modeling device based on large model fine-tuning, including: an acquisition module for acquiring the smart contract code and the business process data of the business process model; a processing module for converting the smart contract code and the business process data into the input sequence of the fine-tuning framework, where the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feed-forward module, and the decoder includes a cross-attention module and a fully connected feed-forward module; a fine-tuning module for adding low-rank matrix layers after the attention mechanism module and the fully connected feed-forward module of the encoder, and designing a comprehensive loss function for the target large model after the fully connected feed-forward module of the decoder, and iteratively training the target large model under the fine-tuning framework using the input sequence. During the training process, adjusting the model parameters during the training process of the target large model based on the low-rank matrix layer and the comprehensive loss function until the iterative training ends; a deployment module for encapsulating the model parameters after the training of the target large model, deploying the target large model to the smart contract design platform, and implementing the smart contract modeling function based on the target large model.

[0017] An embodiment of the third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the smart contract modeling method based on large model fine-tuning as described in the above embodiments.

[0018] Thus, the present invention has the following beneficial effects:

[0019] Embodiments of the present invention can fine-tune a basic pre-trained large language model in a sequence-to-sequence framework, add a low-rank matrix layer at an appropriate position, and construct a comprehensive loss function based on semantic similarity loss, structural loss, and generation loss, enabling the large model to better understand the behavioral semantics and regulatory logic in smart contract code. Furthermore, it optimizes the smart contract code modeling process, reduces the smart contract conversion cost, and thus can fine-tune the large language model by constructing a smart contract-specific domain corpus, effectively overcoming the limitations of equivalent modeling in the smart contract code field in related technologies, improving the modeling efficiency, and reducing the modeling cost.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 is a flowchart of a smart contract modeling method based on large model fine-tuning according to an embodiment of the present invention;

[0023] Figure 2 is a structural diagram of sequential execution according to an embodiment of the present invention;

[0024] Figure 3 is a structural diagram of a parallel gateway according to an embodiment of the present invention;

[0025] Figure 4 is a structural diagram of an exclusive gateway according to an embodiment of the present invention;

[0026] Figure 5 is a flowchart of a smart contract modeling method based on large model fine-tuning according to an embodiment of the present invention;

[0027] Figure 6 is an example diagram of a smart contract modeling device based on large model fine-tuning according to an embodiment of the present invention;

[0028] Figure 7 is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present invention, but should not be construed as limiting the present invention.

[0030] The technical terms used in the following embodiments are explained as follows:

[0031] (1) Smart contract: It is a computer code stored on a blockchain and automatically executed when predefined terms and conditions are met.

[0032] (2) BPMN business process modeling: It is a standardized method for graphically describing various aspects of business processes. It provides a clear and standardized way to represent and analyze business processes, events, decisions, tasks, etc., helping different stakeholders (such as business personnel, technical personnel, management, etc.) understand and communicate business processes.

[0033] (3) DSL: It is a computer language specifically designed for a specific application domain. Different from general-purpose programming languages (such as Python, Java, C++), the goal of DSL is to solve specific problems in a certain field, providing a simpler, more intuitive and efficient way to describe and implement the logic or operations within the domain.

[0034] (4) Large model fine-tuning: Fine-tuning usually adjusts the parameters of the model through a small amount of specific task data on the basis that the pre-trained large language model has learned general knowledge, so that it can better adapt to specific task requirements.

[0035] As a popular business process modeling language, BPMN provides an intuitive graphical modeling method, reducing the technical threshold of programming and enabling legal experts and business personnel to more easily design and understand the business processes of contracts. At the same time, as a general visualization language, it solves the problem of multi-domain collaboration in development. The large model has the capabilities of complex structure understanding and capture, cross-domain dynamic adaptation and generalization, and efficient modeling and processing.

[0036] Therefore, the embodiments of the present invention propose a smart contract modeling method based on large model fine-tuning, construct a corpus for a specific domain (smart contract code - model), insert LoRA low-rank matrices at appropriate positions near the encoder layer and the decoder layer, and design a loss function based on semantic similarity loss, structural loss and generation loss to fine-tune the parameters of the large model, enabling the large model to better understand the behavioral semantics and regulatory logic in the smart contract code, and then optimizing the smart contract code modeling process. The present invention utilizes the technical advantages of the large model such as complex structure understanding and capture capabilities, cross-domain dynamic adaptation and generalization capabilities, and efficient modeling and processing capabilities, aiming to solve the limitations of equivalent modeling in the field of smart contract code in related technologies, improve the modeling efficiency and reduce the modeling cost.

[0037] The following describes the smart contract modeling method, device and electronic device based on large model fine-tuning according to the embodiments of the present invention with reference to the accompanying drawings. Specifically, Figure 1A schematic flowchart of an intelligent contract modeling method based on large model fine-tuning provided by an embodiment of the present invention.

[0038] As Figure 1 shown, the intelligent contract modeling method based on large model fine-tuning includes the following steps:

[0039] In step S101, obtain the business process data of the intelligent contract code and the business process model.

[0040] It can be understood that in the embodiments of the present invention, contract codes and BPMN business process files can be obtained through various channels such as open source projects, GitHub, and development platforms to collect a large amount of diverse Solidity intelligent contract code and BPMN business process model corpora. Among them, Solidity is a programming language for intelligent contracts, mainly used to write and deploy intelligent contracts on the Ethereum blockchain. It is developed by the Ethereum community and aims to provide a safe, easy-to-understand, and easy-to-use programming language for creating decentralized applications.

[0041] Before performing step S102, that is, before converting the intelligent contract code and business process data into the input sequence of the fine-tuning framework, it further includes: preprocessing the intelligent contract code and business process data, where the preprocessing includes one or more of data cleaning, data format conversion, data alignment, and data splitting processing. In the data alignment stage, at least one of the functions, logics, and processes in the intelligent contract is mapped to at least one of the tasks, gateways, and events in the business process data according to the language mapping principle.

[0042] It can be understood that in the embodiments of the present invention, after obtaining the Solidity intelligent contract code and BPMN business process model corpora, high-quality input data is obtained through data cleaning, format conversion, alignment, and data splitting processing methods.

[0043] Thus, in the embodiments of the present invention, by collecting a large amount of diverse Solidity intelligent contract code and BPMN model corpora and performing data cleaning, format conversion, alignment, and data splitting processing methods, high-quality input data can be obtained, which helps to initially alleviate the problems of semantic association difficulties, information loss, and misunderstanding caused by different semantic fields and structures between the Solidity intelligent contract code and BPMN business process data.

[0044] Specifically, S11: Preprocess and clean the Solidity code and BPMN business process data to remove redundant data. Specifically, data preprocessing and cleaning refer to removing redundant comments and whitespace characters in the contract code, considering normalizing variable names and function names, and ensuring the correct file format. The cleaned BPMN business process model should only retain the process elements related to the conversion, such as tasks, gateways, and events, etc.

[0045] S12: Convert the Solidity smart contract code and BPMN business process data into a format for easy data alignment. Specifically, convert the Solidity smart contract code into an Abstract Syntax Tree (AST) and extract its structure; convert the BPMN model from a graphical or file format (such as SVG, PNG) into a standard structured format, where each BPMN element (task, event, gateway, etc.) should have clear labels and relationships. This is to initially alleviate the problems of semantic association difficulties, information loss, and misunderstanding caused by the different semantic domains and structures between the Solidity smart contract code and the BPMN business process file.

[0046] S13: Align the formatted samples so that each code block of the Solidity contract corresponds one-to-one with the corresponding element in the BPMN diagram. For example, in the Solidity code, each operation (such as payment, transfer, logical judgment, etc.) should have a corresponding BPMN task or event, and the conditional judgment should correspond to the gateway in BPMN. Figure 2 shows the sequential execution structure, Figure 3 shows the parallel gateway structure, Figure 4 shows the exclusive gateway structure, the alignment corpus of three types of "Solidity code - BPMN model", and the corresponding BPMN business process model diagram transformed on Camunda.

[0047] S14: Split the data according to a certain ratio. For example, split the data into 80% training set, 10% validation set, and 10% test set. Those skilled in the art can choose the splitting ratio according to the actual situation, and no specific limitation is made.

[0048] In step S102, convert the smart contract code and business process data into the input sequence of the fine-tuning framework, where the fine-tuning framework includes a decoder and an encoder. The encoder includes an attention mechanism module and a fully connected feed-forward module, and the decoder includes a cross-attention module and a fully connected feed-forward module.

[0049] Among them, the fine-tuning framework can include a Seq2Seq framework, and the decoder and encoder can be based on the Transformer architecture.

[0050] It can be understood that the embodiments of the present invention can convert the corpus of smart contracts and business process data into a format suitable for input under the Seq2Seq framework to construct a good input sequence.

[0051] In the embodiments of the present invention, converting the smart contract code and business process data into an input sequence of the fine-tuning framework includes: identifying the input format of the fine-tuning framework; performing word segmentation on the smart contract code and business process data according to the input format; loading the target large model under the fine-tuning framework, marking the word tokens after word segmentation according to the tokenizer of the target large model, and assigning a unique index to the marked word tokens; converting the marked word tokens into an input sequence through the embedding layer of the fine-tuning framework.

[0052] It should be noted that the target large model can be a pre-trained large language model that performs well in code-related tasks, and compare their performance in the current smart contract modeling task, and select a model according to the comparison results. For example, the pre-trained large language model can be CodeBERT, etc.

[0053] Specifically, S21: For Solidity and BPMN corpora, they first need to be converted into a format suitable for input under the Seq2Seq framework.

[0054] For example, Solidity code may need to be segmented, splitting keywords, function names, variable names, etc. in the code into independent word tokens. For example, for the function function transferFunds(address from,addressto,uint amount), after segmentation, it may obtain ["function","transferFunds","(","address","from",",","address","to",",","uint","amount",")"].

[0055] S23: Load the pre-trained large language model under the fine-tuning framework, and use a tokenizer compatible with CodeBERT to tokenize the segmented Solidity and BPMN corpora. This process will convert the word tokens into corresponding tokens and assign a unique index to each token. For example, TransferFunds may be converted into a specific token index, such as

[102] .

[0056] Table 1 summarizes the comparison of the applicable scenarios, advantages and disadvantages of three types of basic models: CodeBERT / GraphCodeBERT, GPT series / Llama series, and T5 / CodeT5. After comprehensive comparison, the embodiment of the present invention selects CodeBERT as the pre-trained large language model. CodeBERT is a large language model pre-trained on a large-scale code corpus and has good congenital understanding ability of the syntax structure and semantics of Solidity code. In addition, although BPMN semantics focuses on business processes, CodeBERT can serve as a bridge to establish the semantic association between the two. Fine-tuning CodeBERT in the Seq2Seq framework can further alleviate the semantic understanding differences between smart contract code and BPMN business process files.

[0057] Table 1

[0058]

[0059] S23: Encode these tokens into vector representations through the embedding layer. These vectors will serve as the basic units of the subsequent model input and contain the semantic and position information of the tokens.

[0060] S24: Add a decoder based on the Transformer structure.

[0061] In step S103, a low-rank matrix layer is added after the attention mechanism module and the fully connected feed-forward module of the encoder, and after the cross-attention module of the decoder. A comprehensive loss function for the target large model is designed after the fully connected feed-forward module of the decoder.

[0062] It can be understood that in the embodiment of the present invention, LoRA low-rank matrices are added near the self-attention module, the fully connected layer of the encoder of the basic pre-trained model, and the attention layer of the decoder, and the loss function is designed by comprehensively considering three aspects: semantic similarity loss, structure matching loss, and generation loss. Therefore, the LoRA layer is added at the appropriate position and the appropriate loss function is designed for fine-tuning, so that the large model can better understand the behavioral semantics and regulatory logic in the smart contract code, and then optimize the smart contract code modeling process.

[0063] In the embodiment of the present invention, the comprehensive loss function includes semantic similarity loss, structure matching loss, and generation loss. The comprehensive loss function constructed based on semantic similarity loss, structure matching loss, and generation loss guides the target large model to learn the corresponding relationship between the smart contract and the business process of the business process model.

[0064] It can be understood that the embodiments of the present invention can fine-tune the large model under the fine-tuning framework of the Seq2Seq Trainer, and calculate based on the comprehensive loss function constructed by the semantic similarity loss, the structure matching loss, and the generation loss, further guiding the model to learn the fine correspondence between the smart contract code and the BPMN process. This enables, when generating the BPMN process description, the detailed behavioral semantics in the smart contract to be mapped to elements such as tasks and gateways in the BPMN process with a finer granularity, thereby improving the modeling granularity.

[0065] Specifically, as Figure 5 shown, add the LoRA layer at the corresponding position and design the loss function as follows:

[0066] S31: Add the LoRA layer after the attention mechanism and before the fully connected layer respectively.

[0067] The encoder is a key part for the target large model to understand the input sequence (Solidity and BPMN corpus). After the multi-head attention mechanism, the semantic and structural information in the corpus has been initially attended to and extracted. Adding the LoRA layer at this time can fine-tune the parameters in this crucial feature extraction stage. The feature represented by the output of the multi-head attention mechanism shows the correlation between each element in the input sequence. The LoRA layer can, based on these correlation relationships, adjust the learning of the specific correlations of the Solidity and BPMN data with a lower-rank update.

[0068] After the multi-head attention mechanism, the fully connected layer performs further non-linear transformation on the features, enhancing the feature representation ability. Adding the LoRA layer at this position can adjust the parameters in the high-dimensional feature space after the complex non-linear transformation. For example, for the complex Solidity smart contract code structure and BPMN process structure, after being processed by the fully connected layer, the features have incorporated various semantic and structural clues in the code and the process. The LoRA layer can adjust the parameters at this stage to help the model better learn how to integrate these clues to understand the deep correlations between the code and the process, such as semantic matching and structural transformation in nested structures.

[0069] S32: Add the LoRA layer after the cross-attention mechanism of the decoder and initialize it.

[0070] In the decoder of the Seq2Seq architecture, the cross-attention mechanism is used to combine the information transmitted from the encoder (including the information of the Solidity smart contract code corpus) and the partially generated decoder output (used to generate the sequence related to the BPMN corpus). Adding the LoRA layer at this position can directly fine-tune the cross-sequence correlation learning during the process of the model constructing the BPMN corpus prediction. The features output by the cross-attention mechanism are crucial for generating accurate BPMN sequences. The LoRA layer can optimize the parameters in this process, making the generated BPMN sequences more semantically and structurally consistent with the Solidity smart contract corpus, and optimizing the smart contract modeling process.

[0071] S33: Based on the differences between the Solidity smart contract code and the BPMN data, considering comprehensively from three aspects: semantic similarity loss, structure matching loss, and generation loss, design a comprehensive loss function.

[0072] Regarding the calculation of semantic similarity loss, a pre-trained semantic model (such as Sentence-BERT) can be used to obtain the semantic vectors of the Solidity smart contract code snippet and the corresponding BPMN process description. Assume that the semantic vector of the Solidity smart contract code snippet is The semantic vector of the BPMN process description is The semantic similarity is measured by cosine similarity, and the formula is

[0073] Regarding the calculation of structure loss, for the smart contract code, based on the structure information extracted from the abstract syntax tree, such as the hierarchy of function calls, conditional judgment structures, etc.; for the BPMN process, based on the structure information of tasks, gateways, events, etc. parsed from the JSON file, use the graph edit distance where N is a normalization factor.

[0074] Regarding the calculation of generation loss, the cross-entropy loss function is used. Assume that the probability distribution of the vocabulary generated by the Decoder is p(y i |x), where x is the input smart contract code, y i is the i-th vocabulary of the BPMN business process data, and the true vocabulary distribution is q(y i |x), then the generation loss function is L g =-∑q(y i |x)logp(y i |x).

[0075] Therefore, the comprehensive loss function is L = αL ss + βL sm + γL gAmong them, α, β, and γ are weight parameters used to balance the importance of different loss functions during the overall fine-tuning process. The specific weight values can be adjusted according to the performance of the model on the validation set.

[0076] S34: Add the calculation of the comprehensive loss function to the fully connected layer of the decoder and the LoRA low-rank matrix after the encoder.

[0077] S35: Initialize the parameters of the LoRA layer, and combine the initialized LoRA layer (A and B matrices) with the corresponding parameters of the target large model. The LoRA (Low-Rank Adaptation) layer is a technique used for fine-tuning pre-trained models. By introducing low-rank matrices, it fine-tunes the weights of the model, enabling it to better adapt to specific tasks without changing the main structure of the pre-trained model. Matrix A: It is a matrix with a shape of (hidden layer dimension, r), where r is the rank of the low-rank decomposition, which is much smaller than the hidden layer dimension. It maps the input features to a low-dimensional space. Matrix B: It is a matrix with a shape of (r, hidden layer dimension), which maps the features in the low-dimensional space back to the original high-dimensional space.

[0078] In step S104, use the input sequence to iteratively train the target large model in the fine-tuning framework. During the training process, adjust the model parameters in the training process of the target large model based on the low-rank matrix layer and the comprehensive loss function until the iterative training ends.

[0079] In the embodiment of the present invention, using the input sequence to iteratively train the target large model in the fine-tuning framework includes: setting the hyperparameters of the fine-tuning framework and starting the fine-tuning of the target large model; inputting the input sequence into the target large model, and using the input sequence to iteratively train the target large model. During the training process, the low-rank matrix layer participates in the parameter calculation at the corresponding position; optimize the model parameters of the target large model according to the loss function value calculated by the comprehensive loss function, and after meeting the iterative stop condition, complete the iterative training of the target large model.

[0080] Among them, the iterative stop condition may include that the performance of the target large model meets the target requirements, or the number of iterative training reaches the target number, etc.

[0081] It can be understood that after setting the corresponding hyperparameters (such as learning rate, training length, and number of training rounds, etc.) in the embodiment of the present invention, the model fine-tuning can be started, and through multiple iterative trainings, the model can be adjusted according to the results of the validation set, and the optimal model parameters can be saved.

[0082] Specifically: S41: Input the preprocessed smart contract code - BPMN data into the pre-trained large language model.

[0083] In this process, the LoRA layer participates in the parameter calculation at the corresponding position. For example, in the calculation after the fully connected layer, y=(W0 + BA)x is obtained, and the feature representation after passing through the encoder layer is obtained. In the Seq2Seq architecture, the features processed by the encoder are passed to the decoder. Similarly, corresponding parameter calculations are also performed at the LoRA layer position of the decoder to gradually generate the prediction results of the target sequence.

[0084] S42: Calculate the loss according to the comprehensive loss function determined previously, and record the change of the loss value for subsequent analysis of the model training progress.

[0085] S43: According to the obtained loss function value, calculate the gradient using the backpropagation algorithm and update the parameters using the optimization algorithm.

[0086] S44: Repeat the above processes of forward propagation, loss calculation, backpropagation, and parameter update for multiple rounds of training.

[0087] S45: Regularly evaluate the performance of the model on the validation set using relevant evaluation metrics (BLEU, F1 value). If it is found that the model is overfitting, take some regularization measures. If it is found that the model performance improves slowly, try to adjust the loss function weight, learning rate, or other hyperparameters. Among them, BLEU is a performance metric for evaluating the model, mainly used to measure the similarity between the model generation result and the reference result. The value range of BLEU is between 0.0 and 1.0, and the closer the value is to 1.0, the higher the model performance. The F1 value is a metric for evaluating the balance between precision and recall in the model. It is the harmonic mean of precision and recall, providing a single score that considers both. The value range of the F1 value is between 0.0 and 1.0, and the closer the value is to 1.0, the better the model.

[0088] In step S105, the model parameters after training the target large model are encapsulated, and the target large model is deployed on the intelligent contract design platform to implement the intelligent contract modeling function based on the target large model.

[0089] Among them, the intelligent contract modeling function is: Modeling the intelligent contract code into a domain description helps non-technical personnel understand the code behavior and logical structure, facilitates legal experts to verify the legal compatibility of the intelligent contract, and improves the participation and transparency of cross-domain collaborative testing and verification.

[0090] It can be understood that the embodiment of the present invention saves and encapsulates the fine-tuning model parameters, and embeds the contract modeling function into the smart contract design platform. Specifically: an API (Application Programming Interface) framework (such as FastAPI) is used to build a smart contract modeling service, and the smart contract modeling function is embedded in the smart contract design platform to improve usability.

[0091] In summary, (1) in the data alignment stage, according to the language mapping principle, the functions, logic and processes in the smart contract are accurately mapped to the elements such as tasks, gateways, events in BPMN, which initially alleviates the semantic association difficulties, information loss and misunderstanding caused by different semantic domains and structures between Solidity smart contract code and BPMN business process data. (2) Select a suitable basic pre-trained large model, and use parameter efficient fine-tuning method under the fine-tuning framework to fine-tune the Solidity smart contract to the specific large model of BPMN business process. (3) Utilize the powerful feature extraction capability of the Transformer architecture to deeply explore the semantic and structural information in the smart contract code. For example, during the processing, the model uses a multi-head attention mechanism and a fully connected layer to perform multi-level analysis on the code, and is no longer limited to the coarser node and line representation in graphical modeling. (4) A comprehensive loss function is constructed based on semantic similarity loss, structural matching loss and generation loss to further guide the model to learn the fine correspondence between smart contract code and BPMN process. When generating BPMN process description, the detailed behavioral semantics in the smart contract can be mapped to tasks, gateways and other elements in the BPMN process with finer granularity, thereby improving the granularity of modeling. (5) The solution of fine-tuning the large language model for smart contract modeling avoids the complex syntax and strict rules of formal modeling languages, and improves the efficiency of complex smart contract conversion.

[0092] The following will further explain the smart contract modeling method based on large model fine-tuning through a specific implementation. Figure 5 As shown, it mainly includes three stages, S1 data processing stage, S2 model fine-tuning stage and S3 deployment stage. Among them, S2 model fine-tuning stage can include n decoders, as follows:

[0093] (1) In the S1 data processing stage, the smart contract code and BPMN business process data are associated and mapped based on the language mapping principle and other methods. The multimodal data mapping and alignment module in the S1 data processing stage is the foundation, which provides rich and organized data resources for the subsequent LoRA-based parameter efficient fine-tuning module. When the two data with different structures are reasonably mapped and associated, effective feature learning and model training can be carried out based on these data.

[0094] (2) In the S2 model fine-tuning stage, select a suitable pre-trained large model, design a loss function based on the semantic and structural differences between the smart contract code and BPMN data, and add a LoRA layer at an appropriate position for efficient parameter fine-tuning. Adjust the model parameters by calculating the loss value. The S2 model fine-tuning stage depends on the data provided by the S1 data processing stage. By designing the loss function and LoRA fine-tuning, the model is trained and optimized to have a stronger ability to understand the semantic behavior and regulatory logic of smart contract code.

[0095] (3) In the S3 deployment stage, save the optimal large language model parameters, and embed the smart contract modeling function into the smart contract design platform to improve usability. The S3 deployment stage encapsulates the trained model parameters to provide API services and embeds them into the smart contract design platform, while establishing a feedback mechanism to update the model.

[0096] Thus, the embodiments of the present invention can work step by step and cooperate with each other through the above three stages to solve the problems of limited modeling ability, insufficient granularity, high learning threshold, low efficiency, etc. existing in smart contract modeling in related technologies.

[0097] Next, refer to the drawings to describe the smart contract modeling device based on large model fine-tuning according to the embodiments of the present invention.

[0098] Figure 6 It is a block diagram of the smart contract modeling device based on large model fine-tuning according to the embodiments of the present invention.

[0099] As Figure 6 shown, the smart contract modeling device 10 based on large model fine-tuning includes: an acquisition module 100, a processing module 200, a fine-tuning module 300, and a deployment module 400.

[0100] Among them, the acquisition module 100 is used to acquire the business process data of the smart contract code and the business process model; the processing module 200 is used to convert the smart contract code and the business process data into the input sequence of the fine-tuning framework. The fine-tuning framework includes a decoder and an encoder. The encoder includes an attention mechanism module and a fully connected feed-forward module. The decoder includes a cross-attention module and a fully connected feed-forward module; the fine-tuning module 300 is used to add low-rank matrix layers after the attention mechanism module and the fully connected feed-forward module of the encoder, and design the comprehensive loss function of the target large model after the fully connected feed-forward module of the decoder. The input sequence is used to iteratively train the target large model under the fine-tuning framework. During the training process, the model parameters in the training process of the target large model are adjusted based on the low-rank matrix layer and the comprehensive loss function until the iterative training ends; the deployment module 400 is used to encapsulate the model parameters after the target large model is trained, deploy the target large model on the smart contract design platform, and implement the smart contract modeling function based on the target large model.

[0101] It should be noted that the foregoing explanation of the embodiment of the smart contract modeling method based on large model fine-tuning also applies to the smart contract modeling device based on large model fine-tuning of this embodiment, and will not be repeated here.

[0102] The smart contract modeling device based on large model fine-tuning proposed according to the embodiment of the present invention fine-tunes the basic pre-trained large language model under the sequence-to-sequence framework, adds low-rank matrix layers at appropriate positions, and constructs a comprehensive loss function based on semantic similarity loss, structural loss and generation loss, so that the large model can better understand the behavior semantics and regulatory logic in the smart contract code, thereby optimizing the smart contract code modeling process, reducing the smart contract conversion cost, and thus can fine-tune the large language model by constructing a smart contract specific domain corpus, which can solve the limitations of equivalent modeling in the existing smart contract code field, improve the modeling efficiency and reduce the modeling cost.

[0103] Figure 7 The structural schematic diagram of the electronic device provided by the embodiment of the present invention. The electronic device may include:

[0104] A memory 701, a processor 702, and a computer program stored on the memory 701 and executable on the processor 702.

[0105] When the processor 702 executes the program, it implements the smart contract modeling method based on large model fine-tuning provided in the above embodiment.

[0106] Furthermore, the electronic device further includes:

[0107] A communication interface 703 for communication between the memory 701 and the processor 702.

[0108] A memory 701 for storing a computer program that can run on a processor 702.

[0109] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0110] If the memory 701, the processor 702, and the communication interface 703 are implemented independently, the communication interface 703, the memory 701, and the processor 702 can be interconnected through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 7 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0111] Optionally, in a specific implementation, if the memory 701, the processor 702, and the communication interface 703 are integrated on a chip, the memory 701, the processor 702, and the communication interface 703 can communicate with each other through an internal interface.

[0112] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0113] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0114] In addition, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0115] Any process or method description shown in a flowchart or described otherwise herein can be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logical function or process, and the scope of the preferred embodiments of the present invention includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0116] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following technologies well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays, field programmable gate arrays, etc.

[0117] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried by the methods for implementing the above embodiments can be completed by instructing relevant hardware through a program, and the above program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0118] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A smart contract modeling method based on large model fine-tuning, characterized in that: The following steps are involved: Get smart contract code and business process data of business process model; Convert the smart contract code and the business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross attention module and a fully connected feedforward module; A low-rank matrix layer is added after the attention mechanism module and the fully connected feedforward module of the encoder and the cross attention module of the decoder, respectively, and a comprehensive loss function of the target large model is designed after the fully connected feedforward module of the decoder; the comprehensive loss function includes semantic similarity loss, structural matching loss and generation loss, and the comprehensive loss function constructed based on the semantic similarity loss, the structural matching loss and the generation loss guides the target large model to learn the correspondence between the smart contract and the business process of the business process model; the formula of the comprehensive loss function is: ,in, represents the loss function value, , and represents the weight, represents the semantic similarity loss, represents the structural matching loss, represents said generation loss; Iteratively training the target large model under the fine-tuning framework using the input sequence, and during the training process, adjusting the model parameters of the target large model during the training process based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; Encapsulate the model parameters of the trained target large model, deploy the target large model on the smart contract design platform, and implement the smart contract modeling function based on the target large model.

2. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: Before converting the smart contract code and the business process data into an input sequence of a fine-tuning framework, the method further includes: The smart contract code and the business process data are preprocessed, wherein the preprocessing includes one or more of data cleaning, data format conversion, data alignment, and data segmentation processing.

3. The smart contract modeling method based on large model fine-tuning according to claim 2 is characterized in that: In the data alignment stage, according to the language mapping principle, at least one of the functions, logics and processes in the smart contract is mapped to at least one of the tasks, gateways and events in the business process data.

4. The smart contract modeling method based on large model fine-tuning according to claim 1 or 2 is characterized in that: The step of converting the smart contract code and the business process data into an input sequence of a fine-tuning framework includes: identifying an input format of the fine-tuning framework; Perform word segmentation processing on the smart contract code and the business process data according to the input format; Loading the target large model into the fine-tuning framework, marking the word-units after word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word-units; The tokenized tokens are converted into input sequences through the embedding layer of the fine-tuning framework.

5. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: The iterative training of the target large model under the fine-tuning framework using the input sequence includes: Setting the hyperparameters of the fine-tuning framework and starting fine-tuning of the target large model; Inputting the input sequence into the target large model, and iteratively training the target large model using the input sequence, wherein during the training process, the low-rank matrix layer participates in the parameter calculation of the corresponding position; The model parameters of the target large model are optimized according to the loss function value calculated by the comprehensive loss function, and the iterative training of the target large model is completed after the iteration stop condition is met.

6. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: The fine-tuning framework includes a Seq2Seq framework, and the decoder and the encoder are based on a Transformer architecture.

7. A smart contract modeling device based on large model fine-tuning, characterized in that: include: The acquisition module is used to obtain the smart contract code and the business process data of the business process model; A processing module, configured to convert the smart contract code and the business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross attention module and a fully connected feedforward module; A fine-tuning module is used to add low-rank matrix layers after the attention mechanism module and the fully connected feedforward module of the encoder and the cross-attention module of the decoder, respectively, to design a comprehensive loss function of the target large model after the fully connected feedforward module of the decoder, to iteratively train the target large model under the fine-tuning framework using the input sequence, and to adjust the model parameters of the target large model in the training process based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; the comprehensive loss function includes semantic similarity loss, structural matching loss and generation loss, and the comprehensive loss function constructed based on the semantic similarity loss, the structural matching loss and the generation loss guides the target large model to learn the correspondence between the smart contract and the business process of the business process model; the formula of the comprehensive loss function is: ,in, represents the loss function value, , and represents the weight, represents the semantic similarity loss, represents the structural matching loss, represents said generation loss; The deployment module is used to encapsulate the model parameters of the trained target large model, deploy the target large model on the smart contract design platform, and implement the smart contract modeling function based on the target large model.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the smart contract modeling method based on large model fine-tuning as described in any one of claims 1 to 6.

Images