Intelligent contract modeling method and device based on large model fine tuning and electronic equipment

Abstract

The invention relates to the technical field of intelligent contracts, in particular to an intelligent contract modeling method and device based on large model fine tuning and electronic equipment, and the method comprises the steps: converting an intelligent contract code and business process data into an input sequence of a fine tuning framework which comprises a decoder and an encoder, respectively adding low-rank matrix layers behind an attention mechanism module and a full-connection feedforward module of the encoder and a cross attention module of the decoder, and designing a comprehensive loss function of the target large model behind the full-connection feedforward module of the decoder; and iteratively training the target large model under the fine tuning framework by using the input sequence, packaging model parameters after training of the target large model, deploying the target large model on an intelligent contract design platform, and realizing an intelligent contract modeling function based on the target large model. Therefore, the problems of limitation, relatively low modeling efficiency, high modeling cost and the like of equivalent modeling in the field of smart contract codes in related technologies are solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a smart contract modeling method, device and electronic equipment based on large model fine-tuning. Background Art

[0002] Smart contracts are one of the core innovations of blockchain technology. Their emergence has changed the traditional way of transactions and brought efficiency improvements and cost reductions to many fields. Modeling smart contract code as a domain description helps non-technical personnel understand the code behavior and logical structure, facilitates legal experts to verify the legal compatibility of smart contracts, and improves the participation and transparency of cross-domain collaborative testing and verification. However, the logical complexity, element integrity, and domain diversity of smart contract code make smart contract code equivalent modeling face challenges such as poor accuracy, low efficiency, and high cost.

[0003] Related technologies mainly use software engineering methods to achieve domain equivalent modeling of smart contract code. The first type of research uses structured graphical modeling methods, using BPMN (Business Process Model and Notation) or UML (Unified Modeling Language) standardized modeling tools and symbols to describe the behavior and logic of smart contract code. The second type of research focuses on designing DSL (Domain-Specific Language), which is customized according to the specific needs and domain characteristics of smart contracts, aiming to achieve personalized descriptions that are easy to understand and maintain. The third type of research uses formal modeling methods to abstract smart contract code into conditional rules, and use first-order logic or linear temporal logic to describe the logic of smart contract rules.

[0004] However, although the structured graphical modeling method has a high degree of standardization and is easy to model across domains, its modeling capabilities are limited by icon elements, and its granularity is too coarse to fully capture the logical details of complex smart contract codes, and its expression accuracy is poor. Although DSL and formal modeling methods can achieve fine-grained modeling and effectively describe the behavior and logic of smart contracts, the modeling process requires professional technical knowledge, has a high learning threshold, and has a high cost and low efficiency in processing complex smart contract codes. Summary of the invention

[0005] The present invention provides a smart contract modeling method, device and electronic device based on large model fine-tuning to solve the problems of limitations in equivalent modeling in the field of smart contract code, low modeling efficiency and high modeling cost in related technologies.

[0006] The first aspect of the present invention provides a smart contract modeling method based on large model fine-tuning, including the following steps: obtaining smart contract code and business process data of a business process model; converting the smart contract code and business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross-attention module and a fully connected feedforward module; adding a low-rank matrix layer after the attention mechanism module and the fully connected feedforward module of the encoder, and the cross-attention module of the decoder, respectively, and designing a comprehensive loss function of the target large model after the fully connected feedforward module of the decoder; iteratively training the target large model under the fine-tuning framework using the input sequence, and during the training process, adjusting the model parameters of the target large model in the training process based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; encapsulating the model parameters of the trained target large model, deploying the target large model on the smart contract design platform, and realizing the smart contract modeling function based on the target large model.

[0007] Optionally, the comprehensive loss function includes semantic similarity loss, structural matching loss and generation loss. The comprehensive loss function constructed based on semantic similarity loss, structural matching loss and generation loss guides the target large model to learn the correspondence between the smart contract and the business process of the business process model.

[0008] Optionally, the formula for the comprehensive loss function is:

[0009] L=αL ss +βL sm +γL g

[0010] Among them, L represents the loss function value, α, β and γ represent weights, and L ss represents the semantic similarity loss, L sm represents the structure matching loss, L g Represents generation loss.

[0011] Optionally, before converting the smart contract code and business process data into an input sequence of the fine-tuning framework, it also includes: preprocessing the smart contract and business process data, wherein the preprocessing includes one or more of data cleaning, data format conversion, data alignment and segmentation data processing.

[0012] Optionally, in the data alignment stage, at least one of the functions, logics and processes in the smart contract is mapped to at least one of the tasks, gateways and events in the business process data according to the language mapping principle.

[0013] Optionally, the smart contract code and business process data are converted into an input sequence of a fine-tuning framework, including: identifying the input format of the fine-tuning framework; performing word segmentation on the smart contract code and business process data according to the input format; loading the target large model into the fine-tuning framework, marking the word units after word segmentation according to the marker of the target large model, and assigning a unique index to the marked word units; and converting the marked word units into an input sequence through an embedding layer of the fine-tuning framework.

[0014] Optionally, the target large model under the fine-tuning framework is iteratively trained using the input sequence, including: setting hyperparameters of the fine-tuning framework to start fine-tuning the target large model; inputting the input sequence into the target large model, and iteratively training the target large model using the input sequence, during the training process, the low-rank matrix layer participates in the parameter calculation of the corresponding position; the model parameters of the target large model are optimized according to the loss function value calculated by the comprehensive loss function, and the iterative training of the target large model is completed after the iteration stop condition is met.

[0015] Optionally, the fine-tuning framework includes a Seq2Seq framework, and the decoder and encoder are based on the Transformer architecture.

[0016] The second aspect of the present invention provides a smart contract modeling device based on large model fine-tuning, including: an acquisition module, used to acquire smart contract code and business process data of a business process model; a processing module, used to convert the smart contract code and business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross-attention module and a fully connected feedforward module; a fine-tuning module, used to add a low-rank matrix layer after the attention mechanism module and the fully connected feedforward module of the encoder, and the cross-attention module of the decoder, respectively, and design a comprehensive loss function of the target large model after the fully connected feedforward module of the decoder, and use the input sequence to iteratively train the target large model under the fine-tuning framework. During the training process, the model parameters of the target large model in the training process are adjusted based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; a deployment module, used to encapsulate the model parameters of the trained target large model, deploy the target large model on the smart contract design platform, and realize the smart contract modeling function based on the target large model.

[0017] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a smart contract modeling method based on large model fine-tuning as in the above-mentioned embodiment.

[0018] Thus, the present invention includes the following beneficial effects:

[0019] The embodiments of the present invention can fine-tune the basic pre-trained large language model under the sequence-to-sequence framework, add a low-rank matrix layer at an appropriate position, and construct a comprehensive loss function based on semantic similarity loss, structural loss and generation loss, so that the large model can better understand the behavioral semantics and regulatory logic in the smart contract code, thereby optimizing the smart contract code modeling process and reducing the smart contract conversion cost. Therefore, the large language model can be fine-tuned by constructing a smart contract specific domain corpus, effectively addressing the limitations of equivalent modeling in the smart contract code field in related technologies, improving modeling efficiency and reducing modeling costs.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a smart contract modeling method based on large model fine-tuning according to an embodiment of the present invention;

[0023] Figure 2 A sequential execution structure diagram according to an embodiment of the present invention;

[0024] Figure 3 A structural diagram of a parallel gateway according to an embodiment of the present invention;

[0025] Figure 4 This is a structural diagram of an exclusive gateway according to an embodiment of the present invention;

[0026] Figure 5 A flowchart of a smart contract modeling method based on large model fine-tuning according to an embodiment of the present invention;

[0027] Figure 6 This is an example diagram of a smart contract modeling device based on large model fine-tuning according to an embodiment of the present invention;

[0028] Figure 7 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0030] The technical terms used in the following embodiments are explained as follows:

[0031] (1) Smart contract: a computer code stored on a blockchain that automatically executes when predetermined terms and conditions are met.

[0032] (2) BPMN business process modeling: is a standardized method for graphically describing various aspects of business processes. It provides a clear and standardized way to represent and analyze business processes, events, decisions, tasks, etc., helping different stakeholders (such as business personnel, technical personnel, management, etc.) understand and communicate business processes.

[0033] (3) DSL: A computer language designed specifically for a specific application domain. Unlike general-purpose programming languages ​​(such as Python, Java, and C++), DSL aims to solve specific problems in a certain domain and provide a simpler, more intuitive, and more efficient way to describe and implement logic or operations within the domain.

[0034] (4) Fine-tuning of large models: Fine-tuning usually involves adjusting the model parameters based on the general knowledge learned by the pre-trained large language model through a small amount of specific task data, so that it can better adapt to specific task requirements.

[0035] As a popular business process modeling language, BPMN provides an intuitive graphical modeling method, which lowers the technical threshold of programming and enables legal experts and business personnel to design and understand the business process of contracts more easily. At the same time, as a universal visual language, it solves the problem of multi-domain collaboration in development. Large models have the ability to understand and capture complex structures, dynamically adapt and generalize across domains, and efficiently model and process.

[0036] To this end, the embodiment of the present invention proposes a smart contract modeling method based on fine-tuning of a large model, constructs a corpus of a specific field (smart contract code-model), inserts a LoRA low-rank matrix at a suitable position near the encoder layer and the decoder layer, and designs a loss function based on semantic similarity loss, structural loss and generation loss to achieve fine-tuning of the parameters of the large model, so that the large model can better understand the behavioral semantics and regulatory logic in the smart contract code, and thus optimize the smart contract code modeling process. The present invention utilizes the technical advantages of the large model's ability to understand and capture complex structures, cross-domain dynamic adaptation and generalization capabilities, and efficient modeling and processing capabilities, aiming to solve the limitations of equivalent modeling in the smart contract code field in related technologies, improve modeling efficiency, and reduce modeling costs.

[0037] The following describes the smart contract modeling method, device and electronic device based on large model fine-tuning according to the embodiments of the present invention with reference to the accompanying drawings. Specifically, Figure 1A flowchart of a smart contract modeling method based on large model fine-tuning provided in an embodiment of the present invention.

[0038] like Figure 1 As shown, the smart contract modeling method based on large model fine-tuning includes the following steps:

[0039] In step S101, the smart contract code and the business process data of the business process model are obtained.

[0040] It is understandable that the embodiments of the present invention can obtain contract codes and BPMN business process files through various channels such as open source projects, GitHub, and development platforms to collect a large amount of diverse Solidity smart contract codes and BPMN business process model corpus. Among them, Solidity is a programming language for smart contracts, mainly used to write and deploy smart contracts on the Ethereum blockchain. It was developed by the Ethereum community and aims to provide a safe, easy-to-understand and use programming language for creating decentralized applications.

[0041] Before executing step S102, that is, before converting the smart contract code and business process data into an input sequence of the fine-tuning framework, it also includes: preprocessing the smart contract code and the business process data, wherein the preprocessing includes one or more of data cleaning, data format conversion, data alignment and segmentation data processing. In the data alignment stage, at least one of the functions, logics and processes in the smart contract is mapped to at least one of the tasks, gateways and events in the business process data according to the language mapping principle.

[0042] It can be understood that after obtaining the corpus of the Solidity smart contract code and the BPMN business process model, the embodiment of the present invention obtains high-quality input data through data cleaning, format conversion, alignment, and segmentation data processing.

[0043] Therefore, the embodiment of the present invention can obtain high-quality input data by collecting a large amount of diverse Solidity smart contract code and BPMN model corpus, and through data cleaning, format conversion, alignment, and data segmentation processing methods. This is conducive to initially alleviating the semantic association difficulties, information loss and misunderstanding problems caused by different semantic domains and structures between Solidity smart contract code and BPMN business process data.

[0044] Specifically, S11: Preprocess and clean the Solidity code and BPMN business process data to remove redundant data. Specifically, data preprocessing and cleaning refers to removing redundant comments and blank characters in the contract code, considering normalizing variable names and function names, and ensuring the correct file format. The cleaned BPMN business process model should only retain process elements related to the conversion, such as tasks, gateways, and events.

[0045] S12: Convert the format of Solidity smart contract code and BPMN business process data to facilitate data alignment. Specifically, convert the Solidity smart contract code into an abstract syntax tree (AST) and extract its structure; convert the BPMN model from a graphic or file format (such as SVG, PNG) to a standard structured format, and each BPMN element (task, event, gateway, etc.) should have a clear label and relationship. This is done to initially alleviate the semantic association difficulties, information loss and misunderstanding caused by different semantic domains and structures between Solidity smart contract code and BPMN business process files.

[0046] S13: Align the formatted samples so that each Solidity contract code block corresponds to the corresponding element in the BPMN diagram. For example, in the Solidity code, each operation (such as payment, transfer, logical judgment, etc.) should have a corresponding BPMN task or event, and the conditional judgment should correspond to the gateway in BPMN. Figure 2 Demonstrates sequential execution structure, Figure 3 Demonstrates parallel gateway architecture, Figure 4 It shows the exclusive gateway structure, three types of "Solidity code-BPMN model" alignment corpus and the corresponding BPMN business process model diagram converted on Camunda.

[0047] S14: Divide the data according to a certain ratio, for example, divide the data according to 80% training set, 10% validation set, and 10% test set. Those skilled in the art can select the division ratio according to actual conditions, without making specific limitations.

[0048] In step S102, the smart contract code and business process data are converted into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross attention module and a fully connected feedforward module.

[0049] Among them, the fine-tuning framework can include a Seq2Seq framework, and the decoder and the encoder can be based on a Transformer architecture.

[0050] It is understandable that the embodiments of the present invention can convert the corpus of smart contracts and business process data into a format suitable for input under the Seq2Seq framework to construct a good input sequence.

[0051] In an embodiment of the present invention, the smart contract code and business process data are converted into an input sequence of a fine-tuning framework, including: identifying the input format of the fine-tuning framework; performing word segmentation processing on the smart contract code and business process data according to the input format; loading the target large model into the fine-tuning framework, marking the word units after the word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word units; and converting the marked word units into an input sequence through an embedding layer of the fine-tuning framework.

[0052] It should be noted that the target large model can be a pre-trained large language model that performs well on code-related tasks, and their performance on the current smart contract modeling task is compared, and the model is selected based on the comparison results. For example, the pre-trained large language model can be CodeBERT, etc.

[0053] Specifically, S21: For Solidity and BPMN corpora, they first need to be converted into a format suitable for input into the Seq2Seq framework.

[0054] For example, Solidity code may need to be segmented to separate keywords, function names, variable names, etc. in the code into independent tokens. For example, for the function transferFunds(address from,addressto,uint amount), after segmentation, the result may be ["function","transferFunds","(","address","from",",","address","to",",","uint","amount",")"].

[0055] S23: Load the pre-trained large language model into the fine-tuning framework and tokenize the tokenized Solidity and BPMN corpora using a tokenizer compatible with CodeBERT. This process converts tokens into corresponding tokens and assigns a unique index to each token. For example, TransferFunds may be converted to a specific token index, such as

[102] .

[0056] Table 1 summarizes the applicable scenarios and advantages and disadvantages of the three basic models CodeBERT / GraphCodeBERT, GPT series / Llama series and T5 / CodeT5. After comprehensive comparison, the embodiment of the present invention selects CodeBERT as the pre-trained large language model. CodeBERT is a large language model pre-trained on a large-scale code corpus. It has a good innate understanding of the grammatical structure and semantics of Solidity code. In addition, although BPMN semantics focuses on business processes, CodeBERT can serve as a bridge to establish semantic associations between the two. Fine-tuning CodeBERT under the Seq2Seq framework can further alleviate the semantic understanding differences between smart contract code and BPMN business process files.

[0057] Table 1

[0058]

[0059] S23: These tokens are encoded into vector representations through the embedding layer. These vectors will serve as the basic unit of subsequent model input, containing the semantics and position information of the word unit.

[0060] S24: Added a decoder based on the Tansformer structure.

[0061] In step S103, low-rank matrix layers are added after the attention mechanism module and the fully connected feedforward module of the encoder and the cross-attention module of the decoder, and a comprehensive loss function of the target large model is designed after the fully connected feedforward module of the decoder.

[0062] It can be understood that the embodiment of the present invention adds a LoRA low-rank matrix near the self-attention module, fully connected layer of the encoder and the attention layer of the decoder of the basic pre-trained model, and designs the loss function by comprehensively considering the three aspects of semantic similarity loss, structural matching loss and generation loss, thereby adding the LoRA layer at the appropriate position and designing the appropriate loss function for fine-tuning, so that the large model can better understand the behavioral semantics and regulatory logic in the smart contract code, thereby optimizing the smart contract code modeling process.

[0063] In an embodiment of the present invention, the comprehensive loss function includes semantic similarity loss, structural matching loss and generation loss. The comprehensive loss function constructed based on semantic similarity loss, structural matching loss and generation loss guides the target large model to learn the correspondence between the smart contract and the business process of the business process model.

[0064] It is understandable that the embodiment of the present invention can fine-tune the large model under the fine-tuning framework of Seq2Seq Trainer, and further guide the model to learn the fine correspondence between the smart contract code and the BPMN process based on the calculation of the comprehensive loss function constructed based on semantic similarity loss, structural matching loss and generation loss. This allows the detailed behavior semantics in the smart contract to be mapped to the tasks, gateways and other elements in the BPMN process with finer granularity when generating the BPMN process description, thereby improving the granularity of modeling.

[0065] Specifically, if Figure 5 As shown, add the LoRA layer at the corresponding position and design the loss function as follows:

[0066] S31: LoRA layers are added after the attention mechanism and the fully connected layer.

[0067] The encoder is a key part of the target large model to understand the input sequence (Solidity and BPMN corpus). After the multi-head attention mechanism, the semantic and structural information in the corpus has been initially focused on and extracted. At this time, the LoRA layer is added to fine-tune the parameters in this critical feature extraction stage. The features output by the multi-head attention mechanism represent the associations between the elements in the input sequence. The LoRA layer can adjust the model's learning of specific associations of Solidity and BPMN data based on these associations with lower rank updates.

[0068] After the multi-head attention mechanism, the fully connected layer further performs nonlinear transformation on the features, enhancing the ability of feature representation. Adding the LoRA layer at this position allows parameter adjustment in the high-dimensional feature space after complex nonlinear transformation. For example, for complex Solidity smart contract code structures and BPMN process structures, after being processed by the fully connected layer, the features have integrated multiple semantic and structural clues in the code and process. The LoRA layer can adjust parameters at this stage to help the model better learn how to integrate these clues to understand the deep-level associations between code and processes, such as semantic matching and structural transformations in nested structures.

[0069] S32: Add the LoRA layer after the decoder’s cross-attention mechanism and initialize it.

[0070] In the decoder of the Seq2Seq architecture, the cross-attention mechanism is used to combine the information from the encoder (including information from the Solidity smart contract code corpus) and the generated partial decoder output (used to generate sequences related to the BPMN corpus). Adding the LoRA layer at this position allows for fine-tuning of cross-sequence association learning directly during the model construction of the BPMN corpus prediction process. The features output by the cross-attention mechanism are critical for generating accurate BPMN sequences. The LoRA layer can optimize the parameters in this process so that the generated BPMN sequence is more semantically and structurally consistent with the association with the Solidity smart contract corpus, optimizing the smart contract modeling process.

[0071] S33: Based on the differences between Solidity smart contract code and BPMN data, a comprehensive loss function is designed from three aspects: semantic similarity loss, structural matching loss, and generation loss.

[0072] Regarding the calculation of semantic similarity loss, a pre-trained semantic model (such as Sentence-BERT) can be used to obtain the semantic vectors of the Solidity smart contract code snippet and the corresponding BPMN process description. Assume that the semantic vector of the Solidity smart contract code snippet is The semantic vector of BPMN process description is Semantic similarity is measured by cosine similarity, and the formula is

[0073] Regarding the calculation of structural loss, for smart contract codes, we can use the structural information extracted from the abstract syntax tree, such as the hierarchy of function calls, conditional judgment structure, etc.; for BPMN processes, we can use the graph edit distance to calculate the structural information such as tasks, gateways, events, etc. parsed from JSON files. Where N is a normalization factor.

[0074] Regarding the calculation of generation loss, the cross entropy loss function is used. Assume that the probability distribution of the vocabulary generated by the Decoder is p(y i |x), where x is the input smart contract code, y i is the i-th word in the BPMN business process data, and the actual word distribution is q(y i |x), then the generated loss function is L g =-∑q(y i |x)logp(y i |x).

[0075] Therefore, the comprehensive loss function is L = αL ss +βL sm +γL gAmong them, α, β and γ are weight parameters, which are used to balance the importance of different loss functions in the overall fine-tuning process. The specific weight values ​​can be adjusted according to the performance of the model on the validation set.

[0076] S34: Add comprehensive loss function calculation to the fully connected layer of the decoder and the LoRA low-rank matrix after the encoder.

[0077] S35: Initialize the parameters of the LoRA layer, and combine the initialized LoRA layer (A and B matrices) with the corresponding parameters of the target large model. The LoRA (Low-Rank Adaptation) layer is a technology used in fine-tuning pre-trained models. It introduces a low-rank matrix to fine-tune the weights of the model, so that it can better adapt to specific tasks without changing the main structure of the pre-trained model. A matrix: is a matrix with a shape of (hidden layer dimension, r), where r is the rank of the low-rank decomposition, which is much smaller than the hidden layer dimension. It maps the input features to a low-dimensional space. B matrix: is a matrix with a shape of (r, hidden layer dimension), which maps the features in the low-dimensional space back to the original high-dimensional space.

[0078] In step S104, the target large model under the fine-tuning framework is iteratively trained using the input sequence. During the training process, the model parameters of the target large model training process are adjusted based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed.

[0079] In an embodiment of the present invention, an input sequence is used to iteratively train a target large model under a fine-tuning framework, including: setting hyperparameters of the fine-tuning framework to start fine-tuning of the target large model; inputting the input sequence into the target large model, and iteratively training the target large model using the input sequence, during the training process, the low-rank matrix layer participates in the parameter calculation of the corresponding position; the model parameters of the target large model are optimized according to the loss function value calculated by the comprehensive loss function, and the iterative training of the target large model is completed after the iteration stop condition is met.

[0080] Among them, the iteration stopping condition may include that the performance of the target large model meets the target requirements, or the number of iterative training times reaches the target number, etc.

[0081] It is understandable that the embodiment of the present invention can set the corresponding hyperparameters (such as learning rate, training length and training rounds, etc.), start model fine-tuning, through multiple iterative training, and adjust the model according to the results of the validation set, and save the optimal model parameters.

[0082] Specifically: S41: input the preprocessed smart contract code - BPMN data into the pre-trained large language model.

[0083] In this process, the LoRA layer will participate in the parameter calculation of the corresponding position, such as the calculation y=(W0+BA)x after the fully connected layer, and obtain the feature representation after the encoder layer. In the Seq2Seq architecture, the features processed by the encoder will be passed to the decoder. Similarly, the corresponding parameter calculation will be performed at the LoRA layer position of the decoder to gradually generate the prediction results of the target sequence.

[0084] S42: Calculate the loss according to the comprehensive loss function determined previously, and record the change in the loss value for subsequent analysis of the training progress of the model.

[0085] S43: According to the obtained loss function value, the gradient is calculated using the back propagation algorithm, and the parameters are updated using the optimization algorithm.

[0086] S44: Repeat the above process of forward propagation, loss calculation, back propagation and parameter update for multiple rounds of training.

[0087] S45: Use relevant evaluation indicators (BLEU, F1 value) to regularly evaluate the performance of the model on the validation set. If the model is found to be overfitting, take some regularization measures. If the model performance is found to be improving slowly, try to adjust the loss function weight, learning rate or other hyperparameters. Among them, BLEU is a performance indicator used to evaluate the model. It is mainly used to measure the similarity between the model-generated results and the reference results. The value range of BLEU is between 0.0 and 1.0. The closer the value is to 1.0, the higher the model performance. The F1 value is an indicator used to evaluate the balance between precision and recall in the model. It is the harmonic mean of precision and recall, providing a single score that takes both into account. The value range of F1 is between 0.0 and 1.0. The closer the value is to 1.0, the better the model.

[0088] In step S105, the model parameters of the trained target large model are encapsulated, the target large model is deployed on the smart contract design platform, and the smart contract modeling function is implemented based on the target large model.

[0089] Among them, the smart contract modeling function is: modeling the smart contract code as a domain description helps non-technical personnel understand the code behavior and logical structure, facilitates legal experts to verify the legal compatibility of smart contracts, and improves the participation and transparency of cross-domain collaborative testing and verification.

[0090] It can be understood that the embodiment of the present invention saves and encapsulates the fine-tuning model parameters, and embeds the contract modeling function into the smart contract design platform. Specifically: an API (Application Programming Interface) framework (such as FastAPI) is used to build a smart contract modeling service, and the smart contract modeling function is embedded in the smart contract design platform to improve usability.

[0091] In summary, (1) in the data alignment stage, according to the language mapping principle, the functions, logic and processes in the smart contract are accurately mapped to the elements such as tasks, gateways, events in BPMN, which initially alleviates the semantic association difficulties, information loss and misunderstanding caused by different semantic domains and structures between Solidity smart contract code and BPMN business process data. (2) Select a suitable basic pre-trained large model, and use parameter efficient fine-tuning method under the fine-tuning framework to fine-tune the Solidity smart contract to the specific large model of BPMN business process. (3) Utilize the powerful feature extraction capability of the Transformer architecture to deeply explore the semantic and structural information in the smart contract code. For example, during the processing, the model uses a multi-head attention mechanism and a fully connected layer to perform multi-level analysis on the code, and is no longer limited to the coarser node and line representation in graphical modeling. (4) A comprehensive loss function is constructed based on semantic similarity loss, structural matching loss and generation loss to further guide the model to learn the fine correspondence between smart contract code and BPMN process. When generating BPMN process description, the detailed behavioral semantics in the smart contract can be mapped to tasks, gateways and other elements in the BPMN process with finer granularity, thereby improving the granularity of modeling. (5) The solution of fine-tuning the large language model for smart contract modeling avoids the complex syntax and strict rules of formal modeling languages, and improves the efficiency of complex smart contract conversion.

[0092] The following will further explain the smart contract modeling method based on large model fine-tuning through a specific implementation. Figure 5 As shown, it mainly includes three stages, S1 data processing stage, S2 model fine-tuning stage and S3 deployment stage. Among them, S2 model fine-tuning stage can include n decoders, as follows:

[0093] (1) In the S1 data processing stage, the smart contract code and BPMN business process data are associated and mapped based on the language mapping principle and other methods. The multimodal data mapping and alignment module in the S1 data processing stage is the foundation, which provides rich and organized data resources for the subsequent LoRA-based parameter efficient fine-tuning module. When the two data with different structures are reasonably mapped and associated, effective feature learning and model training can be carried out based on these data.

[0094] (2) In the S2 model fine-tuning phase, select a suitable pre-trained large model, design a loss function based on the semantic and structural differences between the smart contract code and BPMN data, and add a LoRA layer at the appropriate location to efficiently fine-tune the parameters. Adjust the model parameters by calculating the loss value. The S2 model fine-tuning phase relies on the data provided by the S1 data processing phase. By designing the loss function and LoRA fine-tuning, the model is trained and optimized to have a stronger ability to understand the behavioral semantics and regulatory logic of the smart contract code.

[0095] (3) The S3 deployment phase saves the optimal large language model parameters and embeds the smart contract modeling function into the smart contract design platform to improve usability. The S3 deployment phase is to encapsulate the trained model parameters to provide API services and embed them into the smart contract design platform, while establishing a feedback mechanism to update the model.

[0096] Therefore, the embodiments of the present invention can progressively and collaboratively work through the above three stages to solve the problems of limited modeling capabilities, insufficient granularity, high learning threshold, low efficiency, etc. in smart contract modeling in related technologies.

[0097] Secondly, the smart contract modeling device based on large model fine-tuning proposed according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0098] Figure 6 It is a block diagram of a smart contract modeling device based on large model fine-tuning according to an embodiment of the present invention.

[0099] like Figure 6 As shown, the smart contract modeling device 10 based on large model fine-tuning includes: an acquisition module 100, a processing module 200, a fine-tuning module 300 and a deployment module 400.

[0100] Among them, the acquisition module 100 is used to obtain the business process data of the smart contract code and the business process model; the processing module 200 is used to convert the smart contract code and the business process data into an input sequence of the fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross-attention module and a fully connected feedforward module; the fine-tuning module 300 is used to add a low-rank matrix layer after the attention mechanism module and the fully connected feedforward module of the encoder, and the cross-attention module of the decoder, respectively, and design a comprehensive loss function of the target large model after the fully connected feedforward module of the decoder, and use the input sequence to iteratively train the target large model under the fine-tuning framework. During the training process, the model parameters of the target large model in the training process are adjusted based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; the deployment module 400 is used to encapsulate the model parameters of the target large model after training, deploy the target large model on the smart contract design platform, and realize the smart contract modeling function based on the target large model.

[0101] It should be noted that the aforementioned explanation of the embodiment of the smart contract modeling method based on large model fine-tuning is also applicable to the smart contract modeling device based on large model fine-tuning of this embodiment, and will not be repeated here.

[0102] According to the smart contract modeling device based on large model fine-tuning proposed in an embodiment of the present invention, the basic pre-trained large language model is fine-tuned under the sequence-to-sequence framework, a low-rank matrix layer is added at an appropriate position, and a comprehensive loss function is constructed based on semantic similarity loss, structural loss and generation loss, so that the large model can better understand the behavioral semantics and regulatory logic in the smart contract code, thereby optimizing the smart contract code modeling process and reducing the smart contract conversion cost. Therefore, the large language model can be fine-tuned by constructing a smart contract specific domain corpus, which can solve the limitations of equivalent modeling in the existing smart contract code field, improve modeling efficiency and reduce modeling costs.

[0103] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0104] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0105] When the processor 702 executes the program, the smart contract modeling method based on large model fine-tuning provided in the above embodiment is implemented.

[0106] Furthermore, the electronic device further comprises:

[0107] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0108] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0109] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0110] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0111] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0112] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0114] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0115] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0116] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0117] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0118] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A smart contract modeling method based on large model fine-tuning, characterized in that: The following steps are involved: Get smart contract code and business process data of business process model; Convert the smart contract code and the business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross attention module and a fully connected feedforward module; A low-rank matrix layer is added after the attention mechanism module and the fully connected feedforward module of the encoder and the cross attention module of the decoder, and a comprehensive loss function of the target large model is designed after the fully connected feedforward module of the decoder; Iteratively training the target large model under the fine-tuning framework using the input sequence, and during the training process, adjusting the model parameters of the target large model during the training process based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; Encapsulate the model parameters of the trained target large model, deploy the target large model on the smart contract design platform, and implement the smart contract modeling function based on the target large model.

2. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: The comprehensive loss function includes semantic similarity loss, structural matching loss and generation loss. The comprehensive loss function constructed based on the semantic similarity loss, the structural matching loss and the generation loss guides the target large model to learn the correspondence between the smart contract and the business process of the business process model.

3. The smart contract modeling method based on large model fine-tuning according to claim 2 is characterized in that: The formula of the comprehensive loss function is: L=αL ss +βL sm +γL g Among them, L represents the loss function value, α, β and γ represent weights, and L ss represents the semantic similarity loss, L sm represents the structural matching loss, L g represents the generation loss.

4. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: Before converting the smart contract code and the business process data into an input sequence of a fine-tuning framework, the method further includes: The smart contract code and the business process data are preprocessed, wherein the preprocessing includes one or more of data cleaning, data format conversion, data alignment, and data segmentation processing.

5. The smart contract modeling method based on large model fine-tuning according to claim 4 is characterized in that: In the data alignment stage, according to the language mapping principle, at least one of the functions, logics and processes in the smart contract is mapped to at least one of the tasks, gateways and events in the business process data.

6. The smart contract modeling method based on large model fine-tuning according to claim 1 or 4 is characterized in that: The step of converting the smart contract code and the business process data into an input sequence of a fine-tuning framework includes: identifying an input format of the fine-tuning framework; Perform word segmentation processing on the smart contract code and the business process data according to the input format; Loading the target large model into the fine-tuning framework, marking the word-units after word segmentation processing according to the marker of the target large model, and assigning a unique index to the marked word-units; The tokenized tokens are converted into input sequences through the embedding layer of the fine-tuning framework.

7. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: The iterative training of the target large model under the fine-tuning framework using the input sequence includes: Setting the hyperparameters of the fine-tuning framework and starting fine-tuning of the target large model; Inputting the input sequence into the target large model, and iteratively training the target large model using the input sequence, wherein during the training process, the low-rank matrix layer participates in the parameter calculation of the corresponding position; The model parameters of the target large model are optimized according to the loss function value calculated by the comprehensive loss function, and the iterative training of the target large model is completed after the iteration stop condition is met.

8. The smart contract modeling method based on large model fine-tuning according to claim 1 is characterized in that: The fine-tuning framework includes a Seq2Seq framework, and the decoder and the encoder are based on a Transformer architecture.

9. A smart contract modeling device based on large model fine-tuning, characterized in that: include: The acquisition module is used to obtain the smart contract code and the business process data of the business process model; A processing module, configured to convert the smart contract code and the business process data into an input sequence of a fine-tuning framework, wherein the fine-tuning framework includes a decoder and an encoder, the encoder includes an attention mechanism module and a fully connected feedforward module, and the decoder includes a cross attention module and a fully connected feedforward module; A fine-tuning module, used to add low-rank matrix layers after the attention mechanism module and the fully connected feedforward module of the encoder and the cross-attention module of the decoder, respectively, to design a comprehensive loss function of the target large model after the fully connected feedforward module of the decoder, to iteratively train the target large model under the fine-tuning framework using the input sequence, and during the training process, to adjust the model parameters of the target large model during the training process based on the low-rank matrix layer and the comprehensive loss function until the iterative training is completed; The deployment module is used to encapsulate the model parameters of the trained target large model, deploy the target large model on the smart contract design platform, and implement the smart contract modeling function based on the target large model.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the smart contract modeling method based on large model fine-tuning as described in any one of claims 1 to 8.

Images