A model method for characterizing smart contracts

Abstract

The present invention discloses a model method for characterizing smart contracts, which includes the following steps: collecting a large number of open-source smart contract source codes on the blockchain network; cleaning the smart contract source codes and generating a structured code tree; designing a pre-trained model based on a deep neural network; inputting the training set of smart contract codes into the pre-trained model for learning; disassembling the input layer, encoding layer, and output layer in the pre-trained model to form the SmartBERT model; calling the SmartBERT model to input the smart contract source code to generate a characterization. By disassembling the input layer, encoding layer, and output layer in the pre-trained model to construct the SmartBERT model, in the application stage of SmartBERT, when a user inputs smart contract codes, a mathematical matrix representing the entire smart contract can be generated, and this mathematical matrix can more accurately capture the context semantics and structural information of the smart contract, providing strong support for subsequent downstream tasks related to smart contracts.

Description

Technical Field

[0001] The present invention relates to the technical field of smart contracts. Specifically, it relates to a model method for representing smart contracts. Background Art

[0002] In the rapid development of the natural language processing (NLP) field, pre-trained models have become a key technology for achieving excellent results in various text-related tasks. Among them, the neural network architecture Transformer based on the attention mechanism is widely used in pre-trained models. This architecture focuses on processing sequential data and introduces self-attention mechanism to consider the relationships of different positions in the input sequence simultaneously. Two representative pre-trained models using Transformer are BERT (Bidirectional Encoder Representations from Transformers) of Google and GPT (Generative Pre-trained Transformer) of OpenAI.

[0003] However, the current mainstream general pre-trained models focus on too broad areas. Limited by the model parameter scale and performance, they have not been able to provide accurate spatial representations for smart contracts.

[0004] In view of the above problems, how to provide an accurate spatial representation for smart contracts has become a technical problem that people urgently need to solve. Summary of the Invention

[0005] In view of the above technical problems in the related art, the present invention proposes a model method for representing smart contracts. Driven by the deep neural network technology of Transformer, SmartBERT is designed and trained to generate accurate smart contract representations, which can overcome the above deficiencies of the prior art.

[0006] To achieve the above technical objectives, the technical solution of the present invention is realized as follows:

[0007] A model method for representing smart contracts includes the following steps:

[0008] S1 Collect a large number of open-source smart contract source codes on the blockchain network;

[0009] S2 Clean the smart contract source codes and generate a structured code tree;

[0010] S3 Design a pre-trained model based on a deep neural network and train it using at least more than ten thousand smart contract codes;

[0011] The pre-trained model is designed based on a deep neural network with a Transformer architecture. In the data structure output by the model:

[0012] During the training phase, the model outputs the probability distribution of each token in the vocabulary, and selects the token ID with the highest probability as the prediction result. During the application phase, the LM Head layer of the model is removed, and a Pooler output layer is added. The output data is the representation vectors of all tokenized tokens in the input code segment, and after average pooling, it outputs the representation vector of the code segment, with the data type being floating-point.

[0013] S4 inputs the smart contract code in the training set into the pre-trained model for learning;

[0014] S5 disassembles the input layer, encoding layer, and output layer in the pre-trained model to form the SmartBERT model;

[0015] The specific steps of S5 are as follows:

[0016] S501 removes the language model head layer in the training phase, only retains the input layer and encoding layer, and adds an output layer;

[0017] The input layer of S502 converts the input integer sequences of token ID, position, and type into n-dimensional floating-point word vectors. The encoding layer contains the multi-head self-attention encoding layer of the Transformer, which internally includes residual connections, normalization layers, fully connected layers, and Dropout layers;

[0018] The output layer of S503 is a fully connected layer, which outputs the representation vector through the Tanh activation function;

[0019] S6 calls the SmartBERT model to input the smart contract source code to generate a representation.

[0020] Furthermore, the specific steps of step S1 are as follows: Batch download the open-source smart contract source code in the blockchain public chain network, save the downloaded smart contract source code to the MySQL database, set an auto-increment primary key ID in the database, and use the unique address of the smart contract on the blockchain as the index of the table.

[0021] Furthermore, the specific steps of step S2 are as follows: Remove the version declarations and code comments from the collected smart contract source code, and then perform structured processing on the code to generate a tree-shaped data structure diagram of the code. When encountering a smart contract composed of multiple files, multiple code files need to be merged into one smart contract.

[0022] Furthermore, the specific steps of generating the tree-shaped data structure diagram of the code are as follows:

[0023] S201 uses the contract, abstract contract, library, and interface keywords of the smart contract to split the contract-level code segments into the first level of the tree using regular expressions;

[0024] S202 traverses the code segments at the first level and further splits the function-level code segments into the leaf nodes of the tree, i.e., the second level, using the function, event, modifier, and constructor keywords of the smart contract;

[0025] S203 finally saves the function-level code segments of the smart contract functions that actually require the input model in the leaf nodes.

[0026] Furthermore, the basic structure of the pre-trained model in step S3 includes an input layer, an encoding layer, an output layer, and a language model head required for the training phase; the data structure of the model input is a token encoding sequence, a token position sequence, and a token type sequence, and the data types are all integer types; where token is the word segmentation of a long text segment, and the word segmentation tool uses tokenizer, which splits the long text of the smart contract code segment into words and then maps them into the data structure consistent with the model input according to the word table.

[0027] Furthermore, the specific steps of step S4 are as follows:

[0028] S401 The code tree uses tokenizer to generate three sequences of token ID, token position, and token type for the function-level code segments of the leaf nodes of the code tree, and masks some of the token IDs. The ID of the masked token corresponds to <mask>Symbols, represented by a predefined numerical code, form the training set;

[0029] The labeled dataset output by S402 is the original token ID sequence without masking. The cross-entropy loss function is used to calculate the loss value between the predicted token ID and the true label token ID, and the model is trained in batches and multiple times to make the model predict the masked data. Among them, before MLM training, the code segment is tokenized, and the code segment is split into tokens using the tokenizer; during MLM training, a part of the tokens in the tokenized code segment are randomly masked.

[0030] Furthermore, the specific steps for step S6 to call the SmartBERT model to input the smart contract source code to generate a representation are as follows:

[0031] S601 Download or write any new smart contract code, use the algorithm in step S2 to generate the code tree of the smart contract, and input the content of all leaf nodes in order, that is, all function-level code segments. The model generates a representation vector for each code segment;

[0032] S602 Combine the vectors of all leaf node code segments into a matrix in the context order; assume that the representation generated by the pre-trained model has n dimensions and the number of leaf nodes of the smart contract code tree is m, then the shape of the finally output smart contract representation matrix should be: A m×n , m ∈ R, n ∈ R, that is, a matrix with m rows and n columns. At this time, A is the mathematical representation of the smart contract.

[0033] Advantages of the present invention: The present invention constructs the SmartBERT model by disassembling the input layer, encoding layer, and output layer in the pre-trained model. In the application stage of SmartBERT, when the user inputs the smart contract code, a mathematical matrix representing the entire smart contract can be generated. This mathematical matrix can more accurately capture the context semantics and structural information of the smart contract, providing strong support for subsequent downstream tasks related to smart contracts.

[0034] More accurate smart contract representation: Compared with general language pre-trained models, SmartBERT has higher representation accuracy in the field of smart contracts. SmartBERT is specially designed and trained to deeply understand the specific semantics and structures of smart contracts, so it can generate more accurate mathematical representations. This feature is crucial for the accuracy of smart contract-related tasks, such as smart contract detection and generation.

[0035] Flexibility of retraining: SmartBERT designed based on deep neural networks has the ability to be retrained. By inputting more smart contract data, the maturity and adaptability of the model can be improved. This flexibility enables the model to continuously adapt to the evolving smart contract field without being restricted by time and technological changes. This is very important for the evolving smart contract ecosystem and helps maintain the practicality and utility of the model.

[0036] Flexibility of transfer learning: As a pre-trained model, SmartBERT can be flexibly applied to various downstream tasks in the smart contract field. Through transfer learning, the model can apply the knowledge and representations learned from large-scale data to specific smart contract tasks, thereby improving the efficiency and accuracy of the tasks. This flexibility gives SmartBERT broad application potential in the smart contract ecosystem. Description of the Drawings

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 It is a basic step diagram of the model method for characterizing smart contracts according to the embodiments of the present invention;

[0039] Figure 2 It is a process diagram of smart contract word segmentation of the model method for characterizing smart contracts according to the embodiments of the present invention;

[0040] Figure 3 It is a structural diagram of the SmartBERT model of the model method for characterizing smart contracts according to the embodiments of the present invention;

[0041] Figure 4 It is a visualization diagram of the regular expression for extracting the contract level of the smart contract in the generated code tree according to the embodiments of the present invention;

[0042] Figure 5 It is a visualization diagram of the regular expression for extracting the function level of the smart contract in the generated code tree according to the embodiments of the present invention;

[0043] Figure 6 It is a visualization example diagram of the smart contract code tree according to the embodiments of the present invention;

[0044] Figure 7 It is a call example diagram of using the SmartBERT API according to the embodiments of the present invention. Detailed implementation manners

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0046] This patent invents a Transformer-based pre-trained model called SmartBERT. The training dataset and the generated representation vector structure of SmartBERT are designed specifically for smart contracts, aiming to provide a more accurate method for representing smart contracts.

[0047] Similar to BERT, the model structure of SmartBERT also relies on multiple layers of Transformer encoders and the Masked Language Model (MLM) task. The difference is that its input is the entire smart contract, and the output is a mathematical matrix representing the entire smart contract, rather than a simple statement representation vector. The training stage of SmartBERT includes the following steps: First, download and store a large number of smart contract source codes in a local database. Then, perform structured processing on the smart contract source codes to generate a code tree, where the leaf nodes of the tree correspond to the function-level code segments in the contract. After that, randomly mask some tokens in the input code segments using MLM and input the processed data into the model to predict the masked tokens. Finally, split the fully trained pre-trained model to obtain the input layer, encoding layer, and output layer that are truly used for generating representations, that is, to form SmartBERT. In the application stage of SmartBERT, when a user inputs a smart contract code, a mathematical matrix representing the entire smart contract can be generated. This mathematical matrix can more accurately capture the context semantics and structural information of the smart contract, providing strong support for subsequent downstream tasks related to smart contracts, such as code detection and generation.

[0048] As Figure 1 shown, a model method for representing smart contracts according to the embodiments of the present invention, the model is named SmartBERT, and a pre-trained model is constructed based on a Transformer deep neural network. In the training stage, a large number of smart contract source codes are input to enable the model to learn the context semantics of smart contracts; in the application stage, new smart contract codes are input into the fully trained SmartBERT model to obtain its mathematical representation. The representation generated by SmartBERT can be applied to various downstream tasks related to smart contracts, such as the detection and generation of smart contract codes. The method described in this patent is divided into the following main steps:

[0049] (1) Collect a large number of open-source smart contract source codes on the blockchain network;

[0050] (2) Clean the smart contract source codes and generate a structured code tree;

[0051] (3) Design a pre-trained model based on a deep neural network;

[0052] (4) Input the smart contract codes in the training set into the pre-trained model for learning;

[0053] (5) Disassemble the input layer, encoding layer, and output layer in the pre-trained model to form SmartBERT;

[0054] (6) Call the SmartBERT model to input the smart contract source code to generate a representation.

[0055] In step (1), collect a large number of open-source smart contract source codes on the blockchain network: Batch download the open-source (verified) smart contract source codes in the current mainstream blockchain public chain networks. The codes are generally written in Solidity language or Vyper language. The websites providing smart contract code downloads include but are not limited to the following blockchain browser websites: (1) https: / / bscscan.com (2) https: / / etherscan.io. Write scripts in general programming languages to batch download smart contract source codes from the above websites, or use the APIs officially provided by the above websites to download the source codes, and download at least tens of thousands of non-repeating smart contracts. Store the downloaded smart contract source code files and their related information in a local database, use an auto-incrementing ID as the primary key, and the smart contract's on-chain address as the unique index.

[0056] In step (2), clean the smart contract source codes and generate a structured code tree: Remove the version declaration (pragma) and code comment (comment) parts from the smart contract source codes collected in step (1). Then perform structured processing on the codes to generate a tree-shaped data structure diagram of the codes. For smart contracts composed of multiple files, the multiple code files need to be merged into one smart contract before performing the above steps. Among them, the process of generating the code tree is as follows: First, use the contract, abstract contract, library, and interface keywords of the smart contract to split the contract-level code segments into the first level of the tree using regular expressions; then traverse the code segments at the first level, and further split the code segments at the function, event, modifier, and constructor keywords of the smart contract into function-level code segments as the leaf nodes of the tree, that is, the second level; finally, the code segments at the function level of the smart contract actually required to be input into the model are saved in the leaf nodes.

[0057] The step (3) designs a pre-training model based on a deep neural network: designs a pre-training model based on the Transformer encoder. The basic structure of the model includes: an input layer (Embedding Layer), an encoding layer (TransformerEncoder), an output layer (Pooler Layer), and a language model head (LM Head) required in the training phase. The data structure of the model input is a token encoding sequence (token ID), a token position sequence (token position), and a token type sequence (token type), and the data types are all integers. A token is a word segmentation of a long text segment, and the tokenizer tool is used as the word segmentation tool. The tokenizer splits the long text of the code segment of the smart contract into words and then maps them into the data structure consistent with the model input according to the vocabulary. The data structure of the model output is different in the training and application phases: in the training phase, the model outputs the probability distribution of each token in the vocabulary, and the token ID with the highest probability is selected as the prediction result; during application, the LMHead layer of the model is removed, and a Pooler output layer is added. The output data is the representation vector of all the word segments (tokens) of the input code segment, and after average pooling, it outputs the representation vector of the code segment, and the data type is floating point.

[0058] The step (4) inputs the smart contract code in the training set into the pre-training model for learning: cleans and generates a code tree for the smart contract code obtained in steps (1) and (2), uses the tokenizer to generate three groups of sequences of token ID, token position, and token type for the function-level code segments at the leaf nodes of the code tree, and then masks some of the token IDs. Generally, the ID of the masked token corresponds to <mask>Symbols are represented by a pre - agreed numerical code. Repeat the above steps for at least ten thousand smart contracts to form a training dataset. The output labeled dataset is the original token ID sequence without masking. The cross - entropy loss function is used to calculate the loss value between the predicted token ID and the true label token ID. The model is trained in batches and multiple times on a server equipped with a GPU or an acceleration card so that the model can predict the masked data. Specifically, during the MLM process, the masking algorithm is dynamic, and the masked part will randomly change for each training input, and the masked coverage rate is 15%; 80% of the masked token IDs are replaced by <mask>Symbols, 10% are replaced with other tokens, and 10% remain the original tokens unchanged. In addition, the training set of the model should ensure data diversity, and there are no duplicate smart contract code segments in the dataset.

[0059] In step (5), the input layer, encoding layer, and output layer in the pre-trained model are disassembled to form SmartBERT: The final form of the SmartBERT model is a well-trained pre-trained model. First, we remove the language model head (LM Head) layer in the training phase, only retain the input layer (Embedding Layer), encoding layer (TransformerEncoder), and add an output layer (PoolerLayer). In this structure, the input layer (Embedding Layer) converts the input integer sequence of token ID, position, and type into an n-dimensional floating-point word vector. The encoding layer contains the multi-head self-attention encoding layer of Transformer, which internally includes residual connections, normalization layer (NormalizationLayer), fully connected layer (Dense Layer), and Dropout layer, etc. The encoding layer is stacked 12 layers or more. Finally, the output layer is a fully connected layer that outputs a representation vector through the Tanh activation function.

[0060] In step (6), call the SmartBERT model to input the smart contract source code to generate its representation: Download or write any new smart contract code, and use the algorithm described in step (2) to generate the code tree of the smart contract. Input the content of all leaf nodes in sequence, that is, all function-level code segments, and the model generates a representation vector for each code segment. Then, combine the vectors of all leaf node code segments into a matrix in the context order. Assume that the representation generated by the pre-trained model has n dimensions, and the number of leaf nodes in the smart contract code tree is m, then the shape of the final output smart contract representation matrix should be: A m×n , m ∈ R, n ∈ R, that is, a matrix with m rows and n columns. At this time, A is the mathematical representation of this smart contract.

[0061] To facilitate the understanding of the above technical solution of the present invention, the above technical solution of the present invention will be described in detail below through specific usage methods.

[0062] In specific use, according to a model method for representing smart contracts described in the present invention, as Figure 1 shown, it is divided into a training phase and an application phase:

[0063] Training phase:

[0064] 1. Data collection: First, collect a large number of open-source smart contract source codes from the blockchain network.

[0065] 2. Data cleaning and structuring: Clean the source code of smart contracts and then generate a structured code tree so that the model can better understand the organizational structure of the contracts.

[0066] 3. Pre-trained model design: Design a pre-trained model for smart contracts based on deep neural networks, which will be a core component of SmartBERT.

[0067] 4. Model training: Input the smart contract code in the training set to train the pre-trained model so that it can understand and represent the semantic and structural information of smart contracts.

[0068] Application stage:

[0069] 5. SmartBERT construction: In the application stage, disassemble the trained pre-trained model to obtain the input layer, encoding layer, and output layer respectively, and form the SmartBERT model.

[0070] 6. Representation generation: Invoke the SmartBERT model, input the source code of smart contracts, and generate corresponding smart contract representations, which are mathematical matrices that can accurately express the semantic and structural features of smart contracts.

[0071] To ensure the accuracy and effectiveness of the application stage, this method emphasizes fully training the model in the training stage. At least more than ten thousand pieces of smart contract code should be used for training to ensure that the SmartBERT model can show highly accurate and robust characteristics in the application stage and provide strong support for downstream tasks in the field of smart contracts.

[0072] The specific implementation is as follows:

[0073] After unsupervised learning on a large number of smart contract codes, SmartBERT can understand the context semantics of smart contract texts and generate accurate representations of them.

[0074] The first step: Data collection and storage

[0075] In the first step, we widely collect the source code of smart contracts by using blockchain browsers (such as etherscan and bscscan). Subsequently, we save the downloaded source code of smart contracts and related information to a MySQL database. We set an auto-increment primary key ID in the database and use the unique address of the smart contract on the blockchain as the index of the table.

[0076] The second step: Data cleaning and structuring

[0077] We first clean the downloaded source code of smart contracts. This step includes the following sub-steps:

[0078] 1. Duplication removal: Delete duplicate smart contract source codes to ensure data uniqueness.

[0079] 2. Merging multi - file contracts: Merge the source codes of multi - file smart contracts into a single file.

[0080] 3. Removing redundant and irrelevant codes: Delete codes that are irrelevant to the functional logic, such as the pragma version declaration at the contract header and all comment annotations.

[0081] The cleaned code is structured and a code tree of the smart contract is generated. This process mainly uses regular expressions. The construction of the smart contract code tree includes the following two main steps:

[0082] 1. Extracting contract - level code segments: We use the following regular expression to extract contract - level code segments. For the visualization of the regular expression, refer to Figure 4 。

[0083] / (^|\s|)(contract|interface|library|abstract\s*contract)[^;{}]*{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{(?:[^{}]+|{[^{}]*})*})*})*})*})*})*} / gm

[0084] 2. Extracting function - level code segments: For the contract - level code segments extracted in the previous step, we use the following regular expression to further extract function - level code segments. For the visualization of the regular expression, refer to Figure 5 。

[0085] / (^|\s|)(function|event|modifier|constructor)[^{};]*({(?:[^}{]+|{(?:[^}{]+|{(?:[^}{]+|{(?:[^}{]+|{(?:[^}{]+|{(?:[^}{]+|{(?:[^}{]+|{[^}{]*})*})*})*})*})*})*}|;) / gm

[0086] These extracted code segments are mapped according to the hierarchical structure as Figure 6 An example of the code tree shown. The first layer of the tree represents the smart contract file, which contains only one root node. The second layer represents the contract-level code segment, and the third layer represents the function-level code segment. The nodes are leaf nodes. Each leaf node actually stores the text content to be input into the model.

[0087] Step 3: Design the pre-trained model

[0088] The pre-trained model is designed as a deep neural network based on the Transformer architecture. The model has different structures during the training phase and the application phase. In the present invention, we will specifically discuss SmartBERT, which refers to the model that has been fully trained during the application phase.

[0089] During the training phase, the pre-trained model includes the following three main layers:

[0090] 1. Input Layer (Embedding Layer): Converts text tokenization into a high-dimensional vector representation so that the model can understand and process text data. The input data is a sequence of token IDs, token positions, and token types generated by a tokenizer, and the data type is integer. In the input layer, the data is first transformed by the embedding layer and then passed to the normalization layer and the dropout layer.

[0091] 2. Encoding Layer (Transformer Encoder): Implements a bidirectional multi-head self-attention mechanism, including self-attention layers, multiple fully connected layers, normalization layers, dropout layers, and residual connections with the output data of the previous layer. The encoding layer structure is stacked into multiple layers, usually 12 layers or more.

[0092] 3. Language Model Head (LM Head Layer): Outputs the probability distribution of each predicted token in the vocabulary, and the token with the highest probability is the final predicted target token. The structure of the LM Head includes a fully connected layer, a normalization layer, and a linear transformation layer (decoder).

[0093] During the application phase, some structural adjustments are made to the model structure:

[0094] 1. Remove the Language Model Head (LM Head Layer): During the application phase, there is no longer a need to predict the next token, so the language model head can be removed.

[0095] 2. Add an output layer (Pooler Layer): Its function is to output the word vector representation of the entire text. Its structure includes a fully connected layer and a Tanh activation function.

[0096] In addition, there is some data processing logic at both ends of the model input and output, including using a tokenizer to tokenize the text at the input end and mapping the relationship between text words and integer numbers, referring to Figure 2 the tokenization process of smart contracts. At the output end, average pooling is performed on the word vectors to generate a sentence-level representation vector, and then multiple sentences are combined into a complete representation smart contract matrix, or inserted into the code tree to generate a tree-structured smart contract representation.

[0097] Step 4: Model training

[0098] In the model training stage, we first batch-read the function-level code segments of the leaf nodes in the code tree in the second step. To ensure the full training of the model, we obtained at least ten thousand function-level code segments under different smart contracts and used them as the training dataset to input into the model described in the third step for training on the MLM (Masked Language Modeling) task. The process refers to Figure 1 the training stage in

[0099] Before MLM training, we first tokenize the code segments, using the tokenizer to split the code segments into tokens. During MLM training, we randomly mask a part of the tokens in the tokenized code segments. Specifically, we perform a masking operation on 15% of the tokens. Among these 15%: 80% of the tokens are replaced with special tokens in the vocabulary <mask>; 10% of the tokens remain with their original token IDs unchanged; 10% of the tokens are replaced with random tokens from the vocabulary. To enhance the model's understanding of the context of smart contract texts, we adopted the method of dynamic masking. This means that in each training batch of the model, 15% of the tokens to be randomly masked are dynamically updated.

[0100] The output of the model is the probability distribution of each token in the vocabulary, and the probability is calculated using softmax. The formula is as follows:

[0101]

[0102] softmax(z) i represents the i-th element of the probability distribution, that is, the probability of a certain word in the vocabulary;

[0103] z i is the i-th element of the vector z which is the output of the LM Head;

[0104] K is the total number of tokens in the vocabulary.

[0105] We then obtain the predicted token ID sequence according to the token with the maximum probability in this distribution. Compare the predicted token ID sequence with the original token ID sequence input to the model, and calculate the loss value using the cross-entropy loss function shown in the following formula. represents the predicted token ID sequence of the model, and y represents the actual token ID sequence. Through the built-in backpropagation algorithm of the neural network, optimize the model parameters to reduce the loss value. The smaller the loss value, the better the model.

[0106]

[0107] We repeat the above steps for the function-level code segments of ten thousand smart contracts to ensure that the model is fully trained.

[0108] Step 5: Build the SmartBERT model

[0109] To build the SmartBERT model, only need to split and reorganize the pre-trained model that has been fully trained in the previous step, remove the LM Head layer, and then add a Pooler output layer at the end of the model. The Pooler layer contains a fully connected layer, and the activation function used is tanh.

[0110] Step 6: Invoke the SmartBERT model

[0111] Call the SmartBERT model. According to the data format input during training, first use the algorithm of the code tree to split all function-level code segments from the newly input smart contract. Then use the tokenizer to tokenize the code segments to obtain sequences of token ID, token position, and token type, and input them into SmartBERT. At the output end of SmartBERT, word vectors of all tokens are obtained, and then average pooling is performed on all words to calculate the average value of all token vectors to represent the complete function-level code segment. After completing the above process for all function code segments of the smart contract, matrix A is obtained. m×n , m ∈ R, n ∈ R. This matrix has m rows and n columns. m is equal to the number of function-level code segments included in the smart contract, and n is the representation dimension set by the model. By default, n = 768. At this time, A is the mathematical representation of this smart contract. In the actual production environment, the present invention provides an API of SmartBERT. By requesting this service through the HTTP protocol, the representation matrix of the smart contract can be obtained. The usage effect of the API is as Figure 7 shown. It is required to send the complete smart contract code text, and then the JSON data returned by the server side is obtained. Its representation matrix is distributed in the code tree structure of the smart contract.

[0112] In summary, by means of the above technical solution of the present invention, the SmartBERT model is constructed by disassembling the input layer, encoding layer, and output layer in the pre-trained model. In the application stage of SmartBERT, when the user inputs the smart contract code, a mathematical matrix representing the entire smart contract can be generated. This mathematical matrix can more accurately capture the context semantics and structural information of the smart contract, providing strong support for subsequent downstream tasks related to smart contracts.

[0113] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.< / mask> < / mask> < / mask> < / mask>

Claims

1. A model method for characterizing smart contracts, characterized in that, It includes the following steps: S1: Collect a large number of open-source smart contract source codes on the blockchain network; S2: Clean the smart contract source codes and generate a structured code tree; S3: Design a pre-training model based on a deep neural network, and train it using at least more than ten thousand smart contract codes; The pre-training model is designed based on a deep neural network with a Transformer architecture. Among them, in the data structure output by the model: In the training stage, the model outputs the probability distribution of each token in the vocabulary, and selects the token ID with the highest probability as the prediction result; in the application stage, the LM Head layer of the model is removed, and a Pooler output layer is added. The output data is the representation vector of all tokenized tokens in the input code segment, and after average pooling, it outputs the representation vector of the code segment, and the data type is floating-point; S4: Input the smart contract codes in the training set into the pre-training model for learning; S5: Decompose the input layer, encoding layer, and output layer in the pre-training model to form the SmartBERT model; The specific steps of S5 are as follows: S501: Remove the language model head layer in the training stage, only retain the input layer and encoding layer, and add an output layer; S502: The input layer converts the input integer sequence of token ID, position, and type into an n-dimensional floating-point word vector; the encoding layer contains the multi-head self-attention encoding layer of Transformer, which internally contains a residual connection, a normalization layer, a fully connected layer, and a Dropout layer; S503: The output layer is a fully connected layer, and outputs a representation vector through the Tanh activation function; S6: Call the SmartBERT model to input the smart contract source code to generate a representation.

2. The model method for characterizing a smart contract according to claim 1, wherein The specific steps of step S1 are as follows: Batch download the open-source smart contract source codes in the blockchain public chain network, save the downloaded smart contract source codes to the MySQL database, set an auto-increment primary key ID in the database, and use the unique address of the smart contract on the blockchain as the index of the table.

3. The model method for characterizing a smart contract according to claim 1, wherein The specific steps of step S2 are as follows: Remove the version declarations and code comments from the collected smart contract source codes, and then perform a structured process on the codes to generate a tree-shaped data structure diagram of the codes. When encountering a smart contract composed of multiple files, multiple code files need to be merged into one smart contract.

4. The model method for characterizing a smart contract according to claim 3, wherein The specific steps of generating the tree-shaped data structure diagram of the codes are as follows: S201: Use the contract, abstract contract, library, and interface keywords of the smart contract to split the contract-level code segments into the first level of the tree using regular expressions; S202: Traverse the code segments at the first level, and further split the function-level code segments using the function, event, modifier, and constructor keywords of the smart contract as the leaf nodes of the tree, that is, the second level; S203: Finally, the code segments at the function level of the smart contract required for actual input into the model are saved in the leaf nodes.

5. The model method for characterizing a smart contract according to claim 1, wherein The basic structure of the pre-trained model in step S3 includes an input layer, an encoding layer, an output layer, and a language model head required in the training phase; the data structure input to the model is a token encoding sequence, a token position sequence, and a token type sequence, and the data types are all integers; among them, a token is a word segmentation of a long text segment, and the word segmentation tool uses a tokenizer. The tokenizer splits the long text of the smart contract code segment into words, and then maps them into a data structure consistent with the model input according to the vocabulary table.

6. The model method for characterizing an intelligent contract according to claim 1, wherein The specific steps of step S4 are as follows: The S401 code tree uses a tokenizer to generate three sequences of token ID, token position, and token type for the function-level code segments of the leaf nodes of the code tree, masks some of the token IDs, and the ID of the masked token corresponds to <mask>Symbols, represented by a predefined numerical encoding, form the training set; < / mask> The labeled dataset output by S402 is an unmasked original token ID sequence. The cross-entropy loss function is used to calculate the loss value between the predicted token ID and the true label token ID, and the model is trained batch by batch and multiple times to make the model predict the masked data; among them, before MLM training, the code segment is tokenized, and the code segment is split into tokens using a tokenizer; during MLM training, a part of the tokens in the tokenized code segment are randomly masked.

7. The model method for characterizing a smart contract according to claim 1, wherein The specific steps of step S6 for calling the SmartBERT model to input the smart contract source code to generate a representation are as follows: S601 Download or write any new smart contract code, use the algorithm in step S2 to generate the code tree of the smart contract, input the content of all leaf nodes in sequence, that is, all function-level code segments, and the model generates a representation vector for each code segment; S602 combines the vectors of all leaf node code segments into a matrix in the context order; assuming that the representation generated by the pre-trained model has n dimensions and the number of leaf nodes of the smart contract code tree is m, the shape of the finally output smart contract representation matrix should be: A m×n , m ∈ R, n ∈ R, that is, a matrix with m rows and n columns. At this time, A is the mathematical representation of this smart contract.

Images