Cross-modal pre-training model-based smart contract similarity detection method

Abstract

The present invention provides a cross-modal pre-training model-based smart contract similarity detection method, comprising: constructing a control flow graph of a smart contract; extracting comment information from the smart contract, and enabling the control flow graph to correspond to the comment information so as to acquire a corresponding graph; inputting information of the corresponding graph into a preset cross-modal pre-training model so as to acquire a feature representation vector of the contract; and, on the basis of the feature representation vector, determining a contract similarity by means of a threshold value. The present invention can implement high-precision smart contract semantic similarity detection.

Description

Smart contract similarity detection method based on cross-modal pre-training model Technical Field

[0001] The present invention belongs to the technical field of blockchain smart contract security detection, and in particular relates to a smart contract similarity detection method based on a cross-modal pre-training model. Background Art

[0002] With the rapid development of blockchain technology, smart contracts have been widely used in digital payments, financial assets, the internet, supply chains, and other fields. In recent years, the number of smart contracts in Ethereum has continued to grow, bringing significant economic benefits while also posing significant security risks. Due to the open-source nature of smart contracts, code cloning has become a common phenomenon in contract development. While contract cloning improves blockchain application development efficiency, it also has serious negative consequences, including code base redundancy and the risk of contract vulnerabilities spreading. Therefore, research on the security of smart contracts is crucial. Breakthroughs in similarity detection technology can help address the various hazards (security vulnerabilities, malicious code) posed by contract cloning, ensuring the security and trustworthiness of digital assets.

[0003] Currently, code similarity detection focuses on traditional programming languages, while semantic similarity detection for smart contracts remains challenging. Existing detection technologies have achieved good results for textual similarity at the source code level. However, facing the challenge of semantic similarity detection at a higher level of abstraction, effectively mining the deep semantic information of contract source code has become a pressing issue for smart contract source code similarity detection. With the advancement of deep learning and natural language processing technologies, the use of neural network models to learn program feature representations has been applied to many code similarity detection tasks. However, research on cross-modal semantic similarity for blockchain smart contracts is still in its infancy.

[0004] In this context, this paper proposes a similarity detection method based on a cross-modal pre-trained model. Specifically, it uses the Masked Multimodal Attention (MMA) mechanism to help adjust the contract weight by annotating the modal information. A similarity detection scheme based on a cross-modal pre-trained model (BERT) is proposed.

[0005] Summary of the Invention

[0006] To address the aforementioned technical problem of a single technical means for smart contract similarity detection, the present invention provides a smart contract similarity detection solution based on a cross-modal pre-training model. Through a fusion training process of multimodal contract semantic information, a more comprehensive similarity feature of smart contracts is obtained. This technology interacts with the CFG (Control Flow Graph) modality through semantic annotations of the source code, generates a control flow graph by decompiling the bytecode, and converts the CFG graph features into a vector representation through a cross-modal pre-training model. Combined with the annotation information in the source code, a neural network is used to capture semantic information and program-wide control flow information, thereby achieving high-precision smart contract semantic similarity detection.

[0007] To achieve the above objectives, the present invention proposes a smart contract similarity detection method based on a cross-modal pre-training model, comprising:

[0008] Build the control flow graph of the smart contract;

[0009] Extracting annotation information from the smart contract, matching the control flow graph with the annotation information, and obtaining a corresponding graph;

[0010] Input the information in the corresponding graph into a preset cross-modal pre-training model to obtain a feature representation vector of the contract;

[0011] Based on the feature representation vector, the contract similarity is determined by using a threshold.

[0012] Optionally, constructing the control flow graph includes:

[0013] Obtaining assembly instructions for the smart contract source code;

[0014] The assembly instructions are converted into preset elements, and the control flow graph is generated based on the preset elements; wherein the preset elements include: a block set and an edge set, the block set is a set of node instructions in the graph, and the edge set is a combination of the starting point, end point and type of the edge.

[0015] Optionally, obtaining the assembly instructions for the smart contract source code includes:

[0016] Compile the smart contract source code to obtain a bytecode file;

[0017] The bytecode file is decompiled to generate the assembly instructions.

[0018] Optionally, the information in the corresponding graph includes: graph information, block instruction set, edge attributes, timing information, contract annotation information and function name; wherein the graph information includes: starting point, edge type and end point.

[0019] Optionally, the preset cross-modal pre-training model includes: annotation features of CFG event sequence and function alignment;

[0020] Inputting the information in the corresponding graph into the preset cross-modal pre-training model includes

[0021] Generate events based on the information in the corresponding graph;

[0022] The generated event sequence is passed through the output of the last encoder layer of the cross-modal pre-trained model as the contract feature;

[0023] After performing one-dimensional convolution processing on the contract features, expansion processing is performed;

[0024] The expanded features are input into the masked multimodal attention mechanism to adjust the contract weight by combining the contract performance under different modalities;

[0025] A residual connection is used on the output of the masked multimodal attention mechanism and the contract feature to maintain the original structure of the data. After passing through a linear layer and a normalization layer, the output of the last linear layer, i.e., the feature representation vector, is obtained.

[0026] Optionally, determining contract similarity by using a threshold includes:

[0027] The feature representation vector is mapped within [0, 1] through a mapping function, and the contract similarity is determined by a threshold.

[0028] Optionally, generating an event from the information in the corresponding graph includes:

[0029] The information between nodes is updated and generated into events by using the temporal relationship of the edges in the input data. Each event has four features: the source node v l , target node v j 、Event type e ij , and timing information t ij ;

[0030] Construct the event by the time sequence of the event, and convert the source node v l , target node v j 、Event type e ij , and timing information t ij After modeling by pre-set connections and generating event codes, a series of events with timestamps are obtained in chronological order.

[0031] The preset connection is:

[0032] Among them, || represents the splicing operation, and Represents node v l and node v j The hidden state at time n is represented by represents the hidden state represented by the edge connecting each pair of nodes, represents the hidden state of temporal information, Represents events between nodes.

[0033] Optionally, the output of the masked multimodal attention mechanism is: X Att =W m V m

[0034] Among them, X Att represents the attention matrix, i.e., the output of the masked multimodal attention mechanism, V m represents the output of the last encoder layer of the cross-modal pre-training model, W m represents the multimodal attention matrix;

[0035] The multimodal attention matrix W m is: W m =Softmax(W f +M)

[0036] Among them, W f represents the weighted fusion attention matrix, M represents the mask matrix;

[0037] The weighted fusion attention matrix W f is: W f =w s *α s +w n *β n +b

[0038] Among them, w s and w n represents the weight difference between the contract and annotation modes, b represents the deviation, α s represents the contract attention matrix, β n represents the annotation attention matrix.

[0039] Optionally, the one-dimensional convolution process is:

[0040] Among them, k {t,a} Represents the convolution kernel size of the contract and annotation, X n represents annotation features, X s Represents the contract characteristics, Represents the contract features after one-dimensional convolution, Represents the annotation features after one-dimensional convolution, Conv1D represents a one-dimensional convolutional neural network;

[0041] The extension process is as follows:

[0042] in, Represents the extended contract features, Represents an expanded annotation feature.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] The present invention proposes a smart contract similarity detection method based on a cross-modal pre-training model. First, the cross-modal pre-training model BERT is trained using CFG graph information and annotation information. By using Mask Language Model (MLM), the result obtained by model encoding can also contain contextual information of the context, which can effectively represent the information in the CFG graph and the annotation information. Secondly, the Masked Multimodal Attention mechanism MMA (Masked Multimodal Attention) is used to help the contract modality adjust the contract weight, and a multimodal invisible alignment method is designed to obtain a general semantic representation of the contract source code. Finally, the general contract semantic representation output by bimodal pre-training is integrated into the contract source code representation to improve the similarity detection capability of the smart contract source code. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0046] FIG1 is a schematic diagram of a smart contract data processing flow according to an embodiment of the present invention;

[0047] FIG2 is a diagram showing the correspondence between CFG and annotations according to an embodiment of the present invention;

[0048] FIG3 is a CFG event generation diagram according to an embodiment of the present invention;

[0049] FIG4 is a framework diagram of the cross-modal pre-training model BERT according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0051] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0052] The purpose of this embodiment is achieved as follows: First, the bytecode file needs to be decompiled. The decompiled bytecode generates EVM instructions (i.e., opcodes) and corresponding parameters. Based on the decompiled EVM instructions, the control flow graph (CFG) is constructed using the Octopus tool. Secondly, the annotation information in each contract is extracted, and the CFG graph and its contract annotation information are mapped at the function level. The cross-modal pre-trained model BERT is trained on the CFG graph and annotation information. The CFG graph representation information and annotation information are used as input, and the cross-modal BERT encoder converts both features into vector representations. The masked multimodal attention mechanism (MMA) is used to help the contract modality adjust the contract weight and obtain the semantic features of the contract. Finally, the contract semantic features are represented as vectors, and the calculated values ​​are mapped between [0, 1] using a mapping function. The threshold for whether the contract graphs are similar is determined by determining the calculated values ​​of each graph vector, ultimately obtaining the contract graph similarity threshold.

[0053] The overall overview of the smart contract similarity detection method based on the cross-modal pre-training model includes the following steps:

[0054] First, decompilation and CFG construction. The decompiled bytecode will generate EVM instructions (i.e., opCode) and corresponding parameters, and then the control flow graph CFG will be reconstructed based on the decompiled EVM instructions.

[0055] Second, extract the comments in the contract and map the CFG graph to the comments in units of functions.

[0056] Third, the information in the CFG graph and the contract annotation information are input into the cross-modal pre-training model, and the feature representation vector of the contract is obtained after processing by the model.

[0057] Fourth, the contract semantic feature vector is mapped to [0, 1] through the mapping function, and the contract similarity is determined by the threshold.

[0058] The smart contract similarity detection method based on the cross-modal pre-training model proposed in this embodiment specifically includes the following steps:

[0059] Data processing flow

[0060] The smart contract source code is compiled using the solc tool to generate a bytecode file. Converting the bytecode to a CFG graph requires two steps: decompiling to generate assembly instructions and using the assembly instructions to build the CFG. Decompiling the bytecode allows for parsing of EVM instructions and their parameters. The Octopus tool is used for decompilation and CFG construction. Octopus first converts the bytecode into an assembly instruction set. Then, it converts the assembly instruction set into the elements required for the CFG graph: the block set and edge set. The CFG file is then generated. The block set is the set of node instructions in the graph. The edge set not only indicates the starting and ending points of the edge, but also the edge type. There are five edge types: unconditional, conditional_true, conditional_false, fallthrough, and call. Figure 1 illustrates the data processing flow of a smart contract.

[0061] For the source code annotation information of the smart contract, the corresponding annotation information in the contract is extracted through a script and mapped to the CFG graph constructed by Octopus. Figure 2 shows the CFG graph and annotation correspondence. The information and annotation information in the CFG graph are extracted and the required data is exported as shown in Table 1. Graph represents the graph information, and its parameters are the starting point, edge type, and end point; Instruction is the instruction set of the block; Attribute represents the edge type; Time represents the timing information; Comment represents the contract annotation information; Function_name represents the function name.

[0062] Table 1: Dataset parameter description

[0063] Cross-modal pre-training model BERT

[0064] By integrating a multimodal pre-trained model with source code control flow graphs and contract annotations, we capture deep semantic information from the contract source code. We also design a multimodal implicit alignment method based on a masked multimodal attention mechanism to obtain a universal semantic representation of the contract source code. Finally, we construct a contract semantic similarity detection model based on deep graph learning, integrating the universal contract semantic representation output by the bimodal pre-trained model into the contract source code representation to improve the similarity detection capabilities of smart contract source code. The cross-modal pre-trained model includes an encoder layer, which encodes the source code and annotations using two encoders, respectively. A one-dimensional convolutional layer is used to keep annotation features and source code features of the same size. A masked multimodal attention mechanism adjusts contract weights by combining contracts from different modalities, while a residual connection is used to preserve the original data structure. A linear layer and a normalization layer are used to generate a linear layer output.

[0065] 1.CFG graph generation event

[0066] For the CFG graph modality, since its graph structure is not suitable for the BERT model input port, it is proposed to generate events from the CFG graph, as shown in Figure 3. In the event generation stage, the information between nodes is updated to generate events using the temporal relationship of the edges in the input data; each event has four features: the source node v l , target node v j 、Event type e ij , and timing information t ij . For the hidden state of temporal information The calculation formula is as follows:

[0067] Here F represents the dimension of the time vector, and f is the Fth dimension. Time2Vec is used to encode time, which is denoted as T2V. T2V uses the first dimension of the time vector as a linear time representation and the others as periodic nonlinear representations. f and is a learnable linear parameter. Construct events by the time sequence of events. Given a source node v l , target node v j 、Event type e ij , and timing information t ij Modeling is done by connecting the following four features, and the calculation formula is as follows:

[0068] Among them, || represents the splicing operation, and Represents node v l and node v j represents the hidden state at time n, and Denotes the hidden state represented by the edge connecting each pair of nodes. After the encoding of the events is generated one after another, a series of events with timestamps are obtained in chronological order.

[0069] 2. Definition of the contract source code multimodal pre-training problem

[0070] Given a CFG graph event sequence G=[g1,g2,…,g n ], where n is the sequence length. Since the embedding layer of the BERT model appends a special classification embedding ([CLS]) to the input sequence and a separator ([SEP]) at the end, the output of the last encoder layer is a sequence of length n+2, expressed as: X s =[E [CLS] ,E1,E2,…,E n ,E [SEP] ] (3)

[0071] To be consistent with the contract model, we append a zero vector before the function alignment annotation feature, and the separator annotation feature is represented as: X n =[A [CLS] ,A1,A2,…,A n ,A [SEP] ] (4)

[0072] Among them A [CLS] is a zero vector, the method aims to use X s and X n The interactions between them are used to adjust each contract in order to better fine-tune the pre-trained BERT model and improve the similarity analysis performance.

[0073] 3. Masked Multimodal Attention Mechanism

[0074] As the core of cross-modal BERT, the Masked Multimodal Attention (MMA) mechanism is designed to use information from the annotation modality to help the contract modality adjust the contract weight and fine-tune the pre-trained BERT model. First, the weight of each contract is evaluated. In the contract modality, the query Q and keyword K are defined as:

[0075] in is the proportionality characteristic of the contract. The query Q of the annotation n and keyword K n is defined as in is the scaled function aligning the annotation features. Contract attention matrix α s and annotation attention matrix β n is defined as: s =Relu(Q s K n T ) (6)

[0076] β n =Relu(Q n K n T ) (7)

[0077] In order to adjust the weight of each word through the interaction between the contract and the annotation, the contract attention matrix α s and annotation attention matrix β n Perform weighted summation to obtain the weighted fusion attention matrix: W f =w s *α s +wn *β n +b (8)

[0078] where w s and w n Indicates the weight difference between the contract and annotation modalities, and b represents the bias. In order to reduce the impact of padding on the sequence, a mask matrix M is introduced, which uses 0 to represent the sample position and -∞ to represent the padding position (after the Softmax function, the attention value of the padding matrix will be changed to 0). Multimodal attention matrix W m Defined as: W m =Softmax(W f +M) (9)

[0079] After obtaining the multimodal attention matrix, W m Multiply by the output V of the last encoder layer of BERT m , which is defined as V m =X s , get the attention matrix X Att : X Att =W m V m (10)

[0080] 4. Contract source code multimodal pre-training model

[0081] As shown in Figure 4, the input to the cross-modal BERT model consists of two parts: the CFG event sequence and the function-aligned annotation features. First, the event sequence is passed through the output of the last encoder layer of the pre-trained BERT model as the contract feature. a The dimension is significantly smaller than the contract feature X s , use a one-dimensional convolutional layer to control it to the same size:

[0082] where k {t,a} Represents the convolution kernel size of the contract and annotation. s The dimension is significantly higher than X n , during the training process, The value will become larger and larger In order to prevent the dot product from becoming larger and push the Softmax function to a very small gradient area, we contract the features Expand to Annotation Features Expand to

[0083] In getting and Finally, in order to make the contract and annotation information fully interactive, they are input into MMA (Masked Multimodal Attention). This attention mechanism can adjust the weight of the contract by combining the contract performance under different modalities. Att After that, in X s and X Att A residual connection is used to maintain the original structure of the data. Then it passes through a linear layer and a normalization layer, and finally the output of the last linear layer is obtained. Y1 = [L [CLS] ,L1,L2,…,L n ,L n+1 ]. Because the first symbol L [CLS] The expression of is learned based on other symbolic information, which is used as the aggregate representation and input into the mapping function to generate the final detection result.

[0084] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A smart contract similarity detection method based on a cross-modal pre-training model, characterized in that: include: Build the control flow graph of the smart contract; Extracting annotation information in the smart contract, matching the control flow graph with the annotation information, and obtaining a corresponding graph; Inputting the information in the corresponding graph into a preset cross-modal pre-training model to obtain a feature representation vector of the contract; Based on the feature representation vector, the contract similarity is determined by a threshold.

2. According to claim 1, the smart contract similarity detection method based on the cross-modal pre-training model is characterized in that: Constructing the control flow graph includes: Obtaining assembly instructions for the smart contract source code; The assembly instructions are converted into preset elements, and the control flow graph is generated based on the preset elements; wherein the preset elements include: a block set and an edge set, the block set is a set of node instructions in the graph, and the edge set is a combination of the starting point, end point and type of the edge.

3. The smart contract similarity detection method based on the cross-modal pre-training model according to claim 2 is characterized in that: The assembly instructions for obtaining the smart contract source code include: Compile the smart contract source code to obtain a bytecode file; The bytecode file is decompiled to generate the assembly instructions.

4. According to claim 1, the smart contract similarity detection method based on the cross-modal pre-training model is characterized in that: The information in the corresponding graph includes: graph information, block instruction set, edge attributes, timing information, contract annotation information and function name; wherein the graph information includes: starting point, edge type and end point.

5. According to claim 1, the smart contract similarity detection method based on the cross-modal pre-training model is characterized in that: The preset cross-modal pre-training model includes: annotation features of CFG event sequence and function alignment; Inputting the information in the corresponding graph into the preset cross-modal pre-training model includes: Generate events based on the information in the corresponding graph; The generated event sequence is passed through the output of the last encoder layer of the cross-modal pre-trained model as the contract feature; The contract features are subjected to one-dimensional convolution processing and then expansion processing; The expanded features are input into the masked multimodal attention mechanism to adjust the contract weight by combining the contract performance under different modalities; A residual connection is used on the output of the masked multimodal attention mechanism and the contract feature to maintain the original structure of the data. After passing through a linear layer and a normalization layer, the output of the last linear layer, i.e., the feature representation vector, is obtained.

6. The smart contract similarity detection method based on the cross-modal pre-training model according to claim 1 is characterized in that: The determination of contract similarity through thresholds includes: The feature representation vector is mapped into [0,1] through a mapping function, and the contract similarity is determined by a threshold.

7. The method for detecting similarity of smart contracts based on a cross-modal pre-training model according to claim 5 is characterized in that: Generating events from the information in the corresponding graph includes: The information between nodes is updated to generate events using the temporal relationship of the edges in the input data. Each event has four features: source node v l 、Target node v j 、Event Type ij , and timing information t ij ; Construct the event by the time sequence of the event, and convert the source node v l 、Target node v j 、Event Type ij , and timing information t ij After modeling through preset connections and generating event codes, a series of events with timestamps are obtained in chronological order; The preset connection is: Among them, || represents the splicing operation, and Respectively represent the node v l and node v j The hidden state at time n is represented by represents the hidden state represented by the edge connecting each pair of nodes, represents the hidden state of the timing information, Represents events formed between nodes.

8. The method for detecting similarity of smart contracts based on a cross-modal pre-training model according to claim 5 is characterized in that: The output of the masked multimodal attention mechanism is: X Att =W m V m Among them, X Att represents the attention matrix, i.e., the output of the masked multimodal attention mechanism, V m represents the output of the last encoder layer of the cross-modal pre-training model, W m represents the multimodal attention matrix; The multimodal attention matrix W m for: W m =Softmax(W f +M) Among them, W f represents the weighted fusion attention matrix, M represents the mask matrix; The weighted fusion attention matrix W f for: W f =w s *a s +w n *b n +b Among them, w s and w n represents the weight difference between the contract and annotation modes, b represents the deviation, and α s represents the contract attention matrix, β n Represents the annotation attention matrix.

9. The method for detecting similarity of smart contracts based on a cross-modal pre-training model according to claim 5 is characterized in that: The one-dimensional convolution process is: Among them, k {t,a} Represents the convolution kernel size of contracts and annotations, X n represents the annotation feature, X s Represents the characteristics of the contract, represents the contract features after one-dimensional convolution, Represents the annotation features after one-dimensional convolution, Conv1D represents a one-dimensional convolutional neural network; The extension process is as follows: in, Represents the extended contract features, Represents an expanded annotation feature.

Images