An explainable smart contract vulnerability detection and location method based on reinforcement learning

Abstract

The present invention provides an interpretable smart contract vulnerability detection and positioning method based on reinforcement learning, which is applied to blockchain supervision in order to solve the problem of security vulnerabilities in smart contracts deployed on the blockchain. The present invention first generates a contract graph based on the smart contract source code, mines the contract graph features based on the graph neural network model to classify vulnerabilities, and extracts the subgraph that has the greatest impact on the classification results based on reinforcement learning. The present invention mines the features in the smart source code graph through deep learning technology, and detects whether there are security vulnerabilities in the contract source code before the smart contract is deployed on the blockchain, thereby ensuring the security of the smart contract. At the same time, through reinforcement learning technology, it is possible to extract the key subgraphs that cause contract vulnerabilities, thereby locating the code parts related to the vulnerabilities, and providing interpretable risk code fragment positioning results for smart contract vulnerability detection.

Description

Technical Field

[0001] The present invention belongs to the field of blockchain, and more specifically, relates to an explainable smart contract vulnerability detection and positioning method based on reinforcement learning. Background Art

[0002] As one of the core innovations of blockchain technology, smart contracts have achieved remarkable development in different fields. This self-executing computing code not only ensures the reliability and transparency of transactions, but also provides a wide range of possibilities for decentralized applications. However, as its application scope expands, the potential security risks of smart contracts are becoming increasingly prominent, which makes smart contract vulnerability detection a critical area. Through a comprehensive review and analysis of smart contract codes, potential vulnerabilities can be discovered early to avoid possible risks. With the continuous evolution of blockchain technology, the development of smart contracts will continue to attract attention and promote more innovative applications in different fields.

[0003] In the existing technologies, there are many studies on smart contract vulnerability detection on blockchain, such as Huang, Jianjun, et al. "Precise Dynamic Symbolic Execution for Nonuniform Data Access in Smart Contracts." IEEE Transactions on Computers 71.7(2021):1551-1563. Considering the problem of multiple addressing models in smart contracts, a Dynamic symbolic execution (DSE) method is proposed to solve the problem of non-uniform data access in smart contracts, and an integer overflow vulnerability detector and a multi-transaction vulnerability detector are implemented; Li, Bixin, Zhenyu Pan, and Tianyuan Hu. "Redefender: detecting reentrancy vulnerabilities in smart contracts automatically." IEEE Transactions on Reliability 71.2(2022):984-999. A smart contract vulnerability detection tool ReDefender based on fuzz testing is proposed. It executes smart contracts by generating fuzzing input data and determines whether there are vulnerabilities in smart contracts by analyzing their execution logs; Wu, Hongjun, et al. "Peculiar: Smart contract vulnerability detection based on crucial data flow graph and pre-training techniques."2021IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2021. proposes a deep learning method based on natural language processing pre-training technology to achieve vulnerability detection by extracting the features of key data flow graphs in smart contracts. However, the above solution has the following problems:

[0004] (1) Lack of multi-classification vulnerability detection results. The vulnerability detection results of existing research work are mostly binary classification, that is, they can only determine whether there is a vulnerability in the smart contract, but cannot break down the specific type of vulnerability.

[0005] (2) Lack of fine-grained vulnerability localization results. Most existing research works only consider the implementation of vulnerability detection results and cannot provide the location results of specific code snippets with vulnerabilities.

[0006] (3) Poor interpretability of detection results. Existing deep learning-based methods are difficult to provide interpretable vulnerability detection results because most of the models they use are black boxes. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a smart contract vulnerability detection and positioning method that can ensure the security of smart contract code and the reliability of blockchain operation.

[0008] The technical solution adopted by the present invention to solve the above technical problems is an explainable smart contract vulnerability detection and positioning method based on reinforcement learning. The method mines the contract graph features based on the graph neural network model to classify vulnerabilities, and extracts the subgraph with the greatest impact on the classification results based on reinforcement learning, thereby achieving multi-classification vulnerability detection results for smart contracts and locating specific function code fragments with vulnerabilities, thereby ensuring the code security of smart contracts and the operational reliability of blockchain.

[0009] The specific steps include:

[0010] (1) First, we collect a smart contract dataset from Ethereum. The dataset includes the smart contract source code and the corresponding vulnerability types. Then, we convert the smart contract source code into a contract graph, where the nodes in the contract graph are the variables in the source code and the edges in the contract graph are the call relationships in the source code. Then, we obtain the dataset from the contract graph and the corresponding vulnerability types. in, Represents the i-th contract graph G i Corresponding vulnerability type, y i ={0,1,2,…,C},y i =0 represents the contract graph G i There is no loophole, otherwise it represents the contract graph G i The types of vulnerabilities that exist, C represents the total number of vulnerability types, and N represents the number of contract graphs in the dataset;

[0011] (2) Divide the subgraphs based on the code snippets of each function in the contract graph; each function code snippet corresponds to a subgraph, and obtain the subgraph set {g1, g2, …, g j ,…,g M}, where M represents the number of subgraphs; each subgraph is input into the graph neural network, which first obtains the node feature representation in the subgraph, and the j-th subgraph g j The node is represented as H(g j ), j = {1, 2, ..., M}; then the node feature representation H(g j ) is weighted fusion to obtain the feature representation r of the subgraphj ; Finally, the feature representation r of each subgraph j After weighted fusion, the feature representation c of the contract graph is obtained i , Represents the subgraph weight matrix W subgraph The jth row of is the trainable parameter;

[0012] (3) The vulnerability multi-classification model of the graph neural network represents the features of the contract graph c i Through the fully connected layer, multi-classification vulnerability detection of the contract graph is realized, and the multi-classification contract graph vulnerability detection results are output;

[0013] (4) Obtain the weight matrix of each subgraph based on the trained graph neural network and define the importance of each subgraph im j : Sort the weight coefficients of the contract graph vulnerability detection results from high to low according to the importance of the subgraphs, and select the first m subgraphs. The functions corresponding to the first m subgraphs are code snippets that may have vulnerabilities.

[0014] Among them, during the training process of the graph neural network, the subgraph weight matrix W subgraph It will be continuously updated, and finally the importance score of each sub-graph to the vulnerability detection result will be obtained; then, the model parameters of the vulnerability multi-classification model are fixed, and the previous Sub-graphs are fused for vulnerability detection. represents rounding up, k is a ratio value in the range of (0-1), and is a trainable parameter. When the vulnerability detection accuracy of the m+1 subgraphs is less than the preset iteration threshold compared with the vulnerability detection accuracy of the first m subgraphs, the reinforcement learning iteration process is stopped.

[0015] The present invention discloses an interpretable smart contract vulnerability detection and positioning method based on reinforcement learning, which is applied to the field of blockchain supervision in order to solve the problem of security vulnerabilities in smart contracts deployed on the blockchain. The present invention works based on reinforcement learning technology. First, a contract graph is generated according to the smart contract source code, and the contract graph features are mined based on a graph neural network model to classify vulnerabilities, and the subgraph with the greatest impact on the classification results is extracted based on reinforcement learning. The present invention mines the features in the smart source code graph through deep learning technology, and detects whether there are security vulnerabilities in the contract source code before the smart contract is deployed on the blockchain, thereby ensuring the security of the smart contract. At the same time, through reinforcement learning technology, it is possible to extract the key subgraphs that cause contract vulnerabilities, thereby locating the code parts related to the vulnerabilities, and providing interpretable risk code fragment positioning results for smart contract vulnerability detection.

[0016] The beneficial effects of the present invention are:

[0017] (1) Multi-classification detection. The vulnerability detection and location model proposed in the present invention can output multi-classification vulnerability detection results, detect multiple vulnerability types at the same time, and output corresponding detection results, thereby improving vulnerability detection efficiency.

[0018] (2) Fine-grained positioning. The vulnerability detection and positioning model proposed in the present invention can output the top m subgraphs that have the greatest impact on vulnerability detection. The function fragments corresponding to these subgraphs are the code fragments that are most likely to have vulnerabilities, thereby achieving fine-grained vulnerability positioning.

[0019] (3) Interpretable results. The vulnerability detection and localization model proposed in the present invention is based on the reinforcement learning method. By setting reward parameters and continuously iterating the selection of the reinforcement learning agent, it can achieve interpretable vulnerability localization and provide evidence for the vulnerability detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of a scenario provided by an embodiment of the present invention.

[0021] Figure 2 is a flow chart of an embodiment. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0023] like Figure 1 As shown, the blockchain scenario of the embodiment is: In blockchain applications based on smart contracts, security is always a key concern. After designing and writing the contract code, the development team will perform vulnerability detection and repair to capture and resolve potential vulnerabilities and ensure the reliability and stability of the contract. Only after rigorous vulnerability detection and confirmation that there are no problems will the contract be deployed on the blockchain network to ensure the security of users' funds and transaction data. This security-first development approach helps build user trust and promote the sustainable application of blockchain technology in key areas.

[0024] The present invention considers the following factors that affect the efficiency of smart contract deployment:

[0025] First, there are many types of vulnerabilities in the code of smart contracts, including but not limited to reentrancy attacks, integer overflows, permission control issues, etc. Therefore, it is necessary to develop a model that can realize multi-classification vulnerability detection and quickly identify various potential vulnerability patterns in the contract.

[0026] Secondly, smart contracts usually have a very complex code structure. Due to the length of the contract code, developers may need to spend a lot of time locating the problem when they find a vulnerability. In order to help developers fix vulnerabilities faster, it is necessary to provide accurate vulnerability location results.

[0027] Finally, vulnerability detection models usually produce some detection results, but these results often need to be understandable and convincing to developers. It is not enough to simply tell a contract that there is a certain vulnerability. Developers need to understand why the model classifies it as such a vulnerability and how it is detected. Therefore, vulnerability detection models need to provide interpretable results.

[0028] Based on the above points, the present invention designs an interpretable smart contract vulnerability detection and positioning method based on reinforcement learning, which is applied to the field of blockchain supervision, in order to solve the problem of security vulnerabilities in smart contracts deployed on the blockchain. The present invention works based on reinforcement learning technology. First, a contract graph is generated according to the smart contract source code, and the contract graph features are mined based on the graph neural network model to classify vulnerabilities, and the subgraph with the greatest impact on the classification results is extracted based on reinforcement learning. The present invention mines the features in the smart source code graph through deep learning technology, and detects whether there are security vulnerabilities in the contract source code before the smart contract is deployed on the blockchain, thereby ensuring the security of the smart contract. At the same time, through reinforcement learning technology, it is possible to extract the key subgraphs that cause contract vulnerabilities, thereby locating the code parts related to the vulnerabilities, and providing interpretable risk code fragment positioning results for smart contract vulnerability detection.

[0029] Combine the following Figure 2 , a method for detecting and locating an explainable smart contract vulnerability based on reinforcement learning is described in detail, which specifically includes the following steps:

[0030] (1) First, we collect a smart contract dataset from Ethereum. The dataset includes the smart contract source code and the vulnerability type corresponding to each contract. Then, we convert the smart contract source code into a contract graph, where the nodes are composed of variables in the source code and the edges are composed of call relationships in the source code. Each constructed contract graph is represented as G = (V, X, A), where V = {v1, v2, …, v n} represents the node set of the contract graph, Represents node characteristics, Represents the adjacency matrix. The initial features of the nodes in the contract graph are obtained by the word2vec algorithm, n represents the number of nodes, and d represents the dimension of the node features. Thus, the data set can be obtained in, Representative contract diagram G i Corresponding vulnerability type, y i ={0,1,2,…,C},yi =0 represents the contract graph G i There is no loophole, otherwise it represents the contract graph G i The types of vulnerabilities that exist, C represents the number of vulnerability types, and N represents the number of samples in the dataset.

[0031] (2) Subgraph Partitioning

[0032] The subgraphs are divided based on the code snippets contained in each function in the contract graph. Each function corresponds to a subgraph, which contains the variables and call relationships in the function. Thus, the subgraph set {g1, g2, …, g j ,…,g M}, where M represents the number of subgraphs. The number of nodes in each subgraph is limited to s. Based on the graph neural network, the node feature matrix in each subgraph is expressed as H(g j ) is obtained by the following formula:

[0033] H(g j )=f(g j )={h j |v j ∈V(g j )}

[0034] Among them, v j ,j={1,2,…,s} represents the subgraph g i For each node in h j represents the hidden vector corresponding to each node, and f(·) represents the message propagation mechanism of the graph neural network, as shown in the following formula:

[0035]

[0036] Among them, U(·) represents the state update function of the graph neural network, AGG(·) represents the message aggregation function of the graph neural network, and h j′ Represents node v j The adjacent node v j′ The superscript l represents the current aggregation state, and the superscript l+1 represents the next aggregation state.

[0037] (3) Constructing subgraph feature representation

[0038] Through step (2), we can obtain the node feature representation H(g j ), the feature representation r of the subgraph is obtained based on the weighted fusion method j , the node weight matrix is ​​defined as W node , then the subgraph representation is calculated as follows:

[0039]

[0040] Among them, the node weight matrix is a trainable parameter, T is transposed.

[0041] (4) Constructing the feature representation of the contract graph

[0042] Similarly, through step (3), the feature representation r of each subgraph in the contract graph is obtained j After that, the feature representation c of the contract graph is still calculated based on the weighted fusion method. i , the weight matrix is ​​defined as W subgraph , then the contract graph representation calculation is as follows:

[0043]

[0044] Among them, sub(G i ) represents the set of all subgraphs of the i-th contract graph, and the subgraph weight matrix is a trainable parameter, represents the j-th row of the matrix.

[0045] (5) Multi-classification detection of contract vulnerabilities

[0046] After obtaining the feature representation of the contract graph through step (4), it is passed through the fully connected layer of the neural network to realize multi-classification vulnerability detection of the contract graph. The cross entropy loss function of the vulnerability multi-classification model is defined as:

[0047]

[0048] Among them, C represents the number of vulnerability types, y represents the real vulnerability type of the sample, and y ′ Represents the classification result of the model output; y i,c represents the actual result of the cth vulnerability type corresponding to the i-th contract graph, y i ′ ,c It represents the probability of occurrence of vulnerability of the c-th vulnerability type corresponding to the i-th contract graph output by the graph neural network, that is, the multi-classification smart contract vulnerability detection result.

[0049] (6) Sub-image selection

[0050] In order to determine the top m subgraphs that have the greatest impact on the vulnerability classification results and locate the vulnerability function fragments, the present invention selects the top-m subgraphs based on the reinforcement learning method. First, the importance of each subgraph is defined according to the weight matrix of each subgraph {im j |g j ∈G}:

[0051]

[0052] That is, the weight coefficients of the contract graph vulnerability detection results are sorted from high to low according to the subgraphs, and then the top m most important subgraphs are selected based on the reinforcement learning method. Then the functions corresponding to the top m subgraphs are the code snippets that are most likely to have vulnerabilities.

[0053] (7) Reinforcement Learning Process

[0054] In the process of model training for multi-classification of vulnerabilities in contract graphs, the weight matrix W subgraph The importance score of each subgraph to the vulnerability detection result will be obtained continuously. Then, the model parameters of the vulnerability multi-classification model are fixed, and the previous Subgraphs are merged to detect vulnerabilities. When the vulnerability detection accuracy of the current m+1 subgraphs does not change much compared to the vulnerability detection accuracy of the previous m subgraphs, the reinforcement learning iteration process is stopped. At this time, the m subgraphs are considered to be the code snippets that have the greatest impact on the vulnerability results, thereby locating the vulnerability function code snippets of the contract graph. The present invention uses a reinforcement learning method to find the optimal solution for the ratio k value, and models the Markov decision process as a cyclic process in which the agent (Agent) takes action (Action) to change its own state (State), obtains rewards (Reward), and continuously interacts with the environment (Environment).

[0055] State: The state s of each cycle e e It is defined as: The sub-graphs are spliced ​​for vulnerability detection, where M sub-graphs are classified according to their importance. i To sort, Indicates rounding up.

[0056] Action: The reinforcement learning agent takes action a in each cycle e according to the reward e . Define a e To add or subtract the adjustment value Δk∈[0,1] from the current k value,

[0057] Reward: This method rewards each action a based on the accuracy of vulnerability detection. e Define the reward function reward(s e ,a e ). According to the current state s e Medium accuracy e and the previous state s e-1 Medium accuracy e-1 The change relationship of the reward function reward(s e ,a e) will have the following three values: e >accuracy e-1 , then the reward function takes the value +1; if accuracy e =accuracy e-1 , then the reward function takes the value 0; if accuracy e <accuracy e-1 , then the reward function takes the value -1.

[0058] Terminal. The termination condition of reinforcement learning is defined as: when the value of m remains unchanged in 5 consecutive cycles, the reinforcement learning algorithm will terminate. This means that reinforcement learning has reached the optimization threshold.

[0059] This section uses the Q-learning method to learn this Markov decision process. Q-learning is a non-policy reinforcement learning algorithm that aims to find the best action given the current state, which conforms to Bellman's optimality formula. This method adopts an ε-greedy search strategy, which means that the reinforcement learning agent explores new states by randomly selecting actions with probability ε, rather than selecting actions based on the maximum future reward.

[0060] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for detecting and locating explainable smart contract vulnerabilities based on reinforcement learning, characterized in that: The following steps are involved: (1) First, we collect a smart contract dataset from Ethereum. The dataset includes the smart contract source code and the corresponding vulnerability types. Then the smart contract source code is converted into a contract graph, where the nodes in the contract graph are variables in the source code, and the edges in the contract graph are call relationships in the source code; then the data set is obtained from the contract graph and the corresponding vulnerability types. in, Represents the i-th contract graph G i Corresponding vulnerability type, y i ={0,1,2,…,C},y i =0 represents the contract graph G i There is no loophole, otherwise it represents the contract graph G i The types of vulnerabilities that exist, C represents the total number of vulnerability types, and N represents the number of contract graphs in the dataset; (2) Divide the subgraphs based on the code snippets of each function in the contract graph; each function code snippet corresponds to a subgraph, and obtain the subgraph set {g1, g2, …, g j ,…,g M }, where M represents the number of subgraphs; each subgraph is input into the graph neural network, which first obtains the node feature representation in the subgraph, and the j-th subgraph g j The node is represented by H(g j ), j = {1, 2, ..., M}; then the node feature representation H(g j ) is weighted fusion to obtain the feature representation r of the subgraph j ; Finally, the feature representation r of each subgraph j After weighted fusion, the feature representation c of the contract graph is obtained i , Represents the subgraph weight matrix W subgraph The jth row of is the trainable parameter; (3) The vulnerability multi-classification model of the graph neural network represents the features of the contract graph c i Through the fully connected layer, multi-classification vulnerability detection of the contract graph is realized, and the multi-classification contract graph vulnerability detection results are output; (4) Obtain the weight matrix of each subgraph based on the trained graph neural network and define the importance of each subgraph im j : Sort the weight coefficients of the contract graph vulnerability detection results from high to low according to the importance of the subgraphs, and select the first m subgraphs. The functions corresponding to the first m subgraphs are code snippets that may have vulnerabilities. Among them, during the training process of the graph neural network, the subgraph weight matrix W subgraph It will be continuously updated, and finally the importance score of each sub-graph to the vulnerability detection result will be obtained; then, the model parameters of the vulnerability multi-classification model are fixed, and the previous Sub-graphs are fused for vulnerability detection. represents rounding up, and the value range of k is a ratio of (0,1], which is a trainable parameter. When the vulnerability detection accuracy of the m+1 subgraphs is less than the preset iteration threshold compared with the vulnerability detection accuracy of the first m subgraphs, the reinforcement learning iteration process is stopped.

2. The method according to claim 1, characterized in that: In step (1), each contract graph is represented as G = (V, X, A), where V represents the node set of the contract graph, X represents the node features, and A represents the adjacency matrix; the node features in the contract graph are obtained by encoding the source code corresponding to the node through word2vec.

3. The method according to claim 1, characterized in that: The specific method for obtaining the feature representation of the contract graph in step (2) is: The graph neural network represents the node feature H(g j ) is weighted fusion to obtain the feature representation r of the subgraph j Specifically: W node is the node weight matrix in the graph neural network, which is a trainable parameter; T is transposed.

4. The method according to claim 3, characterized in that: Node feature representation H(g j ) is obtained through the message propagation mechanism f(·) of the graph neural network: H(g i )=f(g i )={h j };h j Represents subgraph g j For each node v j The hidden vector, h j The update method during training is: Among them, U(·) represents the state update function of the graph neural network, AGG(·) represents the message aggregation function of the graph neural network, and h j′ Represents node v j The adjacent node v j′ The superscript l represents the current aggregation state, and the superscript l+1 represents the next aggregation state.

5. The method according to claim 1, characterized in that: The cross entropy loss function L used by the multi-classification model in the training process of the vulnerability in the graph neural network CE for: Among them, y i,c represents the actual result of the cth vulnerability type corresponding to the i-th contract graph, y′ i,c It represents the probability of occurrence of vulnerability of the c-th vulnerability type corresponding to the i-th contract graph output by the graph neural network, that is, the multi-classification smart contract vulnerability detection result.

Images