Smart contract vulnerability detection method and system based on large language model guidance

Abstract

The present invention relates to the fields of network security and vulnerability detection, and specifically to a smart contract vulnerability detection method and system based on a large language model. The method converts initial vulnerability detection data into standard vulnerability detection data and then performs classification and connection processing. Then, by performing data flow and control flow analysis on the smart contract, data dependencies in the smart contract are mined, and these data dependencies are represented as a data dependency tree. Directed edges reflecting the data dependencies in the data dependency tree and two nodes connected by the directed edges are extracted and saved to a triple table, and the elements in the triple table are marked in an abstract syntax tree to obtain an abstract syntax tree containing the smart contract data dependencies. Then, the large language model is used to learn the data dependencies marked in the abstract syntax tree and analyze the code structure of the smart contract to perform vulnerability detection on the smart contract, thereby significantly improving the accuracy of vulnerability detection.

Description

Technical Field

[0001] The present invention relates to the fields of network security and vulnerability detection technology, and specifically to a smart contract vulnerability detection method and system based on large language model guidance. Background Art

[0002] With the rapid development of blockchain technology in recent years, the security of smart contracts deployed on blockchains has garnered widespread attention. Attackers frequently discover and exploit security vulnerabilities in smart contracts, launching attacks and stealing assets. Due to the immutability of blockchains, accurate and efficient tools are essential for detecting security vulnerabilities in smart contracts before they are deployed.

[0003] Currently, mainstream vulnerability detection methods can be categorized into two categories: static analysis and dynamic detection. Static analysis methods primarily include symbolic execution, formal verification, and intermediate representations, while dynamic detection methods primarily rely on fuzz testing. Existing vulnerability detection tools have limitations. For example, while static analysis can identify potential vulnerabilities in contracts by analyzing them without executing the code, enabling rapid inspection of contract code, it is limited when dealing with complex dynamic behaviors and interactions between contracts (such as reentrancy attacks and delegate calls). It cannot capture vulnerabilities that may arise under specific conditions, resulting in missed detections.

[0004] While dynamic analysis can capture complex interactions that are difficult to identify with static analysis, simulating a more realistic runtime environment and uncovering potential vulnerabilities in smart contracts under specific conditions, it is still expensive and slow when working with large contracts. Furthermore, dynamic analysis requires the creation of numerous test cases to cover diverse inputs and states, consuming significant time and resources during development and increasing the complexity of vulnerability detection.

[0005] In recent years, with the emergence of large language model technology, some studies have applied it to vulnerability detection in smart contracts. However, smart contract vulnerabilities are often closely related to the variables and functions within the contract and the interactions between them. Large language models such as ChatGPT may have difficulty understanding these data dependencies and code logic. In actual vulnerability detection, there will be insufficient analysis, resulting in the inability to accurately identify potential vulnerabilities. Summary of the Invention

[0006] In order to solve the above problems mentioned in the background technology, the present invention provides a smart contract vulnerability detection method and system based on large language model guidance.

[0007] The technical solutions of the present invention are as follows:

[0008] A smart contract vulnerability detection method based on a large language model includes the following steps:

[0009] S1. Use different vulnerability detection tools to perform vulnerability detection on the smart contract to obtain initial vulnerability detection data. Convert the format of the initial vulnerability detection data to obtain standard vulnerability detection data. The standard vulnerability detection data includes vulnerability number and vulnerability location information.

[0010] Based on the set vulnerability classification standards, standard vulnerability detection data is classified and deduplicated to obtain classified vulnerability data, and the classified vulnerability data is connected in a linked list manner to obtain merged vulnerability data;

[0011] S2. Perform data flow analysis on the smart contract to obtain a data flow diagram. Mark the variables that cause vulnerabilities in the data flow diagram based on the merged vulnerability data to obtain marked variables. Starting from the marked variables, traverse the data flow diagram and mark several variables in the data flow diagram in order based on the order of data flow to obtain several influencing variables at different levels.

[0012] Several influencing variables at different levels are treated as nodes at different levels. Based on the data flow relationship, directed edges are added between nodes at adjacent levels to build a data dependency tree.

[0013] S3. Perform control flow analysis on the smart contract to obtain a control flow graph. Based on the codes in the nodes at different levels in the data dependency tree, mark the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, backtrack along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks.

[0014] Extract key variables that affect the execution order of the code blocks in the predecessor code blocks, and add the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree;

[0015] S4. Compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by the directed edges based on the updated data dependency tree, construct a triple table, and mark the abstract syntax tree based on the elements of the triple table to obtain a second abstract syntax tree;

[0016] S5. Set a prompt template for the large language model based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

[0017] Specifically, in S5, a prompt template of the large language model is set based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set response template. The specific content of the prompt template is:

[0018] You are a smart contract vulnerability detection tool. Below are several common smart contract vulnerabilities [VULS]. Next, I will provide a smart contract source code and its abstract syntax tree containing dependencies. The tags in the abstract syntax tree represent the data dependencies in the smart contract. Please carefully analyze the abstract syntax tree of the smart contract and its data dependencies, and determine whether the smart contract has the above-mentioned vulnerability types. Your response should conform to the response template I provide. Please analyze it in detail step by step. The source code of the smart contract is [CONTRACT], the abstract syntax tree is [AST], and the response template is [REPLY].

[0019] In S2, several influencing variables of different levels are obtained, and the specific process is as follows:

[0020] Starting from the marked variable, traverse the data flow diagram, mark the variables whose data flow points to the marked variable, and obtain the first-level influencing variables. Continue to traverse the data flow diagram, mark the variables whose data flow points to the first-level influencing variable, and obtain the second-level influencing variables. Repeat the above process until all variables with a data flow relationship with the marked variable are graded and marked, and several influencing variables of different levels are obtained.

[0021] In S1, the standard vulnerability detection data is classified based on the set vulnerability classification standard. The process is as follows:

[0022] Create three types of dictionaries: the first type is the vulnerability with the same number but different locations, the second type is the vulnerability with the same number and location, and the third type is the vulnerability with different numbers but the same location. According to the vulnerability number and location information, the classified standard vulnerability detection data are stored in the dictionary of the corresponding category.

[0023] In S1, the classified vulnerability data is connected in a linked list manner, specifically:

[0024] Create an empty table T. For each newly encountered vulnerability number in the classified vulnerability data, create a linked list L to store the vulnerability location information. Add the vulnerability number and the corresponding linked list L to table T. Traverse all classified vulnerability data. For classified vulnerability data that matches the existing vulnerability number in table T: check whether its vulnerability location information has been stored in the corresponding linked list L. If not saved, add this vulnerability location information to the corresponding linked list L.

[0025] The data dependency tree is constructed in S2, and the specific process is as follows:

[0026] The label variable is used as the root node, the first-level influencing variable as the first-level leaf node, the second-level influencing variable as the second-level leaf node, and so on, to obtain nodes of different levels with data flow relationships. Based on the data flow relationship, directed edges are added between nodes of adjacent levels to construct a data dependency tree.

[0027] The key variables in S3 include: conditional judgment statements and timestamps.

[0028] In S3, the control flow analysis of the smart contract is performed to obtain a control flow graph, specifically:

[0029] Based on the executable code blocks in the smart contract and the execution paths between different executable code blocks, a control flow graph is obtained. The nodes of the control flow graph are the executable code blocks, and the edges of the control flow graph are the execution order of the code blocks.

[0030] In S1, different vulnerability detection tools are used to detect vulnerabilities in smart contracts, specifically:

[0031] Use Oyente and Mythril to perform static analysis on the compiled smart contracts, and use Confuzzius and sFuzz to perform dynamic analysis on the compiled smart contracts to obtain initial vulnerability detection data.

[0032] The present invention also provides a smart contract vulnerability detection system guided by a large language model, comprising:

[0033] Vulnerability data processing module: used to perform vulnerability detection on smart contracts using different vulnerability detection tools, obtain initial vulnerability detection data, convert the format of the initial vulnerability detection data, obtain standard vulnerability detection data, which includes vulnerability number and vulnerability location information; classify and remove duplicates from the standard vulnerability detection data based on the set vulnerability classification standard to obtain classified vulnerability data, and connect the classified vulnerability data in a linked list manner to obtain merged vulnerability data;

[0034] Explicit dependency data extraction module: used to perform data flow analysis on smart contracts, obtain a data flow graph, mark the variables that cause vulnerabilities in the data flow graph based on the merged vulnerability data, obtain marked variables, traverse the data flow graph with the marked variables as the starting point, and mark several variables in the data flow graph in order based on the order of data flow to obtain several influencing variables of different levels. The influencing variables of different levels are respectively used as nodes of different levels. Directed edges are added between nodes of adjacent levels based on the data flow relationship to construct a data dependency tree;

[0035] Implicit dependency data extraction module: This module is used to perform control flow analysis on smart contracts and obtain a control flow graph. Based on the code in the nodes at different levels in the data dependency tree, the module marks the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, the module backtracks along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks. The module extracts the key variables in the predecessor code blocks that affect the execution order of the code blocks, and adds the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree.

[0036] Abstract syntax tree construction module: used to compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by directed edges based on the updated data dependency tree, build a triple table, mark the abstract syntax tree based on the elements of the triple table, and obtain a second abstract syntax tree;

[0037] Vulnerability detection module: used to set the prompt template of the large language model based on the second abstract syntax tree, smart contract source code, vulnerability type, and set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

[0038] The beneficial effects of the present invention are:

[0039] 1. The present invention provides a smart contract vulnerability detection method based on a large language model. By performing data flow and control flow analysis on the smart contract, the data dependencies in the smart contract are deeply explored, and these data dependencies are represented by a data dependency tree. The directed edges reflecting the data dependencies in the data dependency tree and the two nodes connected by the directed edges are extracted and saved to a triple table. The elements in the triple table are marked in an abstract syntax tree to obtain an abstract syntax tree containing the smart contract data dependencies. The large language model is then used to learn the data dependencies marked in the abstract syntax tree and analyze the code structure of the smart contract to perform vulnerability detection on the smart contract. This method not only solves the problem of high omissions or false positives in traditional vulnerability detection tools, but also overcomes the problem of insufficient analysis when using a single large language model for vulnerability detection, significantly improving the accuracy of vulnerability detection.

[0040] 2. The present invention uses traditional vulnerability detection tools to obtain initial vulnerability detection data, converts the initial vulnerability detection data into standard vulnerability detection data and then classifies it to obtain classified vulnerability data. The classified vulnerability data is then connected and processed to obtain valid vulnerability information in the smart contract. DETAILED DESCRIPTION

[0041] Exemplary embodiments of the present disclosure are described in more detail below.

[0042] Example

[0043] This embodiment provides a smart contract vulnerability detection method based on a large language model, including the following steps:

[0044] S1. Use different vulnerability detection tools to perform vulnerability detection on the smart contract to obtain initial vulnerability detection data. Convert the format of the initial vulnerability detection data to obtain standard vulnerability detection data. The standard vulnerability detection data includes vulnerability number and vulnerability location information.

[0045] Based on the set vulnerability classification standards, the standard vulnerability detection data is classified and deduplicated to obtain classified vulnerability data, and the classified vulnerability data is connected in a linked list manner to obtain merged vulnerability data.

[0046] In step S1, different vulnerability detection tools are used to detect vulnerabilities in the smart contract. Specifically, you can choose to use Oyente and Mythril to perform static analysis on the compiled smart contract, and use Confuzzius and sFuzz to perform dynamic analysis on the compiled smart contract to obtain initial vulnerability detection data.

[0047] The reasons for selecting these vulnerability detection tools are as follows: Oyente is one of the most "classic" symbolic execution tools. Through simulation and symbolic execution technology, it can detect potential security vulnerabilities in smart contracts. Mythril is the officially recommended smart contract vulnerability detection tool of Ethereum and has been widely recognized. Mythril combines symbolic execution and control flow analysis technologies to discover vulnerabilities in contracts. Confuzzius and sFuzz are both currently popular fuzz testing tools.

[0048] By using the above four vulnerability detection tools to perform vulnerability detection on the same smart contract, we can obtain two vulnerability detection reports generated by static analysis tools and two vulnerability detection reports generated by dynamic analysis tools, a total of four original vulnerability detection reports. Based on the vulnerability detection data in all original vulnerability detection reports, we can obtain the initial vulnerability detection data.

[0049] The initial vulnerability detection data is then formatted to obtain standard vulnerability detection data, which includes vulnerability numbers and location information. This step primarily converts the initial vulnerability detection data into a unified format. During the format conversion process, the existing SWC-ID is specifically selected as the vulnerability type standard. SWC-ID is a smart contract vulnerability classification and testing system that provides a series of standardized vulnerability classifications with a clear vulnerability numbering system and classification method. Each vulnerability number (ID) corresponds to a unique vulnerability type, which facilitates standardized vulnerability identification.

[0050] Furthermore, in the obtained standard vulnerability detection data, there are some vulnerability records with the same error reporting location but different error types, so a vulnerability classification standard is set to classify the standard vulnerability detection data to ensure the integrity and validity of the vulnerability information.

[0051] Classify standard vulnerability detection data based on the set vulnerability classification standards. The process is as follows:

[0052] Create three types of dictionaries: the first type is the vulnerability with the same number but different locations, the second type is the vulnerability with the same number and location, and the third type is the vulnerability with different numbers but the same location. According to the vulnerability number and location information, the classified standard vulnerability detection data are stored in the dictionary of the corresponding category.

[0053] In the process of classifying standard vulnerability detection data, vulnerability information is also deduplicated. Vulnerability information is filtered during the classification process so that only one piece of identical vulnerability information is retained.

[0054] The classified vulnerability data is linked in a linked list fashion. Specifically, an empty table T is created. For each newly encountered vulnerability number in the classified vulnerability data, a linked list L is created to store the vulnerability location information. The vulnerability number and the corresponding linked list L are added to table T. All classified vulnerability data are traversed. For classified vulnerability data that matches an existing vulnerability number in table T, the vulnerability location information is checked to see if it has been stored in the corresponding linked list L. If not, the vulnerability location information is added to the corresponding linked list L. Finally, the merged vulnerability data is obtained from table T.

[0055] The process of step S1 corresponds to the processing of vulnerability data, and the algorithm used is shown in Table 1.

[0056] Table 1 Vulnerability data processing algorithm

[0057]

[0058] S2. Perform data flow analysis on the smart contract to obtain a data flow graph. Mark the variables that cause vulnerabilities in the data flow graph based on the merged vulnerability data to obtain marked variables. Starting from the marked variables, traverse the data flow graph and mark several variables in the data flow graph in order based on the order of data flow to obtain several influencing variables at different levels. The influencing variables at different levels are respectively used as nodes at different levels. Directed edges are added between nodes at adjacent levels based on the data flow relationship to construct a data dependency tree.

[0059] In this step, data flow analysis is performed on the smart contract to obtain a data flow graph. Specifically, a data flow graph is constructed based on variables, function calls, and their relationships. Nodes in the data flow graph represent variables, and edges represent the data flows between variables. The specific algorithm for constructing the data flow graph is shown in Table 2.

[0060] Table 2 Algorithm for constructing data flow graph

[0061]

[0062] A data flow diagram (DFD) is a graphical representation used to describe the flow and processing of data within a smart contract. It reveals the path of information within the smart contract. Each smart contract can be represented as a DFD. By analyzing the variables, function calls, and their relationships within the smart contract, the contract's data flow can be represented using nodes and edges. Data flow analysis of smart contracts can reveal explicit dependencies within the contract. Explicit dependencies are an inherent property of smart contracts and refer to the explicit dependencies between one module or variable in the system and another, explicitly stated in the smart contract code. These dependencies are implicit in the code. A DFD containing explicit dependencies provides a foundation for security analysis of smart contracts.

[0063] After the data flow graph is generated, the variables that cause vulnerabilities in the data flow graph are marked based on the merged vulnerability data to obtain marked variables. Starting from the marked variables, the data flow graph is traversed and several variables in the data flow graph are graded and marked in turn based on the order of data flow to obtain several influencing variables of different levels. The specific process is as follows:

[0064] Starting from the marked variable, traverse the data flow diagram, mark the variables whose data flow points to the marked variable, and obtain the first-level influencing variables. Continue to traverse the data flow diagram, mark the variables whose data flow points to the first-level influencing variable, and obtain the second-level influencing variables. Repeat the above process until all variables with a data flow relationship with the marked variable are graded and marked, and several influencing variables of different levels are obtained.

[0065] Furthermore, several influencing variables at different levels are treated as nodes at different levels. Based on the data flow relationship, directed edges are added between nodes at adjacent levels to construct a data dependency tree. The specific process is as follows:

[0066] The label variable is used as the root node, the first-level influencing variable as the first-level leaf node, the second-level influencing variable as the second-level leaf node, and so on, to obtain nodes of different levels with data flow relationships. Based on the data flow relationship, directed edges are added between nodes of adjacent levels to construct a data dependency tree.

[0067] S3. Perform control flow analysis on the smart contract to obtain a control flow graph. Based on the codes in the nodes at different levels in the data dependency tree, mark the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, backtrack along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks. Extract the key variables in the predecessor code blocks that affect the execution order of the code blocks, and add the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree.

[0068] Specifically, in this step, control flow analysis is performed on the smart contract to obtain a control flow graph. Specifically, based on the executable code blocks in the smart contract and the execution paths between different executable code blocks, a control flow graph is obtained. The nodes of the control flow graph are executable code blocks, and the edges of the control flow graph are the execution order of the code blocks.

[0069] The algorithm for generating the control flow graph is shown in Table 3.

[0070] Table 3 Control flow graph construction algorithm

[0071]

[0072] A control flow graph (CFG) is a graphical representation used to describe the control flow of code within a smart contract. It helps better understand smart contract behavior and identify vulnerabilities. Its implementation is based on the relationship between a program's basic blocks and control flow. The nodes of a CFG are basic blocks, which are continuous sections of executable code within a program.

[0073] The edges of a control flow graph represent control flow transfers, reflecting the execution order of smart contract programs. Smart contract programs have two execution orders: the first is sequential execution between adjacent basic blocks, which depends solely on the type of opcode. The second is jump execution between non-adjacent basic blocks, which determines the specific jump block location by analyzing the relationship between instruction opcodes and operands during execution. By graphically representing the functions, conditionals, and loop structures in smart contracts, the control flow graph clearly illustrates the code execution path, thereby uncovering implicit dependencies within smart contracts that are not directly visible.

[0074] Based on the code in the nodes at different levels of the data dependency tree, the corresponding executable code blocks in the control flow graph are marked as tainted blocks. Starting from the tainted blocks, the control flow graph is backtracked along the execution order of the code blocks, obtaining the predecessor code blocks pointing to the tainted blocks. The predecessor code blocks are inspected, and the key variables in the predecessor code blocks that affect the execution order of the code blocks are extracted. These key variables are added as new nodes to the data dependency tree to obtain an updated data dependency tree. Key variables that affect the execution order of code blocks include: variables that may affect dependencies, such as conditional statements; and dependencies on external environment variables, such as timestamps. These functions may affect the execution order of code blocks, and thus affect the variables in the data dependency sequence.

[0075] S4. Compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by the directed edges based on the updated data dependency tree, construct a triple table, mark the abstract syntax tree based on the elements of the triple table, and obtain a second abstract syntax tree.

[0076] In step S4, the Solidity compiler is first used to generate the first abstract syntax tree (AST) of the smart contract. The AST is used as the basis for representing data dependencies because its tree structure clearly displays the syntax and logical hierarchy of the smart contract. This structure not only facilitates analysis of the various components of the smart contract but also improves the large language model's ability to understand the smart contract. Furthermore, by marking data dependencies on the AST, data dependency information in the data stream can be integrated into the grammatical structure, further supporting the efficient analysis of smart contracts by the large language model.

[0077] Based on the updated data dependency tree obtained in step S3, directed edges and the two nodes connected by the directed edges are extracted to construct a dependency table in the form of triples, namely a triple table. The element data in the triple table is then marked in the first abstract syntax tree to obtain a second abstract syntax tree containing the smart contract data dependency.

[0078] This step not only integrates the grammatical structure and data dependencies of smart contracts, but also provides an intuitive and efficient foundation for subsequent vulnerability detection and smart contract analysis.

[0079] S5. Set a prompt template for the large language model based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

[0080] The large language model used in this invention is the existing ChatGPT, an advanced natural language processing model based on the Generative Pre-Trained Transformer (GPT) architecture. By analyzing and learning massive amounts of text data, it masters the basic structure, grammatical rules, and semantic relationships of language, and possesses the fundamental ability to understand and generate natural language. The main advantages of ChatGPT, a large language model, lie in its powerful language understanding capabilities, flexible adaptability, and rapid response capabilities. It has been applied in the field of vulnerability detection. Through deep semantic analysis, ChatGPT can understand the overall logic of smart contract code based on context. Potential vulnerabilities can be identified by analyzing the code context, and its language generation capabilities enable vulnerability detection results to be presented in a coherent and easy-to-understand format. Combining contextual understanding with real-time feedback capabilities, ChatGPT can quickly adapt to different code environments, thereby improving the accuracy and efficiency of vulnerability detection.

[0081] Considering that the model based on the ChatGPT architecture relies on the prompt project, the quality of the prompt has a significant impact on the model performance and vulnerability detection effect. Therefore, this embodiment sets a prompt template for the large language model based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set reply template. The specific content of the prompt template is:

[0082] You are a smart contract vulnerability detection tool. Below are several common smart contract vulnerabilities [VULS]. Next, I will provide a smart contract source code and its abstract syntax tree containing dependencies. The tags in the abstract syntax tree represent the data dependencies in the smart contract. Please carefully analyze the abstract syntax tree of the smart contract and its data dependencies, and determine whether the smart contract has the above-mentioned vulnerability types. Your response should conform to the response template I provide. Please analyze it in detail step by step. The source code of the smart contract is [CONTRACT], the abstract syntax tree is [AST], and the response template is [REPLY].

[0083] Among them, [VULS] represents the four common vulnerabilities detected and their aliases, the information comes from the DASP10 official website; [CONTRACT] is the smart contract source code to be tested; [AST] is the second abstract syntax tree containing dependency markers; [REPLY] is the defined vulnerability detection result reply template.

[0084] This paper mainly considers the four common smart contract vulnerabilities: reentrancy vulnerability, timestamp dependency vulnerability, integer overflow vulnerability, and delegate call vulnerability.

[0085] Timestamp dependency vulnerabilities occur when smart contracts rely on block timestamps for logical reasoning. Attackers can exploit this ability to manipulate timestamps to select specific times for block generation, thereby affecting the contract's execution conditions, leading to improper profit or bypassing security restrictions.

[0086] Integer overflow vulnerabilities occur when smart contracts perform arithmetic operations using fixed-size integer types. When the result of an operation exceeds the maximum value of the integer type, the value wraps around to its minimum value, resulting in unpredictable behavior. Attackers can exploit this to manipulate contract logic. For example, if a smart contract allows users to deposit funds and checks whether the balance is greater than zero, an attacker could trigger an integer overflow, causing the balance to become negative, thereby bypassing security checks and withdrawing funds.

[0087] Delegate call vulnerabilities arise from improper use of the delegate call instruction. Delegate calls allow one smart contract to execute code within the context of another smart contract, meaning the called smart contract can access the storage and state of the calling contract. While this feature is flexible, calling a malicious smart contract can lead to tampering with the calling smart contract's storage or theft of assets. An attacker can control the code of the called smart contract and modify key variables of the calling smart contract. For example, if a smart contract uses a delegate call to execute functions in an external smart contract, an attacker could deploy a malicious smart contract and tamper with the state of the original smart contract through the delegate call.

[0088] The present invention also provides a smart contract vulnerability detection system guided by a large language model, comprising:

[0089] Vulnerability data processing module: used to perform vulnerability detection on smart contracts using different vulnerability detection tools, obtain initial vulnerability detection data, convert the format of the initial vulnerability detection data, obtain standard vulnerability detection data, which includes vulnerability number and vulnerability location information; classify and remove duplicates from the standard vulnerability detection data based on the set vulnerability classification standard to obtain classified vulnerability data, and connect the classified vulnerability data in a linked list manner to obtain merged vulnerability data;

[0090] Explicit dependency data extraction module: used to perform data flow analysis on smart contracts, obtain a data flow graph, mark the variables that cause vulnerabilities in the data flow graph based on the merged vulnerability data, obtain marked variables, traverse the data flow graph with the marked variables as the starting point, and mark several variables in the data flow graph in order based on the order of data flow to obtain several influencing variables of different levels. The influencing variables of different levels are respectively used as nodes of different levels. Directed edges are added between nodes of adjacent levels based on the data flow relationship to construct a data dependency tree;

[0091] Implicit dependency data extraction module: This module is used to perform control flow analysis on smart contracts and obtain a control flow graph. Based on the code in the nodes at different levels in the data dependency tree, the module marks the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, the module backtracks along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks. The module extracts the key variables in the predecessor code blocks that affect the execution order of the code blocks, and adds the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree.

[0092] Abstract syntax tree construction module: used to compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by directed edges based on the updated data dependency tree, build a triple table, mark the abstract syntax tree based on the elements of the triple table, and obtain a second abstract syntax tree;

[0093] Vulnerability detection module: used to set the prompt template of the large language model based on the second abstract syntax tree, smart contract source code, vulnerability type, and set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

[0094] Experimental content

[0095] This paper tests four types of vulnerabilities that commonly occur in smart contracts and compares them with common traditional vulnerability detection methods, the direct use of a single large language model (ChatGPT), and other traditional vulnerability detection methods.

[0096] Dataset: The SmartBugs Curated dataset was used to compare and analyze the proposed smart contract vulnerability detection method based on a large language model with other traditional vulnerability detection methods and vulnerability detection methods using a single large language model. SmartBugs Curated is a smart contract dataset that contains vulnerability markers for smart contracts. The smart contracts in this dataset encompass essentially all vulnerability categories. This dataset was used as a prototype, and four major vulnerabilities were screened at the reliability source level: reentrancy, timestamp dependency, integer overflow, and delegatecall. A total of 13,207 smart contracts, 83 of which had security vulnerabilities, were selected as the dataset for smart contract vulnerability detection and analysis.

[0097] To reasonably evaluate the test results of the method, this paper uses precision (PR), recall (RE), and F1-score (F1) evaluation metrics to demonstrate its detection performance. Precision is a classification metric; recall is a measure of the ability to identify positive samples; and F1-score is a commonly used metric in statistics to measure binary (or multi-task binary) accuracy, calculated based on precision and recall. As a binary classification problem, the reliability of vulnerability detection results is primarily related to the degree of match between the actual situation and the detection results. For a specific smart contract with a specific vulnerability, the experimental results include the following four matching cases:

[0098] True Positive (TP): When the test result indicates that a vulnerability exists and the vulnerability does exist, the test result is correct.

[0099] False Positive (FP): When the test result indicates that a vulnerability exists but no vulnerability actually exists, the test result is a false positive.

[0100] False Negative (FN): When the test result indicates that the vulnerability does not exist but it actually exists, the test result is a false negative.

[0101] True Negative (TN): A test result is correct when the test result indicates that the vulnerability does not exist but it actually does not exist.

[0102] To facilitate the performance evaluation of the improved smart contract detection solution, this paper uses true positives, false positives, and false negatives as indicators of detection effectiveness, and selects precision, recall, and F1 score as evaluation indicators.

[0103] Precision Rate (PR) refers to the ratio of the number of real vulnerabilities detected in all sample smart contracts to the total number of vulnerabilities in the detection report. The calculation formula is as follows:

[0104] ;

[0105] Recall (RE) refers to the ratio of the number of correctly detected vulnerabilities in all samples to all smart contract vulnerabilities. The calculation formula is as follows:

[0106] ;

[0107] F1-score (F1), the harmonic mean of precision (PR) and recall (RE), is used to combine these two metrics to provide a single classification performance measure. It is calculated as follows:

[0108] .

[0109] The detection effects of the present invention and four traditional vulnerability detection methods commonly used in the vulnerability detection process on the same data set are compared, as shown in Table 4.

[0110] Table 4 Comparison with four traditional vulnerability detection methods

[0111]

[0112] The results show that the present invention has excellent results in detecting different types of vulnerabilities. Next, the data in Table 4 will be analyzed one by one based on the vulnerability type to further illustrate the detection advantages of the present invention.

[0113] Comparison of reentrancy vulnerability detection results:

[0114] In detecting reentrancy vulnerabilities, our method achieved an outstanding precision (PR) of 65.78% and a recall (RE) of 96.30%, demonstrating its ability to discover and identify potential reentrancy vulnerabilities. While its precision is lower than Oyente's 76.01%, its high recall means it captures more actual vulnerabilities, enhancing its overall identification capabilities. Oyente's low recall (33.97%) suggests that despite its good precision, it may miss some potential risks when identifying reentrancy vulnerabilities in complex contracts.

[0115] Comparison of timestamp vulnerability detection results:

[0116] The present invention also performed well in detecting timestamp vulnerabilities, with a recall rate of 83.14%, demonstrating considerable sensitivity in practical identification. In comparison, Mythril and Slither performed worse than the present invention in this regard, particularly in terms of recall, failing to capture much vulnerability information. This demonstrates that the present invention is more effective than other vulnerability detection tools in uncovering timestamp-related logic vulnerabilities. The present invention also demonstrates greater adaptability to different contracts.

[0117] Comparison of integer overflow vulnerability detection results:

[0118] The present invention performed slightly worse in detecting integer overflow vulnerabilities. Although its recall rate reached 93.73%, its precision rate fell short of the highest. This suggests that while the present invention has a high sensitivity in identifying integer overflows, its judgment of whether these risks actually exist may be biased, resulting in some unrelated code being mistakenly identified as overflow vulnerabilities. This situation stems from a lack of understanding of mathematical operations and state changes, limiting its accuracy in detecting these vulnerabilities.

[0119] Comparison of access control vulnerability detection results:

[0120] For delegate call vulnerabilities, our detection results also demonstrate a high recall rate (94.35%) and relatively modest precision. This suggests that while many potential delegate call vulnerabilities can be identified, there are also a number of false positives. These false positives may be due to the fact that delegate calls in smart contracts often involve dynamic addresses or function calls, potentially misinterpreting these dynamic behaviors as vulnerabilities, even if they are considered safe in the specific context.

[0121] Based on the information in Table 4, it can be seen that the smart contract vulnerability detection method guided by a large language model used in the present invention has shown good results in terms of vulnerability detection accuracy, especially when detecting reentry vulnerabilities and access control vulnerabilities, showing better results compared to other vulnerability detection methods. At the same time, the data in the table also shows that although the present invention has a high recall rate, it may not have the highest accuracy in vulnerability detection. This phenomenon may be caused by excessive sensitivity to certain semantic features, resulting in false positives, which in turn leads to its relatively low accuracy. However, overall, the present invention still demonstrates excellent comprehensive capabilities in smart contract vulnerability detection.

[0122] Furthermore, by integrating ChatGPT into vulnerability detection, the present invention, with its code analysis capabilities and analysis of contract dependencies, is able to better uncover potential vulnerabilities when performing smart contract vulnerability detection. Furthermore, compared to other vulnerability detection tools, the present invention has a better overall effect.

[0123] The vulnerability detection results of the present invention were compared with those of a single large language model (ChatGPT) on the same dataset, and the results are shown in Table 5. When using a single large language model for vulnerability detection, the source code of the smart contract is passed to ChatGPT using an API interface, and a detailed vulnerability analysis is requested. First, the source code of the smart contract to be tested is stored as a string. Then, the smart contract is analyzed using a prompt template. The prompt template format uploaded to ChatGPT is the same as the prompt template format used in the present invention, except that the abstract syntax tree with contract dependencies is no longer attached to the prompt template. The smart contract source code and prompt template are passed to ChatGPT using the API interface. ChatGPT then returns a vulnerability detection report through the API interface, which includes the SWC-ID number of the vulnerability in the smart contract and the location where the vulnerability occurs.

[0124] Table 5 Comparison with using a single large language model

[0125]

[0126] As can be seen from Table 5, although the recall rate can reach up to 96.04% when ChatGPT is used directly to detect vulnerabilities in smart contracts, its precision is much lower than that of the present invention, especially when detecting timestamp-dependent vulnerabilities, the precision is only 18.04%. ChatGPT's higher recall rate and lower precision indicate that directly using ChatGPT for vulnerability detection will lead to a large number of false positives. For example, when there is a "reentrancy lock" protection mechanism in a smart contract, ChatGPT may not fully understand its role in contract execution and thus mistakenly mark it as a reentrancy vulnerability. ChatGPT's excessive sensitivity and insufficient understanding of code structure cause it to mistakenly mark many potential problems or uncertain behaviors as vulnerabilities. Therefore, many codes marked as vulnerabilities do not actually have vulnerabilities. In contrast, the method of the present invention is significantly better than ChatGPT in terms of precision, highlighting the effectiveness of our method. In addition, ChatGPT's F1 score is also lower than that of the present invention in all types of vulnerability detection.

[0127] The detection effects of the present invention and other traditional vulnerability detection methods on the same data set are compared, as shown in Table 6.

[0128] Table 6 Comparison with other traditional vulnerability detection methods

[0129]

[0130] Based on the statistics in Table 6, the following conclusions are drawn. First, the comprehensive performance of the other four existing traditional vulnerability detection methods in detecting these four types of vulnerabilities is relatively limited, with the highest accuracy rate being only 70.21%. In contrast, the accuracy of the present invention reaches a maximum of 78.96%, which is 9.75% higher than that of traditional tools. In addition, the average recall rate of the present invention exceeds 90%, indicating that it can detect most contract vulnerabilities. However, the recall rates of other vulnerability detection methods for these four types of vulnerabilities vary significantly, with the lowest being only 23.04%, which indicates that the use of these methods may not be able to fully discover or completely detect potential vulnerabilities in the contract.

[0131] The above phenomenon can be explained by analyzing the characteristics of each method as follows: SmartCheck relies primarily on strict logical rules to detect vulnerabilities, but due to its relatively simple logical rules, it performs relatively low in terms of accuracy and F1 score. Osiris combines symbolic execution and taint analysis techniques, focusing on detecting security vulnerabilities related to arithmetic problems. As a result, its F1 score for integer overflow vulnerability detection is superior to other traditional tools. In contrast, Slither generates the inheritance graph and control flow graph of the contract and converts the code into the internal representation language Slither, which enables taint and value tracking. Therefore, it outperforms other traditional detection tools in vulnerability detection accuracy.

[0132] The present invention provides a smart contract vulnerability detection method based on large language model guidance. By performing data flow and control flow analysis on the smart contract, the data dependency relationships in the smart contract are deeply mined, and these data dependency relationships are represented by a data dependency tree. The directed edges reflecting the data dependency relationships in the data dependency tree and the two nodes connected by the directed edges are extracted and saved to a triple table. The elements in the triple table are further marked in an abstract syntax tree to obtain an abstract syntax tree containing the data dependency relationships of the smart contract. Then, the large language model is used to learn the data dependency relationships marked in the abstract syntax tree and analyze the code structure of the smart contract to perform vulnerability detection on the smart contract. This method not only solves the problem of many missed reports or false positives in traditional vulnerability detection tools, but also overcomes the problem of insufficient analysis when using a single large language model for vulnerability detection, and significantly improves the accuracy of vulnerability detection.

Claims

1. A smart contract vulnerability detection method based on a large language model, characterized in that: The following steps are involved: S1. Use different vulnerability detection tools to perform vulnerability detection on the smart contract to obtain initial vulnerability detection data. Convert the format of the initial vulnerability detection data to obtain standard vulnerability detection data. The standard vulnerability detection data includes vulnerability number and vulnerability location information. Based on the set vulnerability classification standards, standard vulnerability detection data is classified and deduplicated to obtain classified vulnerability data, and the classified vulnerability data is connected in a linked list manner to obtain merged vulnerability data; S2. Perform data flow analysis on the smart contract to obtain a data flow diagram. Mark the variables that cause vulnerabilities in the data flow diagram based on the merged vulnerability data to obtain marked variables. Starting from the marked variables, traverse the data flow diagram and mark several variables in the data flow diagram in order based on the order of data flow to obtain several influencing variables at different levels. Several influencing variables at different levels are treated as nodes at different levels. Based on the data flow relationship, directed edges are added between nodes at adjacent levels to build a data dependency tree. S3. Perform control flow analysis on the smart contract to obtain a control flow graph. Based on the codes in the nodes at different levels in the data dependency tree, mark the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, backtrack along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks. Extract key variables that affect the execution order of the code blocks in the predecessor code blocks, and add the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree; S4. Compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by the directed edges based on the updated data dependency tree, construct a triple table, and mark the abstract syntax tree based on the elements of the triple table to obtain a second abstract syntax tree; S5. Set a prompt template for the large language model based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

2. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S5, a prompt template of the large language model is set based on the second abstract syntax tree, the smart contract source code, the vulnerability type, and the set response template. The specific content of the prompt template is: You are a smart contract vulnerability detection tool. The following are several common smart contract vulnerabilities [VULS]. Next, I will provide a smart contract source code and its abstract syntax tree containing dependencies. The tags in the abstract syntax tree represent the data dependencies in the smart contract. Please carefully analyze the smart contract's abstract syntax tree and its data dependencies, and determine whether the smart contract contains the aforementioned vulnerability types. Your response should conform to the provided response template, and please analyze it in detail step by step. The smart contract's source code is [CONTRACT], the abstract syntax tree is [AST], and the response template is [REPLY].

3. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S2, several influencing variables of different levels are obtained, and the specific process is as follows: Starting from the marked variable, traverse the data flow diagram, mark the variables whose data flow points to the marked variable, and obtain the first-level influencing variables. Continue to traverse the data flow diagram, mark the variables whose data flow points to the first-level influencing variable, and obtain the second-level influencing variables. Repeat the above process until all variables with a data flow relationship with the marked variable are graded and marked, and several influencing variables of different levels are obtained.

4. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S1, the standard vulnerability detection data is classified based on the set vulnerability classification standard. The process is as follows: Create three types of dictionaries: the first type is the vulnerability with the same number but different locations, the second type is the vulnerability with the same number and location, and the third type is the vulnerability with different numbers but the same location. According to the vulnerability number and location information, the classified standard vulnerability detection data are stored in the dictionary of the corresponding category.

5. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S1, the classified vulnerability data is connected in a linked list manner, specifically: Create an empty table T. For each newly encountered vulnerability number in the classified vulnerability data, create a linked list L to store the vulnerability location information. Add the vulnerability number and the corresponding linked list L to table T. Traverse all classified vulnerability data. For classified vulnerability data that matches the existing vulnerability number in table T: check whether its vulnerability location information has been stored in the corresponding linked list L. If not saved, add this vulnerability location information to the corresponding linked list L.

6. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: The data dependency tree is constructed in S2, and the specific process is as follows: The label variable is used as the root node, the first-level influencing variable as the first-level leaf node, the second-level influencing variable as the second-level leaf node, and so on, to obtain nodes of different levels with data flow relationships. Based on the data flow relationship, directed edges are added between nodes of adjacent levels to construct a data dependency tree.

7. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: The key variables in S3 include: conditional judgment statements and timestamps.

8. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S3, the control flow analysis of the smart contract is performed to obtain a control flow graph, specifically: Based on the executable code blocks in the smart contract and the execution paths between different executable code blocks, a control flow graph is obtained. The nodes of the control flow graph are the executable code blocks, and the edges of the control flow graph are the execution order of the code blocks.

9. The smart contract vulnerability detection method based on large language model guidance according to claim 1 is characterized in that: In S1, different vulnerability detection tools are used to detect vulnerabilities in smart contracts, specifically: Use Oyente and Mythril to perform static analysis on the compiled smart contracts, and use Confuzzius and sFuzz to perform dynamic analysis on the compiled smart contracts to obtain initial vulnerability detection data.

10. A smart contract vulnerability detection system based on a large language model, characterized in that: include: Vulnerability data processing module: used to perform vulnerability detection on smart contracts using different vulnerability detection tools, obtain initial vulnerability detection data, convert the format of the initial vulnerability detection data, obtain standard vulnerability detection data, which includes vulnerability number and vulnerability location information; classify and remove duplicates from the standard vulnerability detection data based on the set vulnerability classification standard to obtain classified vulnerability data, and connect the classified vulnerability data in a linked list manner to obtain merged vulnerability data; Explicit dependency data extraction module: used to perform data flow analysis on smart contracts, obtain a data flow graph, mark the variables that cause vulnerabilities in the data flow graph based on the merged vulnerability data, obtain marked variables, traverse the data flow graph with the marked variables as the starting point, and mark several variables in the data flow graph in order based on the order of data flow to obtain several influencing variables of different levels. The influencing variables of different levels are respectively used as nodes of different levels. Directed edges are added between nodes of adjacent levels based on the data flow relationship to construct a data dependency tree; Implicit dependency data extraction module: This module is used to perform control flow analysis on smart contracts and obtain a control flow graph. Based on the code in the nodes at different levels in the data dependency tree, the module marks the corresponding executable code blocks in the control flow graph as tainted blocks. Starting from the tainted blocks, the module backtracks along the execution order of the code blocks in the control flow graph to obtain the predecessor code blocks pointing to the tainted blocks. The module extracts the key variables in the predecessor code blocks that affect the execution order of the code blocks, and adds the key variables as new nodes to the data dependency tree to obtain an updated data dependency tree. Abstract syntax tree construction module: used to compile the smart contract to generate a first abstract syntax tree, extract directed edges and two nodes connected by directed edges based on the updated data dependency tree, build a triple table, mark the abstract syntax tree based on the elements of the triple table, and obtain a second abstract syntax tree; Vulnerability detection module: used to set the prompt template of the large language model based on the second abstract syntax tree, smart contract source code, vulnerability type, and set response template, use the large language model to perform vulnerability detection, and obtain vulnerability detection results.

Images