A coverage-guided smart contract test case generation method
Through the coverage-guided smart contract test case generation method, combined with static analysis and genetic algorithm, test cases covering more vulnerable statements are generated, which solves the problem of high false alarms and missed response rates in smart contract detection, and improves detection efficiency and coverage Rate.
Patent Information
- Application Number
- CN202111311599.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-08
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-11-08
AI Technical Summary
The existing smart contract vulnerability detection methods have problems such as high false alarm rate and few branches covered by test cases, which lead to poor detection effect, high false alarm rate and false alarm rate, and low detection efficiency.
Using the smart contract test case generation method with coverage-guided smart contracts, scan the smart contract source code through static analysis tools, extract suspicious vulnerability statements, and combine code coverage information to generate test cases covering more vulnerability statements using genetic algorithms to reduce false alarms and underreport rate.
It effectively reduces the false alarm and missed rate in vulnerability detection, improves the efficiency of smart contract vulnerability detection, significantly reduces the false alarm and missed rate, and improves the coverage of test cases.
Smart Images

Figure CN113971135B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of software testing technology, and specifically relates to the problem of vulnerability detection in smart contracts. It provides a coverage-guided smart contract test case generation method, which mainly solves the problems of high false alarm rate and small number of branches covered by test cases in existing smart contract vulnerability static detection methods, and realizes efficient detection of smart contract vulnerabilities. Background Art
[0002] At present, blockchain technology has entered the blockchain 2.0 era marked by smart contracts. Smart contracts are Turing-complete programs executed on blockchains. At the same time, they have broad application prospects in digital payments, financial asset disposal, cloud computing, the Internet and shared economy, and they have the characteristics of determinism, real-time and decentralization. Like traditional Java and C programs, smart contracts have vulnerabilities, but unlike them, once smart contracts are deployed on blockchains, they cannot be modified, so the vulnerabilities in smart contracts will always exist, making them more vulnerable to attacks. In recent years, there have been more and more news reports on attacks on smart contracts. On June 17, 2016, the largest crowdfunding project in the blockchain industry, the Decentralized Autonomous Organization (The DAO), was attacked. The DAO was attacked due to a vulnerability in the splitDAO function, which eventually led to the separation of more than 3 million ether assets from the DAO asset pool. Parity wallet suffered two vulnerabilities, the first of which resulted in a loss of $60 million, and the second resulted in the freezing of more than $150 million in ether. In 2018, the US$10 billion project of MeChain was instantly reduced to zero. The security issues of smart contracts have caused serious losses to the blockchain community. Therefore, it is particularly important to study the security issues of smart contracts.
[0003] The 10 common security issues in smart contracts summarized by NCC Group are: code reentry, unchecked function call return value, integer overflow, ether freezing, short address attack, improper access control, timestamp dependency, block variable dependency, transaction order dependency (TOD), and block gas limit. Relying on the skill level of developers and the security of code audits based on expert experience can no longer meet the needs of vulnerability analysis and discovery under the new situation of the continuous expansion of the number and scale of smart contracts and the increase in the difficulty of vulnerability mining. Therefore, a large number of smart contract vulnerability automated detection tools continue to emerge. The use of automated vulnerability detection methods can not only more efficiently discover security vulnerabilities in contracts, but also meet the needs of vulnerability analysis and discovery. At present, the mainstream methods for automated detection of smart contract vulnerabilities include static analysis, symbolic execution, and fuzz testing.
[0004] Static analysis does not require dynamic execution of contract code. It only extracts and analyzes the semantic information in the source code through static analysis of the contract source code to detect vulnerabilities in the contract. Slither is a static detection tool for smart contract vulnerabilities. Based on the static analysis of the contract source code, it provides users with two modules: information printing and anomaly detection for smart contract analysis. Its anomaly detection can detect anomalies in more than 40 smart contracts, but it fails to detect business logic vulnerabilities such as Ethereum freezing, and there are false positives.
[0005] As a program analysis technology, the key idea of symbolic execution is to use symbolic values instead of input data, thereby converting the input of the program into a function of symbolic input. After years of continuous development, symbolic execution has played a very important role in software testing, vulnerability detection and other fields. It is also used in the field of smart contract vulnerability detection. Oyente is a smart contract vulnerability analysis tool based on dynamic symbolic execution. It can run directly on the Ethereum virtual machine bytecode. Oyente currently supports the detection of TOD, timestamp dependency, integer overflow and code reentrancy vulnerabilities. In order to prevent path explosion, Oyente simplifies the processing of loops, resulting in the underreporting of some vulnerabilities.
[0006] Fuzz testing is a security testing method that runs the program under test by inputting some unexpected data to monitor whether the contract will be abnormal in the Ethereum virtual machine. ContractFuzzer is the first fuzz testing tool based on Ethereum platform smart contract vulnerability detection. The vulnerability detection tool first compiles the smart contract code into bytecode instructions, statically analyzes the smart contract application binary interface, and generates test cases. Then the instruction code is dynamically executed through the test case to detect whether there are vulnerabilities. The tool can currently only detect 7 types of vulnerabilities, but due to the randomness of test case generation, it cannot achieve the ideal coverage rate and it is difficult to find all potential vulnerabilities.
[0007] In order to improve the coverage of test cases in fuzz testing, sFuzz adopts a feedback adaptive fuzz strategy to achieve the maximum path coverage of the generated test cases. sFuzz draws on the traditional fuzz testing tool AFL (American FuzzyLop, AFL), uses genetic algorithms to improve the search-based strategy to obtain its fitness algorithm, and then proposes a feedback-guided adaptive fuzz strategy, that is, to find the test case closest to the uncovered path. This detection method improves the path coverage of test cases. sFuzz supports the detection of most common vulnerabilities, and can also detect vulnerabilities in business logic such as Ethereum freezing, but the coverage of sFuzz test cases is only about 80%, and there are still underreporting for some vulnerabilities.
[0008] As mentioned above, although there are some research tools for Ethereum smart contract vulnerability detection, they also show some obvious shortcomings: poor detection effect, high false positive rate and false positive rate, large detection overhead, low detection efficiency, the reason is that the coverage of test cases is not enough. Therefore, the present invention proposes a coverage-guided smart contract test case generation method, which guides the evolution direction of test cases according to static analysis results, thereby generating test cases that cover more vulnerability statements and reduce false negatives in vulnerability detection. Summary of the invention
[0009] Problem solved by the present invention: Overcome the problems of high false alarm rate and missed alarm rate in the existing smart contract vulnerability detection method, perform static analysis on the smart contract, and guide the evolution direction of test cases in the genetic algorithm according to the static detection results, so that it can generate test cases covering more vulnerability statements, effectively reduce the false alarms and missed alarms in the vulnerability detection method, and improve the vulnerability detection efficiency of the smart contract.
[0010] The smart contract in Ethereum runs in the Ethereum virtual machine. For the smart contract under test, a blockchain network will be automatically configured for it and the contract will be deployed. The process of the coverage-guided smart contract test case generation method proposed in the present invention is as follows: Figure 1 As shown:
[0011] (1) Use the Slither static vulnerability analysis tool to perform static analysis on the smart contract source code, and record the line number information of the vulnerable statements in the static analysis results and the detected contract vulnerabilities in the static vulnerability detection information file;
[0012] (2) Use the Solc compiler to compile the smart contract under test, generate the smart contract application binary interface and Ethereum virtual machine bytecode, and randomly generate test cases based on the smart contract application binary interface;
[0013] (3) Insert stubs into the bytecode of the smart contract under test in the Ethereum virtual machine, insert function stubs into each function of the smart contract under test, insert branch stubs into each branch point in the function, and deploy the inserted smart contract code in the Ethereum Geth node;
[0014] (4) According to the generated test case, run the smart contract code after the stub, monitor the coverage information of each test case in the Ethereum virtual machine, and determine whether the test case detects a contract vulnerability after all the current test cases are executed or within the user-defined vulnerability detection time. If a vulnerability is detected, output the value of each incoming parameter and the detected contract vulnerability. Otherwise, save the number of branches covered by the test case in the smart contract vulnerability detection library. The information stored in the library includes the number of vulnerable statements covered by each test case, the line number information of the vulnerable statements, and the number of branches covered. At the same time, according to the static vulnerability detection information file, determine whether the line number information of the branches covered by the test case is equal to the line number information of the vulnerable statements in the static vulnerability detection information file. If they are equal, the number of vulnerable statements covered by the test case is also saved in the smart contract vulnerability detection library, and enter step (5);
[0015] (5) According to the number of vulnerability statements and branches covered by each test case in the smart contract vulnerability detection library, the fitness value of the test case is calculated. The fitness value calculation is represented by Fitness.
[0016] Fitness=α1*vCov+α2*nCov
[0017] Among them, vCov represents the number of vulnerability statements covered by the test case, nCov represents the number of branches covered by the test case, α1 and α2 represent the proportion of vCov and nCov in Fitness, and the fitness value is calculated according to the coverage information of the test case. The test cases are sorted from large to small according to the fitness value. The larger the fitness value, the better the gene it contains.
[0018] (6) Calling the crossover mutation operation to generate new test cases; using the crossover operation on the first half of the sorted test case set: selecting the first test case and the second test case as a group, the third test case and the fourth test case as a group, and so on for the rest, and then dividing the selected test cases in the same group into two parts at a randomly selected position, and exchanging the second part to form two new test cases; using one of the following mutation operations on the second half of the sorted test case set: flipping randomly selected bits or bytes, adding or replacing selected random bytes, replacing a value at a random position with a constant in the smart contract, and replacing the address or balance in the test case with a randomly generated address or balance;
[0019] (7) Send the new test cases one by one to the inserted executable code and return to step (4).
[0020] Beneficial effects:
[0021] At present, the existing static analysis vulnerability detection methods have the problem of many false positives and missed negatives, the main reason for which is that the coverage of test cases is insufficient. For this reason, the present invention adopts a coverage-guided smart contract test case generation method. First, a static analysis tool is used to scan the code to extract suspicious contract vulnerability statements, which may be false positives. Then, these contract vulnerability statements are combined with code coverage information and introduced into the fitness calculation, thereby shifting the test direction from achieving a higher code coverage to contract vulnerability statement coverage, avoiding a lot of useless work and improving test efficiency.
[0022] The experimental results of typical examples confirm the effectiveness of this method: all contracts in the Smart Contract Weakness Classification (SWC) are selected to count the smart contract vulnerability detection results of Slither, sFuzz, and this method. SWC is a necessary knowledge base for Ethereum developers. It is a classification and test case set of Ethereum smart contract security vulnerabilities provided by the Smart Contract Security Alliance. It contains descriptions and consequences of 37 common security vulnerabilities in Ethereum Solidity smart contract development, such as reentrancy vulnerabilities, integer overflow vulnerabilities, etc. At the same time, the crossover probability is set to 0.9, the mutation probability is 0.05, the number of individuals in the population is 1000, the number of individual genes is 14, and the weights of the fitness function are α1=2, α2=1. The smart contract vulnerability detection results of the three methods are shown in Table 1. Among them, TW, FW, and FN represent the number of correct vulnerability detections, the number of false positives in vulnerability detection, and the number of missed vulnerability detections, respectively.
[0023] Table 1 Smart contract vulnerability detection status of three detection methods
[0024]
[0025] The false positive and false negative rates of the three detection methods are as follows: Figure 2 As shown, the false alarm rate = FW / (FW+TW), and the false negative rate = FN / (TW+FN). Compared with Slither, the false alarm rate of this method is reduced by 73.7%, and compared with sFuzz, the false negative rate of this method is reduced by 48.3%. Therefore, this method can effectively reduce the false alarm rate and false negative rate of vulnerability detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Attached Figure 1 Flow chart of the method of the present invention.
[0027] Attached Figure 2 Vulnerability detection missed detection rate and false positive rate graph for three detection methods. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the technical solution of the present invention, the coverage-guided smart contract test case generation method provided by the present invention is described in detail below.
[0029] (1) Save the tested smart contract in Table 2 in the FreezingEtherSample.sol file;
[0030] Table 2 Tested smart contract code
[0031]
[0032] (2) Use Slither to statically detect the contract under test in the FreezingEtherSample.sol file. Slither runs in a docker container. Use the following docker command to enter its running environment, where 965b1e4ca92e represents the id number of the docker container corresponding to Slither:
[0033] sudo docker start 965b1e4ca92e;
[0034] sudo docker attach 965b1e4ca92e;
[0035] Enter the Slither runtime environment and enter the slither FreezingEtherSample.sol command (where FreezingEtherSample.sol represents the contract under test) to output the static detection results of the smart contract under test and save them in the static vulnerability detection information file. The results are marked with the line number information of the vulnerable statement and the detected contract vulnerability.
[0036] (3) Use the Solc compiler to compile the smart contract under test into a FreezingEtherSample.json file, which contains the application binary interface of the smart contract and the compiled bytecode of the smart contract. Parse the application binary interface of the smart contract under test and extract all function signatures. The function signature includes the function description and the data type of each function parameter. If the data type of the smart contract function parameter is an address type, first generate a private smart contract pool for each application binary interface of the smart contract under test. The smart contract pool stores the addresses of all smart contracts that support the functions in the application binary interface. Then randomly select the smart contract address stored in the smart contract pool as the test address. Example; if the data type of the smart contract function parameter is a fixed-length parameter, randomly select a set of values from the parameter's legal test data domain as test cases; if the data type of the smart contract function parameter is an indefinite-length parameter, first randomly generate a positive number as the length of the parameter, and then randomly select a set of values from the legal test case domain for each element in the parameter as test cases. The test case contains σ0 and ∑, σ0 is the blockchain network configuration, ∑ is the transaction sequence (function call), σ0 contains all information about the network settings related to the execution of the smart contract, σ0 = (b, ts, SA, SB, v), where b represents the current block number, ts represents the current block timestamp, SA represents a series of smart contract addresses, SB is a function that assigns an initial balance to each address, and v is the initial valuation of the constant variable. It is a series of public function calls of the smart contract under test, each call has an optional specific input test sequence, and each test has an optional specific input parameter sequence, where m0 must be a call to the constructor;
[0037] (4) The initial σ0 blockchain network configuration is set to:
[0038] b=0,ts=0,sa=0xf0,sb=0xff000000000000000000000000,v=0. The contract under test generates 2 sequences and r function call sequences (1<r<2×n). Then, according to the application binary interface in FreezingEtherSample.json, a set of values is randomly selected from the legal test data field of the parameter for the value variable of type uint256 as the parameter value of the function call.
[0039] (5) Insert branch stubs for each branch point in the function in the bytecode of the smart contract under test, where the trace_cov array used records the coverage count of each branch of the current test case; the virgin_cov global coverage information array is used to record the coverage information of all executed test cases;
[0040] (6) Deploy the instrumented contract executable code in the Ethereum Geth node, monitor the coverage information of each test case in the Ethereum virtual machine, set the vulnerability detection time to 1800 milliseconds, execute the test case, and determine whether the test case detects a contract vulnerability. If a vulnerability is detected, output the value of each incoming parameter and the detected contract vulnerability. Otherwise, save the number of branches covered by the test case in the smart contract vulnerability detection library. The information stored in the library includes the number of vulnerable statements covered by each test case, the line number information of the vulnerable statements, and the number of branches covered. At the same time, based on the static vulnerability detection information file, determine whether the line number information of the branches covered by the test case is equal to the line number information of the vulnerable statements in the static vulnerability detection information file. If they are equal, the number of vulnerable statements covered by the test case is also saved in the smart contract vulnerability detection library, and enter step (7);
[0041] (7) According to the number of vulnerability statements and branches covered by each test case in the smart contract vulnerability detection library, the first test case vCov = 0, nCov = 2, the second test case vCov = 0, nCov = 4, the third test case vCov = 1, nCov = 6, and the fourth test case vCov = 3, nCov = 9 are obtained. Finally, the fitness value is calculated, and the parameters α1 = 2, α2 = 1 are taken. The fitness of the first test case is calculated to be 2, the fitness of the second test case is 4, the fitness of the third test case is 8, and the fitness of the fourth test case is 15. Finally, the test case set is sorted from large to small according to the fitness value;
[0042] (8) Call crossover and mutation operations to generate new test cases. For the first two test cases in the sorted test case set, the crossover operation is used: the two test cases are divided into two parts at the selected position, and the second part is exchanged to form two new test cases; the mutation operator is shown in Table 3. For the last two test cases in the sorted test case set, one of the following mutation operations is used: flip the randomly selected bit or byte, add or replace the selected random byte, replace a value with a constant in the smart contract at a random position, and replace the address or balance in the test case with a randomly generated address or balance;
[0043] Table 3 Mutation operators
[0044]
[0045] Send the new test cases one by one to the instrumented executable code, and finally return to step (6).
Claims
1. A coverage-guided smart contract test case generation method, characterized by the following steps: (1) Use the Slither static vulnerability analysis tool to perform static analysis on the smart contract source code, and record the line number information of the vulnerable statements in the static analysis results and the detected contract vulnerabilities in the static vulnerability detection information file; (2) Use the Solc compiler to compile the smart contract under test, generate the smart contract application binary interface and Ethereum virtual machine bytecode, and randomly generate test cases based on the smart contract application binary interface; (3) Insert stubs into the bytecode of the smart contract under test in the Ethereum virtual machine, insert function stubs into each function of the smart contract under test, insert branch stubs into each branch point in the function, and deploy the inserted smart contract code in the Ethereum Geth node; (4) According to the generated test case, run the smart contract code after the stub, monitor the coverage information of each test case in the Ethereum virtual machine, and determine whether the test case detects a contract vulnerability after all the current test cases are executed or within the user-defined vulnerability detection time. If a vulnerability is detected, output the value of each incoming parameter and the detected contract vulnerability. Otherwise, save the number of branches covered by the test case in the smart contract vulnerability detection library. The information stored in the library includes the number of vulnerable statements covered by each test case, the line number information of the vulnerable statements, and the number of branches covered. At the same time, according to the static vulnerability detection information file, determine whether the line number information of the branches covered by the test case is equal to the line number information of the vulnerable statements in the static vulnerability detection information file. If they are equal, the number of vulnerable statements covered by the test case is also saved in the smart contract vulnerability detection library, and enter step (5); (5) According to the number of vulnerability statements and branches covered by each test case in the smart contract vulnerability detection library, the fitness value of the test case is calculated. The fitness value calculation is represented by Fitness. Fitness=α1*vCov+α2*nCov Among them, vCov represents the number of vulnerability statements covered by the test case, nCov represents the number of branches covered by the test case, α1 and α2 represent the proportion of vCov and nCov in Fitness, and the fitness value is calculated according to the coverage information of the test case. The test cases are sorted from large to small according to the fitness value. The larger the fitness value, the better the gene it contains. (6) Calling the crossover mutation operation to generate new test cases, and applying the crossover operation to the first half of the sorted test case set: select the first test case and the second test case as a group, the third test case and the fourth test case as a group, and so on, and then divide the selected test cases in the same group into two parts at a randomly selected position, and exchange the second part to form two new test cases; applying one of the following mutation operations to the second half of the sorted test case set: flipping randomly selected bits or bytes, adding or replacing selected random bytes, replacing a value at a random position with a constant in the smart contract, and replacing the address or balance in the test case with a randomly generated address or balance; (7) Send the new test cases one by one to the inserted executable code and return to step (4).
Citation Information
Patent Citations
Smart contract testing method based on path coverage sufficiency criterion
CN110046089A
Intelligent contract transaction sequence dependent vulnerability variation fuzzy test method
CN111563742A
Intelligent contract test case generation method applying improved genetic algorithm
CN111797010A
Intelligent contract vulnerability detection method based on combination of neural network and dynamic fuzzy test
CN113326050A
