A smart contract state extraction method combining static and dynamic analysis

Abstract

The present invention discloses a method for extracting the state of a smart contract by combining static and dynamic analysis. The smart contract source code is parsed by a static analysis tool to generate a state record table, and the state record table is deployed together with the smart contract in the blockchain. A full node with the state record table is deployed, and all transactions in the block are run in sequence. A dynamic analysis tool is used to maintain the storage layout information of the dynamic type state in real time, and the state record table is updated. After all transactions in the block are executed, the full node with the state record table deployed persists the state record table in the memory to the database. The present invention only needs to deploy the service on a full node, and brings almost no overhead to the blockchain. The full node synchronization mechanism is used instead of using an API to obtain historical transactions, and the problems of API speed limit and failure are avoided.

Description

Technical Field

[0001] This application belongs to the field of computer data information processing technology, and in particular relates to a method for extracting the state of smart contracts by combining static and dynamic analysis. Background Technology

[0002] The rise of blockchain technology in recent years has brought tremendous changes to various industries. Smart contracts, which can execute arbitrary business logic without relying on a trusted third party, are widely used in finance, supply chain, healthcare, and insurance. A smart contract is a program that runs on a specific address on the blockchain; it consists of data and functions and can be executed upon receiving a transaction. Solidity, a high-level programming language created for implementing smart contracts, has emerged and is widely used. Once deployed on the blockchain, smart contracts cannot be updated or upgraded at the same address. However, in real-world scenarios, it is necessary to upgrade smart contracts to fix vulnerabilities in the code or add new functionality. The key challenge is not only making code changes in the upgraded version but also reusing and retrieving the data (state) stored in the variables of the original smart contract, but the blockchain system does not maintain the storage location of this state.

[0003] To address this issue, researchers have sought various solutions to accurately extract all states from smart contracts. In smart contracts, state variables are allocated storage space in 32-byte slots according to the order of their declaration; two adjacent states smaller than 32 bytes are stored in the same slot. Therefore, some static analysis tools and decompilers take smart contract code or bytecode as input and calculate the storage location of states by analyzing the Ethereum Virtual Machine's (EVM) storage management. However, they cannot extract key-value mapping states because the storage location of the mapping variable's "value" is determined by the "key," which may be dynamically generated during transaction execution and therefore cannot be statically obtained. To extract mapping-type states, some dynamic analysis tools retrieve and replay all historical transactions from the blockchain API, performing reachability analysis and backtracking algorithms on the code's control flow graph to approximate the possible paths for key generation, thereby obtaining the set of keys and their storage locations. However, due to API rate limiting and failures, these tools struggle to ensure fast and accurate acquisition of all historical transactions. More importantly, their accuracy is not 100%, which could mean the loss of users' assets in real-world scenarios. Therefore, there is an urgent need for a tool that can accurately extract the state of smart contracts to meet the needs of contract upgrades and state migrations in real-world scenarios. Summary of the Invention

[0004] The purpose of this application is to overcome the shortcomings of existing technologies and provide a combined static and dynamic analysis approach to extract the state of all variable types in smart contracts. This method eliminates the need for developers to possess any additional EVM storage management knowledge. The developed smart contract's relevant state information is automatically extracted by static analysis technology, and only a single full node with state extraction capabilities is required to obtain any state information, without needing to fork and upgrade all nodes in the entire blockchain.

[0005] To achieve the above objectives, the technical solution of this application is as follows:

[0006] A method for extracting the state of smart contracts by combining static and dynamic analysis, comprising:

[0007] Static analysis tools are used to parse the source code of the smart contract into an abstract syntax tree. Then, the storage layout information of each state is recorded in the state record table, and the state record table is deployed to the blockchain along with the smart contract.

[0008] A full node with a state record table is deployed, and a fully synchronized process is started to listen to the blockchain network, download historical blocks, read transactions in the blocks, identify the smart contract address to which the transaction was sent, and load the corresponding state record table into memory.

[0009] Full nodes with deployed state log tables run all transactions in a block sequentially, and use dynamic analysis tools to maintain the storage layout information of dynamic type states in real time and update the state log tables.

[0010] After all transactions within a block have been executed, the full node that deployed the state record table persists the state record table in memory to the database.

[0011] Furthermore, the state record table uses the following general data structure to record the storage layout information of the state:

[0012] Status record table = {variable name, variable type, {slot, offset}, initial slot, layer number, pointer}.

[0013] Furthermore, the deployment of the state record table along with the smart contract into the blockchain includes:

[0014] Smart contracts send regular transactions to the blockchain through the developer's account and are deployed on all full nodes;

[0015] The status log table is deployed to a full node with dynamic analytics enabled. Developers specify the IP address and port number of the full node with dynamic analytics enabled, and send a special transaction to deploy the status log table.

[0016] Furthermore, the step of using dynamic analysis tools to maintain the storage layout information of dynamic type states in real time and updating the state record table includes:

[0017] Obtain the opcode. If it is a SHA3 opcode, determine whether the length of the SHA3 opcode parameter is greater than or equal to 32.

[0018] If the length of the SHA3 opcode parameter is greater than or equal to 32, the initial slot is calculated based on the SHA3 opcode parameter. Then, the status record table is queried. If the initial slot is matched in the status record table, the slot in the status record table is updated with the slot result calculated by the SHA3 opcode.

[0019] Furthermore, the step of using static analysis tools to parse the smart contract source code into an abstract syntax tree, and then recording the storage layout information of each state in a state record table, also includes:

[0020] The variables in the abstract syntax tree are traversed and analyzed sequentially according to their declaration order;

[0021] Variables are classified into static or dynamic types, and the storage layout information of the corresponding states is extracted and recorded in the state record table.

[0022] For composite type variables, they are parsed into static or dynamic types, and then the storage layout information of the corresponding states is extracted and recorded in the state record table.

[0023] This application proposes a method for extracting smart contract states that combines static and dynamic analysis. This method clearly and completely records the storage space of smart contract states, providing technical support for contract upgrades or data migration. Its advantages include: developers do not need to understand the storage space management mechanisms of various variable types; they can easily and quickly obtain all the states of the smart contract by directly accessing the database; the system is installed as an external service on a full node, ensuring that all transaction runtime information is monitored and identified, thus accurately identifying the storage space of mapping variables; the system implementation only requires deploying the service on a single full node, incurring almost no overhead on the blockchain; and it utilizes the full node synchronization mechanism instead of using an API to obtain historical transactions, avoiding issues such as API rate limiting and failures. Attached Figure Description

[0024] Figure 1 This is a flowchart of the smart contract state extraction method in this application.

[0025] Figure 2 This is a flowchart of the static analysis of an embodiment of this application.

[0026] Figure 3 This is a flowchart illustrating the dynamic analysis process in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] One embodiment of this application proposes a method for extracting the state of smart contracts that combines static and dynamic analysis, such as... Figure 1 As shown, it includes:

[0029] Step S1: Use static analysis tools to parse the smart contract source code into an abstract syntax tree, then record the storage layout information of each state into a state record table, and deploy the state record table and the smart contract together into the blockchain.

[0030] The smart contracts that developers write locally are named with the .sol extension. These are then used as input to run static analysis tools, requiring no additional work.

[0031] This application uses a smart contract written in Solidity as an example. Solidity is a contract-oriented high-level programming language used to implement smart contracts. In a smart contract written in Solidity, variables declared outside the function body are called state variables. Their data will be persistently stored on the blockchain, and this data is also called state.

[0032] To comprehensively handle all variable types in smart contracts written in Solidity, variable types are divided into two categories: static types and dynamic types. The basis for this is whether the storage space of the variable changes after the contract is deployed. Variables whose storage space does not change after contract deployment are static types, while those whose storage space changes are dynamic types.

[0033] Static types include boolean types, integer types, address types, static arrays, enumeration types, etc.; dynamic types include dynamic arrays (array, string, byte) and mapping types. Notably, there are user-defined types and struct types, which are renamings or combinations of the aforementioned static and dynamic types.

[0034] To comprehensively and accurately maintain the state storage space for various variable types, this application records the storage layout information of each state in a state record table. A general data structure is introduced into the state record table to record the storage layout information of the state:

[0035] Status record table = {variable name, variable type, {slot, offset}, initial slot, layer number, pointer};

[0036] The variable name and variable type are used to record which variable's state is maintained in the current state record table. The slot is used to record the storage address allocated to the variable. The offset is used to record the precise position of the state within the slot when multiple states share a slot. The pointer is used to handle the inner variable when dealing with nested dynamic arrays or mapping variables. The initial slot records the initial slot of the dynamic type state, and the level number is used to record the number of times the variable is nested, i.e., it is composed of several basic dynamic or static types.

[0037] In one specific embodiment, the step of using static analysis tools to parse the smart contract source code into an abstract syntax tree, and then recording the storage layout information of each state into a state record table, further includes:

[0038] The variables in the abstract syntax tree are traversed and analyzed sequentially according to their declaration order;

[0039] Variables are classified into static or dynamic types, and the storage layout information of the corresponding states is extracted and recorded in the state record table.

[0040] For composite type variables, they are parsed into static or dynamic types, and then the storage layout information of the corresponding states is extracted and recorded in the state record table.

[0041] Specifically, such as Figure 2 As shown, the static analysis tool first automatically parses the smart contract code and generates an Abstract Syntax Tree (AST). The AST stores the various components of the code in a tree structure, and its traversability enables static analysis. Then, it sequentially traverses and analyzes the name and type of each variable v according to the order of variable declaration, recording them in the first two fields of the state record table.

[0042] For statically typed variables, if their state occupies 32 bytes or more, one or more slots can be directly allocated. If it's less than 32 bytes, slot calculation is used to determine whether the variable's state shares a slot with other variables. Specifically, slot calculation is implemented using a 32-byte stack. Each variable less than 32 bytes is pushed onto the stack. If the next variable can be pushed onto the stack, their states are considered to be stored in the same slot, and their respective offsets are recorded; otherwise, the variable is popped from the stack. At this point, the storage space for statically typed variables has been fully extracted and will not change subsequently.

[0043] For dynamically typed variables, in addition to the variable name and type, their initial slot and nesting level are extracted. The initial slot is the slot occupied by the variable at its declaration location, and the nesting level is the number of times it is nested. For example, the variable `mapping(string=>mapping(uint=>bool))` will be resolved by the EVM into an outer `mapping(string=>uint)` and an inner `mapping(uint=>bool)`, thus having a nesting level of 2. During the static analysis phase, the state log table only needs to record the initial slot of the outer `mapping` variable, while the initial slot of the inner variable is dynamically maintained during the dynamic analysis phase.

[0044] For composite type variables, they are parsed into their internal basic type (static or dynamic) and then processed accordingly.

[0045] The state log table generated by static analysis is deployed to the blockchain along with the smart contract. The smart contract sends ordinary transactions to the blockchain through the developer's account and is deployed on all full nodes. Unlike the smart contract, the state log table needs to be deployed to special full nodes that have dynamic analysis services enabled.

[0046] When deploying the state log table, the developer specifies the IP address and port number of the full node and sends a special transaction for deploying the state log table. Similar to a normal transaction, the special transaction contains three fields: "To" specifies the contract address, "DATA" contains the state log table, and "FROM" indicates the developer's address. To distinguish it from a normal transaction, the "DATA" field begins with an additional 5-byte identifier (0x0000000022). In a normal transaction, the first four bytes of the "DATA" field are the signature of the external function and cannot be zero, thus distinguishing which transactions are used to deploy the state log table. Upon receiving the special transaction, the node verifies its identity by checking if the deployers in "FROM" and "To" match.

[0047] Step S2: Deploy a full node with a state record table, start a full synchronization process to listen to the blockchain network, download historical blocks, read transactions in the blocks, identify the smart contract address to which the transaction was sent, and load the corresponding state record table into memory.

[0048] This embodiment deploys a full node with a state record table. The full synchronization process is initiated through a local blockchain client program. After full synchronization is initiated, the node will listen to the blockchain network and download historical blocks.

[0049] A full node with a deployed state log table will read all transactions in a block. Based on the "TO" field in the transaction structure, it can identify which smart contract address the transaction is destined for. Using these contract addresses, the full node loads the corresponding state log table into memory, waiting for the EVM to execute these transactions and simultaneously update the state log table.

[0050] Thanks to the blockchain's own full synchronization mechanism, this full node does not need to actively call the blockchain API to obtain historical transaction data, so there is no need to worry about data loss or API rate limits. The full synchronization mechanism verifies multiple hash values ​​in the block header to ensure the global consistency and integrity of the block data. Therefore, the node will not miss any on-chain transactions, and the obtained block data is consistent with the latest data on the current blockchain.

[0051] Step S3: The full node with the deployed state record table runs all transactions in the block sequentially, and uses dynamic analysis tools to maintain the storage layout information of dynamic type states in real time and update the state record table.

[0052] For each block downloaded, a full node that has deployed a state record table needs to execute all transactions in the block inside the virtual machine to update its local blockchain database.

[0053] During the sequential execution of all transactions in the blocks, the dynamic analysis tool will maintain the storage layout information of the dynamic type variable state in real time and update the state record table.

[0054] To maintain the storage space for dynamically typed state in real time during transaction runtime, a straightforward solution is to monitor the entire opcode sequence of every assignment statement in the Solidity code. However, a simple statement will compile into a very long opcode sequence. For example, an assignment statement for a string variable (e.g., s = "1";) requires approximately 200 opcodes in the execution flow. Worse still, different statements can be compiled into different opcode sequences. Therefore, monitoring complex opcode sequences of various types is extremely complex.

[0055] Through in-depth research, it was discovered that all storage access to dynamically typed variables involves the execution of an opcode, namely SHA3, which is used to calculate the slot address. Therefore, this embodiment only needs to monitor SHA3 to discover which slots will be accessed by certain dynamically typed variables.

[0056] After loading the state record table into memory, the full node will sequentially execute each transaction in the block. For each transaction, the Virtual Machine (EVM) sequentially executes each opcode in the interpreter to implement the contract code logic. This application embeds dynamic analysis into the interpreter to monitor whether the state storage space has changed in real time during transaction execution.

[0057] In one specific embodiment, a dynamic analysis tool is used to maintain the storage layout information of dynamic type states in real time and update the state record table, including:

[0058] Obtain the opcode. If it is a SHA3 opcode, determine whether the length of the SHA3 opcode parameter is greater than or equal to 32.

[0059] If the length of the SHA3 opcode parameter is greater than or equal to 32, the initial slot is calculated based on the SHA3 opcode parameter. Then, the status record table is queried. If the initial slot is matched in the status record table, the slot in the status record table is updated with the slot result calculated by the SHA3 opcode.

[0060] Specifically, such as Figure 3 As shown, during transaction execution, when the SHA3(m) opcode is executed (if it is not a SHA3 opcode, the next opcode is checked), the program verifies its parameter (m). If the length is greater than 32 bytes, the current execution flow may be accessing a mapping type; if the length is equal to 32 bytes, the current execution flow may be accessing a dynamic array. The reason behind this is that the storage location of the mapping value is calculated using SHA3(h(key), initial slot), and the length of the SHA3 parameter in this case must be greater than 32 bytes; while the storage location of the first element of the dynamic array is calculated using SHA3(initial slot), and the length of the SHA3 parameter in this case must be equal to 32 bytes.

[0061] The initial slot can be calculated based on the SHA3 opcode parameters. If m is greater than 32, the key and the initial slot are split into 32-byte segments, with the key occupying a multiple of 32 bytes. The last 32 bytes (or less than 32 bytes) is the initial slot. If m equals 32, then m is the initial slot.

[0062] After obtaining a possible initial slot, the status record table is traversed and queried. If a matching initial slot obtained from the previous static analysis is found in the status record table, the SHA3 calculation result is the slot address to be stored, and the SHA3 calculation result is updated in the status record table. If no matching initial slot obtained from the previous static analysis is found in the status record table, the system returns to obtain the next opcode.

[0063] It should be noted that for dynamic arrays, the SHA3 calculation result is only the storage location of the first element of the array. It is also necessary to record the slots occupied by all elements of the array based on the array length stored in the initial slot.

[0064] In reality, complex nested types often exist. Taking a variable `m` of type `mapping(string => mapping(uint => bool))` as an example, this illustrates how dynamic analysis identifies its slot. At this moment, the interpreter is executing an assignment statement `m[k...` str [k] int ]=true;), the interpreter within the EVM treats it as a combination of two mapping variables to calculate the storage location. First, by calculating s′=SHA3(h(k str The outer mapping slot s′ is obtained by hashing the initial slot of the inner mapping. Then, s′ is used as the initial slot of the inner mapping, and the hash operation s = SHA3(h(k) is performed again. int The slot address s is obtained by s′), which is the location where the boolean value true is stored. To handle this situation, the state record table is designed with pointers and layers. The layer number is used to record whether the variable is a multidimensional array or a combination of several mappings, and the pointer is used to point to the state record table of the inner mapping variable. For variable m, the layer number in the state record table obtained by static analysis is 2. When the first SHA3 operation is performed to calculate s′, the state record table of the inner mapping variable is created and its address is recorded in the pointer field of the outer state record table. During the second SHA3 run, dynamic analysis can capture s = SHA3(h(k) through the inner state record table. int ),s′), and then record the final storage location. Similarly, multidimensional dynamic arrays can also be analyzed in this way.

[0065] Step S4: After all transactions within the block have been executed, the full node that deployed the state record table persists the state record table in memory to the database.

[0066] During persistence, data needs to be formatted to fit the database table structure. Each table is named after the smart contract's address, and each row in the table represents the state information of a state variable, including the variable name, variable type, storage location (including slot and offset), initial slot, layer number, pointer, and block height. This information is stored in the database as strings and linked to the row number of the inner variable via pointers, allowing for the tracking of relationships between state variables and historical data when needed. To enable snapshot reading, each record in the database must include block height information. This allows the smart contract to accurately retrieve state record table data from a specific block based on the block height, thus satisfying the contract's historical data query needs.

[0067] Furthermore, to efficiently persist the state record table data in memory to the database while ensuring the main process is not blocked due to database writes, this application designs a child process synchronization mechanism. Specifically, persisting the state record table in memory to the database includes:

[0068] The main process creates a separate child process, which is responsible for writing data to the database to prevent the main process from being blocked by database write operations. The main process uses the operating system's fork() function to copy itself, creating a child process that is identical to the main process.

[0069] The child process runs independently. After receiving a copy of the data from the main process, it begins writing the data to the database. This process can be done in batches or row-by-row, depending on system performance and requirements. Because the child process runs independently of the main process, the main process is not blocked by database write operations.

[0070] Database update complete: After the child process successfully writes the data to the database, it can send a signal or notification to the main process indicating that the data has been persisted. This communication mechanism can be implemented using inter-process communication tools, such as Unix domain sockets or pipes.

[0071] Main process continues: After receiving a successful write notification from the child process, the main process can continue executing the next batch of transactions or other tasks. Since the child process is responsible for data persistence, the main process does not need to wait for the data write operation to complete.

[0072] Error handling: If the child process encounters an error or exception while writing to the database, it can send an error message to the main process. The main process can then take appropriate measures based on the error message, such as retrying the write, rolling back the transaction, or notifying the administrator for manual intervention.

[0073] This design, where a child process forks the main process, allows data persistence and smart contract execution to proceed in parallel, improving system performance and availability. The main process continuously executes smart contract transactions, while the child processes efficiently write data from memory to the database, and also possess exception handling capabilities. This design ensures data consistency and reliability without hindering the main process due to database writes. In practical applications, the system should periodically create database backups to prevent data loss. In the event of a single node failure or data corruption, the most recent database backup can be used for recovery, thereby reducing the risk of data loss. When persisting data to the database, database transactions can be used to ensure data integrity. If an error occurs during data writing, the transaction can be rolled back, maintaining the consistency of the database state and avoiding data inconsistency caused by partial writes. Furthermore, a master-slave architecture can be used to improve system high availability. The master node is responsible for synchronizing blocks and writing the state record table to the database, while the slave nodes are responsible for handling user requests to read the state record table.

[0074] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A smart contract state extraction method combining static and dynamic analysis, characterized in that: The smart contract state extraction method combining static and dynamic analysis includes: Use static analysis tools to parse the smart contract source code into an abstract syntax tree, then record the storage layout information of each state into the state record table, and deploy the state record table together with the smart contract to the blockchain; A full node with a state record table is deployed, and the full synchronization process is started to monitor the blockchain network, download historical blocks, read transactions in the blocks, identify the smart contract address to which the transaction is sent, and load the corresponding state record table into memory; The full node that has deployed the state record table runs all transactions in the block in sequence, uses dynamic analysis tools to maintain the storage layout information of dynamic type states in real time, and updates the state record table; After all transactions in the block are executed, the full node that has deployed the state record table will persist the state record table in memory to the database; The method of using a static analysis tool to parse the smart contract source code into an abstract syntax tree, and then recording the storage layout information of each state into a state record table, also includes: Traverse and analyze the variables in the abstract syntax tree in the order of declaration; Classify variables into static type or dynamic type, extract storage layout information of corresponding states and record them in state record table; For composite type variables, parse them into static type or dynamic type, and then extract the storage layout information of the corresponding state and record them in the state record table; The state record table uses the following general data structure to record the storage layout information of the state: State record table = {variable name, variable type, {slot, offset}, initial slot, number of layers, pointer}; The method of using a dynamic analysis tool to maintain the storage layout information of the dynamic type state in real time and updating the state record table includes: Get the opcode. If it is a SHA3 opcode, determine whether the length of the SHA3 opcode parameter is greater than or equal to 32. If the length of the SHA3 opcode parameter is greater than or equal to 32, the initial slot is calculated based on the SHA3 opcode parameter, and then the state record table is queried. If the initial slot is matched in the state record table, the slot in the state record table is updated with the slot result calculated by the SHA3 opcode.

2. The method for extracting smart contract status by combining static and dynamic analysis as claimed in claim 1, characterized in that: The step of deploying the state record table and the smart contract to the blockchain includes: Smart contracts send ordinary transactions to the blockchain through the developer's account and are deployed in all full nodes; The status record table is deployed to the full node with the dynamic analysis service enabled. The developer specifies the IP address and port number of the full node with the dynamic analysis service enabled, and sends a special transaction to deploy the status record table.

Images