Data processing method and device, equipment, storage medium and computer program product
By storing the snapshot processing results of blockchain operation data in the buffer and directly reading data from the buffer, the problem of low blockchain data reading efficiency is solved and more efficient data access is achieved.
Patent Information
- Application Number
- CN202310665811.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing blockchain data reading method traverses multiple nodes in the Merkle Patricia Trie tree, resulting in low data reading efficiency and high complexity.
By storing the snapshot processing results of blockchain operation data in the buffer, the operation data corresponding to the index information is directly read from the buffer, reducing the reading complexity and improving efficiency.
It achieves the efficiency of data reading, reduces the number of accesses to the hard disk, and improves the speed and performance of data reading.
Smart Images

Figure CN119088500A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of blockchain, and in particular to a data processing method, a data processing apparatus, a data processing device, a readable storage medium, and a computer program product. Background Art
[0002] As a decentralized distributed ledger, blockchain has been applied in more and more fields due to its reliability that data cannot be tampered with.
[0003] The existing technology stores the data generated by blocks in the blockchain through the Merkle Patricia Trie (MPT). When data needs to be read from the MPT tree, it is necessary to traverse multiple nodes in the MPT tree to determine the index information of the data to be read. The deeper the MPT tree, the more nodes need to be read. This method of reading data is highly complex, resulting in low data reading efficiency. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, apparatus, device, storage medium, and computer program product, which can reduce the complexity of reading data and thus improve the efficiency of data reading.
[0005] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:
[0006] receiving a first data read instruction, wherein the first data read instruction includes first index information, and the first index information is used to indicate operation data to be read;
[0007] Reading operation data corresponding to the first index information from a buffer; wherein the buffer includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by performing snapshot processing on a corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record operations generated when executing a block in a blockchain network;
[0008] If the operation data corresponding to the first index information is successfully read from the buffer, the read operation data is output.
[0009] On the other hand, an embodiment of the present application provides a data processing device, the device comprising:
[0010] A communication unit, configured to receive a first data reading instruction, wherein the first data reading instruction includes first index information, and the first index information is used to indicate operation data to be read;
[0011] A processing unit, configured to read operation data corresponding to the first index information from a buffer; wherein the buffer includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by performing snapshot processing on a corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record an operation generated when executing a block in a blockchain network;
[0012] The communication unit is further configured to output the read operation data if the operation data corresponding to the first index information is successfully read from the buffer.
[0013] On the other hand, an embodiment of the present application provides a data processing device, which includes: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to execute the data processing method in the embodiment of the present application.
[0014] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the data processing method in the embodiment of the present application is executed.
[0015] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the data processing method in the embodiment of the present application is implemented.
[0016] The embodiment of the present application directly reads the operation data corresponding to the first index information included in the first data read instruction from the cache after receiving the first data read instruction. Since any operation data and the index information of any operation data in the cache are obtained by snapshot processing of the corresponding operation record (the operation generated when executing the block in the blockchain network), the any operation data and the index information of any operation data are a mapping relationship, that is, the operation data in the cache and the index information corresponding to the operation data are stored in a flat manner, so when reading the operation data corresponding to the first index information from the cache, it is only necessary to access the hard disk once, thereby reducing the complexity of reading data and improving the data reading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 is a schematic diagram of a data storage structure exemplarily provided in an embodiment of the present application;
[0019] Figure 2 is a schematic diagram of another data storage structure exemplarily provided in an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of the architecture of a data processing system exemplarily provided in an embodiment of the present application;
[0021] Figure 4 This is a schematic diagram of a data processing flow chart provided by way of example in an embodiment of the present application;
[0022] Figure 5 is a flowchart of another data processing method provided by way of example in an embodiment of the present application;
[0023] Figure 6 This is another data processing flow diagram provided by way of example in an embodiment of the present application;
[0024] Figure 7 is a structural diagram of a data processing device exemplarily provided in an embodiment of the present application;
[0025] Figure 8 It is a structural diagram of a data processing device exemplarily provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0027] In order to better understand the embodiments of the present application, some professional terms involved in the embodiments of the present application are introduced below:
[0028] Blockchain: Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer and application service layer. With the rapid development of blockchain, blockchain has been applied to many businesses. For example, wallets are used to provide functions for electronic currency transactions, including initiating transactions; shared ledgers are used to provide functions such as storage, query and modification of account data; smart contracts are computerized protocols that can execute the terms of a contract. They are implemented by deploying codes on shared ledgers to execute when certain conditions are met. According to actual business needs, the codes are used to complete automated transactions.
[0029] Blockchain is generally divided into three types: public blockchain, private blockchain and consortium blockchain.
[0030] Among them, the public chain has the highest degree of decentralization. Common public chains include Ethereum. Nodes / participants who join the public chain can read data on the chain, publish transactions, and compete for the right to record new blocks. Moreover, each node / participant can freely join and exit the public chain. Private chains are the opposite. The accounting authority of private chains is controlled by a certain organization or institution, and the data reading authority is also controlled by the organization or institution. There are few participants and they cannot join the private chain at will. They must be reviewed by the organization or institution.
[0031] A consortium chain is also called a community blockchain. It is a blockchain whose consensus process is controlled by pre-selected nodes. It is a hybrid of public and private chains and can achieve "partial decentralization". Each node on the chain usually has a corresponding entity or organization; participants join the network through authorization and form a stakeholder alliance to jointly maintain the operation of the blockchain. Through the consortium chain, new participants can join an already formed blockchain and share data without having to build it from scratch. At the same time, by jointly solving common problems, companies reduce development costs and time expenses.
[0032] Whether it is a public chain, a private chain or a consortium chain, it may provide the function of smart contracts. Smart contracts on the blockchain are contracts that can be triggered and executed by transactions on the blockchain system. Smart contracts can be defined in the form of code.
[0033] Smart contract: also known as chaincode or application code, is a computer protocol designed to disseminate, verify or execute contracts in an information-based manner. It is a program deployed in the nodes of the blockchain network, carries the business logic for executing transactions, and runs in an isolated operating environment (such as a container or virtual machine). The contract program automatically executed by each node in the blockchain system according to specific conditions can operate on the data stored on the chain. It is an important way for users to interact with the blockchain and use the blockchain to implement business logic. The purpose of smart contracts is to provide a security method that is superior to traditional contracts and reduce other transaction costs associated with contracts. It allows trusted transactions without a third party, and these transactions are irreversible.
[0034] Merkle Patricia Trie (MPT): A data structure used to store and retrieve key-value pairs. The MPT tree is a combination of the Merkle Tree and the Patricia Trie. To better understand the MPT tree, the following will introduce the Merkle Tree and the Patricia Trie respectively.
[0035] Merkle Tree hashes the data block as a leaf node, then merges multiple nodes and calculates the hash to get a new node, repeats the above steps until a root node (root hash) is obtained, forming a tree structure. Figure 1 As shown, after four data blocks 101 (data block L1, data block L2, data block L3 and data block L4) are hashed respectively, four leaf nodes corresponding to the four data blocks are obtained (Hash0-0, Hash0-1, Hash1-0 and Hash1-1), and then the four leaf nodes are merged and hashed to obtain two new nodes (Hash0 and Hash1), and finally the two new nodes are merged to obtain the root node (Top Hash).
[0036] As can be seen above, the root of the Merkle Tree hashes all the data, and the root of the Merkle Tree can be used to make a Merkle Proof. A Merkle Proof contains a data block, a tree root, and all the hashes of the path from the data block to the tree root. Through this Merkle Proof, it can be quickly verified that a piece of data does exist at a certain location in the tree. An attacker cannot forge a Merkle Proof because the root hash depends on the hashes of all other nodes, and any modification of a node will cause the root hash to be inconsistent. For example, Figure 1The root hash 102 in depends on Hash0 and Hash1, and Hash0 and Hash1 depend on Hash0-0, Hash0-1, Hash1-0 and Hash1-1. If any of the Hash is modified (such as Hash0-1 is modified), the root hash 102 will cause inconsistency.
[0037] Trie is an ordered tree structure used to store and retrieve key-value pairs. Each node in the Trie records a character and points to the next character. Each path in the Trie can form a complete key, thereby achieving efficient storage and retrieval of key sets with the same prefix. Figure 2 As shown, Figure 2 The Trie shown stores key values (A, to, tea, ten, ted, i, in, and inn), and each key corresponds to a data as a value (for example, the value corresponding to the key value to is 7). It should be noted that Figure 2 The nodes in the Trie in the example are labeled with complete words (such as the word ten) to demonstrate the principle. In practice, the complete key consists of the path and is not stored in the node.
[0038] Merkle Tree and Patricia Trie are introduced above. The following is a further explanation of MPT tree. MPT tree can be regarded as an implementation of Trie. MPT tree defines the following node types: blank node, branch node, leaf node and extended node. Among them, branch node is the node in the above Trie, and extended node is the newly added node after optimizing Trie. Leaf node is a special case of extended node. The key value ends at this node, and value is the value corresponding to this node.
[0039] An embodiment of the present application provides a data processing method, which involves cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to achieve data calculation, storage, processing, and sharing.
[0040] In particular, it involves cloud storage and database in cloud technology. Cloud storage is a new concept that extends and develops from the concept of cloud computing. Distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network to work together through application software or application interfaces to provide data storage and business access functions. In short, a database can be regarded as an electronic filing cabinet - a place to store electronic files. Users can add, query, update, delete, and other operations on the data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
[0041] In a feasible embodiment, the data processing method provided in the embodiment of the present application can be implemented based on cloud data, and can specifically involve cloud storage and private cloud in cloud data.
[0042] See also Figure 3 , Figure 3 The data processing method provided in the embodiment of the present application can be applied to Figure 3 The system architecture is shown in Figure 1. Figure 3 As shown, Figure 3 The device marked with 301 is the terminal device 301. Figure 3 The device marked with 302 is a data processing device 302, and the data processing device 302 can execute the data processing method proposed in this application.
[0043] Optionally, the terminal device 301 may send a first data reading instruction to the data processing device 302. After successfully acquiring the operation data corresponding to the first index information, the data processing device 302 may output the operation data corresponding to the first index information to the terminal device 301.
[0044] Optionally, the cache may exist in the data processing device 302 or in other devices, and the data processing device 302 may access and read the data in the cache.
[0045] Optionally, the terminal device 301 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a car terminal, etc., but is not limited to these, and this application does not impose any restrictions on this.
[0046] Optionally, the data processing device 302 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0047] Optionally, the data processing device 302 may be a terminal device, specifically a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., but is not limited thereto and this application does not impose any restrictions on this.
[0048] The above is a brief introduction to the data processing method provided in the embodiment of the present application. Figure 4 The data processing method provided in the embodiment of the present application is described in detail. Figure 4 , Figure 4 The data processing method provided in the embodiment of the present application is a flowchart. The data processing method can be executed by the above-mentioned data processing device 302, and the data processing method includes but is not limited to the following steps S401 to S403:
[0049] S401. Receive a first data read instruction, where the first data read instruction includes first index information, and the first index information is used to indicate operation data to be read.
[0050] Among them, the first data read instruction is used to instruct the data processing device to perform data processing, and the data processing can specifically be reading data. The first data read instruction can be sent by the above-mentioned terminal device, or it can be sent to the data processing device by the operation object through operating the data processing device.
[0051] S402. Read operation data corresponding to the first index information from a cache; wherein the cache includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by snapshot processing of the corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record the operation generated when executing a block in a blockchain network.
[0052] Among them, the operation data stored in the buffer and the index information of the operation data may be a mapping relationship, and the index information of one operation data corresponds to one operation data. Snapshot processing includes converting the operation data and all information in the operation record into a mapping relationship between the operation data and the index information of the operation data. For example, the operation record is: number (index information of the operation data)-268 (operation data)-update (operation type of the operation data). After snapshot processing of the operation record, the operation data and the index information of the operation data stored in the buffer are as follows: number (index information of the operation data)-268 (operation data). The operation data may be blockchain status data, such as the number of users in the blockchain, the deposits of users recorded in the blockchain, etc. The operation type in the operation record may be deletion, update, addition, etc. For example, if the block in the blockchain network changes 268 (operation data) corresponding to number (index information of the operation data) to 375, the operation record corresponding to the operation is: number (index information of the operation data)-375 (operation data)-update (operation type of the operation data).
[0053] It should be noted that the operation data is not limited to the form of numbers, but may also exist in characters or other forms. The above-mentioned digital form is only for example. This application does not limit the specific form of the operation data. Similarly, the index information of the operation data is not limited to the form of characters, but may also be in other forms, such as digital form, etc. This application does not limit this.
[0054] Exemplarily, the mapping relationship between the operation data stored in the buffer and the index information of the operation data is shown in the following table:
[0055] Index information of operation data Operational Data number 375 account 28 tea 4 inn 96
[0056] If the first index information included in the first data read instruction is specifically "number", the operation data corresponding to the first index information read from the cache storing the above-mentioned table based on the first index information is 375; similarly, the first index information included in the first data read instruction is specifically "account", and the operation data corresponding to the first index information read from the cache storing the above-mentioned table based on the first index information is 28, and so on, which is not elaborated in this application.
[0057] S403: If the operation data corresponding to the first index information is successfully read from the buffer, the read operation data is output.
[0058] Among them, the output of the read operation data can be specifically sending the read operation data to a terminal device, the terminal device can be a device that sends a first data read instruction to a data processing device, or the terminal device can be another independent terminal device, and there is device indication information in the first data read instruction to indicate the independent terminal device, and the output of the read operation data is specifically sending the read operation data to the independent terminal device. The output of the read operation data can also be specifically displaying the read operation data, specifically, the data processing device includes a display screen, and the read operation data is displayed on the display screen of the data processing device. The present application does not limit the form of outputting the read operation data.
[0059] Optionally, successfully reading the operation data corresponding to the first index information from the cache includes: searching the cache for index information of the operation data identical to the first index information; and reading the operation data corresponding to the index information of the operation data identical to the first index information.
[0060] That is to say, if index information of the operation data identical to the first index information can be found in the cache, the operation data corresponding to the first index information can be successfully read from the cache.
[0061] In a possible embodiment, after successfully reading the operation data corresponding to the first index information from the cache and before outputting the read operation data, the data processing method may further include: based on the first index information, reading the operation data corresponding to the first index information from a Merkle tree, the Merkle tree including at least one operation data and index information of each operation data; comparing the operation data read from the Merkle tree with the operation data read from the cache; if the operation data read from the cache is the same as the operation data read from the Merkle tree, triggering the step of outputting the read operation data.
[0062] Among them, the Merkle tree can refer to the introduction above. The at least one operation data and index information of each operation data included in the Merkle tree are obtained by snapshot processing of the corresponding operation record. Any operation record includes operation data, index information of the operation data, and operation type of the operation data. Any operation record is used to record the operations generated when executing blocks in the blockchain network.
[0063] Exemplarily, the following three pairs of data are stored in the Merkle tree: number-395, account-27, and inn-366, where 395, 27, and 366 are operation data, and number, account, and inn are index information of the corresponding operation data. The first index information is number, and the operation data read from the cache based on the first index information is 395, and the operation data read from the Merkle tree based on the first index information is 395. Since the operation data read from the cache and the operation data read from the Merkle tree are the same (both are 395), the step of outputting the operation data is triggered.
[0064] Since the Merkle tree supports Merkle proof, the data in the Merkle tree is reliable. The operation data read from the cache is compared with the operation data read from the Merkle tree, and the operation data that passes the comparison is output, ensuring the reliability of the output operation data.
[0065] Optionally, when a data verification instruction is received, the data verification instruction includes second index information and operation data to be verified, and the operation data corresponding to the second index information is read from the Merkle tree; the operation data read from the Merkle tree and the operation data to be verified included in the data verification instruction are compared; and the comparison result is output.
[0066] Among them, if the operation data read from the Merkle tree is the same as the operation data to be verified included in the data verification instruction, then the operation data to be verified included in the data verification instruction is considered to be reliable; conversely, if the operation data read from the Merkle tree is different from the operation data to be verified included in the data verification instruction, then the operation data to be verified included in the data verification instruction is considered to be unreliable. The operation data to be verified can be read from a cache, and the first index information and the second index information can be the same or different, and this application does not limit this.
[0067] After receiving the first data read instruction, the operation data corresponding to the first index information included in the first data read instruction is directly read from the cache. Since any operation data and the index information of any operation data in the cache are obtained by snapshot processing of the corresponding operation record (the operation generated when executing the block in the blockchain network), the any operation data and the index information of any operation data are a mapping relationship, that is, the operation data in the cache and the index information corresponding to the operation data are stored in a flat manner, so when reading the operation data corresponding to the first index information from the cache, only one hard disk needs to be accessed, thereby reducing the complexity of reading data and improving the data reading efficiency.
[0068] The above embodiment describes a method for successfully reading operation data from a buffer. The above embodiment can reduce the complexity of reading data, thereby improving the efficiency of data reading. Figure 5 An embodiment in which the operation data is not successfully read from the buffer is introduced.
[0069] S501. Receive a first data read instruction, where the first data read instruction includes first index information, and the first index information is used to indicate operation data to be read.
[0070] Please refer to the detailed description in the above step S401, which will not be repeated here.
[0071] S502. Read operation data corresponding to the first index information from a cache; wherein the cache includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by snapshot processing of the corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record the operation generated when executing a block in a blockchain network.
[0072] Please refer to the detailed description in the above step S402, which will not be repeated here.
[0073] S503: If the operation data corresponding to the first index information is successfully read from the buffer, the read operation data is output.
[0074] Please refer to the detailed description in the above step S403, which will not be repeated here.
[0075] S504. If the operation data corresponding to the first index information is not successfully read from the cache, the operation data corresponding to the first index information is read from the memory; wherein the memory at least includes the operation data contained in the cache and the index information of each operation data, and the storage validity period of the operation data and the index information of each operation data in the memory is greater than or equal to the storage validity period of the operation data and the index information of each operation data in the cache.
[0076] Among them, the number of operation data and index information of each operation data stored in the memory is greater than or equal to the number of operation data and index information of each operation data stored in the cache. For example, the memory contains 10 operation data and index information of each operation data, and the cache contains 7 operation data and index information of each operation data. The storage effective duration is the duration allowed for storage. For example, if the storage effective duration of the cache is 30 days, the data stored in the cache for more than 30 days will be cleared. The storage effective duration of the operation data and index information of each operation data in the memory is greater than or equal to the storage effective duration of the operation data and index information of each operation data in the cache, that is, the duration allowed for storing the operation data and index information of the operation data in the memory is greater than the duration allowed for storing the operation data and index information of the operation data in the cache.
[0077] In a possible embodiment, the storage validity period may be a preset period, for example, the period for storing the operation data and the index information of the operation data in the cache is 10 days, and the period for storing the operation data and the index information of the operation data in the memory is 30 days. It may also be a period determined according to the operating state of the device where the memory and the cache are located, for example, when the device where the memory and the cache are located is restarted, the operation data and the index information of the operation data stored in the cache will be cleared, while the operation data and the index information of the operation data stored in the memory will continue to be retained.
[0078] In a possible embodiment, the memory stores the operation data and the index information of each operation data in the form of a key-value database such as leveldb, and the cache stores the operation data and the index information of each operation data in the form of a Map as a data structure.
[0079] In the above manner, since the number of operation data and index information of operation data stored in the memory is equal to or greater than the number of operation data and index information of operation data stored in the buffer, if the operation data corresponding to the first index information is not found in the buffer, the operation data corresponding to the first index information can be read from the memory. Since the number of operation data and index information of operation data in the buffer is relatively smaller, the buffer is read first, which can improve the efficiency of reading operation data.
[0080] S505: If the operation data corresponding to the first index information is successfully read from the memory, the read operation data is output.
[0081] Among them, the output of the read operation data can be specifically sending the read operation data to a terminal device, the terminal device can be a device that sends a first data read instruction to a data processing device, or the terminal device can be another independent terminal device, and there is device indication information in the first data read instruction to indicate the independent terminal device, and the output of the read operation data is specifically sending the read operation data to the independent terminal device. The output of the read operation data can also be specifically displaying the read operation data, specifically, the data processing device includes a display screen, and the read operation data is displayed on the display screen of the data processing device. The present application does not limit the form of outputting the read operation data.
[0082] In a possible embodiment, after successfully reading the operation data corresponding to the first index information from the memory and before outputting the read operation data, the data processing method may further include: based on the first index information, reading the operation data corresponding to the first index information from the Merkle tree, the Merkle tree including at least one operation data and index information of each operation data; comparing the operation data read from the Merkle tree with the operation data read from the memory; if the operation data read from the memory is the same as the operation data read from the Merkle tree, triggering the step of outputting the read operation data.
[0083] Since the Merkle tree supports Merkle proof, the data in the Merkle tree is reliable. The operation data read from the memory is compared with the operation data read from the Merkle tree, and the operation data that passes the comparison is output, ensuring the reliability of the output operation data.
[0084] Optionally, when a data verification instruction is received, the data verification instruction includes second index information and operation data to be verified, and the operation data corresponding to the second index information is read from the Merkle tree; the operation data read from the Merkle tree and the operation data to be verified included in the data verification instruction are compared; and the comparison result is output.
[0085] Among them, if the operation data read from the Merkle tree is the same as the operation data to be verified included in the data verification instruction, then the operation data to be verified included in the data verification instruction is considered to be reliable; conversely, if the operation data read from the Merkle tree is different from the operation data to be verified included in the data verification instruction, then the operation data to be verified included in the data verification instruction is considered to be unreliable. The operation data to be verified can be read from a memory, and the first index information and the second index information can be the same or different, and this application does not limit this.
[0086] In a possible embodiment, after successfully reading the operation data corresponding to the first index information from the memory, the method also includes: writing the operation data read from the memory and the first index information into a cache; when receiving a second data read instruction, the second data read instruction includes the first index information, and reading the operation data corresponding to the first index information from the cache; and outputting the read operation data.
[0087] The second data reading instruction and the first data reading instruction may be sent by the same device or by different devices.
[0088] Exemplarily, the memory contains the following operation data and index information of the operation data: number-395, account-27, ten-58, and inn-366, and the cache contains the following operation data and index information of the operation data: number-395 and account-27. The first index information is inn, and the operation data corresponding to inn is not successfully read from the cache. The operation data corresponding to inn read from the storage is 366. After the operation data 366 is output, inn-366 is written into the cache. The operation data and index information of the operation data contained in the written cache are: number-395, account-27, and inn-366. When the second data read instruction containing inn is received again, the operation data corresponding to inn can be successfully read from the cache.
[0089] Since the number of operation data and index information of the operation data in the cache is less than or equal to the number of operation data and index information in the memory, the operation data and the first index information read from the memory are written to the cache so that the next time the operation data corresponding to the first index information is read, it can be read faster from the cache.
[0090] The above two embodiments describe methods for reading operation data. The above two embodiments can reduce the complexity of reading data, thereby improving the efficiency of data reading. Figure 6 This section describes how to write operation data.
[0091] S601. When a block in the blockchain network is detected to be executed, a block write set corresponding to the block is obtained, where the block write set includes at least one operation record of the block.
[0092] Among them, when the execution of the block in the blockchain network is detected, it can be specifically when the execution of the block in the blockchain network is detected. The detection method can be to obtain the execution status of the block in the blockchain network in real time, and when the execution status of the block in the blockchain network is completed, generate a block write set corresponding to the block. The detection method can also be when the execution of the block in the blockchain network is completed, send an execution completion notification to the data processing device to inform the data processing device that the execution of the block in the blockchain network is completed, and when the data processing device receives the execution completion notification, generate a block write set corresponding to the block. The block write set includes the data generated by each transaction operation in a block, and one block corresponds to one block write set.
[0093] S602: Write the block write set into the buffer.
[0094] In a possible embodiment, when a block in a blockchain is committed, a block write set corresponding to the block in the blockchain is written into a cache.
[0095] Optionally, when the block write set corresponding to the block in the blockchain is written into the cache, the block write set can also be written into the memory and the block write set can be written into the MPT tree.
[0096] In a possible embodiment, writing a block write set into a cache includes: if there is a first operation record with an operation type of deletion in the block write set, deleting the operation data in the cache that is identical to the index information of the operation data in the first operation record and the index information of the operation data that is identical to that in the first operation record; if there is a second operation record with an operation type of update in the block write set, updating the operation data in the cache that is identical to the index information of the operation data in the second operation record to the operation data in the second operation record; if there is a third operation record with an operation type of addition in the block write set, writing the operation data in the third operation record and the index information of the operation data into the cache.
[0097] Exemplarily, the operation data and index information of the operation data originally contained in the cache are as follows: number-395, account-27, ten-58, and inn-366.
[0098] If there is a first type operation record whose operation type is deletion, the first type operation record is as follows: ten-58-deletion, then the operation data and the index information of the operation data (ten-58) in the cache are deleted, and the operation data and the index information of the operation data contained in the cache after deletion are as follows: number-395, account-27 and inn-366.
[0099] If there is a second type operation record whose operation type is update, the second type operation record is as follows: inn-874-update, then the operation data and the index information of the operation data (inn-366) in the cache are updated, and the operation data and the index information of the operation data contained in the updated cache are as follows: number-395, account-27, ten-58 and inn-874.
[0100] If there is a third type operation record whose operation type is new, and the third type operation record is as follows: tea-21-new, then the operation data and the index information of the operation data (tea-21) are written into the cache, and the operation data and the index information of the operation data contained in the newly added cache are as follows: number-395, account-27, ten-58, inn-366 and tea-21.
[0101] Optionally, writing a block write set into a memory includes: if there is a first operation record with an operation type of deletion in the block write set, deleting the operation data in the memory that is identical to the index information of the operation data in the first operation record and the index information of the operation data that is identical to that of the operation data in the first operation record; if there is a second operation record with an operation type of update in the block write set, updating the operation data in the memory that is identical to the index information of the operation data in the second operation record to the operation data in the second operation record; if there is a third operation record with an operation type of addition in the block write set, writing the operation data in the third operation record and the index information of the operation data into the memory.
[0102] Optionally, writing the block write set into the MPT tree includes: if there is a first operation record with an operation type of deletion in the block write set, deleting the operation data in the MPT tree that is identical to the index information of the operation data in the first operation record and the index information of the operation data that is identical to that in the first operation record; if there is a second operation record with an operation type of update in the block write set, updating the operation data in the MPT tree that is identical to the index information of the operation data in the second operation record to the operation data in the second operation record; if there is a third operation record with an operation type of addition in the block write set, writing the operation data in the third operation record and the index information of the operation data into the MPT tree.
[0103] In a possible embodiment, after obtaining a block write set corresponding to a block in the blockchain network, the data processing method further includes: if block write sets corresponding to multiple blocks in the blockchain network are obtained, then filtering the operation records in the block write sets corresponding to the multiple blocks to obtain filtered operation records; and writing the filtered operation records into a cache.
[0104] Among them, one block corresponds to one block write set, and multiple blocks correspond to multiple block write sets. The screening process is to screen out non-duplicate operation records from the block write sets corresponding to the multiple blocks, or to screen out one operation record from the repeated operation records in the block write sets corresponding to the multiple blocks for retention. The repeated operation records include operation records with the same index information of the operation data, for example, operation record 1 is: number-236-update, operation record 2 is: number-577-delete, the index information of the operation data in operation record 1 and operation record 2 is both number, and operation record 1 and operation record 2 are repeated operation records.
[0105] Exemplarily, three block write sets corresponding to three blocks in the blockchain network are obtained: block write set 1, block write set 2, and block write set 3. Among them, block write set 1 includes the following operation records: number-395-update, account-27-update, ten-58-update, and inn-366-deletion, block write set 2 includes the following operation records: tea-21-update, ten-87-deletion, and A-177-update, and block write set 3 includes the following operation record: account-77-update. The index information of the operation data of the three operation records in the above three block write sets is the same: ten-58-update in block write set 1 and ten-87-deletion in block write set 2; account-27-update in block write set 1 and account-77-update in block write set 3. The three block write sets are filtered, and ten-87-deletion is filtered out from ten-58-update in block write set 1 and ten-87-deletion in block write set 2, and account-77-update is filtered out from account-27-update in block write set 1 and account-77-update in block write set 3. The filtered operation records are: number-395-update, account-77-update, ten-87-deletion, inn-366-deletion, tea-21-update, A-177-update.
[0106] It takes multiple writes to directly write the block write sets corresponding to multiple blocks in the blockchain network into the cache. For example, three block write sets need to be written into the cache three times. However, it only takes one write to write the block write sets corresponding to multiple blocks in the blockchain network into the cache after filtering. For example, after filtering the three block write sets, the obtained operation records are written into the cache at one time, thereby improving the efficiency of writing into the cache.
[0107] Optionally, if block write sets corresponding to multiple blocks in the blockchain network are obtained, the operation records in the block write sets corresponding to the multiple blocks are filtered and processed to obtain filtered operation records; and the filtered operation records are written into the memory.
[0108] Optionally, if block write sets corresponding to multiple blocks in the blockchain network are obtained, the operation records in the block write sets corresponding to the multiple blocks are filtered and processed to obtain filtered operation records; and the filtered operation records are written into the MPT tree.
[0109] In a possible embodiment, operation records in a block write set corresponding to multiple blocks are screened to obtain screened operation records, including: if there are at least two operation records to be screened with the same index information in the block write set corresponding to the multiple blocks, a target block is determined from blocks corresponding to the block write set to which the at least two operation records to be screened belong, wherein a block height of the target block is greater than block heights of other blocks; and the operation record to be screened in the block write set corresponding to the target block is used as the screened operation record.
[0110] Among them, the block height refers to the block number, that is, the number of a block after the genesis block (the block height of the genesis block is 0). For example, if the height of a block is 3, the block number is also 3. The block is the third block after the genesis block. The larger the block height, the larger the corresponding block number. Similarly, the smaller the block height, the smaller the corresponding block number. The index information of the operation data of the operation record to be filtered is the same.
[0111] Exemplarily, if there are three operation records to be screened with the same index information in the block write sets corresponding to the three blocks: operation record 1 to be screened, operation record 2 to be screened, and operation record 3 to be screened, the blocks corresponding to the block write sets described by the three operation records to be screened are: block 1, block 2, and block 3, respectively, wherein the height of block 1 is 3, the height of block 2 is 5, and the height of block 3 is 9. From the above three blocks, it is determined that the target block is block 3 (the block height of block 3 is greater than that of block 1 and block 2), and the operation record 3 to be screened corresponding to block 3 is used as the operation record after screening.
[0112] S603: Receive a first data read instruction, where the first data read instruction includes first index information, and the first index information is used to indicate operation data to be read.
[0113] Please refer to the detailed description in the above step S401, which will not be repeated here.
[0114] S604. Read operation data corresponding to the first index information from a cache; wherein the cache includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by snapshot processing of the corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record the operation generated when executing a block in a blockchain network.
[0115] Please refer to the detailed description in the above step S402, which will not be repeated here.
[0116] S605: If the operation data corresponding to the first index information is successfully read from the buffer, the read operation data is output.
[0117] Please refer to the detailed description in the above step S403, which will not be repeated here.
[0118] In the above manner, the block write set is written into the cache, so that when the operation data needs to be read later, it can be read from the cache quickly and efficiently.
[0119] See also Figure 7 , Figure 7 701 and 702 are schematic diagrams of the structure of the data processing device provided by the present application. The data processing device provided by the embodiment of the present application includes: a communication unit 701 and a processing unit 702.
[0120] The communication unit 701 is used to receive a first data reading instruction, where the first data reading instruction includes first index information, where the first index information is used to indicate the operation data to be read;
[0121] The processing unit 702 is used to read the operation data corresponding to the first index information from the buffer; wherein the buffer includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by performing snapshot processing on the corresponding operation record, any operation record includes the operation data, the index information of the operation data, and the operation type of the operation data, and any operation record is used to record the operation generated when executing a block in the blockchain network;
[0122] The communication unit 701 is further configured to output the read operation data if the operation data corresponding to the first index information is successfully read from the buffer.
[0123] In another implementation, the processing unit 702 is further configured to read, based on the first index information, the operation data corresponding to the first index information from the Merkle tree, where the Merkle tree includes at least one operation data and index information of each operation data;
[0124] The processing unit 702 is further used to compare the operation data read from the Merkle tree with the operation data read from the buffer;
[0125] The communication unit 701 is also used to trigger the step of outputting the read operation data if the operation data read from the buffer is the same as the operation data read from the Merkle tree.
[0126] In another implementation, the processing unit 702 is further configured to read the operation data corresponding to the first index information from a memory if the operation data corresponding to the first index information is not successfully read from the cache; wherein the memory at least includes the operation data contained in the cache and the index information of each operation data, and the storage validity period of the operation data and the index information of each operation data in the memory is greater than or equal to the storage validity period of the operation data and the index information of each operation data in the cache;
[0127] The communication unit 701 is further configured to output the read operation data if the operation data corresponding to the first index information is successfully read from the memory.
[0128] In another implementation, the processing unit 702 is further configured to write the operation data read from the memory and the first index information into the buffer;
[0129] The processing unit 702 is further configured to, upon receiving a second data read instruction, the second data read instruction including the first index information, read the operation data corresponding to the first index information from the buffer;
[0130] The communication unit 701 is also used to output the read operation data.
[0131] In another implementation, the processing unit 702 is further configured to, when detecting that a block in the blockchain network is executed, obtain a block write set corresponding to the block, the block write set including at least one operation record of the block;
[0132] The processing unit 702 is further configured to write the block write set into the buffer.
[0133] In another implementation, the processing unit 702 is further configured to delete the operation data and the index information of the same operation data in the buffer as the index information of the operation data in the first operation record if there is a first operation record whose operation type is deletion in the block write set;
[0134] The processing unit 702 is further configured to update the operation data in the buffer with the same index information as the operation data in the second operation record to the operation data in the second operation record if there is a second operation record with the operation type of update in the block write set;
[0135] The processing unit 702 is further configured to write the operation data and index information of the operation data in the third operation record into the buffer if there is a third operation record whose operation type is newly added in the block write set.
[0136] In another implementation, the processing unit 702 is further configured to, if block write sets corresponding to multiple blocks in the blockchain network are obtained, filter the operation records in the block write sets corresponding to the multiple blocks to obtain filtered operation records;
[0137] The processing unit 702 is further configured to write the filtered operation records into a buffer.
[0138] In another implementation, the processing unit 702 is further configured to determine a target block from blocks corresponding to the block write sets to which the at least two operation records to be filtered belong if there are at least two operation records to be filtered with the same index information in the block write sets corresponding to the multiple blocks, wherein the block height of the target block is greater than the block heights of the other blocks;
[0139] The processing unit 702 is further configured to use the operation record to be filtered in the block write set corresponding to the target block as the filtered operation record.
[0140] It can be understood that the functions of each functional unit of the data processing device provided in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment, and its specific implementation process can refer to the relevant description in the above method embodiment, which will not be repeated here.
[0141] In a feasible embodiment, the data processing device provided in the embodiment of the present application can be implemented in software. The data processing device can be stored in a memory. It can be software in the form of programs and plug-ins, and includes a series of units, including a communication unit and a processing unit; wherein the communication unit and the processing unit are used to implement the data processing method provided in the embodiment of the present application.
[0142] In other feasible embodiments, the data processing device provided in the embodiments of the present application may also be implemented in a combination of software and hardware. As an example, the data processing device provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may adopt one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.
[0143] See also Figure 8 , Figure 8 It is a structural diagram of a data processing device provided in an embodiment of the present application. The data processing device 800 may include a processor 801, a memory 802, a network interface 803 and at least one communication bus 804. Among them, the processor 801 is used to schedule computer programs, and may include a central processing unit, a controller, and a microprocessor; the memory 802 is used to store computer programs, and may include a high-speed random access memory (Random Access Memory, RAM), a non-volatile memory, such as a disk storage device, a flash memory device, etc.; the network interface 803 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), provide data communication functions, and the communication bus 804 is responsible for connecting various communication elements, etc. The data processing device 800 may correspond to the data processing device in the foregoing text. The memory 802 is used to store a computer program, and the computer program includes program instructions. The processor 801 is used to execute the program instructions stored in the memory 802 to execute the process described in steps S401 to S403 in the above embodiment.
[0144] In a specific implementation, the above-mentioned recognition data processing device can execute the above-mentioned Figure 4 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.
[0145] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, Figure 4For the data processing methods provided in each step, please refer to the implementation methods provided in the above steps, which will not be repeated here.
[0146] The computer-readable storage medium may be a data processing device provided in any of the aforementioned embodiments or an internal storage unit of the data processing device, such as a hard disk or memory of an electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the electronic device. Further, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0147] The terms "first", "second", "third", "fourth", etc. in the claims, specification and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0148] In the specific implementation of this application, data related to user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0149] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. Displaying the phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations. It can be appreciated by those of ordinary skill in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0150] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.
Claims
1. A data processing method, characterized in that: The method comprises: receiving a first data read instruction, wherein the first data read instruction includes first index information, and the first index information is used to indicate operation data to be read; Reading operation data corresponding to the first index information from a buffer; wherein the buffer includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by performing snapshot processing on a corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record operations generated when executing a block in a blockchain network; If the operation data corresponding to the first index information is successfully read from the buffer, the read operation data is output.
2. The method according to claim 1, characterized in that The method further comprises: Based on the first index information, read the operation data corresponding to the first index information from a Merkle tree, where the Merkle tree includes at least one operation data and index information of each operation data; Comparing the operation data read from the Merkle tree with the operation data read from the buffer; If the operation data read from the buffer is the same as the operation data read from the Merkle tree, the step of outputting the read operation data is triggered.
3. The method according to claim 1, characterized in that: The method further comprises: If the operation data corresponding to the first index information is not successfully read from the buffer, the operation data corresponding to the first index information is read from the memory; wherein the memory at least includes the operation data contained in the buffer and the index information of each operation data, and the storage validity period of the operation data and the index information of each operation data in the memory is greater than or equal to the storage validity period of the operation data and the index information of each operation data in the buffer; If the operation data corresponding to the first index information is successfully read from the memory, the read operation data is output.
4. The method according to claim 3, characterized in that After successfully reading the operation data corresponding to the first index information from the memory, the method further includes: Writing the operation data read from the memory and the first index information into the buffer; When receiving a second data read instruction, the second data read instruction includes the first index information, reading the operation data corresponding to the first index information from the buffer; Output the read operation data.
5. The method according to claim 1, characterized in that The method further comprises: When detecting that a block in the blockchain network is executed, obtaining a block write set corresponding to the block, the block write set including at least one operation record of the block; The block write set is written into the buffer.
6. The method according to claim 5, characterized in that Writing the block write set into the buffer includes: If there is a first operation record whose operation type is deletion in the block write set, deleting the operation data in the buffer that has the same index information as the operation data in the first operation record and the index information of the same operation data; If there is a second operation record whose operation type is update in the block write set, updating the operation data in the buffer with the same index information as the operation data in the second operation record to the operation data in the second operation record; If there is a third operation record whose operation type is newly added in the block write set, the operation data and index information of the operation data in the third operation record are written into the buffer.
7. The method according to claim 6, characterized in that After obtaining the block write set corresponding to the block, the method further includes: If block write sets corresponding to multiple blocks in the blockchain network are obtained, the operation records in the block write sets corresponding to the multiple blocks are filtered and processed to obtain filtered operation records; The filtered operation records are written into the buffer.
8. The method according to claim 7, characterized in that The filtering and processing the operation records in the block write set corresponding to the multiple blocks to obtain the filtered operation records includes: If there are at least two to-be-screened operation records with the same index information in the block write sets corresponding to the multiple blocks, determine a target block from the blocks corresponding to the block write sets to which the at least two to-be-screened operation records belong, wherein the block height of the target block is greater than the block heights of other blocks; The operation record to be filtered in the block write set corresponding to the target block is used as the filtered operation record.
9. A data processing device, characterized in that: The device comprises: A communication unit, configured to receive a first data reading instruction, wherein the first data reading instruction includes first index information, and the first index information is used to indicate operation data to be read; A processing unit, configured to read operation data corresponding to the first index information from a buffer; wherein the buffer includes at least one operation data and index information of each operation data, any operation data and index information of any operation data are obtained by performing snapshot processing on a corresponding operation record, any operation record includes operation data, index information of the operation data, and operation type of the operation data, and any operation record is used to record an operation generated when executing a block in a blockchain network; The communication unit is further configured to output the read operation data if the operation data corresponding to the first index information is successfully read from the buffer.
10. A data processing device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the data processing method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the data processing method according to any one of claims 1 to 8.
12. A computer program product, characterized in that The computer program product comprises a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 8 are implemented.
