Data storage method and device, terminal equipment and computer readable storage medium

Abstract

The invention is suitable for the technical field of data storage of a block chain, and provides a data storage method and device, terminal equipment and a computer readable storage medium, and the method comprises the steps: obtaining first state data of a plurality of first units in the block chain; wherein each first unit is a block unit subjected to data updating in the block chain; performing duplicate removal processing on the multiple pieces of first state data to obtain multiple pieces of second state data; and the multiple pieces of second state data are stored in the corresponding storage positions in the storage area in batches. The method can reduce the waste of storage space and improve the utilization rate of storage resources.

Description

Technical Field

[0001] The present application belongs to the field of blockchain data storage technology, and in particular, relates to a data storage method, apparatus, terminal device, and computer-readable storage medium. Background Art

[0002] With the widespread application of blockchain technology in finance, Internet of Things, supply chain and other fields, the amount of data in the blockchain has exploded. Each participating node needs to store all transaction data and status updates starting from the blockchain genesis block, which has led to a sharp increase in the storage requirements of the blockchain.

[0003] In blockchain-related scenarios, in order to improve data storage performance, a write-time copy strategy is adopted in related technologies. When a new version is generated, the old data will not be directly overwritten. Instead, new nodes are created to retain historical data, ensuring the consistency of version data. However, since nodes are stored in pages, when a small amount of data in a page needs to be modified, the entire page must be copied and rewritten, resulting in excessive data replication and inefficient use of storage resources. Summary of the invention

[0004] The embodiments of the present application provide a data storage method, apparatus, and computer-readable storage medium for a terminal device, which can eliminate the need to repeatedly store unchanged data, reduce waste of storage space, and improve utilization of storage resources.

[0005] In a first aspect, an embodiment of the present application provides a blockchain data storage method, comprising:

[0006] Obtaining first state data of multiple first units in the blockchain; wherein each first unit is a block unit in which data update occurs in the block unit;

[0007] Deduplication processing is performed on the plurality of first state data to obtain a plurality of second state data;

[0008] The plurality of second state data are stored in batches in corresponding storage locations in the storage area.

[0009] In an embodiment of the present application, information of all blocks on the blockchain where data updates occur is collected, and then the collected block information is deduplicated, that is, the data of duplicate records is filtered out, and finally all the deduplicated data is written to the storage area in batches, which is equivalent to accumulating multiple write operations and writing them to the storage area after deduplication, which can avoid repeated storage of unchanged data, reduce the waste of storage space, and improve the utilization rate of the storage medium.

[0010] In a possible implementation manner of the first aspect, deduplication processing is performed on the plurality of first state data to obtain the plurality of second state data, including:

[0011] Acquire, from the first state data of each first unit, coded data for data identification of the first state data;

[0012] The repeated coded data and first state data are filtered out from the multiple coded data and the multiple first state data to obtain multiple second state data.

[0013] In the embodiment of the present application, by deduplicating the stored data and storing only incremental data, the waste of storage space can be reduced and the utilization rate of the storage medium can be improved.

[0014] In one embodiment, filtering out repeated coded data and repeated first state data from the plurality of coded data and the plurality of first state data to obtain the plurality of second state data includes:

[0015] Traverse multiple encoded data and filter out duplicate encoded data;

[0016] The first state data corresponding to the plurality of coded data are traversed, and repeated first state data in the first state data corresponding to the coded data are filtered out to obtain a plurality of second state data.

[0017] In the embodiment of the present application, by performing deduplication processing on the stored data, the write amplification effect can be reduced and the data writing performance can be improved.

[0018] In one embodiment, storing the plurality of second state data in batches to corresponding storage locations in the storage area includes:

[0019] sorting the plurality of first units according to an identifier corresponding to each first unit;

[0020] Classifying the plurality of second state data according to the sorted first units to obtain classified second state data;

[0021] The classified second state data is stored in batches to corresponding storage locations in the storage area.

[0022] In an embodiment of the present application, multiple status data can be classified and sorted for block height according to the block identifier, and a specific block can be quickly located by the block height, thereby improving the processing performance of the status data.

[0023] In one embodiment, classifying the plurality of second state data according to the sorted first units to obtain the classified second state data includes:

[0024] Sorting the plurality of second state data according to the coded data corresponding to each second state data to obtain sorted second state data;

[0025] The sorted second state data are respectively classified into each sorted first unit according to the identifier corresponding to the first unit.

[0026] In an embodiment of the present application, multiple second state data in each block identifier (height) are sorted according to the code (which can be a hash value), which can optimize subsequent batch processing and query performance.

[0027] In one embodiment, storing the classified second state data in batches to corresponding storage locations in the storage area includes:

[0028] At preset intervals, filtering the plurality of second state data in each first unit to remove duplicate second state data in each first unit;

[0029] The second state data in the plurality of first cells after filtering out duplicate data are stored in corresponding storage locations in the storage area.

[0030] In one embodiment, after a plurality of second state data are stored in batches to corresponding storage locations in the storage area, and when there is third state data of an updated second unit in the blockchain, the third state data is deduplicated, and the deduplicated third state data is stored in a new storage location in the storage area, wherein the second unit is a block unit in the blockchain where data is updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

[0031] In an embodiment of the present application, after each data write operation, the system generates a new node, that is, a new storage location or area for subsequent updated status data write operations. This dynamic update mechanism can ensure the stability of the system and the orderliness of the data.

[0032] In a second aspect, an embodiment of the present application provides a data storage device, including:

[0033] A data acquisition module, used to acquire first state data of multiple first units in the blockchain; wherein each of the first units is a block unit where data update occurs in the blockchain;

[0034] A data deduplication module, used for performing deduplication processing on a plurality of the first state data to obtain a plurality of the second state data;

[0035] The data storage module is used to store a plurality of the second state data in batches into a corresponding storage area.

[0036] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data storage method as described in any one of the first aspects above is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the data storage method as described in any one of the first aspects above.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the data storage methods of the first aspect.

[0039] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a flowchart of a data storage method provided in an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of a process for obtaining second state data provided in an embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of Batch structure data provided in an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the process of batch storage of status data provided by the embodiment of the present application Figure 1 ;

[0045] Figure 5 It is a schematic diagram of using the BlockQueue structure to sort block units provided in an embodiment of the present application;

[0046] Figure 6 This is a schematic diagram of using BlockQueue to sort state data provided in an embodiment of the present application;

[0047] Figure 7It is a schematic diagram of the BlockQueue data structure provided in an embodiment of the present application;

[0048] Figure 8 This is a schematic diagram of the process of batch storage of status data provided by the embodiment of the present application Figure 2 ;

[0049] Fig. 9 is a schematic diagram of a data storage device provided in an embodiment of the present application;

[0050] Fig.10 It is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0053] It should also be understood that the term “and / or” used in the specification and appended claims of the present application refers to any combination of one or of the associated listed items and all possible combinations, and includes these combinations.

[0054] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or but not all embodiments", unless otherwise specifically emphasized in other ways.

[0057] With the widespread application of blockchain technology in finance, Internet of Things, supply chain and other fields, the amount of data in the blockchain has exploded. Each participating node needs to store all transaction data and status updates starting from the blockchain genesis block, which has led to a sharp increase in the storage requirements of the blockchain.

[0058] In the blockchain scenario, in order to improve the storage performance of data, the relevant technology adopts a tree structure (Time-Structured Multi-version Merkle Tree, TSMM-Tree) that stores multiple versions according to the time structure, and combines it with a write-time copy strategy. When generating a new version, the old data will not be directly overwritten, but historical data will be retained by creating new nodes to ensure the consistency of version data. However, since nodes are stored in pages, when a small amount of data in a page needs to be modified, the entire page must be copied and rewritten. In addition, in the relevant technology, when a leaf node changes, the change will propagate upward along the tree hierarchy, updating the parent node in turn until the root node. Such a hierarchical propagation mechanism ensures the integrity of the tree structure and can reflect the latest status changes at all levels. The problems caused by the above are as follows:

[0059] ① Excessive data replication

[0060] Since nodes are stored in Page units, when a small amount of data in a Page needs to be modified, the entire Page must be copied and rewritten. Even if the actual modified data only occupies a small part of the Page, the entire 4KB Page needs to be rewritten, resulting in a large amount of data duplication and redundancy.

[0061] ② Redundant overhead of copy-on-write: Whenever a new version is generated, the system must copy the leaf nodes of the previous version, even though the actual modified data only accounts for a small part of the node. This operation mode increases unnecessary disk I / O and storage overhead.

[0062] ③ Write amplification effect of hierarchical propagation: Changes to leaf nodes not only affect themselves, but also recursively update their parent nodes until the root node. Each update causes the nodes on the entire path to be copied and rewritten, further increasing the write overhead. This recursive update causes each modification to involve rewriting the node, amplifying the write cost.

[0063] In order to solve the above-mentioned related technical problems, the present application provides a data storage method. In combination with the actual application scenarios of blockchain, the present application introduces batch processing (Batch), blocking queue (BlockQueue), checkpoint (Checkpoint) mechanism, and append-type log (Vlog) storage strategy when using tree structure for data storage, thereby reducing unnecessary Page copy and node update operations, effectively alleviating the write amplification effect, and improving the system's write performance and overall efficiency.

[0064] Among them, the above-mentioned Batch mechanism is a mechanism that processes multiple tasks or data items as a whole, rather than one by one; the blocking queue is a special queue. When the queue is empty, the task execution thread will wait until data is available in the queue. When the queue is full, the task generation thread will wait until there is free space in the queue; the Checkpoint mechanism refers to a snapshot of the system's state saved at a specific point in time, and an event trigger operation is performed when the system saves certain data at a specific time; the append-type log storage strategy refers to writing new log records (such as operation logs, transaction logs, etc.) to the log file in an appended manner, rather than overwriting or modifying existing data. The above mechanism can be used to store data that has been updated in the blockchain scenario without the need to repeatedly store unchanged data, reducing the waste of storage space and improving the utilization of storage resources.

[0065] See also Figure 1 , is a flow chart of a data storage method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0066] S101, obtaining first state data of multiple first units in the blockchain, wherein each first unit is a block unit where data update occurs in the blockchain.

[0067] In the embodiment of the present application, the state data of multiple block units (referred to as first units) where data updates occur are obtained from the blockchain. "State data" generally refers to the current information stored on the blockchain, such as account balances, asset ownership, the status of smart contracts, etc., and "first state data" refers to the state data after the transaction occurs.

[0068] Specifically, a complete record of the transaction can be retrieved by using a blockchain browser or communicating with a blockchain node, and the transaction record can be analyzed to determine the state of the blockchain before the transaction occurred, and relevant state data can be extracted from the transaction record, which may include but is not limited to account balances, tokens owned by an address, storage variables of smart contracts, etc.

[0069] S102, performing deduplication processing on the plurality of first state data to obtain a plurality of second state data.

[0070] In an embodiment of the present application, when storing the first state data that has been updated, in order to improve the storage space utilization of the system, it is necessary to perform data cleaning, i.e., data deduplication processing, on the updated first state data, which can clean some duplicate data and reduce the writing operations of duplicate data.

[0071] Specifically, the batch mechanism can be used to deduplicate the first state data. The batch mechanism is a data structure table implemented by using a skip list. Among them, a skip list is a data structure that improves search efficiency by adding multi-level indexes on the basis of a linked list. Here, it is used to implement a "Batch", that is, a batch-processed data set. Batch is equivalent to a container for collecting transaction data in the blockchain. It is implemented internally using a table adjustment structure. The reason for using a skip list is to improve the efficiency of data writing and retrieval. The skip list can quickly locate the location of the data through multi-level indexes, thereby reducing the search time and realizing deduplication operations.

[0072] After obtaining multiple updated data (first state data) updated in the blockchain, all transaction data of each unit shall be written into a Batch structure data table in priority. After writing into the Batch structure data table, all transaction data shall be batch deduplicated using the skip table structure inside the Batch structure data table to obtain the deduplicated second state data. The process of deduplicating the first state data using the Batch mechanism is detailed in steps S201-S202.

[0073] In one embodiment, Figure 2 FIG. 1 is a flow chart of obtaining second state data provided by an embodiment of the present application, wherein step S102 includes:

[0074] S201, acquiring, from the first state data of each first unit, coded data for data identification of the first state data.

[0075] In the embodiments of the present application, the encoded data generally refers to a specific field in the transaction record, such as the Key field, which is used to uniquely identify a transaction. The Key field may be the transaction ID itself, which is the unique identifier of the transaction record. In many systems, the transaction ID is generated by a hash algorithm, such as SHA-256, to ensure its uniqueness and immutability.

[0076] In one implementation, the Key field may be a simplified representation of the transaction hash value. In the blockchain, each transaction has a hash value, which is a summary of the transaction content. For ease of reference and storage, a substring or specific length of the hash value is sometimes used, such as Figure 3 As shown, it is a schematic diagram of the Batch structure provided in an embodiment of the present application, including multiple Key fields and the Value value (status data) corresponding to each field.

[0077] In another implementation, if the transaction involves the transfer of funds between accounts, the KEY field may be the account address involved in the transaction. The account address is public and usually has a fixed length, for example, the address in Bitcoin is 26 to 35 characters.

[0078] S202, filtering out repeated coded data and first state data from the plurality of coded data and the plurality of first state data to obtain a plurality of second state data.

[0079] In one implementation, step S202 includes:

[0080] Traverse multiple coded data and filter out repeated coded data; traverse the first state data corresponding to the multiple coded data and filter out repeated first state data in the first state data corresponding to the coded data to obtain multiple second state data.

[0081] In the embodiment of the present application, by traversing all the state data in the Batch, some identical data, such as key1=A, key1=B, etc., can be removed based on the first coded data, and the same key1 (coded data) can be filtered out, and only one key1 code can be retained. At the same time, the data values ​​(values) corresponding to the same key field, i.e., the first state data A or B, etc., are removed, and the deduplicated data will be regarded as the "second state data".

[0082] In the above method, the first state data is initially deduplicated through the Batch structure, which reduces the complexity and repeated operations of subsequent processing.

[0083] S103, storing the plurality of second state data in batches into corresponding storage locations in the storage area.

[0084] In one embodiment, Figure 4The figure is a schematic diagram of the process of batch storage of status data provided by the embodiment of the present application. Figure 1 , step S103 includes:

[0085] S301, sorting a plurality of first units according to an identifier corresponding to each first unit.

[0086] In the embodiment of the present application, in order to reduce the frequent copying of Page during the write operation and to reduce the waste of storage space, BlockQueue can be used to store the second state data in batches. BlockQueue is a queue structure for storing Batch data. One Batch data stores the transaction data of a block unit, and BlockQueue can store multiple Batch data.

[0087] BlockQueue uses the block height (identifier) ​​of the block unit for organization, where the block height refers to the position of a block in the blockchain, usually represented by an integer. Specifically, the block height is calculated from the Genesis Block of the blockchain. The height of the Genesis Block is 0, and the height of each subsequent block increases by 1..., where the block height 0, 1... can be the identifier corresponding to each block unit, and the identifiers are sorted in the BlockQueue in ascending order. The first unit after the sorting of the BlockQueue is as follows Figure 5 As shown in FIG. 1 , it is a schematic diagram of using the BlockQueue structure to sort block units provided in an embodiment of the present application, such as Figure 5 As shown, Seq0...Seqn are the block heights corresponding to the first unit where data update occurs in the blockchain.

[0088] S302, classifying the plurality of second state data according to the sorted first units to obtain classified second state data.

[0089] In one embodiment, step S302 includes:

[0090] The plurality of second state data are sorted according to the coded data corresponding to each second state data to obtain sorted second state data; and the sorted second state data are respectively classified into each sorted first unit according to the identifier corresponding to the first unit.

[0091] In the embodiment of the present application, whenever the transaction data of a block is written in batch, the second state data (the data after deduplication processing) in the batch is sorted according to the code corresponding to each state data (that is, the hash value can be the key field), such as Figure 6As shown in FIG. 1 , it is a schematic diagram of using BlockQueue to sort state data provided by an embodiment of the present application, such as Figure 6 is a batch of sorted data, and the encoded data of the sorted second state data are: key1, key2..., keyN, and ValueA, ValueA...keyZ are the values ​​corresponding to each encoded data, namely the second state data.

[0092] The data of a block unit is written into a Batch data. After the Batch data of multiple block units are sorted as above, the multiple Batch data are temporarily stored in the BlockQueue data structure, and the data is classified according to the block height, and the classified second state data is temporarily stored in the corresponding position of the BlockQueue. The sorted third state data is as follows Figure 7 The figure shows a schematic diagram of the BlockQueue data structure provided by the implementation of this application.

[0093] like Figure 7 The Seq in the BlockQueue shown is the current block height, Key is the transaction Key field in the current block, Len is the total number of Keys in the current block, and Vtr is the storage location pointer information of the Value data corresponding to the current Key on Vlog. In a BlockQueue, all data are arranged in a continuous growth manner according to Seq. In a Seq, all keys are arranged in the order of the data after Hash. The purpose is to control the query efficiency when querying data and store it according to the BlockQueue data in the subsequent writing process.

[0094] In the above method, the system will sort the data by hash value according to the height of the block, and store them in BlockQueue after sorting. BlockQueue is used to temporarily store data to avoid frequent write operations.

[0095] S303, storing the classified second state data in batches to corresponding storage locations in the storage area.

[0096] In the embodiment of the present application, after BlockQueue caches the data of the Batch structure, the data of the BlockQueue can be written into the corresponding storage area.

[0097] In the above method, multiple block data are accumulated through BlockQueue and organized by block height (Seq), and sorted by the hash value of the Key to facilitate batch processing. Through BlockQueue, multiple small batches of data can be written to the storage area uniformly when a Checkpoint is triggered, effectively reducing the frequency of data duplication and write amplification effect.

[0098] In one embodiment, see Figure 8 , is a schematic diagram of the process of batch storage of status data provided in the embodiment of the present application Figure 2 , step S303 includes:

[0099] S401, filtering the plurality of second status data in each first unit at a preset time interval to remove the repeated second status data in each first unit.

[0100] In an embodiment of the present application, in order to avoid frequent write operations, the present application provides an optimization mechanism for Checkpoint batch writing, and the trigger conditions of Checkpoint can be set by preset thresholds, such as triggering based on time (such as every 1 minute) or data volume (such as the amount of data stored in the BlockQueue reaches a certain threshold or every n block heights).

[0101] Whenever a Checkpoint is triggered, the second-state data will be deduplicated again before being written to the storage medium to avoid writing duplicate data. The batch write operation significantly reduces the number of Page copies for a single write and reduces the write amplification effect. When multiple deduplicated second-state data are written to the storage location, an append-type Vlog mechanism is used for writing. The data in the Vlog is appended in chronological order and does not involve random write operations.

[0102] S402: Store the second state data in the plurality of first units after filtering out duplicate data into corresponding storage locations in a storage area.

[0103] In an embodiment of the present application, when a Checkpoint is triggered and after deduplication processing is performed on the second-state data, the system will batch process the data in multiple first units in the BlockQueue (second-state data). After the Checkpoint cycle ends, the system will write the data in the BlockQueue into the storage area in batches.

[0104] In the process of data storage, a tree structure is used to store data. When the blockchain is initialized, a genesis block is generated. Based on the genesis block, a reference tree is generated in the state tree, and the updated state data is used to write the current reference tree. The state tree is a tree structure containing multiple nodes, each node represents a state data, and the state tree can be used to represent the state change of the system.

[0105] During the storage process, the benchmark tree writes data according to the transaction data (Key-Value pair) of the BlockQueue, and the Value part is stored in an independent Vlog. The Vlog uses an append-write mechanism to write to the first storage location corresponding to the storage area, which can effectively avoid the performance loss caused by random writes. The storage area can be a disk. The data in the Vlog is appended in chronological order and does not involve random write operations. This method can greatly improve the write performance and avoid the storage fragmentation problem caused by write operations. Each Key will record the position of its corresponding Value in the Vlog (Vtr pointer) in the tree structure to ensure that the data can be quickly located. During query operations, the system can quickly find the Value data through the Vtr pointer to optimize query efficiency.

[0106] In the above method, the Checkpoint mechanism is introduced to delay the data writing operation until the Checkpoint cycle ends before batch writing. This batch writing strategy significantly reduces the frequency of node replication and update, and reduces the write amplification effect of the system. Delayed writing not only reduces the number of Page replications, but also reduces unnecessary disk I / O operations, improving the system's write performance.

[0107] In one embodiment, the method further comprises:

[0108] After storing multiple second state data in batches to corresponding storage locations in the storage area, and when there is updated third state data of the second unit in the blockchain, the third state data is deduplicated, and the deduplicated third state data is stored in a new storage location in the storage area, wherein the second unit is a block unit in the blockchain where data is updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

[0109] In an embodiment of the present application, after each Checkpoint operation is completed, the tree structure will be updated to generate a new baseline tree as a new storage location in the storage area where data is written. This is because within a Checkpoint cycle, the initial baseline tree has stored enough status data of the block units. When the status data of a new block unit, i.e., the second unit, is updated, a baseline tree of information needs to be regenerated for subsequent writing operations of the new updated data (third single status data). This baseline tree update strategy ensures the stability of the tree structure and avoids frequent node updates and data disorder problems.

[0110] It should be noted that when the benchmark tree in the memory stores enough state data, in order to reduce storage pressure, some historical data can be transferred to some other devices to reduce memory pressure.

[0111] It should be noted that after each Checkpoint operation, the latest state tree generated by the system will be used as the new benchmark number, i.e., the second node, for the next round of data writing operations. This dynamic update of the benchmark tree ensures the structural stability of the tree structure and the orderliness of the data.

[0112] This application aims to use the tree structure to solve the problem of large-scale duplication of duplicate data during data storage. Combined with the actual application scenarios of blockchain, this application proposes an innovative optimization solution. By introducing Batch, BlockQueue, Checkpoint mechanism, and append-type Vlog storage strategy, it reduces unnecessary Page copy and node update operations, effectively alleviates the write amplification effect, and improves the system's write performance and overall efficiency.

[0113] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0114] In one embodiment, a data storage device is provided, and the data storage device corresponds one-to-one with the data storage method in the above embodiment. Fig. 9 FIG. 1 is a schematic diagram of a data storage device provided in an embodiment of the present application, including:

[0115] A data acquisition module 91 is used to acquire first state data of multiple first units in the blockchain; wherein each first unit is a block unit where data update occurs in the blockchain;

[0116] A data deduplication module 92, configured to perform deduplication processing on the plurality of first state data to obtain a plurality of second state data;

[0117] The data storage module 93 is used to store a plurality of the second state data in batches to corresponding storage locations in the storage area.

[0118] Optionally, the data deduplication module 92 is further used for:

[0119] Acquire, from the first state data of each first unit, coded data for data identification of the first state data;

[0120] Repeated encoded data and repeated first state data are filtered out from the multiple encoded data and the multiple first state data to obtain multiple second state data.

[0121] Optionally, the data deduplication module 92 is further used for:

[0122] Traversing the plurality of coded data and filtering out repeated coded data;

[0123] The first state data corresponding to the plurality of coded data are traversed, and repeated first state data in the first state data corresponding to the coded data are filtered out to obtain a plurality of second state data.

[0124] Optionally, the data storage module 93 is further used for:

[0125] sorting the plurality of first units according to an identifier corresponding to each first unit;

[0126] Classifying the plurality of second state data according to the sorted first units to obtain classified second state data;

[0127] The classified second state data is stored in batches to corresponding storage locations in the storage area.

[0128] Optionally, the data storage module 93 is further used for:

[0129] Sorting the plurality of second state data according to the coded data corresponding to each second state data to obtain sorted second state data;

[0130] The sorted second state data are respectively classified into each sorted first unit according to the identifier corresponding to the first unit.

[0131] Optionally, the data storage module 93 is further used for:

[0132] At preset intervals, filtering the plurality of second state data in each first unit to remove duplicate second state data in each first unit;

[0133] The second state data in the plurality of the first units after filtering out duplicate data are stored in corresponding storage locations in the storage area.

[0134] Optionally, the data storage module 93 is further used for:

[0135] After storing a plurality of second state data in batches to the first storage location in the storage area, and when there is third state data of an updated second unit in the blockchain, deduplication processing is performed on the third state data, and the deduplication-processed third state data is stored in a new storage location in the storage area, wherein the second unit is a block unit in the blockchain where data is updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

[0136] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0137] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0138] Fig.10 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig.10 As shown, the terminal device 10 of this embodiment includes: at least one processor 100 ( Fig.10 Only one is shown in the figure) a processor, a memory 101, and a computer program 102 stored in the memory 101 and executable on at least one processor 100, and when the processor 100 executes the computer program 102, the steps of any of the above-mentioned data storage method embodiments are implemented.

[0139] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Fig.10 It is only an example of the terminal device 10 and does not constitute a limitation on the terminal device 10. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0140] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] In some embodiments, the memory 101 may be an internal storage unit of the terminal device 10, such as a hard disk or memory of the terminal device 10. In other embodiments, the memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 10. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as program codes of a computer program, etc. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0142] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0143] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data storage method, characterized in that: Applied to blockchain, the method includes: Obtaining first state data of a plurality of first units in a blockchain; wherein each of the first units is a block unit in which data update occurs in the blockchain; Deduplication processing is performed on the plurality of first state data to obtain a plurality of second state data; The plurality of second state data are stored in batches to corresponding storage locations in the storage area.

2. The data storage method according to claim 1, characterized in that: The deduplication processing of the plurality of the first state data to obtain the plurality of the second state data includes: Acquire, from the first state data of each of the first units, coded data for data identification of the first state data; The repeated coded data and repeated first state data are filtered out from the plurality of coded data and the plurality of first state data to obtain a plurality of second state data.

3. The storage method according to claim 2, characterized in that: The filtering out repeated coded data and repeated first state data from the plurality of coded data and the plurality of first state data to obtain a plurality of second state data includes: Traversing the plurality of coded data and filtering out repeated coded data; The first state data corresponding to the plurality of coded data are traversed, and repeated first state data in the first state data corresponding to the coded data are filtered out to obtain a plurality of second state data.

4. The storage method according to claim 1, characterized in that: The step of storing the plurality of second state data in batches to corresponding storage locations in the storage area includes: sorting the plurality of first units according to an identifier corresponding to each of the first units; Classifying the plurality of second-state data according to the sorted first units to obtain classified second-state data; The classified second state data are stored in batches to corresponding storage locations in the storage area.

5. The data storage method according to claim 4, characterized in that: The step of classifying the plurality of second state data according to the sorted first units to obtain the classified second state data includes: sorting the plurality of second state data according to the coded data corresponding to each of the second state data to obtain sorted second state data; The sorted second state data are respectively classified into each of the sorted first units according to the identifiers corresponding to the first units.

6. The data storage method according to claim 4, characterized in that: The step of storing the classified second state data in batches to corresponding storage locations in a storage area includes: At preset intervals, filtering the plurality of second state data in each of the first units to remove duplicate second state data in each of the first units; The second state data in the plurality of the first units after filtering out duplicate data are stored in corresponding storage locations in the storage area.

7. The data storage method according to claim 6, characterized in that: The method further comprises: After storing multiple second-state data in batches to corresponding storage locations in the storage area, and when there is third-state data of an updated second unit in the blockchain, deduplication processing is performed on the third-state data, and the deduplication-processed third-state data is stored in a new storage location in the storage area, wherein the second unit is a block unit in the blockchain where data is updated after the first unit, and the third-state data is the updated state data corresponding to the second unit.

8. A data storage device, characterized in that: include: A data acquisition module, used to acquire first state data of multiple first units in the blockchain; wherein each of the first units is a block unit where data update occurs in the blockchain; A data deduplication module, used for performing deduplication processing on a plurality of the first state data to obtain a plurality of the second state data; The data storage module is used to store a plurality of the second state data in batches to corresponding storage locations in the storage area.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images