Data storage method and device, terminal equipment and computer readable storage medium

Abstract

The application is suitable for the technical field of data storage of blockchains, and provides a data storage method, device, terminal equipment and computer readable storage medium, comprising: acquiring first state data of a plurality of first units in a blockchain; wherein each first unit is a block unit in the blockchain that has data update; performing deduplication processing on the plurality of first state data to obtain a plurality of second state data; and storing the plurality of second state data in a storage area in corresponding storage positions in batches. The above method can reduce the waste of storage space and improve the utilization rate of storage resources.

Description

Technical Field

[0001] This application belongs to the field of blockchain data storage technology, and in particular relates to a data storage method, apparatus, terminal equipment and computer-readable storage medium. Background Technology

[0002] With the widespread application of blockchain technology in finance, the Internet of Things, supply chain and other fields, the amount of data in blockchains is growing explosively. Each participating node needs to store all transaction data and state updates from the blockchain genesis block, which leads to a sharp increase in the storage requirements of blockchains.

[0003] In blockchain-related scenarios, in order to improve data storage performance, the copy-on-write strategy is adopted in related technologies. When a new version is generated, the old data is not directly overwritten. Instead, historical data is retained by creating new nodes, which ensures the consistency of version data. However, since nodes are stored in units of pages, when a small amount of data in a page needs to be modified, the entire page must be copied and rewritten, resulting in excessive data copying and inefficient use of storage resources. Summary of the Invention

[0004] This application provides a data storage method, apparatus, and computer-readable storage medium for terminal devices, which can eliminate the need to repeatedly store unchanged data, reduce storage space waste, and improve the utilization rate of storage resources.

[0005] In a first aspect, embodiments of this application provide a blockchain data storage method, including:

[0006] Obtain the first state data of multiple first units in the blockchain; where each first unit is a block unit in the block unit where data updates have occurred;

[0007] Multiple first-state data are deduplicated to obtain multiple second-state data.

[0008] Multiple second-state data are stored in batches to the corresponding storage locations in the storage area.

[0009] In this embodiment of the application, information on all blocks on the blockchain that have undergone data updates is collected. Then, the collected block information is deduplicated, that is, duplicate data is filtered out. Finally, all the deduplicated data is written to the storage area in batches. This is equivalent to writing the deduplicated data to the storage area after accumulating multiple write operations, which can avoid the duplicate storage of unchanged data, reduce the waste of storage space, and improve the utilization rate of storage media.

[0010] In one possible implementation of the first aspect, multiple first state data are deduplicated to obtain multiple second state data, including:

[0011] From the first state data of each first unit, obtain the encoded data used to identify the first state data;

[0012] Repeated encoded data and first-state data are filtered out from multiple encoded data and multiple first-state data to obtain multiple second-state data.

[0013] In this embodiment of the application, by deduplicating the stored data and storing only incremental data, the waste of storage space can be reduced and the utilization rate of the storage medium can be improved.

[0014] In one embodiment, filtering out duplicate encoded data and duplicate first state data from a plurality of encoded data and a plurality of first state data to obtain a plurality of second state data includes:

[0015] Iterate through multiple encoded data sets and filter out duplicate encoded data.

[0016] The first state data corresponding to multiple encoded data is traversed, and duplicate first state data is filtered out in the first state data corresponding to the encoded data to obtain multiple second state data.

[0017] In this embodiment of the application, by deduplicating the stored data, the write amplification effect can be reduced and the data write performance can be improved.

[0018] In one embodiment, the step of batch storing multiple second state data to corresponding storage locations in the storage area includes:

[0019] Sort the multiple first units according to the identifier corresponding to each first unit;

[0020] Multiple second-state data are classified according to the sorted first unit to obtain classified second-state data;

[0021] The categorized second-state data is stored in batches to the corresponding storage location in the storage area.

[0022] In this embodiment, multiple state data can be classified and sorted according to block identifiers based on block height. A specific block can be quickly located by block height, thereby improving the processing performance of state data.

[0023] In one embodiment, multiple second-state data are classified according to the sorted first unit to obtain classified second-state data, including:

[0024] The multiple second-state data are sorted according to the encoded data corresponding to each second-state data to obtain the sorted second-state data;

[0025] The sorted second state data are classified into each sorted first unit according to the identifier corresponding to the first unit.

[0026] In this embodiment of the application, sorting multiple second state data in each block identifier (height) according to the encoding (which can be a hash value) can optimize the performance of subsequent batch processing and querying.

[0027] In one embodiment, storing the categorized second state data in batches to the corresponding storage location in the storage area includes:

[0028] At preset intervals, multiple second state data in each first unit are filtered to remove duplicate second state data in each first unit.

[0029] The second state data from multiple first units after filtering out duplicate data is stored in the corresponding storage location in the storage area.

[0030] In one embodiment, after multiple second state data are stored in batches to the corresponding storage location in the storage area, and when there is updated third state data of the second unit in the blockchain, the third state data is deduplicated, and the deduplicated third state data is stored in a new storage location in the storage area. The second unit is the block unit in the blockchain that has been updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

[0031] In this embodiment, after each data write operation, the system generates a new node, i.e., a new storage location or region, for subsequent update state data write operations. This dynamic update mechanism can ensure the stability of the system and the orderliness of the data.

[0032] Secondly, embodiments of this application provide a data storage device, including:

[0033] The data acquisition module is used to acquire the first state data of multiple first units in the blockchain; wherein each first unit is a block unit in the blockchain that has undergone data updates;

[0034] The data deduplication module is used to deduplicatize multiple first state data to obtain multiple second state data.

[0035] The data storage module is used to store multiple sets of the second state data in batches to the corresponding storage area.

[0036] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data storage method as described in any of the first aspects above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a data storage method as described in any of the first aspects above.

[0038] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute any of the data storage methods described in the first aspect above.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the data storage method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the process for obtaining second state data provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the Batch structure data provided in the embodiments of this application;

[0044] Figure 4 This is a schematic diagram of the process for batch storage of status data provided in the embodiments of this application. Figure 1 ;

[0045] Figure 5 This is a schematic diagram of block unit sorting using the BlockQueue structure provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of sorting state data using BlockQueue provided in an embodiment of this application;

[0047] Figure 7This is a schematic diagram of the BlockQueue data structure provided in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of the process for batch storage of status data provided in the embodiments of this application. Figure 2 ;

[0049] Figure 9 This is a schematic diagram of a data storage device provided in an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0051] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0052] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0053] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any one or any combination of the associated listed items and all possible combinations, and includes such combinations.

[0054] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0055] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or not all embodiments," unless otherwise specifically emphasized.

[0057] With the widespread application of blockchain technology in finance, the Internet of Things, supply chain and other fields, the amount of data in blockchains is growing explosively. Each participating node needs to store all transaction data and state updates from the blockchain genesis block, which leads to a sharp increase in the storage requirements of blockchains.

[0058] In blockchain scenarios, to improve data storage performance, related technologies employ a time-structured multi-version Merkle Tree (TSMM-Tree) for multi-version storage, combined with a copy-on-write strategy. When a new version is generated, old data is not directly overwritten; instead, historical data is preserved by creating new nodes, ensuring consistency across versions. However, since nodes are stored in units of pages, when a small amount of data in a page needs modification, the entire page must be copied and rewritten. Furthermore, in these technologies, when a leaf node changes, the change propagates upwards along the tree hierarchy, updating parent nodes sequentially until the root node. This hierarchical propagation mechanism ensures the integrity of the tree structure and reflects the latest state changes at all levels. The problems arising from this are as follows:

[0059] ① Excessive data duplication

[0060] Because nodes are stored in units of pages, when a small amount of data in a page needs to be modified, the entire page must be copied and rewritten. Even if the actual modified data only accounts for a small portion of the page, the entire 4KB page needs to be rewritten, resulting in redundancy due to extensive data copying.

[0061] ② Redundancy overhead of copy-on-write: Whenever a new version is generated, the system must copy the leaf nodes of the previous version, even though the actual modified data only accounts for a small portion of the nodes. This operation increases unnecessary disk I / O and storage overhead.

[0062] ③ Hierarchical propagation write amplification effect: Changes to leaf nodes not only affect themselves but also recursively update their parent nodes, up to the root node. Each update causes nodes along the entire path to be copied and rewritten, further increasing write overhead. This recursive update results in each modification involving the rewriting of nodes, amplifying the write cost.

[0063] To address the aforementioned technical issues, this application provides a data storage method. Combining practical application scenarios of blockchain, this application utilizes a tree structure for data storage by introducing batch processing, block queue, checkpoint mechanisms, and an append-only log (Vlog) storage strategy. This reduces unnecessary page copying and node update operations, effectively mitigating the write amplification effect and improving the system's write performance and overall efficiency.

[0064] The Batch mechanism described above processes multiple tasks or data items as a whole, rather than processing them one by one. A blocking queue is a special type of queue where, when empty, the task execution thread waits until data becomes available; when full, the task generation thread waits until free space becomes available. The Checkpoint mechanism saves a snapshot of the system's state at a specific point in time, triggering events when certain data is saved at that time. The append-only log storage strategy appends new log records (such as operation logs and transaction logs) to the log file, rather than overwriting or modifying existing data. These mechanisms allow for the storage of updated data in blockchain scenarios without repeatedly storing unchanged data, reducing storage space waste and improving storage resource utilization.

[0065] See Figure 1 This is a flowchart illustrating a data storage method provided in an embodiment of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0066] S101, obtain the first state data of multiple first units in the blockchain, where each first unit is a block unit in the blockchain where data updates have occurred.

[0067] In this embodiment of the application, state data of multiple block units (referred to as first units) that have undergone data updates are obtained from the blockchain. "State data" generally refers to the current information stored on the blockchain, such as account balance, asset ownership, and the state of smart contracts. "First state data" refers to the state data after the transaction has occurred.

[0068] Specifically, a blockchain explorer or communication with a blockchain node can be used to retrieve the complete record of the transaction, analyze the transaction record to determine the state of the blockchain before the transaction occurred, and extract relevant state data from the transaction record, which may include, but is not limited to, account balances, smart contract storage variables, etc.

[0069] S102, deduplication of multiple first state data is performed to obtain multiple second state data.

[0070] In this embodiment of the application, when storing the first state data that has been updated, in order to improve the storage space utilization of the system, it is necessary to perform data cleaning, i.e. data deduplication, on the updated first state data. This can clean up some duplicate data and reduce the writing operations of duplicate data.

[0071] Specifically, the batch mechanism can be used to deduplicate the first-state data. The batch mechanism is a data structure implemented using skip lists. A skip list is a data structure that improves search efficiency by adding multi-level indexes to a linked list. Here, it is used to implement a "batch," that is, a collection of data to be processed in batches. A batch is essentially a container for collecting transaction data within the blockchain, implemented internally using a skip list structure. The reason for using skip lists is to improve the efficiency of data writing and retrieval. Skip lists can quickly locate the position of data through multi-level indexes, thereby reducing search time, and can also achieve deduplication.

[0072] After obtaining multiple updated data (first state data) that have been updated in the blockchain, all transaction data of each unit should be written into a Batch structure data table first. After writing into the Batch structure data table, the skip list structure inside the Batch structure data table is used to perform batch deduplication of all transaction data to obtain the deduplicated second state data. The process of using the Batch mechanism to perform deduplication of the first state data is detailed in steps S201-S202.

[0073] In one embodiment, such as Figure 2 The diagram shown is a schematic flowchart of obtaining second state data provided in an embodiment of this application. Step S102 includes:

[0074] S201, obtain coded data for data identification of the first state data from the first state data of each first unit.

[0075] In the embodiments of this application, the encoded data typically refers to a specific field in a transaction record, such as a Key field. This field is used to uniquely identify a transaction, and the Key field may be the transaction ID itself, which is a unique identifier for the transaction record. In many systems, the transaction ID is generated using a hash algorithm, such as SHA-256, ensuring its uniqueness and immutability.

[0076] Specifically, the Key field may be a simplified representation of the transaction hash value. In a blockchain, each transaction has a hash value, which is a summary of the transaction content. For ease of referencing and storage, sometimes a substring or a specific length of the hash value is used, such as... Figure 3 The diagram shown is a schematic of the Batch structure provided in this application embodiment, which includes multiple Key fields and the corresponding Value (status data) for each field.

[0077] S202, filter out duplicate encoded data and first state data from multiple encoded data and multiple first state data to obtain multiple second state data.

[0078] In one implementation, step S202 includes:

[0079] Iterate through multiple encoded data and filter out duplicate encoded data; iterate through the first state data corresponding to multiple encoded data and filter out duplicate first state data in the first state data corresponding to the encoded data to obtain multiple second state data.

[0080] In this embodiment, by traversing all state data in the batch and removing some identical data based on the first encoded data, such as key1=A, key1=B, etc., identical key1 (encoded data) can be filtered out, leaving only one key1 encoding. At the same time, data values ​​(values) corresponding to the same key field, i.e., first state data A or B, are removed. The deduplicated data will be regarded as "second state data".

[0081] In the above method, the first state data is deduplicated using a batch structure, which reduces the complexity of subsequent processing and redundant operations.

[0082] S103, batch store multiple second state data to the corresponding storage location in the storage area.

[0083] In one embodiment, such as Figure 4 The diagram shown is a schematic representation of the process for batch storage of status data provided in an embodiment of this application. Figure 1 Step S103 includes:

[0084] S301, sort the multiple first units according to the identifier corresponding to each first unit.

[0085] In this embodiment, to reduce frequent copying of Pages during write operations and to minimize storage space waste, a BlockQueue can be used to store the second state data in batches. A BlockQueue is a queue structure used to store batch data. Each batch stores the transaction data of a block unit, and a BlockQueue can store multiple batches.

[0086] BlockQueue organizes blocks using their block heights (identifiers). The block height refers to the position of a block on the blockchain and is typically represented by an integer. Specifically, the block height is calculated from the genesis block, which has a height of 0. Subsequent blocks increase in height by 1, and so on. The block heights 0, 1, etc., can be used as identifiers for each block unit, and they are sorted in ascending order based on these identifiers within the BlockQueue. The first unit after sorting in the BlockQueue is as follows: Figure 5 The diagram shown is a schematic diagram of block unit sorting using the BlockQueue structure provided in an embodiment of this application. Figure 5 As shown, Seq0……Seqn represent the block heights corresponding to the first unit in the blockchain where data updates have occurred.

[0087] S302, classify the multiple second state data according to the sorted first unit to obtain the classified second state data.

[0088] In one embodiment, step S302 includes:

[0089] Multiple second-state data are sorted according to the encoded data corresponding to each second-state data to obtain sorted second-state data; the sorted second-state data are then classified into each sorted first unit according to the identifier corresponding to the first unit.

[0090] In this embodiment of the application, after each block of transaction data is written in a batch, the second state data (data after deduplication) in the batch is sorted according to the code corresponding to each state data (i.e., the hash value can be the key field), such as... Figure 6 The diagram shown is a schematic diagram of using BlockQueue for sorting state data according to an embodiment of this application. Figure 6Given a batch of sorted data, the encoded data of the second state data after sorting are key1, key2, ..., keyN, and their ValueA, ValueA...keyZ are the numerical values ​​corresponding to each encoded data, i.e., the second state data.

[0091] Data from one block unit is written to a batch of data. After sorting multiple batches of data according to the above method, the batches are temporarily stored in a BlockQueue data structure. Data is then categorized according to block height, and the categorized second-state data is temporarily stored in the corresponding position in the BlockQueue. The organized third-state data is as follows: Figure 7 The diagram shown is a schematic of the BlockQueue data structure provided in this application.

[0092] like Figure 7 In the BlockQueue shown, Seq represents the current block height, Key represents the transaction key field in the current block, Len represents the total number of keys in the current block, and Vtr represents the storage location pointer information of the value data corresponding to the current key on the Vlog. In a BlockQueue, all data is arranged in a continuously increasing order according to Seq. Within a Seq, all keys are arranged in the order of the hashed data. The purpose of this is to control the efficiency of data query during the subsequent writing process and to store the data according to the BlockQueue.

[0093] In the above method, the system sorts the data by hash value according to the block height, organizes it and stores it in BlockQueue. BlockQueue is used to temporarily store data to avoid frequent write operations.

[0094] S303, Store the classified second state data in batches to the corresponding storage location in the storage area.

[0095] In this embodiment of the application, after the BlockQueue caches the data of the Batch structure, the data of the BlockQueue can be written to the corresponding storage area.

[0096] In the above method, multiple blocks of data are accumulated and organized by block height (Seq) using BlockQueue, and then sorted by the hash value of the key, which facilitates batch processing. Through BlockQueue, multiple small batches of data can be written to the storage area simultaneously when a checkpoint is triggered, effectively reducing the frequency of data copying and the write amplification effect.

[0097] In one embodiment, see Figure 8This is a schematic diagram of the process for batch storage of status data provided in the embodiments of this application. Figure 2 Step S303 includes:

[0098] S401, at preset time intervals, multiple second state data in each first unit are filtered to remove duplicate second state data in each first unit.

[0099] In this application embodiment, in order to avoid frequent write operations, this application provides an optimization mechanism for checkpoint batch writes. The triggering conditions for checkpoints can be set by preset thresholds, such as based on time (e.g., every 1 minute) or data volume (e.g., the amount of data stored in the BlockQueue reaches a certain threshold or every n block heights).

[0100] Whenever a checkpoint is triggered, the second-state data is deduplicated again before being written to the storage medium to avoid writing duplicate data. Batch write operations significantly reduce the number of page copies per write, reducing the write amplification effect. When writing multiple deduplicated second-state data to the storage location, an append-only Vlog mechanism is used. The data in the Vlog is appended in chronological order and does not involve random write operations.

[0101] S402, the second state data of the multiple first units after filtering out duplicate data is stored in the corresponding storage location in the storage area.

[0102] In this embodiment of the application, when a Checkpoint is triggered and after deduplication of the second state data, the system will batch process the data (second state data) in multiple first units in the BlockQueue. After the Checkpoint cycle ends, the system will batch write the data in the BlockQueue to the storage area.

[0103] Specifically, a tree structure is used for data storage. During blockchain initialization, a genesis block is generated. Based on the genesis block, a baseline tree is generated in the state tree, and updated state data is written into the current baseline tree. The state tree is a tree structure containing multiple nodes, each node representing a piece of state data. The state tree can be used to represent the state changes of the system.

[0104] During storage, the baseline tree writes data according to the transaction data (key-value pairs) of the BlockQueue. The value portion is stored in a separate Vlog. The Vlog uses an append-only write mechanism to write to the first storage location corresponding to the storage area. This effectively avoids the performance loss caused by random writes. The storage area can be a disk. Data in the Vlog is appended in chronological order, without random write operations. This method significantly improves write performance and avoids storage fragmentation caused by write operations. Each key records the position of its corresponding value in the Vlog (Vtr pointer) in the tree structure, ensuring that data can be quickly located. During query operations, the system can quickly find the value data through the Vtr pointer, optimizing query efficiency.

[0105] The method described above incorporates a checkpoint mechanism, delaying data write operations until after the checkpoint period ends before performing batch writes. This batch write strategy significantly reduces node replication and update frequency, mitigating the system's write amplification effect. Delayed writing not only reduces the number of page copies but also reduces unnecessary disk I / O operations, improving system write performance.

[0106] In one embodiment, the method further includes:

[0107] After multiple second-state data are stored in batches to the corresponding storage locations in the storage area, and when there is updated third-state data for the second unit in the blockchain, the third-state data is deduplicated, and the deduplicated third-state data is stored in a new storage location in the storage area. Here, the second unit is the block unit in the blockchain that has been updated after the first unit, and the third-state data is the updated state data corresponding to the second unit.

[0108] In this embodiment, after each Checkpoint operation, the tree structure is updated to generate a new baseline tree as the new storage location in the storage area where data is written. This is because the initial baseline tree has stored enough block unit state data within a Checkpoint cycle. When a new block unit, i.e. the second unit's state data, is updated, a new baseline tree needs to be generated for subsequent new updated data (third unit state data) writing operations. This baseline tree update strategy ensures the stability of the tree structure and avoids frequent node updates and data disorder problems.

[0109] It should be noted that when there is enough state data stored in the baseline tree in memory, in order to reduce storage pressure, some historical data can be transferred to other devices to reduce memory pressure.

[0110] It should be noted that after each Checkpoint operation, the latest state tree generated by the system serves as the new baseline, i.e., the second node, for use in the next round of data write operations. This dynamic updating of the baseline tree ensures the structural stability of the tree structure and the order of the data.

[0111] This application aims to address the problem of extensive duplication of data during data storage using tree structures. Combining this with practical blockchain applications, it proposes an innovative optimization solution. By introducing Batch, BlockQueue, and Checkpoint mechanisms, as well as an append-only Vlog storage strategy, it reduces unnecessary page copying and node update operations, effectively mitigating the write amplification effect and improving the system's write performance and overall efficiency.

[0112] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0113] In one embodiment, a data storage device is provided, which corresponds one-to-one with the data storage methods described in the above embodiments. For example... Figure 9 The diagram shown is a schematic diagram of a data storage device provided in an embodiment of this application, including:

[0114] The data acquisition module 91 is used to acquire the first state data of multiple first units in the blockchain; wherein each first unit is a block unit in the blockchain in which data has been updated;

[0115] The data deduplication module 92 is used to deduplicatize multiple first-state data to obtain multiple second-state data;

[0116] The data storage module 93 is used to store multiple second state data in batches to the corresponding storage locations in the storage area.

[0117] Optionally, the data deduplication module 92 is also used for:

[0118] From the first state data of each first unit, obtain the encoded data used to identify the first state data;

[0119] Repeated encoded data and repeated first-state data are filtered out from multiple encoded data and multiple first-state data to obtain multiple second-state data.

[0120] Optionally, the data deduplication module 92 is also used for:

[0121] Iterate through multiple sets of encoded data and filter out duplicate encoded data;

[0122] The first state data corresponding to multiple encoded data is traversed, and duplicate first state data are filtered out from the first state data corresponding to the encoded data to obtain multiple second state data.

[0123] Optionally, the data storage module 93 is also used for:

[0124] Sort the multiple first units according to the identifier corresponding to each first unit;

[0125] Multiple second-state data are classified according to the sorted first unit to obtain the classified second-state data;

[0126] The categorized second-state data is stored in batches to the corresponding storage location in the storage area.

[0127] Optionally, the data storage module 93 is also used for:

[0128] The multiple second-state data are sorted according to the encoded data corresponding to each second-state data to obtain the sorted second-state data;

[0129] The sorted second state data are classified into each sorted first unit according to the identifier corresponding to the first unit.

[0130] Optionally, the data storage module 93 is also used for:

[0131] At preset intervals, multiple second state data in each first unit are filtered to remove duplicate second state data in each first unit;

[0132] The second state data from the multiple first units after filtering out duplicate data is stored in the corresponding storage location in the storage area.

[0133] Optionally, the data storage module 93 is also used for:

[0134] After multiple second state data are stored in batches to the first storage location in the storage area, and there is updated third state data of the second unit in the blockchain, the third state data is deduplicated, and the deduplicated third state data is stored in a new storage location in the storage area. The second unit is the block unit in the blockchain that has been updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

[0135] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] Figure 10 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 10 As shown, the terminal device 10 of this embodiment includes: at least one processor 100 ( Figure 10 (Only one is shown in the diagram) a processor, a memory 101, and a computer program 102 stored in the memory 101 and executable on at least one processor 100, wherein the processor 100 executes the computer program 102 to implement the steps in any of the above-described data storage method embodiments.

[0138] The terminal device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 10 This is merely an example of terminal device 10 and does not constitute a limitation on terminal device 10. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0139] The processor 100 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] In some embodiments, memory 101 may be an internal storage unit of terminal device 10, such as a hard disk or memory of terminal device 10. In other embodiments, memory 101 may be an external storage device of terminal device 10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on terminal device 10. Furthermore, memory 101 may include both internal and external storage units of terminal device 10. Memory 101 is used to store operating system, applications, boot loader, data, and other programs, such as program code of computer programs. Memory 101 may also be used to temporarily store data that has been output or will be output.

[0141] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0142] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0144] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A data storage method, characterized in that, Applied to blockchain, the method includes: Obtain first state data of multiple first units in the blockchain; wherein each first unit is a block unit in the blockchain that has undergone data update; the first state data is the state data of the blockchain after the data update; Multiple first state data are deduplicated to obtain multiple second state data; Batch storage of multiple sets of the second state data to the corresponding storage locations in the storage area includes: The first units are sorted according to the identifier corresponding to each first unit to obtain the sorted first units; The multiple second state data are sorted according to the encoded data corresponding to each second state data to obtain the sorted second state data; The sorted second state data is classified into each sorted first unit according to the identifier corresponding to the first unit, thus obtaining the classified second state data; At preset intervals, the second state data after classification in each first unit is filtered to remove duplicate second state data in each first unit; The second state data from the multiple first units after filtering out duplicate data is stored in the corresponding storage location in the storage area.

2. The data storage method as described in claim 1, characterized in that, The step of deduplicating multiple first state data to obtain multiple second state data includes: From the first state data of each of the first units, obtain encoded data for identifying the first state data; By filtering out duplicate encoded data and duplicate first state data from multiple encoded data and multiple first state data, multiple second state data are obtained.

3. The storage method as described in claim 2, characterized in that, The step of filtering out duplicate encoded data and duplicate first state data from multiple encoded data and multiple first state data to obtain multiple second state data includes: Iterate through multiple sets of encoded data and filter out duplicate encoded data; The first state data corresponding to multiple encoded data is traversed, and duplicate first state data is filtered out from the first state data corresponding to the encoded data to obtain multiple second state data.

4. The data storage method as described in claim 3, characterized in that, The method further includes: After multiple second state data are stored in batches to the corresponding storage location in the storage area, and when there is updated third state data of the second unit in the blockchain, the third state data is deduplicated, and the deduplicated third state data is stored in a new storage location in the storage area. The second unit is the block unit in the blockchain that has been updated after the first unit, and the third state data is the updated state data corresponding to the second unit.

5. A data storage device, characterized in that, include: The data acquisition module is used to acquire first state data of multiple first units in the blockchain; wherein each first unit is a block unit in the blockchain that has undergone data update; and the first state data is the state data of the blockchain after the data update. The data deduplication module is used to deduplicatize multiple first state data to obtain multiple second state data. The data storage module is used to batch store multiple second state data to the corresponding storage location in the storage area; The data storage module is also used for: The first units are sorted according to the identifier corresponding to each first unit to obtain the sorted first units; The multiple second state data are sorted according to the encoded data corresponding to each second state data to obtain the sorted second state data; The sorted second state data is classified into each sorted first unit according to the identifier corresponding to the first unit, thus obtaining the classified second state data; At preset intervals, the classified second state data in each first unit is filtered to remove duplicate second state data in each first unit. The second state data from the multiple first units after filtering out duplicate data is stored in the corresponding storage location in the storage area.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images