A hierarchical storage method and terminal based on blockchain

Abstract

The present invention discloses a hierarchical storage method and terminal based on blockchain. The method comprises the following steps: receiving a data on-chain request and caching the data on-chain request; calculating a hash value for data corresponding to the data on-chain request that meets the preset condition and storing the hash value; storing the data corresponding to the data on-chain request, and upon receiving information that the data is successfully stored, uploading the hash value corresponding to the successfully stored data to the chain; uploading only the hash value corresponding to the data to be uploaded to the chain, thereby greatly reducing the amount of storage on the chain; and calculating and storing the hash value for the corresponding data only when the cached data on-chain request meets the preset condition, and uploading the hash value corresponding to the successfully stored data to the chain, thereby realizing relay processing of the data on-chain request, realizing unified organization and management of the data on-chain request, avoiding the need to execute the upload once for each data on-chain request, and improving the data processing effect.

Description

Technical Field

[0001] The present invention relates to the field of data storage, and in particular to a hierarchical storage method and terminal based on blockchain. Background Art

[0002] Given its advantages of decentralization and immutability, blockchain technology is increasingly being used to meet security requirements. However, to ensure data security and decentralization, each participating node must store and synchronize a complete copy of the ledger data. When blockchain technology is applied to business scenarios with large data volumes, storage resources between nodes are inevitably strained, significantly reducing the efficiency of on-chain node synchronization and verification. Furthermore, the massive amount of data makes complex on-chain queries and analysis difficult and inefficient. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a hierarchical storage method and terminal based on blockchain, which can significantly reduce the demand for storage resources while improving data processing efficiency.

[0004] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0005] A hierarchical storage method based on blockchain, comprising the steps of:

[0006] Receive a data upload request and cache the data upload request;

[0007] When the cached data on-chain request meets the preset conditions, a hash value is calculated for the data corresponding to the data on-chain request that meets the preset conditions and the hash value is stored;

[0008] The data corresponding to the data upload request is stored on the chain. When the information of successful data storage is received, the hash value corresponding to the successfully stored data is uploaded to the chain.

[0009] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0010] A blockchain-based hierarchical storage terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the above-mentioned blockchain-based hierarchical storage method is implemented.

[0011] The beneficial effects of the present invention are as follows: before uploading data to the chain, the data uploading request is first cached; when the cached data uploading request meets the preset conditions, the hash value of the data corresponding to the data uploading request that meets the preset conditions is calculated and stored; when the data corresponding to the data uploading request is successfully stored, the corresponding hash value is uploaded to the chain; through the above-mentioned hierarchical storage method, on the one hand, only the hash value corresponding to the data to be uploaded is uploaded to the chain, which greatly reduces the on-chain storage amount and reduces the resource overhead of data storage and data synchronization of each blockchain node; on the other hand, when the cached data uploading request meets the preset conditions, the hash value of the corresponding data is calculated and stored, and the hash value corresponding to the successfully stored data is uploaded to the chain, thereby realizing relay processing of the data uploading request, realizing unified organization and management of the data uploading request, avoiding the need to execute an uploading for each data uploading request, and improving the data processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flowchart of a hierarchical storage method based on blockchain according to an embodiment of the present invention;

[0013] Figure 2 This is a schematic diagram of the structure of a blockchain-based hierarchical storage terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0015] Please refer to Figure 1 , a hierarchical storage method based on blockchain, comprising the steps of:

[0016] Receive a data upload request and cache the data upload request;

[0017] When the cached data on-chain request meets the preset conditions, a hash value is calculated for the data corresponding to the data on-chain request that meets the preset conditions and the hash value is stored;

[0018] The data corresponding to the data upload request is stored on the chain. When the information of successful data storage is received, the hash value corresponding to the successfully stored data is uploaded to the chain.

[0019] From the above description, it can be seen that the beneficial effect of the present invention is that: before the data is uploaded to the chain, the data upload request is first cached; when the cached data upload request meets the preset conditions, the hash value of the data corresponding to the data upload request that meets the preset conditions is calculated and stored; when the data corresponding to the data upload request is successfully stored, the corresponding hash value is uploaded to the chain; through the above-mentioned hierarchical storage method, on the one hand, only the hash value corresponding to the data to be uploaded is uploaded to the chain, which greatly reduces the on-chain storage amount and reduces the resource overhead of data storage and data synchronization of each blockchain node; on the other hand, when the cached data upload request meets the preset conditions, the hash value of the corresponding data is calculated and stored, and the hash value corresponding to the successfully stored data is uploaded to the chain, thereby realizing relay processing of the data upload request, realizing unified organization and management of the data upload request, avoiding the need to execute an upload for each data upload request, and improving the data processing effect.

[0020] Furthermore, the caching of the data uplink request includes:

[0021] Grouping and caching the data uplink requests;

[0022] When a data upload request of any group buffer meets a preset condition, a hash value is calculated for the data corresponding to the data upload request that meets the preset condition and the hash value is stored;

[0023] Storing the data corresponding to the data upload request includes:

[0024] Based on the group to which the data uplink request belongs, the data corresponding to the data uplink request is stored in corresponding groups.

[0025] From the above description, it can be seen that data chain requests are cached based on groups. When the data chain request cached by any group meets the preset conditions, the hash value can be calculated and stored, and when the data corresponding to any group is stored successfully, its corresponding hash value is uploaded to the chain. The data chain request is cached, the hash value is calculated, and the data is stored in groups. This not only improves the efficiency of data processing, but also can quickly hit the data to be queried during subsequent data tracing, thereby improving the efficiency of data query.

[0026] Furthermore, the grouping and caching the data uplink request includes:

[0027] The data uplink request is grouped and cached based on a sending end of the data uplink request.

[0028] From the above description, it can be seen that based on the sender of the data uplink request, the data uplink request is grouped and cached, and the data uplink requests from the same sender are divided into the same group for caching, which can improve the effective management of data and is also conducive to rapid positioning during subsequent data backtracking.

[0029] Furthermore, the method further comprises the steps of:

[0030] The cached data uplink request is monitored, and when the cache time of the data uplink request reaches a preset time or when the cached data uplink request reaches a preset number, it is determined that the cached data uplink request meets the preset condition.

[0031] From the above description, it can be seen that the hash value calculation can be set when the cache time reaches a preset time, or the hash value calculation can be set when the cached data chain request reaches a preset number. It can be flexibly set according to actual needs. If the time requirement is relatively strict, it can be determined based on the preset time whether to calculate the hash value. If the quantity requirement is relatively strict, it can be determined based on the preset number whether to calculate the hash value. It can balance time and quantity well to ensure timely or sufficient calculation of hash values.

[0032] Furthermore, storing the data corresponding to the data uplink request includes:

[0033] The data corresponding to the data chain request is stored in a distributed manner.

[0034] From the above description, we can see that the data corresponding to the hash value is distributedly stored off-chain. By storing data through a distributed system, we can use big data technology to perform complex retrieval queries, statistical reports, intelligent analysis, modeling training, and other means on file data to maximize the value of data, and also to a certain extent ensure the security and credibility of off-chain storage.

[0035] Furthermore, the step of uploading the hash value corresponding to the successfully stored data to the chain includes:

[0036] Establishing a connection with a blockchain network and sending a data upload request to the blockchain network, wherein the data upload request includes a hash value corresponding to the successfully stored data;

[0037] Receive the transaction identifier returned by the blockchain network after successfully receiving the data on-chain request, and store the corresponding relationship between the transaction identifier and the hash value corresponding to the successfully stored data.

[0038] As can be seen from the above description, when the hash value is uploaded to the chain, a correspondence is established between the transaction identifier returned by the blockchain network and the corresponding hash value, and the correspondence is stored, which helps to quickly and conveniently locate the data to be queried based on the correspondence when performing subsequent data queries.

[0039] Furthermore, the method further comprises the steps of:

[0040] Receive a data query request, wherein the data query request includes a target hash value corresponding to the data to be queried;

[0041] Determining a target transaction identifier corresponding to the target hash value according to the target hash value and the stored correspondence between the transaction identifier and the hash value;

[0042] Determine the corresponding block according to the target transaction identifier, download the data stored in the block, and parse it to obtain the hash value to be verified;

[0043] Compare the target hash value and the hash value to be verified to see if they are consistent. If they are consistent, obtain the data corresponding to the target hash value.

[0044] From the above description, it can be seen that when a data query request is received, the target transaction identifier corresponding to the target hash value can be determined conveniently and quickly based on the correspondence between the target hash value in the query request and the stored transaction identifier and the hash value, so that the corresponding block can be determined based on the target transaction identifier to download the hash value to be verified stored in the block. After the download is completed, the target hash value is compared with the hash value to be verified. Only when the comparison is consistent can the data to be queried be obtained. Due to the consistency of the hash value and the corresponding data, by comparing the hash value, it can be determined whether the data corresponding to the hash value has been tampered with, thereby ensuring the non-tamperability of the off-chain file content and improving the security of the obtained data.

[0045] Furthermore, the method further comprises the steps of:

[0046] receiving a historical transaction query request, wherein the historical transaction query request includes a data sending end key value;

[0047] Query all historical transaction record identifiers corresponding to the data sending end key value according to the data sending end key value;

[0048] Determine the hash values ​​corresponding to all historical transaction records based on the corresponding relationship between all historical transaction record identifiers and stored transaction identifiers and hash values, and obtain a hash value set;

[0049] The data corresponding to each target hash value is obtained according to the hash value set.

[0050] From the above description, it can be seen that based on the key value of the data sending end, all historical transaction identifiers associated with the key value can be queried, so that according to the correspondence between the stored transaction identifier and the hash value, the hash value set corresponding to all historical transaction identifiers can be obtained, and the corresponding data can be obtained based on the hash value set, so that the historical transaction data of the data sending end can be queried conveniently and quickly.

[0051] Furthermore, the method further comprises the steps of:

[0052] The group identifier of the data corresponding to the stored hash value;

[0053] The data corresponding to the target hash value is obtained as follows:

[0054] Get the group identifier of the data corresponding to the target hash value;

[0055] Data corresponding to the target hash value is obtained from the corresponding data group according to the group identifier.

[0056] From the above description, it can be seen that when the hash value is stored, the group identifier of the corresponding data is also stored. When querying data, the group identifier can be directly obtained to accurately and quickly locate the group, thereby quickly obtaining the data to be queried.

[0057] Please refer to Figure 2 A blockchain-based hierarchical storage terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the above-mentioned blockchain-based hierarchical storage method is implemented.

[0058] The above-mentioned blockchain-based hierarchical storage method and terminal of the present invention can be applied to various application scenarios that require the use of blockchain for data storage. The following is an explanation through specific implementation methods:

[0059] Example 1

[0060] Please refer to Figure 1 , a hierarchical storage method based on blockchain, comprising the steps of:

[0061] S1. Receive a data upload request and cache the data upload request;

[0062] Specifically, a middleware service is established to connect with the data upload requests of each data sender. Each data sender submits the data upload request to the middleware service, and the middleware service uniformly processes the data upload requests sent by each data sender.

[0063] The middleware groups and caches the data upload requests;

[0064] The data uplink request may be grouped and cached based on the sending end of the data uplink request. Specifically, different uplink queues may be established according to different data sending ends. For example, a corresponding uplink queue A may be established for data sending end a, and a corresponding uplink queue B may be established for data sending end b. Then, when a data uplink request is received from a, it may be cached in queue A. When a data uplink request is received from b, it may be cached in queue B.

[0065] S2. When the cached data on-chain request meets the preset conditions, calculate a hash value for the data corresponding to the data on-chain request that meets the preset conditions and store the hash value;

[0066] Specifically, when grouping and caching data uplink requests, when any of the grouped cached data uplink requests meets a preset condition, a hash value is calculated for the data corresponding to the data uplink request that meets the preset condition and the hash value is stored;

[0067] Among them, the data link request can be stored in the corresponding redis according to the group it is in. When the corresponding group meets the preset conditions, all the data link requests in the group that meets the conditions are taken out from redis, and then the data corresponding to all the data link requests are read, and the read data is written into the file. Then, the data of the entire file is hashed to obtain a hash value, and the hash value is used to name the file, realizing the storage of the hash value and facilitating subsequent backtracking queries. That is, the hash value calculated based on all the data in a file is used as the file name to name the file;

[0068] The monitoring rules of each queue can be dynamically set according to actual needs to determine whether the data on-chain requests cached in each queue meet the preset conditions. For example, the preset conditions can be set based on the existence time of the data on-chain requests cached in the queue, or based on the number of data on-chain requests cached in the queue;

[0069] Specifically, the data on-link requests cached by each queue are monitored in real time. When the cache time of the data on-link requests in a queue reaches a preset time, such as 5 hours, or when the cached data on-link requests reach a preset number, such as 50,000 requests, it is determined that the data on-link requests cached by the queue meet the preset conditions;

[0070] Steps S1 and S2 are equivalent to grouping and dividing massive amounts of data according to corresponding rules and related users, i.e., data senders, and storing them in files. Subsequently, uploading the text string of this hash value significantly reduces the amount of data on the chain compared to the uploaded file content data. At the same time, the uniqueness of the hash calculation ensures that the file content off-chain cannot be tampered with.

[0071] S3. Store the data corresponding to the data upload request. When receiving the information that the data is successfully stored, upload the hash value corresponding to the successfully stored data to the chain.

[0072] Among them, it is possible to build or use an existing highly reliable distributed file storage system, such as HDFS or IPFS Interstellar File System, to store the data corresponding to the data on-chain request to ensure that the file is not lost and difficult to tamper with. Specifically, the text file with the hash value calculated can be stored in the distributed storage system through the middleware calling API, etc.

[0073] In an implementation method of grouping and caching the data uplink request, data corresponding to the data uplink request is stored in corresponding groups based on the group to which the data uplink request belongs;

[0074] For example, when the data upload request of the corresponding group meets the conditions, the data corresponding to all the data upload requests of the group are stored in a text file, and the text file is named with the hash value calculated based on the stored data, and the text file corresponding to the data with the calculated hash value is stored in the distributed file storage system;

[0075] When the middleware receives the response message from the distributed file storage system that the file has been successfully stored, it starts to try to connect to the blockchain network. At this time, it only needs to send the calculated hash value as a data request to the blockchain.

[0076] Specifically, establishing a connection with a blockchain network, and after the blockchain connection is successful, sending a data upload request to the blockchain network, wherein the data upload request includes a hash value corresponding to the successfully stored data;

[0077] When the hash value is successfully uploaded to the chain, the middleware will receive a response message from the blockchain network indicating that the data upload request is successful.

[0078] Example 2

[0079] This embodiment further defines how to trace data. When the middleware receives a response message from the blockchain network indicating a successful data upload request, the response message includes a corresponding transaction identifier, i.e., a transaction ID. The middleware stores the corresponding relationship between the transaction ID and the hash value corresponding to the successfully stored data, i.e., the file name.

[0080] When conducting data tracing, there are several ways:

[0081] One way is:

[0082] Receive a data query request, wherein the data query request includes a target hash value corresponding to the data to be queried;

[0083] Determining a target transaction identifier corresponding to the target hash value according to the target hash value and the stored correspondence between the transaction identifier and the hash value;

[0084] Determine the corresponding block according to the target transaction identifier, download the data stored in the block, and parse it to obtain the hash value to be verified;

[0085] Compare the target hash value and the hash value to be verified to see if they are consistent. If they are consistent, obtain the data corresponding to the target hash value;

[0086] Specifically, the file name corresponding to the data to be searched, that is, the calculated hash value, can be determined through big data analysis and positioning, and then a data query request can be sent. The data query request contains the hash value. After receiving the data query request containing the hash value, the middleware determines the transaction identifier corresponding to the hash value based on the calculated hash value and the correspondence between the stored transaction identifier and the hash value, and then reversely checks the block number where the transaction identifier is located based on the transaction identifier. Then, the corresponding block is retrieved through the block number, and the corresponding block is downloaded. After parsing the downloaded data, the hash value stored on the blockchain is obtained, and the obtained hash value on the blockchain is compared with the hash value in the data query request. If they are consistent, it indicates that the data has not been tampered with, and the queried data can be sent to the data sender that sent the data query request. In addition, the corresponding submitting user end, that is, the sender signature, endorsement node signature, packaging node signature information, etc. of the data sender can also be queried in the blockchain.

[0087] Another way is:

[0088] receiving a historical transaction query request, wherein the historical transaction query request includes a data sending end key value;

[0089] Query all historical transaction record identifiers corresponding to the data sending end key value according to the data sending end key value;

[0090] Determine the hash values ​​corresponding to all historical transaction records based on the corresponding relationship between all historical transaction record identifiers and stored transaction identifiers and hash values, and obtain a hash value set;

[0091] Obtain data corresponding to each target hash value according to the hash value set;

[0092] Specifically, you can directly query all the historical transaction records submitted by the user on the blockchain through the user's key. Then, based on the correspondence between the hash value and the transaction ID, you can find the historical hash value (file name) corresponding to the historical transaction record and obtain a file list. The file data in this file list is all the data uploaded by this user.

[0093] You can also add time filter conditions when querying historical transaction records to obtain data submitted by a certain user in a certain time period;

[0094] When storing the hash value of the corresponding group, the group identifier of the data corresponding to the hash value is stored;

[0095] When obtaining the data corresponding to the target hash value, obtain the group identifier of the data corresponding to the target hash value;

[0096] Acquire data corresponding to the target hash value from the corresponding data group according to the group identifier;

[0097] In an implementation method where one file is a group, when a user wants to query certain data in a file, the file containing the data that the user wants to query can be obtained from the distributed data storage system based on the file name, and then the obtained file can be filtered to obtain the data that the user wants to query and return it to the user.

[0098] Example 3

[0099] Please refer to Figure 2 、 one A blockchain-based hierarchical storage terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements the steps of the blockchain-based hierarchical storage method described in Example 1 or Example 2.

[0100] In summary, the present invention provides a blockchain-based hierarchical storage method and terminal, which groups and caches received data on-chain requests. When any group of cached data on-chain requests meets preset conditions, the data corresponding to the data on-chain requests that meet the conditions is written to a file and a hash value is calculated. The calculated hash value is used as the file name of the file. The text file with the data is stored in a distributed file storage system. For the successfully stored data, the corresponding hash value is uploaded to the chain. This optimizes the data storage method in the blockchain, so that the data storage requirements fall within the distributed file storage system that can store massive resources, while still ensuring the data's tamper-proof and decentralized characteristics. By batch processing data on-chain requests, file writing is performed, and then the hash value of the file is calculated. The calculated hash value is used for on-chain storage, which significantly reduces the on-chain storage capacity, reduces the data storage and data synchronization resource overhead of each blockchain node, and improves the verification efficiency between nodes. In addition, the corresponding data processing based on the group also improves the efficiency of subsequent data traceability queries. Storing data based on the distributed file storage system further guarantees the security and credibility of off-chain stored files to a certain extent.

[0101] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A hierarchical storage method based on blockchain, characterized in that: Including steps: Receive a data upload request and cache the data upload request; The caching of the data uplink request includes: performing group buffering on the data uplink request based on a sending end of the data uplink request; When a cached data on-link request meets a preset condition, a hash value is calculated for the data corresponding to the data on-link request that meets the preset condition and the hash value is stored; wherein, when a data on-link request of any group cache meets the preset condition, all the data corresponding to the data on-link request that meets the preset condition are written into the same file, a hash value is calculated and stored, and the file is named with the hash value; The data corresponding to the data upload request is stored on the chain. When the information of successful data storage is received, the hash value corresponding to the successfully stored data is uploaded to the chain; Storing the data corresponding to the data upload request includes: Based on the group to which the data uplink request belongs, storing the data corresponding to the data uplink request in corresponding groups; The step of uploading the hash value corresponding to the successfully stored data to the chain includes: Establishing a connection with a blockchain network and sending a data upload request to the blockchain network, wherein the data upload request includes a hash value corresponding to the successfully stored data; Receiving a transaction identifier returned by the blockchain network after successfully receiving the data upload request, and storing a correspondence between the transaction identifier and a hash value corresponding to the successfully stored data; receiving a historical transaction query request, wherein the historical transaction query request includes a data sending end key value; Query all historical transaction identifiers corresponding to the data sending end key value according to the data sending end key value; Determine the hash values ​​corresponding to all historical transaction identifiers based on the correspondence between all historical transaction identifiers and the stored transaction identifiers and hash values, and obtain a hash value set; The data corresponding to each target hash value is obtained according to the hash value set.

2. A hierarchical storage method based on blockchain according to claim 1, characterized in that: Also includes the steps: The cached data uplink request is monitored, and when the cache time of the data uplink request reaches a preset time or when the cached data uplink request reaches a preset number, it is determined that the cached data uplink request meets the preset condition.

3. A hierarchical storage method based on blockchain according to claim 1, characterized in that: Storing the data corresponding to the data upload request includes: The data corresponding to the data chain request is stored in a distributed manner.

4. A hierarchical storage method based on blockchain according to claim 1, characterized in that: Also includes the steps: Receive a data query request, wherein the data query request includes a target hash value corresponding to the data to be queried; Determining a target transaction identifier corresponding to the target hash value according to the target hash value and the stored correspondence between the transaction identifier and the hash value; Determine the corresponding block according to the target transaction identifier, download the data stored in the block, and parse it to obtain the hash value to be verified; Compare the target hash value and the hash value to be verified to see if they are consistent. If they are consistent, obtain the data corresponding to the target hash value.

5. A hierarchical storage method based on blockchain according to claim 4, characterized in that: Also includes the steps: The group identifier of the data corresponding to the stored hash value; The data corresponding to the target hash value is obtained as follows: Get the group identifier of the data corresponding to the target hash value; Data corresponding to the target hash value is obtained from the corresponding data group according to the group identifier.

6. A hierarchical storage terminal based on blockchain, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the blockchain-based hierarchical storage method as described in any one of claims 1 to 5.

Images