Data distributed storage system based on block chain

Abstract

The invention relates to a data distributed storage system based on a block chain, and belongs to the technical field of storage systems. The distributed storage prototype system based on the block chain is designed by combining a block chain technology and realizing a data integrity proving protocol and a data deterministic deleting protocol based on the block chain, data is stored in the distributed storage network equipment, metadata information is stored on the block chain, and the data integrity proving protocol and the data deterministic deleting protocol based on the block chain are realized. According to the design, the integrity and deterministic deletion function of data is achieved, and out-of-chain distributed storage of mass data is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of storage systems, and specifically relates to a data distributed storage system based on blockchain. Background Art

[0002] According to IDC, the global data production is expected to reach 163ZB (Zettabytes) by 2025, which is 10 times the global data production in 2016. The data generation speed shows a clear upward trend. Such a large amount of data that is still growing exponentially has brought both opportunities and challenges to the data storage industry. The storage capacity, performance and scalability of storage systems have been greatly tested.

[0003] Common centralized storage systems often have the following defects when processing massive data: (1) Small storage capacity. The storage capacity of a single storage server is limited and cannot complete the storage of massive files; (2) Low reliability. Data is stored on a single machine, which is prone to "single point failure". If a machine fails, the entire data center may not be able to operate normally; (3) Poor scalability. In order to meet the storage capacity requirements, the capacity can only be expanded by adding storage disks, and horizontal expansion between storage nodes cannot be achieved; (4) Expensive cost. High-performance single-machine storage devices that meet storage requirements are expensive, which will bring huge cost burdens to users. Therefore, the design of data storage systems needs to consider the following issues: (1) Rapid processing capabilities of massive data; (2) Reliable storage of data; (3) Scalability of the storage system; (4) Cost-effectiveness. After research, it was found that the idea of ​​"divide and conquer" can be adopted to use cheap servers to form a decentralized distributed storage system to solve the above problems.

[0004] At present, the data storage industry presents a variety of development trends. On the one hand, with the continuous development of cloud computing, big data, artificial intelligence and other technologies, the demand for storage capacity continues to rise. On the other hand, enterprises have higher and higher requirements for the security and reliability of data storage, which has prompted continuous innovation in storage technology, such as the use of multiple data backup and encryption technology. At the same time, new storage modes such as distributed storage and software-defined storage are gradually emerging and becoming a new direction for the development of the industry.

[0005] With the widespread application of distributed storage, users can no longer directly manage data after saving it to a remote server. They lose physical control over the data and access control management, and data security faces severe challenges. From the perspective of storage, in order to ensure the security of external data, users must regularly test the stored data to ensure that the data is correctly and completely saved; from the perspective of destruction, when the data ends its life cycle, the server should correctly delete the stored data to ensure that the data cannot be recovered, thereby ensuring data security. In summary, this is the integrity problem and deterministic deletion problem of data storage. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] The present invention aims to solve the data integrity problem and the data deterministic deletion technical problem in distributed storage, and realize off-chain distributed storage of massive data.

[0008] (II) Technical solution

[0009] In order to solve the above technical problems, the present invention provides a data distributed storage system based on blockchain, which includes an application layer module, a control layer module, a routing layer module and a storage layer module;

[0010] The storage layer module is a peer-to-peer network composed of many anonymous storage nodes. Each storage node is a user's PC device, a cluster that can provide storage services, or a cloud service provider. The storage nodes are interconnected through the network. The user's data files are divided into data blocks of the same size and stored in the storage layer module. The storage layer module is responsible for the distributed storage of data blocks, responding to user requests and returning corresponding data blocks.

[0011] The routing layer module is responsible for forwarding user requests to the storage layer module and distributing user data files;

[0012] The control layer module is implemented by blockchain, and the metadata information of the data file is stored on the blockchain;

[0013] The application layer module provides users with an interface to access the distributed data storage system located in the storage layer module; users send requests through different clients of the application layer module, including data upload requests, data download requests, and data deletion requests;

[0014] The system can realize the function of proving the integrity of data and support the public verifiable function. The specific implementation method is: using the blockchain to save the hash value of all data blocks, constructing the MHT structure through the information saved on the blockchain, and then using the generated MHT structure to realize the integrity verification of any data block; in addition, users in the system are identified by public keys;

[0015] The system can achieve data irrecoverability and support the public verifiable function of deletion results. The specific implementation method is: the client sends a data deletion request, and the storage node performs the corresponding deletion operation after receiving the deletion request, and uploads the corresponding deletion evidence generated to the blockchain, and constructs an MHT structure to store it in the blockchain, and sends the generated deletion evidence to the client; the client compares the root value of the MHT structure with the deletion result returned by the storage node. If the comparison result is consistent, it means that the storage node executes the deletion operation successfully. If it is inconsistent, it means that the storage node fails to execute the deletion operation. At this time, the application layer module re-initiates the data deletion request.

[0016] The present invention also provides a data file uploading method based on the system, and the data file uploading process is completed by the following steps: first, the system initializes the instance user; second, the uploaded data file is encrypted at the client; then, the data file is divided into data blocks of equal size, and an MHT structure is constructed according to the hash values ​​of all data blocks; next, all data blocks and the MHT structure are forwarded to the distributed data storage system through the routing layer module, and the system returns the storage address and index to the user client, thus completing the data file uploading process.

[0017] The present invention also provides a data file downloading method based on the system, comprising the following steps: when the data information is successfully saved to the blockchain, the control layer module returns a data storage status flag and a data access flag; when the user needs to download the relevant data file, the flag parameter is input, and the information of the corresponding data file is obtained by calling the smart contract, so as to obtain the storage location information of all data file blocks; then a download request is sent to the corresponding storage node in the storage layer module through the routing layer module, and the data integrity of the data block requested for download is verified, and the data block is obtained after the verification is passed; after the user has downloaded all the data blocks, all the data blocks are spliced, and after the data blocks are spliced, the original ciphertext is obtained, and the user uses the locally stored private key to decrypt the ciphertext to obtain the original data file.

[0018] The present invention also provides a data file deletion method based on the system, comprising the following steps: first, a user initiates a data deletion request; then, the storage node determines the validity of the data deletion request based on the user signature and timestamp; only when the data deletion request is verified, the corresponding data block is deleted, and the corresponding deletion evidence is generated and sent to the blockchain of the control layer module; after the deletion evidence is successfully uploaded to the blockchain, the user obtains information of successful deletion; when the user does not trust the data returned by the storage node, the deletion evidence is verified through the information of the storage node on the blockchain, and if the deletion request is illegal, the user re-initiates the deletion request to achieve the public verifiable function of the deletion result.

[0019] The present invention also provides an application of the system in centralized data security storage.

[0020] The present invention also provides an application of the method in centralized data security storage.

[0021] (III) Beneficial effects

[0022] The present invention combines blockchain technology, and designs a distributed storage prototype system based on blockchain by implementing a data integrity proof protocol and a data deterministic deletion protocol based on blockchain. The data is stored in a distributed storage network device, and the metadata information is saved on the blockchain. The design implements the data integrity and deterministic deletion functions, and realizes off-chain distributed storage of massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A data uploading flow chart based on the system implementation of the present invention;

[0024] Figure 2 A data downloading flow chart based on the system implementation of the present invention;

[0025] Figure 3 The present invention is a flowchart of data deletion based on the system implementation of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0027] In recent years, with the popularity of various cryptocurrencies, blockchain technology has received widespread attention from many industries. The blockchain network is composed of different peer nodes, and all nodes jointly maintain network stability and security through a consensus algorithm. It has many characteristics such as openness, decentralization, tamper-proof, transaction transparency, traceability, and auditability. Using these characteristics, it can effectively solve data security and user privacy protection issues. Therefore, using blockchain technology to solve the problem of data integrity and data deterministic deletion in distributed storage has become a good choice.

[0028] The present invention designs a model of a data distributed storage system based on blockchain, which consists of four layers: application layer, control layer, routing layer and storage layer.

[0029] 1. Storage layer: This layer is one of the important components of the system model. The storage layer is a peer-to-peer network composed of many anonymous storage nodes. Each storage node can be a user's PC device, a cluster that can provide storage services, or a professional cloud service provider. The storage nodes are interconnected through the network to form a distributed data storage system (network). In order to efficiently upload and download user data files, data files are divided into data blocks of the same size (generally 4KB in size) and stored in the storage layer. The storage layer is responsible for the distributed storage of data file blocks, responding to user requests and returning corresponding data blocks.

[0030] 2. Routing layer: This layer is responsible for forwarding user requests to the storage layer and distributing user data files.

[0031] 3. Control layer: This layer is the most important part of the system model and is implemented by blockchain. The metadata information of data files is stored on the blockchain.

[0032] 4. Application layer: The application layer provides users with an interface to access the distributed data storage system located in the storage layer. Users send requests through different clients, including data upload requests, download requests, and deletion requests.

[0033] In view of the data integrity problem in the distributed storage scenario, the present invention proposes a data integrity proof protocol based on blockchain to realize data integrity proof and support public verifiable functions. The specific implementation method is: using blockchain to save the hash values ​​of all data blocks, constructing the MHT (Merkle Hash Tree, MHT) structure through the information saved on the blockchain, and then using the generated MHT structure to realize the integrity verification of any data block. In addition, due to the anonymity of blockchain, users in the system are identified by public keys, and no real identity information of any user will be leaked, thereby protecting user privacy.

[0034] In view of the problem of secure data deletion in distributed storage systems, the present invention proposes a data deterministic deletion protocol based on blockchain to achieve data irrecoverability and support the public verifiable function of deletion results. The specific implementation method is: the client sends a deletion request, and the storage node performs the corresponding deletion operation after receiving the deletion request, and uploads the corresponding deletion evidence to the blockchain and constructs MHT to store it in the blockchain, and sends the generated evidence to the client at the same time. After the storage node executes the data deletion operation, deletion evidence will be generated. The client compares the root value of MHT with the deletion result returned by the storage node. If the comparison result is consistent, it means that the storage node successfully executes the deletion operation. If it is inconsistent, it means that the deletion operation fails. At this time, the application layer needs to re-initiate the deletion request to ensure that the data is irrecoverable.

[0035] The present invention designs a data distributed storage system model based on blockchain, which realizes the mass storage and reliable management of data. By adopting the "off-chain storage" method, the data is stored in the distributed data storage system, and the metadata information is saved on the blockchain, which realizes the data integrity proof protocol and deterministic deletion function, solving the problem of mass data storage while also solving the problem of data security.

[0036] The present invention designs a distributed data storage system model based on blockchain to realize data storage and management. The system supports the integrity and deterministic deletion of data storage, thereby solving the data security problem in the distributed storage scenario. In addition, the system can effectively protect the user's data privacy by using the anonymity of blockchain. In the prototype system, the data upload, download, and deletion processes are designed. The data file upload process is as follows: Figure 1 shown.

[0037] The data file upload process is completed through multiple steps: First, the system initializes the instance user. Second, the uploaded data file is encrypted on the client. Then, the data file is divided into data blocks of equal size, and the MHT structure is constructed based on the hash values ​​of all data blocks. Next, all data blocks and the MHT structure are forwarded to the distributed data storage system through the routing layer, and the system returns the storage address and index to the user client, thus completing the data file upload process.

[0038] Data download process is as follows Figure 2 shown.

[0039] When the data information is successfully saved to the blockchain, the control layer returns the data storage status flag and the data access flag; when the user needs to download the relevant data file, by entering the flag parameter and calling the smart contract to obtain the information of the corresponding data file, the storage location information of all data file blocks is obtained; then the routing layer sends a download request to the corresponding storage node in the storage layer, and verifies the data integrity of the data block requested for download. After the verification is passed, the data block can be obtained; after the user has downloaded all the data blocks, all the data blocks can be spliced ​​together. After the data blocks are spliced ​​together, the original ciphertext is obtained. The user can decrypt the ciphertext with the locally stored private key to obtain the original data file.

[0040] Data deletion process Figure 3 shown.

[0041] When the life cycle of a data file in the system ends, in order to prevent data privacy leakage, the user can initiate a request to delete the data. The data deletion process begins with the user initiating a data deletion request; then, the storage node will determine the validity of the data deletion request based on the user's signature and timestamp; only when the data deletion request is verified will the corresponding data block be deleted, and the corresponding deletion evidence will be generated and sent to the blockchain of the control layer; when the deletion evidence is successfully uploaded to the blockchain, the user will receive information about the successful deletion; when the user does not trust the data returned by the storage node, the deletion evidence can be verified through the information of the storage node on the blockchain. If the deletion request is illegal, the user can re-initiate the deletion request to achieve the public verifiable function of the deletion result.

[0042] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed data storage system based on blockchain, characterized in that: The system includes an application layer module, a control layer module, a routing layer module and a storage layer module; The storage layer module is a peer-to-peer network composed of many anonymous storage nodes. Each storage node is a user's PC device, a cluster that can provide storage services, or a cloud service provider. The storage nodes are interconnected through the network. The user's data files are divided into data blocks of the same size and stored in the storage layer module. The storage layer module is responsible for the distributed storage of data blocks, responding to user requests and returning corresponding data blocks. The routing layer module is responsible for forwarding user requests to the storage layer module and distributing user data files; The control layer module is implemented by blockchain, and the metadata information of the data file is stored on the blockchain; The application layer module provides users with an interface for accessing the distributed data storage system located in the storage layer module; Users send requests through different clients of the application layer module, including data upload requests, data download requests, and data deletion requests; The system can realize the function of proving the integrity of data and support the public verifiable function. The specific implementation method is: using the blockchain to save the hash value of all data blocks, constructing the MHT structure through the information saved on the blockchain, and then using the generated MHT structure to realize the integrity verification of any data block; in addition, users in the system are identified by public keys; The system can achieve data irrecoverability and support the public verification of deletion results. The specific implementation method is: the client sends a data deletion request, and the storage node performs the corresponding deletion operation after receiving the deletion request, and uploads the corresponding deletion evidence to the blockchain, constructs an MHT structure and stores it in the blockchain, and sends the generated deletion evidence to the client at the same time; The client compares the root value of the MHT structure with the deletion result returned by the storage node. If the comparison results are consistent, it means that the storage node has successfully executed the deletion operation. If they are inconsistent, it means that the storage node has failed to execute the deletion operation. At this time, the application layer module re-initiates the data deletion request.

2. A data file uploading method implemented based on the system as claimed in claim 1.

3. The method according to claim 2, characterized in that The data file upload process is completed through the following steps: first, the system initializes the instance user; second, the uploaded data file is encrypted on the client; then, the data file is divided into data blocks of equal size, and the MHT structure is constructed according to the hash values ​​of all data blocks; next, all data blocks and MHT structures are forwarded to the distributed data storage system through the routing layer module, and the system returns the storage address and index to the user client, thus completing the data file upload process.

4. A data file downloading method implemented based on the system as claimed in claim 1.

5. The method according to claim 4, characterized in that The following steps are involved: When the data information is successfully saved to the blockchain, the control layer module returns the data storage status flag and the data access flag; when the user needs to download the relevant data file, by inputting the flag parameter and obtaining the information of the corresponding data file by calling the smart contract, the storage location information of all data file blocks is obtained; then the routing layer module sends a download request to the corresponding storage node in the storage layer module, and verifies the data integrity of the data block requested for download, and obtains the data block after the verification; after the user downloads all the data blocks, all the data blocks are spliced ​​together. After the data blocks are spliced ​​together, the original ciphertext is obtained, and the user uses the locally stored private key to decrypt the ciphertext to obtain the original data file.

6. A method for deleting data files based on the system as claimed in claim 1.

7. The method according to claim 6, characterized in that The following steps are included: first, the user initiates a data deletion request; then, the storage node determines the validity of the data deletion request based on the user signature and timestamp; Only when the data deletion request is verified, the corresponding data block will be deleted, and the corresponding deletion evidence will be generated and sent to the blockchain of the control layer module; when the deletion evidence is successfully uploaded to the blockchain, the user will get the information of successful deletion; When the user does not trust the data returned by the storage node, the deletion evidence is verified through the information of the storage node on the blockchain. If the deletion request is illegal, the user re-initiates the deletion request to achieve the public verifiable function of the deletion result.

8. The method according to claim 7, characterized in that When the life cycle of a data file in the system ends, the user initiates a data deletion request.

9. An application of the system as claimed in claim 1 in centralized data security storage.

10. Application of the method according to any one of claims 2 to 8 in centralized data security storage.

Images