Distributed data storage management method, device and system based on cloud chain integration

Abstract

The present invention discloses a distributed data storage management method based on cloud-chain integration, belonging to the field of data management technology, comprising: a data classification storage step and a data access step, wherein the data classification storage step comprises: (S1) when uploading original data, determining the data domain to which the original data to be uploaded belongs according to a data classification strategy pre-established by the data owner; the data classification strategy is used to determine the data domain to which the data belongs in the cloud based on data attributes; similar original data information stored by a data owner in a cloud computing platform belongs to the same data domain; (S2) storing the original data in the cloud computing platform corresponding to the data domain and generating corresponding key metadata; the key metadata is used to record the location information of the original data in the cloud; (S3) storing the generated key metadata on the blockchain. The present invention can overcome the dependence on a single cloud service provider and improve the security of multi-source heterogeneous data storage.

Description

Technical Field

[0001] The present invention belongs to the field of data management technology, and more specifically, relates to a distributed data storage management method, device and system based on cloud chain integration. Background Art

[0002] With the rapid development of technologies such as the Internet, the Internet of Things, and the Internet of Vehicles, multi-source heterogeneous data represented by network data, media data, and log data has become an important part of people's lives and economic development. Considering the characteristics of multi-source heterogeneous data such as source diversity, type heterogeneity, and complexity of the application environment, it needs to be stored and managed securely and reliably.

[0003] In cloud data environments, data storage is typically outsourced to a cloud data service provider, which then stores and manages the data. However, in this solution, the data owner and the server are not in the same trust domain. The server is directly controlled by the cloud data service provider, leaving the user without control. Furthermore, outsourced data is typically stored and centrally managed by a single cloud service provider, while the data management methods of a single cloud service provider are often tailored to specific data types. This makes effective data storage and management difficult in scenarios with multi-source, heterogeneous data. Therefore, in cloud data environments, traditional data management methods are no longer sufficient to address the storage and management challenges of multi-source, heterogeneous data.

[0004] In recent years, decentralized data management mechanisms based on blockchain technology have emerged to address the inadequacies of centralized computing and storage management models in cloud data. Blockchain technology offers a new research approach to addressing the challenges of security and mutual trust. Blockchain is a new decentralized distributed computing paradigm that uses an encrypted chain-like block structure to verify and store data, generates and updates data based on a consensus algorithm among distributed nodes, and uses smart contracts (automated scripts) to program and manipulate data. It employs cryptographic methods rather than a central authority to establish trust between nodes, thus forming a decentralized distributed architecture. Furthermore, blockchain leverages the immense computing power generated by consensus algorithms such as proof-of-work across the distributed system's nodes to resist external attacks. Only a compromise of more than 51% of the nodes can tamper with information. In a sufficiently large blockchain system, tampering is extremely costly, thus ensuring the immutability and forgery of blockchain data. However, the application requirements of blockchain technology are relatively limited. Existing blockchain technology struggles to meet the unique requirements of cloud environments, such as the diverse data types, complex user security requirements, and diverse sharing needs.

[0005] In general, current research on how to securely and reliably store and manage multi-source heterogeneous data is not in-depth enough. Existing solutions can solve the distributed storage and management of big data to a certain extent, but cannot solve the management of multi-source heterogeneous data under multi-cloud collaboration, and data storage security is limited. Summary of the Invention

[0006] In response to the defects of the existing technology and the need for improvement, the present invention provides a distributed data storage management method, device and system based on cloud-chain integration, which aims to overcome the dependence on a single cloud service provider and improve the security of multi-source heterogeneous data storage.

[0007] To achieve the above object, according to one aspect of the present invention, a distributed data storage and management method based on cloud chain integration is provided, comprising: a data classification and storage step;

[0008] The steps for data classification and storage include:

[0009] (S1) When uploading the original data, the data domain to which the original data to be uploaded belongs is determined according to the data classification strategy pre-defined by the data owner;

[0010] Data classification strategy is used to determine the data domain to which data belongs in the cloud based on data attributes; similar raw data information stored by a data owner on a cloud computing platform belongs to the same data domain;

[0011] (S2) storing the original data in the cloud computing platform corresponding to the data domain and generating corresponding key metadata;

[0012] Key metadata is used to record the location information of the original data in the cloud;

[0013] (S3) The generated key metadata is stored on the blockchain.

[0014] Furthermore, the distributed data storage management method based on cloud chain integration provided by the present invention further includes: a data access step;

[0015] The data access steps include:

[0016] (T1) When a data user requests to access data, extract the data keyword from the request;

[0017] The data keyword is the key of the key metadata and corresponds one-to-one with the key metadata;

[0018] (T2) Searching for key metadata from the blockchain based on the data keyword. If the search is successful, proceed to step (T3); otherwise, returning access failure information and proceeding to step (T4);

[0019] (T3) Obtaining the location information of the original data in the cloud from the found key metadata, and searching for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user and the process proceeds to step (T4); otherwise, an access failure message is returned and the process proceeds to step (T4);

[0020] (T4) The visit ends.

[0021] Furthermore, the data access step also includes:

[0022] After the data user receives the returned original data, he or she uses key metadata to verify whether the obtained original data has not been tampered with.

[0023] Furthermore, the data attributes include data types.

[0024] Furthermore, data attributes also include security partitioning of data.

[0025] According to another aspect of the present invention, there is provided a distributed data storage and management apparatus based on cloud-chain integration, comprising: a data classification storage device;

[0026] The data classification storage device includes: a data classification module and a data storage module;

[0027] The data classification module is used to store data classification strategies pre-defined by data owners. Data classification strategies are used to determine the data domain to which data belongs in the cloud based on data attributes. Similar raw data information stored by a data owner on a cloud computing platform belongs to the same data domain.

[0028] The data classification module is also used to determine the data domain to which the original data to be uploaded belongs according to the data classification strategy pre-defined by the data owner when the original data is uploaded, and then trigger the data storage module;

[0029] The data storage module is used to store the original data in the cloud computing platform corresponding to the data domain and generate corresponding key metadata; the key metadata is used to record the location information of the original data in the cloud;

[0030] The data storage module is also used to store the generated key metadata on the blockchain.

[0031] Furthermore, the distributed data storage and management device based on cloud-chain integration provided by the present invention further includes: a data access device;

[0032] The data access device includes: a data access interface, a first data access module, and a second data access control module;

[0033] The data access interface is used to receive a data access request sent by a data user, extract a data keyword from the request, and then trigger the first data access module; the data keyword is the key of the key metadata and has a one-to-one correspondence with the key metadata;

[0034] The data access interface is also used to return access results to data users;

[0035] The first data access module is used to search for key metadata from the blockchain based on the data keyword. If the search is successful, it triggers the second data access control module. If the search is unsuccessful, it returns access failure information through the data access interface;

[0036] The second data access control module is used to obtain the location information of the original data in the cloud from the key metadata, and search for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user through the data access interface; if the search is unsuccessful, the access failure information is returned through the data access interface.

[0037] According to another aspect of the present invention, a distributed data storage management system is provided, including: a cloud computing platform, a blockchain, and a distributed data storage manager based on a cloud-chain fusion environment provided by the present invention.

[0038] In some optional embodiments, the blockchain is Hyperledger Fabric.

[0039] In general, through the above technical solutions conceived by the present invention, when the original data is uploaded, the data is classified according to the data classification strategy pre-established by the data owner, and the original data is stored in the corresponding data domain based on the classification results. For the same data owner, different data domains correspond to different cloud computing platforms. Therefore, the data uploaded to the cloud will be distributed and stored on different cloud computing platforms based on its attribute classification results, thereby integrating the data storage capabilities of different cloud service providers, avoiding dependence on a single cloud service provider, and ensuring the effective storage of multi-source heterogeneous data. In addition, the location information of the original data in the cloud will be stored as key metadata on the blockchain, and the multi-source heterogeneous data stored in the cloud will be uniformly managed, which can ensure that the stored key information is not tampered with, thereby ensuring the security of multi-source heterogeneous data storage. In addition, the formulation of the data classification strategy can be based on the actual needs of the data owner, avoiding the single division and storage of different data types and data security requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a distributed data storage management method and system based on cloud-chain integration provided by an embodiment of the present invention;

[0041] Figure 2 A flow chart of a distributed data storage management method based on cloud-chain integration provided in an embodiment of the present invention;

[0042] Figure 3 A flow chart of the steps for data classification and storage provided by an embodiment of the present invention;

[0043] Figure 4 A flowchart of data access steps provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0045] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0046] Multi-source heterogeneous data has multiple data owners, and the types and characteristics of the data are inconsistent. The storage requirements of different data owners are also different. The existing centralized storage systems are mostly used to store and manage specific data. It is difficult to uniformly store and manage multi-source heterogeneous data, and single point failures are prone to occur. The security and availability of the data are difficult to guarantee. In response to the problem of distributed storage and management of multi-source heterogeneous data, the present invention proposes a distributed data storage management method, device and system based on cloud chain integration. The overall idea is to pre-formulate a data classification strategy according to the application requirements of the data owner, classify the data according to the data classification strategy when uploading the data, and store the data in the corresponding cloud computing platform based on the classification results. In this way, the multi-source heterogeneous data is dispersedly stored on multiple cloud computing platforms, and the blockchain is used to uniformly manage the key information of the original data stored in the cloud, overcome the dependence on a single cloud service provider, and improve the security of multi-source heterogeneous data storage.

[0047] To realize the above technical concept, the present invention coordinates the differences between cloud computing and blockchain technologies in data organization and storage, effectively integrates the storage, access and verification processes of data and metadata in blockchain and cloud, and realizes the secure storage and management function of multi-source heterogeneous data.

[0048] Before explaining the technical solution of the present invention in detail, the following is a brief introduction to the relevant technical terms:

[0049] Blockchain: It is actually a distributed database maintained by a group of decentralized user nodes. When a block is linked to the previous block, the transaction is further confirmed.

[0050] Blockchain technology: A technical solution that stores, verifies, transmits, and communicates network data through its own distributed nodes, independent of third parties. Blockchain technology can be seen as a large-scale, distributed, open, and decentralized online ledger. Anyone can add their own information at any time using the same technical standards, extending the blockchain and continuously meeting the data entry needs brought about by various needs.

[0051] Hyperledger Fabric: An open-source blockchain framework designed to help companies build private or consortium permissioned blockchain networks, in which multiple organizations can share the authority to control and operate nodes within the network. The present invention can be implemented based on any blockchain. Without loss of generality, the following embodiments use Hyperledger Fabric as an example.

[0052] Data domain: For a single data owner, the domain to which the same type of raw data information stored in a cloud service provider's cloud computing platform belongs constitutes a data domain; different data domains of the same data owner correspond to different types of data and different cloud computing platforms.

[0053] The following are examples.

[0054] Example 1:

[0055] A distributed data storage management method based on cloud chain integration, see Figure 1 and Figure 2 , including: data classification and storage steps;

[0056] like Figure 3 As shown, the data classification and storage steps include:

[0057] (S1) When uploading the original data, the data domain to which the original data to be uploaded belongs is determined according to the data classification strategy pre-defined by the data owner;

[0058] The data classification strategy is used to determine the data domain to which data belongs in the cloud based on data attributes. Similar raw data stored by a data owner on a cloud computing platform belongs to the same data domain. In this embodiment, the so-called similar data refers to data with the same classification results. In other words, raw data belonging to the same data owner and with the same classification results on the same cloud computing platform belongs to the same data domain.

[0059] like Figure 1As shown, the original data that needs to be uploaded to the cloud may be data generated by smart devices, smart homes, and Internet of Vehicles;

[0060] As an optional implementation, in this embodiment, the data attributes used by the data classification strategy for classification specifically include data type and security classification. The data type is used to identify the source and type of data, for example, whether it is smart device data or Internet of Vehicles data, structured data or unstructured data, etc. The security classification is used to identify the security level of the data. When uploading raw data to the cloud, data users can identify the data type and security classification through special data tags. It should be noted that the data attributes described here are only optional data attributes for implementing data classification and should not be understood as the sole limitation of the present invention.

[0061] Data users can flexibly set data classification strategies based on their specific storage needs. For example, data generated by the Internet of Vehicles and classified as top secret can be uploaded to the cloud computing platform provided by cloud service provider A; data generated by the Internet of Vehicles and classified as non-confidential can be uploaded to the cloud computing platform provided by cloud service provider B, and so on. The classification results of data reflect the storage needs of the data owner.

[0062] Optionally, in this embodiment, a configuration file is used to save the data classification strategy formulated by the data user, and the configuration file is specifically an XML (Extensible Markup Language) file. When data is uploaded, the corresponding data attributes, namely the data type and security classification, are extracted from the data tags of the original data. The configuration file is loaded and parsed to complete the classification of the original data and determine the corresponding data domain.

[0063] (S2) storing the original data in the cloud computing platform corresponding to the data domain and generating corresponding key metadata;

[0064] The key metadata is used to record the location information of the original data in the cloud. Optionally, in this embodiment, the information stored in the key metadata includes, in addition to the location information of the original data in the cloud, file hash, file size, data domain, etc.

[0065] After storing the original data in the corresponding cloud computing platform according to the data domain determined by the data classification strategy, multi-source heterogeneous data can be dispersed and stored on different cloud computing platforms in the cloud, thereby avoiding dependence on a single cloud service provider and integrating the processing capabilities of each cloud service provider for different data to achieve effective storage of multi-source heterogeneous data;

[0066] (S3) storing the generated key metadata on the blockchain;

[0067] The present invention stores the storage information of the original data in the cloud as key metadata in the blockchain. The blockchain can be used to uniformly manage multi-source heterogeneous data stored in the cloud, ensuring the security of data storage.

[0068] like Figure 1 and Figure 2 As shown, this embodiment further includes: a data access step;

[0069] like Figure 4 As shown, the data access steps include:

[0070] (T1) When a data user requests to access data, extract the data keyword from the request;

[0071] The data keyword is the key of the key metadata and corresponds one-to-one with the key metadata;

[0072] (T2) Searching for key metadata from the blockchain based on the data keyword. If the search is successful, proceed to step (T3); otherwise, returning access failure information and proceeding to step (T4);

[0073] (T3) Obtaining the location information of the original data in the cloud from the found key metadata, and searching for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user and the process proceeds to step (T4); otherwise, an access failure message is returned and the process proceeds to step (T4);

[0074] (T4) End of visit;

[0075] In this embodiment, the data access step further includes:

[0076] After the data user receives the returned original data, he or she uses key metadata to verify whether the obtained original data has not been tampered with.

[0077] Based on the above data access steps, data access is achieved, and the security of data access is effectively guaranteed during the access process; in addition, key metadata is used to verify the successfully accessed data to determine whether the data has not been tampered with, which can further ensure the security of the data.

[0078] In general, this embodiment adopts a multi-strategy data classification that integrates multi-cloud management and homogeneous classification ideas in a cloud storage environment; based on the metadata hierarchical management strategy, key metadata related to data management is stored on the blockchain, thereby realizing hierarchical and classified security management of multi-source heterogeneous data.

[0079] Example 2:

[0080] A distributed data storage and management device based on cloud-chain integration, comprising: a data classification storage device;

[0081] like Figure 2 As shown, the data classification storage device includes: a data classification module and a data storage module;

[0082] The data classification module is used to store data classification strategies pre-defined by data owners. Data classification strategies are used to determine the data domain to which data belongs in the cloud based on data attributes. Similar raw data information stored by a data owner on a cloud computing platform belongs to the same data domain.

[0083] The data classification module is also used to determine the data domain to which the original data to be uploaded belongs according to the data classification strategy pre-defined by the data owner when the original data is uploaded, and then trigger the data storage module;

[0084] The data storage module is used to store the original data in the cloud computing platform corresponding to the data domain and generate corresponding key metadata; the key metadata is used to record the location information of the original data in the cloud;

[0085] The data storage module is also used to store the generated key metadata on the blockchain;

[0086] The distributed data storage management device based on cloud chain integration provided in this embodiment also includes: a data access device;

[0087] The data access device includes: a data access interface, a first data access module, and a second data access control module;

[0088] The data access interface is used to receive a data access request sent by a data user, extract a data keyword from the request, and then trigger the first data access module; the data keyword is the key of the key metadata and has a one-to-one correspondence with the key metadata;

[0089] The data access interface is also used to return access results to data users;

[0090] The first data access module is used to search for key metadata from the blockchain based on the data keyword. If the search is successful, it triggers the second data access control module. If the search is unsuccessful, it returns access failure information through the data access interface;

[0091] The second data access control module is used to obtain the location information of the original data in the cloud from the key metadata, and search for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user through the data access interface; if the search is unsuccessful, the access failure information is returned through the data access interface;

[0092] In this embodiment, the specific implementation of each module can refer to the description in the above method embodiment and will not be repeated here.

[0093] Example 3:

[0094] A distributed data storage management system, such as Figure 1 As shown, it includes: a cloud computing platform, a blockchain, and a distributed data storage manager based on the cloud-chain fusion environment provided in the above embodiment 1;

[0095] Optionally, in this embodiment, the blockchain is Hyperledger Fabric.

[0096] In general, through the above technical solutions conceived by the present invention, when the original data is uploaded, the data is classified according to the data classification strategy pre-established by the data owner, and the original data is stored in the corresponding data domain based on the classification results. For the same data owner, different data domains correspond to different cloud computing platforms. Therefore, the data uploaded to the cloud will be distributed and stored on different cloud computing platforms based on its attribute classification results, thereby integrating the data storage capabilities of different cloud service providers, avoiding dependence on a single cloud service provider, and ensuring the effective storage of multi-source heterogeneous data. In addition, the location information of the original data in the cloud will be stored as key metadata on the blockchain, and the multi-source heterogeneous data stored in the cloud will be uniformly managed, which can ensure that the stored key information is not tampered with, thereby ensuring the security of multi-source heterogeneous data storage. In addition, the formulation of the data classification strategy can be based on the actual needs of the data owner, avoiding the single division and storage of different data types and data security requirements.

[0097] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed data storage management method based on cloud chain integration, characterized in that: include: Data classification and storage steps; The data classification and storage step includes: (S1) When uploading the original data, the data domain to which the original data to be uploaded belongs is determined according to the data classification strategy pre-defined by the data owner; The data classification strategy is formulated based on the actual needs of the data owner and is used to determine the data domain to which the data belongs in the cloud based on the data attributes. The same type of raw data information stored by a data owner on a cloud computing platform belongs to the same data domain. The data attributes include data type and data security classification. The data type is used to identify the source and type of data, and the security classification is used to identify the security level of the data. (S2) storing the original data in the cloud computing platform corresponding to the data domain and generating corresponding key metadata; The key metadata is used to record the location information of the original data in the cloud; (S3) The generated key metadata is stored on the blockchain.

2. The distributed data storage management method based on cloud chain integration according to claim 1 is characterized in that: Also includes: Data access steps; The data access step includes: (T1) When a data user requests to access data, extract the data keyword from the request; The data keyword is the key of the key metadata and has a one-to-one correspondence with the key metadata; (T2) searching for key metadata from the blockchain based on the data keyword. If the search is successful, proceed to step (T3); otherwise, return access failure information and proceed to step (T4); (T3) Obtaining the location information of the original data in the cloud from the found key metadata, and searching for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user and the process proceeds to step (T4); otherwise, an access failure message is returned and the process proceeds to step (T4); (T4) The visit ends.

3. The distributed data storage management method based on cloud chain integration according to claim 2 is characterized in that: The data access step further includes: After the data user receives the returned original data, he or she uses key metadata to verify whether the obtained original data has not been tampered with.

4. A distributed data storage and management device based on cloud chain integration, characterized in that: include: Data classification storage device; The data classification storage device includes: a data classification module and a data storage module; The data classification module is used to store data classification strategies pre-defined by data owners. These data classification strategies are formulated based on the actual needs of the data owner and are used to determine the data domain to which data belongs in the cloud based on data attributes. Similar raw data information stored by a data owner on a cloud computing platform belongs to the same data domain. Data attributes include data type and data security classification. The data type is used to identify the source and type of data, and the security classification is used to identify the security level of the data. The data classification module is further configured to determine the data domain to which the original data to be uploaded belongs according to the data classification strategy pre-defined by the data owner when the original data is uploaded, and then trigger the data storage module; The data storage module is used to store the original data in the cloud computing platform corresponding to the data domain and generate corresponding key metadata; the key metadata is used to record the location information of the original data in the cloud; The data storage module is further configured to store the generated key metadata on the blockchain.

5. The distributed data storage and management device based on cloud chain integration according to claim 4 is characterized in that: Also includes: Data access devices; The data access device includes: a data access interface, a first data access module, and a second data access control module; The data access interface is used to receive a data access request sent by a data user, extract a data keyword from the request, and then trigger the first data access module; the data keyword is a key of the key metadata and has a one-to-one correspondence with the key metadata; The data access interface is also used to return access results to data users; The first data access module is configured to search for key metadata from the blockchain based on the data keyword, trigger the second data access control module if the search is successful, and return access failure information through the data access interface if the search is unsuccessful; The second data access control module is used to obtain the location information of the original data in the cloud from the key metadata, and search for the corresponding original data in the cloud based on the location information. If the search is successful, the found original data is returned to the data user through the data access interface; if the search is unsuccessful, the access failure information is returned through the data access interface.

6. A distributed data storage management system, characterized in that: include: A cloud computing platform, a blockchain, and a distributed data storage manager based on a cloud-chain fusion environment as described in claim 4 or 5.

7. The distributed data storage management system according to claim 6, wherein: The blockchain is Hyperledger Fabric.

Images