Data management method and related device
By combining the decentralized characteristics of the blockchain network, the data of the multi-cloud platform is uniformly encoded and managed, and the single point of bottleneck problem is solved, and efficient, secure and reliable management and recovery of multi-cloud backup data is achieved.
Patent Information
- Application Number
- PCT/CN2023/136149
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-07-19
- Filing Date
- 2023-12-04
- Publication Date
- 2024-11-14
AI Technical Summary
In the prior art, there is a single point of bottleneck in reliance on a single cloud platform for data backup, which leads to threatening data security and storage reliability. The data management of multi-cloud platforms is difficult and there is a lack of effective unified management solutions.
By combining the decentralization and security and trustworthiness of the blockchain network, a data management method is provided, and the blockchain network is used to uniformly encode and manage the data of the multi-cloud platform, realizing distributed multi-cloud backup data management, eliminating single Point security bottlenecks and improve the robustness and credibility of data recovery through smart contracts and scheduling strategies.
It improves the security and reliability of multi-cloud backup data, realizes decentralized management of multi-cloud platform data, reduces the risk of single point of failure, and improves the efficiency and reliability of data recovery.
Smart Images

Figure CN2023136149_14112024_PF_FP_ABST
Abstract
Description
Data management method and related equipment
[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 10, 2023, with application number 202310524887.3 and invention name “A data management method and related equipment”, and claims priority to the Chinese patent application filed with the State Intellectual Property Office on July 19, 2023, with application number 202310889759.9 and invention name “A data management method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of cloud computing technology, and in particular to a data management method, a data management system, a computing device cluster, a computer-readable storage medium, and a computer program product. Background Art
[0003] With the widespread adoption and growth of cloud computing across various industries, a growing number of individuals, businesses, and organizations are using cloud-based data disaster recovery solutions, effectively reducing the difficulty and cost of building and maintaining data disaster recovery systems. Compared to traditional backup and recovery technologies, cloud-based disaster recovery does not require extensive infrastructure deployment, supports on-demand subscriptions and elastic capacity expansion, and is compatible with backups of databases, files, virtualization platforms, operating systems, and physical environments. It offers advantages such as low investment costs, high scalability, and strong compatibility.
[0004] Considering that relying on a single cloud platform will result in a single point of bottleneck, once the cloud platform fails or completely crashes, a large amount of data will be lost, threatening data security and storage reliability. More and more users choose to back up data on multiple cloud platforms (also referred to as multi-cloud platforms) to reduce the risk of failure of a single cloud platform.
[0005] How to uniformly manage data on multi-cloud platforms has become a key concern in the industry.
[0006] Summary of the Invention
[0007] This application provides a data management method that, by combining the decentralized, secure, and reliable characteristics of blockchain networks, achieves cloud-chain integration and improves the security of multi-cloud backup data. This application also provides a corresponding data management system, computing device cluster, computer-readable storage medium, and computer program product.
[0008] In a first aspect, the present application provides a data management method that can be executed by a data management system. The data management system includes a client, a multi-cloud platform, and a blockchain network, and is used to manage data on the multi-cloud platform.
[0009] Specifically, the client receives a user-configured backup plan for the data to be backed up. The backup plan includes the number n of cloud nodes used for backup in the multi-cloud platform and the number b of backup copies. The client then divides the data to be backed up into c data blocks based on the number n of cloud nodes used for backup and the number b of backup copies, and stores the c data blocks in a distributed manner across the n cloud nodes in the multi-cloud platform. For at least one of the c data blocks, the multi-cloud platform stores b backup copies of at least one data block. The client provides the blockchain network with metadata (also known as backup metadata) of the data stored on the multi-cloud platform, so that the blockchain network encodes the metadata into a backup identifier and stores the backup identifier. The backup identifier is used to address the data stored on the multi-cloud platform.
[0010] This method divides the data to be backed up into blocks and stores them dispersedly across multiple cloud platforms. The metadata is then provided to the blockchain network for unified encoding, resulting in a globally unique backup identifier. This on-chain backup identifier is then used to record and manage the data backed up across multiple cloud platforms, providing unified location and addressing for distributed multi-cloud backup data management. This method combines the decentralized, secure, and trustworthy nature of blockchain networks to achieve cloud-chain integration, improving the security of multi-cloud backup data and eliminating the single-point security bottlenecks inherent in traditional methods.
[0011] In some possible implementations, the backup identifier may include a short identifier and a long identifier, wherein the short identifier may include a data identifier, and the long identifier may include a storage address of the data on the multi-cloud platform. The blockchain network may receive the short identifier provided by the user, search for the long identifier corresponding to the short identifier based on the short identifier, parse the long identifier to obtain the storage address of the data on the multi-cloud platform, and return the storage address of the data on the multi-cloud platform.
[0012] This method uses a combination of short and long identifiers to determine the backup identifier. In this way, users can use the short identifier to query data, reducing the complexity of user operations. The blockchain network determines the storage address of the data through the long identifier corresponding to the short identifier, improving data query efficiency and accuracy.
[0013] In some possible implementations, the backup identifier includes a first short identifier, a first long identifier, and a second short identifier and a second long identifier, the first short identifier includes a data identifier, the first long identifier includes a version set, the second short identifier includes the data identifier and a target version in the version set, and the second long identifier includes the storage address of the target version data on the multi-cloud platform.
[0014] This method takes into account that the data backed up by multi-cloud platforms can have multiple versions. The data version is added to the backup identifier, so that the data with multiple time versions can be recorded and managed through the backup identifier on the chain, thereby supporting the unified positioning and addressing of the data backed up by multi-cloud platforms, so as to realize the management of distributed multi-cloud backup data.
[0015] In some possible implementations, the blockchain network can obtain the state parameters of the cloud nodes in the multi-cloud platform, obtain the weights of the cloud nodes in the multi-cloud platform based on the state parameters, and return the node identifiers of n cloud nodes whose weights meet the requirements to the client.
[0016] This method obtains the weight of the cloud node through the state parameters of the cloud node, thereby determining the n cloud nodes used for distributed storage of data, thereby realizing the scheduling of the multi-cloud platform and further realizing the efficient management of the multi-cloud platform data.
[0017] In some possible implementations, state parameters may include one or more of node bandwidth, node cost, node remaining storage capacity, and node reputation information. In this method, state parameters may include static parameters (node bandwidth and node cost) that remain essentially unchanged after subscribing to the cloud service, as well as dynamic parameters (node remaining storage capacity and node reputation information) whose values can be dynamically adjusted. In this way, as data is backed up across multiple cloud platforms, smart contracts can be updated.
[0018] In some possible implementations, the node reputation information may include a node reputation value, which may be dynamically adjusted as the backup operation triggers the multi-cloud state parameter update contract. For example, the node reputation value may be dynamically adjusted based on the audit results. The more successful audits, the higher the node reputation value (credibility) of the cloud node, and the more failed audits, the lower the node reputation value (credibility) of the cloud node.
[0019] In some possible implementations, the blockchain network can normalize the state parameters of cloud nodes before calculating their weights in a multi-cloud platform. By selecting an appropriate normalization function to process the parameter vectors, the comparability of different parameters can be enhanced. By then determining the influence coefficients of each parameter using appropriate methods (such as the entropy weight method), and processing the impact of each parameter on the weight based on its degree of difference, the reliability of multi-cloud scheduling calculations and the flexibility of scheduling strategies can be improved.
[0020] In some possible implementations, the blockchain network can check the consistency between the actual storage address of the data in the multi-cloud platform and the storage address recorded in the backup identifier stored in the blockchain network. If there is inconsistency, the blockchain network can restore the data on the multi-cloud platform according to the storage address recorded in the backup identifier.
[0021] This method takes into account the inconsistencies caused by accidental deletion of files or directories by the cloud service provider's operation and maintenance personnel after the data is backed up. By setting up a consistency verification mechanism, inconsistencies caused by accidental deletion operations can be discovered in a timely manner and restored in a timely manner.
[0022] In some possible implementations, the client may create a backup storage transaction and execute the backup storage transaction to perform transaction operations of distributing and storing c data blocks on n cloud nodes in the multi-cloud platform and providing metadata of the data stored on the multi-cloud platform to the blockchain network.
[0023] This method introduces the concept of transaction, which can be defined as backing up data in the data management system. By executing backup storage transactions, backup data storage can be realized, thereby ensuring the consistency of the cloud chain before and after backup data storage and improving the security of multi-cloud backup data.
[0024] In some possible implementations, the client may obtain incremental data, create a backup update transaction, and execute the backup update transaction to perform transaction operations of storing the incremental data in the multi-cloud platform and updating the backup identifier.
[0025] This method introduces the concept of transaction, which can be defined as an incremental update of data in the data management system. By executing backup update transactions, incremental backup updates can be achieved, thereby ensuring the consistency of the cloud chain before and after the incremental update and improving the security of multi-cloud backup data.
[0026] In some possible implementations, the client may create a backup deletion transaction in response to a deletion operation and execute the backup deletion transaction to perform a transaction operation of deleting the data corresponding to the specified data identifier and a transaction operation of deleting the backup identifier corresponding to the specified data identifier.
[0027] This method introduces the concept of transaction, which can be defined as deleting data in the data management system. By executing backup deletion transactions, backup data deletion is achieved, thereby ensuring the consistency of the cloud chain before and after backup data deletion and improving the security of multi-cloud backup data.
[0028] In some possible implementations, the client can achieve consistency and atomicity for backup, update, and deletion transactions through retry and rollback mechanisms. For example, when storing data to be backed up on a multi-cloud platform, if a network anomaly on the cloud platform causes the transaction to fail, the client can retry and rollback. For another example, when storing incremental data on a multi-cloud platform, if a network anomaly on the cloud platform causes the transaction to fail, the client can rollback and retry. For another example, when deleting data corresponding to a specified data identifier from a multi-cloud platform, if a network anomaly on the cloud platform causes the transaction to fail, the client can return to the deletion failure location and update the backup identifier in the blockchain network.
[0029] In some possible implementations, the number of data blocks to be backed up is equal to C(n,n-q+1). The client can generate a scheduling allocation table based on the number of data blocks to be backed up. The scheduling allocation table records the data blocks to be stored in n cloud nodes, and then, according to the scheduling allocation table, the c data blocks are dispersed and stored in the n cloud nodes in the multi-cloud platform.
[0030] This method leverages the concept of secret partitioning (also known as threshold secret partitioning) to schedule backup resources. This allows for complete data recovery even when a backup data block within the specified range experiences a failure, improving the robustness of data backup. Furthermore, cloud nodes participating in the backup are identically important. The impact of multiple node crashes is proportional to the number of crashed nodes. No nodes are considered special or exceptionally important, achieving a truly decentralized scheduling strategy.
[0031] In a second aspect, the present application provides a data management system, the system comprising a client, a multi-cloud platform, and a blockchain network, the system being used to manage data on the multi-cloud platform;
[0032] The client is configured to receive a backup plan configured by a user for data to be backed up, wherein the backup plan includes the number n of cloud nodes used for backup and the number b of backup copies in the multi-cloud platform;
[0033] The client is further configured to divide the data to be backed up into c data blocks based on the number of cloud nodes for backup and the number of backup copies, disperse the c data blocks and store them on n cloud nodes in the multi-cloud platform; and for at least one data block among the c data blocks, the multi-cloud platform stores b backup copies of the at least one data block;
[0034] The client is further used to provide the blockchain network with metadata of the data stored on the multi-cloud platform, so that the blockchain network encodes the metadata into a backup identifier and stores the backup identifier, and the backup identifier is used to address the data stored on the multi-cloud platform.
[0035] In some possible implementations, the backup identifier includes a short identifier and a long identifier, the short identifier includes a data identifier, and the long identifier includes a storage address of the data on the multi-cloud platform, and the blockchain network is used to:
[0036] receiving the short identifier provided by the user;
[0037] The long identifier corresponding to the short identifier is searched according to the short identifier, the long identifier is parsed to obtain the storage address of the data on the multi-cloud platform, and the storage address of the data on the multi-cloud platform is returned.
[0038] In some possible implementations, the backup identifier includes a first short identifier, a first long identifier, and a second short identifier and a second long identifier, the first short identifier includes a data identifier, the first long identifier includes a version set, the second short identifier includes a data identifier and a target version in the version set, and the second long identifier includes the storage address of the data of the target version on the multi-cloud platform.
[0039] In some possible implementations, the blockchain network is further used to:
[0040] Obtaining status parameters of cloud nodes in the multi-cloud platform;
[0041] Obtaining weights of cloud nodes in the multi-cloud platform according to the state parameters;
[0042] Return the node identifiers of the n cloud nodes whose weights meet the requirements to the client.
[0043] In some possible implementations, the state parameters include one or more of node bandwidth, node cost, node remaining storage capacity, and node reputation information.
[0044] In some possible implementations, the blockchain network is further used to:
[0045] Checking the consistency between the actual storage address of the data in the multi-cloud platform and the storage address recorded in the backup identifier stored in the blockchain network;
[0046] If they are inconsistent, the data is restored on the multi-cloud platform according to the storage address recorded in the backup identifier.
[0047] In some possible implementations, the client is further configured to:
[0048] Create a backup storage transaction;
[0049] The backup storage transaction is executed to perform the transaction operation of distributing and storing the c data blocks on n cloud nodes in the multi-cloud platform and the transaction operation of providing metadata of the data stored on the multi-cloud platform to the blockchain network.
[0050] In some possible implementations, the client is further configured to:
[0051] Get incremental data;
[0052] A backup update transaction is created and executed to perform a transaction operation of storing the incremental data in the multi-cloud platform and a transaction operation of updating the backup identifier.
[0053] In some possible implementations, the client is further configured to:
[0054] In response to the delete operation, creating a backup delete transaction;
[0055] The backup deletion transaction is executed to perform a transaction operation of deleting the data corresponding to the specified data identifier and a transaction operation of deleting the backup identifier corresponding to the specified data identifier.
[0056] In some possible implementations, the number of data blocks of the data to be backed up is equal to C(n,n-q+1), and the client is specifically configured to:
[0057] Generate a scheduling allocation table based on the number of data blocks to be backed up, wherein the scheduling allocation table records the data blocks to be stored by the n cloud nodes;
[0058] According to the scheduling allocation table, the c data blocks are dispersedly stored on the n cloud nodes in the multi-cloud platform.
[0059] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, each of which includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is configured to execute instructions stored in the at least one memory, causing the computing device or computing device cluster to perform the data management method described in the first aspect or any implementation of the first aspect.
[0060] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions instruct a computing device or a computing device cluster to execute the data management method described in the first aspect or any implementation of the first aspect.
[0061] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device or a computing device cluster, enables the computing device or computing device cluster to execute the data management method described in the first aspect or any one of the implementations of the first aspect.
[0062] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the technical methods of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.
[0064] FIG1 is a schematic diagram of the architecture of a data management system provided in an embodiment of the present application;
[0065] FIG2 is a flow chart of a data management method provided in an embodiment of the present application;
[0066] FIG3 is a schematic diagram of a backup configuration interface provided in an embodiment of the present application;
[0067] FIG4 is a schematic diagram of a backup identifier provided in an embodiment of the present application;
[0068] FIG5 is an operational flow chart of a backup storage transaction, a backup update transaction, and a backup deletion transaction provided in an embodiment of the present application;
[0069] FIG6 is a flow chart of a data management method provided in an embodiment of the present application;
[0070] FIG7 is a flow chart of a data management method provided in an embodiment of the present application;
[0071] FIG8 is a flow chart of a data management method provided in an embodiment of the present application;
[0072] FIG9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;
[0073] FIG10 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;
[0074] FIG11 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application;
[0075] FIG12 is a schematic diagram of the structure of another computer cluster provided in an embodiment of the present application. DETAILED DESCRIPTION
[0076] The terms "first" and "second" in the embodiments of this application are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.
[0077] First, some technical terms involved in the embodiments of this application are introduced.
[0078] Multi-cloud backup refers to backing up data using multiple cloud platforms (e.g., cloud platforms built by different cloud service providers). Data is stored in multiple backup copies across the multi-cloud platforms. Furthermore, if data is lost in a backup copy on one cloud platform, such as due to deletion or tampering, it can be restored from backup copies on other cloud platforms within the multi-cloud platform. This process is also known as multi-cloud recovery. This solves the single-point bottleneck of data backup on a single cloud platform. If a cloud platform fails or completely crashes, a large amount of data will be lost, threatening data security and storage reliability.
[0079] Considering the difficulty of multi-cloud backup and recovery management, the industry has developed a multi-cloud unified management platform to implement multi-cloud backup management, disaster recovery, and automatic verification. Specifically, the multi-cloud unified management platform is a fault-tolerant parallel application scheduling architecture that monitors, coordinates, and manages the resources of cloud service providers through a third-party resource negotiation layer. This approach relies on a trusted third-party platform (i.e., the multi-cloud unified management platform) to manage resources between users and cloud service providers. Due to the high concentration of information resources, the third-party platform is exposed to multi-dimensional network attacks, especially since administrators and third-party platforms have extremely high access rights to data and resources and can modify and delete data.
[0080] In light of this, this application provides a distributed data management method based on blockchain technology. This method can be applied to a data management system. The data management system includes a client, a multi-cloud platform, and a blockchain network. The data management system is used to manage data on the multi-cloud platform. For example, the data management system can back up data on the multi-cloud platform.
[0081] Specifically, the client receives a user-configured backup plan for the data to be backed up. The backup plan includes the number n of cloud nodes used for backup in the multi-cloud platform and the number b of backup copies. The client then divides the data to be backed up into c data blocks based on the number n of cloud nodes used for backup and the number b of backup copies, and stores the c data blocks in a distributed manner across the n cloud nodes in the multi-cloud platform. For at least one of the c data blocks, the multi-cloud platform stores b backup copies of at least one data block. The client provides the blockchain network with metadata (also known as backup metadata) of the data stored on the multi-cloud platform, so that the blockchain network encodes the metadata into a backup identifier and stores the backup identifier. The backup identifier is used to address the data stored on the multi-cloud platform.
[0082] This method divides the data to be backed up into blocks and stores them dispersedly across multiple cloud platforms. The metadata is then provided to the blockchain network for unified encoding, resulting in a globally unique backup identifier. This on-chain backup identifier is then used to record and manage the data backed up across multiple cloud platforms, providing unified location and addressing for distributed multi-cloud backup data management. This method combines the decentralized, secure, and trustworthy nature of blockchain networks to achieve cloud-chain integration, improving the security of multi-cloud backup data and eliminating the single-point security bottlenecks inherent in traditional methods.
[0083] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.
[0084] Referring to FIG1 , which illustrates an architecture diagram of a data management system, the data management system 10 includes a client 100, a multi-cloud platform 200, and a blockchain network 300. The client 100 establishes communication connections with the multi-cloud platform 200 and the blockchain network 300, respectively. The connection can be via wired communication or wireless communication such as a cellular network or WiFi.
[0085] The client 100 can be a program that provides local services to users, or a terminal that deploys the above program, including but not limited to desktop computers, laptops, smartphones, and smart wearable devices. Users can use the client (or terminal) to execute the consistency maintenance protocol to manage data and metadata on the multi-cloud platform 200 and the blockchain network 300, including but not limited to storage, recovery, incremental updates, and deletions. When a user wants to back up data, the data is uploaded and stored to the remote multi-cloud platform 200 through the client 100. When the locally stored data is damaged and needs to be restored from the multi-cloud platform 200, the user can obtain a backup identifier from the blockchain network 300 through the client 100, and obtain a copy of the data stored on the multi-cloud platform 200 based on the backup identifier.
[0086] Multi-cloud platform 200 includes cloud platforms built by various cloud service providers, providing various types of computing and storage resources, such as cloud computing and storage, edge computing and storage, and so on. Figure 1 illustrates multi-cloud platform 200 as an example, including multiple cloud platforms providing cloud storage resources. Users can pre-subscribe to cloud services from multiple cloud service providers to combine different resources to form a multi-cloud platform with complementary advantages.
[0087] The blockchain network 300 includes multiple blockchain nodes that collaboratively store, manage, and maintain backup data. In some possible implementations, the client 100 and the multi-cloud platform 200 (e.g., cloud nodes within the multi-cloud platform 200) can also serve as different types of blockchain nodes to jointly construct the blockchain network 300. The client 100 can be a lightweight node (or light node), and the cloud nodes within the multi-cloud platform 200 can be full nodes (or full nodes). Full nodes can initiate and receive transactions and participate in consensus, while light nodes can connect to and access full nodes through them. The blockchain network 300 of this application supports different underlying blockchain infrastructures, offering high flexibility and enabling the integration of different consensus mechanisms based on application requirements. Smart contracts are an important component of the blockchain network. The system first deploys a backup identity registration contract and a backup metadata management contract on the blockchain network to enable the storage and updating of metadata within the blockchain network. Multi-cloud state parameter management contracts (e.g., a multi-cloud state parameter initialization contract and a multi-cloud state parameter update contract) can also be deployed on the blockchain network to enable scheduling of cloud nodes within the multi-cloud platform 200.
[0088] In a specific implementation, the client 100 is configured to receive a user-configured backup plan for the data to be backed up. The backup plan includes the number n of cloud nodes used for backup and the number b of backup copies in the multi-cloud platform 200. Based on the number of cloud nodes used for backup and the number of backup copies, the client 100 then divides the data to be backed up into c data blocks and stores the c data blocks in a distributed manner across the n cloud nodes in the multi-cloud platform. For at least one of the c data blocks, the multi-cloud platform 200 stores b backup copies of the at least one data block. The client 100 is also configured to provide metadata of the data stored by the multi-cloud platform 200 to the blockchain network 300, so that the blockchain network 300 encodes the metadata into a backup identifier and stores the backup identifier.
[0089] The backup identifier is used to address data stored on the multi-cloud platform 200. In some examples, the backup identifier includes a short identifier and a long identifier. The short identifier includes a data identifier (DataID), and the long identifier includes the storage address of the data on the multi-cloud platform 200. In this way, when a user provides a short identifier through the client 100, the blockchain network 300 can search for the long identifier corresponding to the short identifier based on the short identifier, parse the long identifier to obtain the storage address of the data on the multi-cloud platform 200, and return the storage address of the data on the multi-cloud platform 200.
[0090] The client 100 is described below from the perspective of functional modularization.
[0091] The client 100 may include a backup storage module 102. The backup storage module 102 is used to undertake the storage task of the data to be backed up, store the data to be backed up in the multi-cloud platform 200 according to the multi-cloud scheduling policy, and store the metadata in the blockchain network 300. Specifically, the backup storage module 102 can divide the data to be backed up into c data blocks according to the number n of cloud nodes for backup and the number b of backup copies configured by the user, and store the c data blocks in a dispersed manner on n cloud nodes in the multi-cloud platform 200, and provide the blockchain network 300 with metadata (also called backup metadata) of the data stored in the multi-cloud platform 200, so that the blockchain network 300 encodes the metadata into a backup identifier and stores the backup identifier, which is used to address the data stored in the multi-cloud platform 200.
[0092] In some possible implementations, the client 100 may further include a multi-cloud state parameter upload module 104. The multi-cloud state parameter upload module 104 is used to upload the state parameters of the cloud nodes in the multi-cloud platform 200 to the blockchain network 300, so that the blockchain network 300 can obtain the weights of the cloud nodes in the multi-cloud platform 200 based on the state parameters and return the identifiers of n cloud nodes whose weights meet the requirements to the client 100. For example, the multi-cloud state parameter upload module 104 can upload the initialization state parameters of multiple cloud nodes to the blockchain network 300, which will be recorded in the blockchain ledger by the blockchain network 300 for use in formulating a multi-cloud backup scheduling strategy. Furthermore, after the backup is completed, the global multi-cloud state parameters (e.g., the state of the cloud nodes used for backup) will typically change. The changed multi-cloud state parameters can be submitted to the blockchain network 300, and the blockchain network 300 can record the changed multi-cloud state parameters in the blockchain ledger.
[0093] The client 100 also supports incremental update, deletion, and restoration of data. The following provides examples of functional modules that implement corresponding functions, such as the backup addition and deletion module 106 and the backup restoration module 108.
[0094] The backup, add, delete, and retry module 106 is used to perform incremental updates or deletions of data. The backup, add, delete, and retry module 106 performing incremental updates of data may include the following steps: storing the incremental data in the multi-cloud platform 200 and updating the metadata in the backup identifier. For example, when a user needs to incrementally update data, he or she may enter a short identifier and incremental data. The backup, add, delete, and retry module 106 may store the incremental data in the multi-cloud platform 200 and update the corresponding fields in the long identifier. The backup, add, delete, and retry module 106 performing deletions of data may include the following steps: deleting the data copy corresponding to the backup identifier from the multi-cloud platform 200 and deleting the backup identifier from the blockchain network 300. For example, when a user needs to delete data, he or she may enter a short identifier. The backup, add, delete, and retry module 106 may query the corresponding long identifier based on the end identifier, delete the data from the multi-cloud platform 200 based on the storage address of the data in the long identifier, and delete the short identifier and the long identifier from the blockchain network 300.
[0095] The backup and recovery module 108 is used to restore data based on the backup identifier in the blockchain network 300 and the backup copy stored on the multi-cloud platform 200. Specifically, the backup and recovery module 108 first queries the storage address (storage location) of the backup copy of the data on the multi-cloud platform 200 from the blockchain network 300, and then downloads the backup copy from the corresponding location on the multi-cloud platform 200 to restore the data. Applying the cloud-chain fusion solution of the present application can avoid the single-point bottleneck problem of traditional methods, realize distributed multi-cloud backup management and recovery coordinated by the cloud chain, and ensure the security and availability of backup data.
[0096] In addition, the client 100 may also include a cloud chain consistency collaboration module 109. The cloud chain consistency collaboration module 109 is used to define backup storage transactions, backup update transactions (such as backup incremental update transactions) or backup deletion transactions, and ensure the consistency of backup storage, backup update, and backup deletion through the transaction consistency protocol.
[0097] Based on the data management system 10 shown in Figure 1, the present application further provides a data management method. The data management method of the present application is described below in conjunction with an embodiment.
[0098] Referring to a flow chart of a data management method shown in FIG2 , the method is applied to a data management system 10, which includes a client 100, a multi-cloud platform 200, and a blockchain network 300. The data management system 10 is used to manage data on the multi-cloud platform 200. The method includes:
[0099] S202: The client 100 receives a backup plan configured by the user for the data to be backed up.
[0100] The backup plan includes the number n of cloud nodes used for backup in the multi-cloud platform and the number b of backup copies. The cloud nodes used can be from different cloud platforms. For example, n cloud nodes can be from n cloud platforms, where n is a positive integer. The number b of backup copies can be a positive integer. To ensure storage reliability, the number b of backup copies can be greater than 1. Typically, three-replica storage is used, so the number b of backup copies can be 3.
[0101] Specifically, when the user triggers a backup operation, for example, the user triggers the backup operation for the selected data to be backed up through a shortcut key, voice command or menu control, the client 100 can present a backup configuration interface to the user. The backup configuration interface can be a graphical user interface (GUI) or a command user interface (CUI). The backup configuration interface is used as an example of a GUI. Referring to the schematic diagram of a backup configuration interface shown in Figure 3, the backup configuration interface 30 includes a node usage quantity configuration control 32 and a backup copy quantity configuration control 34. The user can configure the number n of cloud nodes used for backup in the multi-cloud platform through the node usage quantity configuration control 32, and configure the backup copy quantity b through the backup copy quantity configuration control 34. The backup configuration interface 30 also carries a submit control 36 and a cancel control 38. When the user triggers the submit control 36, the above configuration information can be submitted for subsequent processes. When the user triggers the cancel control 38, the configuration of the backup plan can be canceled.
[0102] S204 . The client 100 divides the data to be backed up into c data blocks according to the number n of cloud nodes used for backup and the number b of backup copies.
[0103] S206 , the client 100 stores the c data blocks in a dispersed manner on n cloud nodes in the multi-cloud platform.
[0104] The client 100 (for example, the backup storage module of the client 100) can execute the multi-cloud backup scheduling strategy and the identification encoding and registration method, store multiple backup copies of the data on the multi-cloud platform 200 (for example, n cloud nodes of the multi-cloud platform 200), and encode the metadata of the data into a globally unique backup identifier and register it in the blockchain ledger, so as to achieve addressing, positioning and unified management of distributed stored data.
[0105] Among them, the multi-cloud backup scheduling strategy may include the scheduling of backup resources (data) and the scheduling of the multi-cloud platform 200.
[0106] The scheduling of backup resources (e.g., resources such as data to be backed up) can be that the client 100 determines the number of data blocks to be cut from the data to be backed up based on the backup plan configured by the user, such as the number n of cloud nodes used for backup and the number b of backup copies, and then generates a scheduling allocation table (also called a data block allocation table) based on the number of data blocks to be cut from the data to be backed up. The scheduling allocation table records the data blocks to be stored by the n cloud nodes. Accordingly, the client 100 can cut (split) the data into data blocks according to the number, and the cut data blocks can be divided into multiple groups according to the scheduling allocation table for upload. Specifically, when grouping the data blocks according to the scheduling allocation table, the data blocks can be grouped according to the storage location (e.g., the cloud node storing the data block).
[0107] The scheduling of the multi-cloud platform 200 can be that the blockchain network 300 (for example, the blockchain smart contract deployed in the blockchain node of the blockchain network 300) determines the weight of the cloud nodes in the multi-cloud platform 200 according to the status parameters of the cloud nodes, selects n cloud nodes whose weights meet the requirements (for example, the largest weight or the weight greater than the preset value) after sorting, and returns the node identifier of the cloud node whose weight meets the requirements.
[0108] The following examples illustrate the scheduling of backup resources and the scheduling of the multi-cloud platform 200.
[0109] The scheduling of backup resources is based on the concept of secret splitting. Secret splitting, also known as threshold secret splitting, is a robust key management scheme in cryptographic systems that can operate securely and reliably even if some fragments are damaged. Specifically, a secret s (or key) is divided into n parts, each of which is called a subkey and is held by a participant, such that:
[0110] (1) The subkeys held by q or more participants can reconstruct s;
[0111] (2) Subkeys held by fewer than q participants cannot reconstruct s.
[0112] The scheme is called a (q, n) threshold secret partitioning scheme, where q is called the threshold value.
[0113] In this example, it is assumed that n clouds S1,...S for backup have been determined. n , split the data file O into x data blocks P1, P2, ..., P x Parallel storage, any cloud node cannot have complete data, which can be expressed as At least q clouds are needed to recover the original data file, so the constraints of the threshold secret splitting scheme can be abstracted as:
[0114] Condition 1: The union S of the data block sets S backed up by any k cloud nodes m ∪S m+1 ∪…∪S m+k-1 =O
[0115] Condition 2: The union of the data block sets S backed up by any k cloud nodes
[0116] Where q is the threshold value. It can be considered that when k=q-1 When k=q, S m ∪S m+1 ∪…∪S m+k-1 =O.
[0117] Then for any data block P i , there exists a union S of q-1 sets S m+1 ∪S m+2 ∪…∪S m+q-1 Does not contain P i , the union S of any q sets S m ∪S m+1 ∪S m+2 ∪…∪S m+q-1 Contains P i , which means that the data block P i is divided into a specific n-q+1 set C. i From the arbitrariness of , we know that each data block should have n-q+1 backups, that is, b=n-q+1. Considering that when k=q-1 There must be a data block P j It is not included in the union of any q-1 sets, which means that the number of data blocks of the data segment to be backed up is c=C(n,n-q+1).
[0118] Based on the above deduction, we can conclude that if there are n backup clouds (or n cloud nodes), any q clouds (or q cloud nodes) can restore the original data, and each data block has b backups, then the original data can be divided into C(n,n-q+1) blocks.
[0119] Each data block can be allocated to n-q+1 cloud nodes, and the cloud nodes assigned to each data block are not exactly the same. In some embodiments, the client 100 can allocate according to all situations of C(n,n-q+1) combinations to obtain a scheduling allocation table. For example, for the scenario where n=5 and b=3, a possible allocation method is shown in the following table:
[0120] Table 1 Scheduling allocation table
[0121] Compared to the simple method of backing up data blocks as separate files, the above allocation strategy can reduce the number of backup requests to blockchain nodes, alleviate the pressure on computing resources in the blockchain network 300, and increase bandwidth utilization by uploading data blocks in parallel.
[0122] In the above method, the client 100 automatically calculates the number of data blocks for data segmentation, and uploads different data blocks to different cloud platforms (such as cloud nodes in different cloud platforms) for storage. A single cloud platform cannot recover secret information (such as complete data), and when any backup data block within the corresponding range has a problem, the data can still be fully restored, thereby improving the robustness of data backup. Specifically, the crash of any q cloud nodes is within the tolerance range and does not affect the recoverability of the original data. Under this strategy, the cloud nodes participating in the backup have exactly the same importance. The impact of the crash of multiple nodes is related to the number of crashed nodes. There are no special or unusually important nodes, thus achieving a truly decentralized scheduling strategy.
[0123] The scheduling of the multi-cloud platform 200 can be achieved by calculating the weights of the cloud nodes in the multi-cloud platform 200. Furthermore, parameter normalization can be performed before calculating the weights of the cloud nodes. The specific implementation of the scheduling of the multi-cloud platform 200 is described in detail below.
[0124] A multi-cloud global parameter update smart contract is deployed in the blockchain node, which can upload the state parameters of the cloud node to the blockchain ledger for storage and calculation. Among them, the state parameters can be stored in the form of key-value pairs (KV). In some examples, the globally unique identifier of the cloud node (for example, the node ID) is used as the key, and the state parameters of the cloud node are used as the value. The state parameters of the cloud node can include static parameters or dynamic parameters. Among them, the static parameters include at least one of the node bandwidth and the node cost (also known as the node price or the cloud storage price). The static parameters are basically fixed after subscribing to the cloud service and can be written to the blockchain ledger through the multi-cloud state parameter initialization contract. The dynamic parameters include at least one of the remaining storage capacity of the node and the node reputation information (such as the node reputation value). During the initialization phase, the above dynamic parameters can be written to the blockchain ledger through the multi-cloud state parameter initialization contract. As the data is backed up on the multi-cloud platform 200, the values of the dynamic parameters can be dynamically adjusted as the backup operation triggers the multi-cloud state parameter update contract. Specifically, the multi-cloud state parameter update contract defines the adjustment process of dynamic parameters. For example, the node reputation value can be dynamically adjusted according to the audit results. The more successful audits, the higher the node reputation value (credibility) of the cloud node; the more failed audits, the lower the node reputation value (credibility) of the cloud node.
[0125] In this example, cloud node S i The state parameters are Then the state parameters of all m cloud nodes are recorded as:
[0126] in, A parameter vector representing the state parameters of multiple cloud nodes.
[0127] By parameter vector Perform standardization, such as ZScore standardization, to adjust the state parameters to a standard normal distribution mode so that the state parameters can reasonably maintain the original size relationship, can be processed normally through the nonlinear region of the sigmoid function, and then use the sigmoid function for normalization.
[0128] For benefit-type parameters (the larger the value, the better, such as node bandwidth), the normalization method is:
[0129] For cost parameters (the smaller the value, the better, such as node cost), the normalization method is:
[0130] Here, mean represents the mean and std represents the standard deviation.
[0131] After normalization is completed, the influence coefficients of various parameters on the weights can be determined. In specific implementation, the entropy weight method can be used to adaptively adjust the parameter weights, so that parameters with large differences in value distribution are given a larger influence coefficient, without the need to manually specify the priority of various parameters. The calculation process is as follows:
[0132] For the normalized parameter vector Calculate the normalized parameter vector The entropy value of each element of is obtained to obtain the entropy vector. The entropy value of the element can be determined by the following formula:
[0133] in, represents the normalized parameter vector The jth value of represents the proportion of the jth value in the parameter vector, The "entropy" value representing the j-th value of the normalized parameter vector.
[0134] Next, we can synthesize the difference between the "entropy" value of the cloud node's parameter vector (usually a column vector) and the maximum value 1. Assuming that there are r types of cloud node state parameters, calculate the jth value of the cloud node's parameter vector The influence coefficient One way is: Cloud Node S j The normalization parameter and influence coefficient Weighted summation to calculate the weight of the cloud node: The blockchain network 300 sorts the cloud nodes by weight, selects n cloud nodes whose weights meet the conditions (for example, the highest weight or the weight is greater than a preset value), and returns the node IDs of the n cloud nodes whose weights meet the conditions to the client 100.
[0135] In this way, the client 100 can upload the data blocks to the cloud nodes corresponding to the above node IDs for backup storage according to the scheduling allocation table. For example, the client 100 can upload data blocks P1 to P6 to the cloud node corresponding to one of the N node IDs returned by the blockchain network 300 for backup storage, and upload data blocks P1 to P3 and P7 to P9 to the cloud node corresponding to another of the N node IDs returned by the blockchain network 300 for backup storage. The backup storage of other cloud nodes is similar and will not be repeated here.
[0136] S208. The client 100 provides the blockchain network 300 with metadata of the data stored by the multi-cloud platform 200, so that the blockchain network 300 encodes the metadata into a backup identifier and stores the backup identifier.
[0137] The metadata of the data stored on the multi-cloud platform 200 includes a data identifier (DataID) and a storage address. The storage address may include a Uniform Resource Locator (URL). Because the data is divided into multiple data blocks and stored in different cloud platforms, the storage address may include a URL list formed by multiple URLs. Furthermore, the metadata may also include a user identifier, backup time, number of backup copies, and a list of available clouds. The list of available clouds may include a list of cloud platforms subscribed by the user.
[0138] In a specific implementation, the client 100 can send a coded registration request to the blockchain network 300. The coded registration request carries metadata (also called backup metadata) of the data stored by the multi-cloud platform 200. Accordingly, the blockchain network 300 can extract the backup metadata, generate a backup identifier based on the identifier encoding rules, and store the backup identifier in the blockchain ledger.
[0139] The backup identifier can include a short identifier and a long identifier. The short identifier includes the data identifier, and the long identifier includes the data storage address on the multi-cloud platform. The blockchain network 300 can return the short identifier to the user. In this way, during subsequent data queries, the blockchain network 300 can receive the short identifier provided by the user, search for the long identifier corresponding to the short identifier based on the short identifier, parse the long identifier to obtain the data storage address on the multi-cloud platform 200, and return the data storage address on the multi-cloud platform. The client 100 can download the data from the multi-cloud platform based on the storage address.
[0140] Considering that the data backed up by the multi-cloud platform 200 may have multiple versions, the backup identifier may be divided into multiple levels, for example, two levels (also referred to as two-level), thereby implementing version management of the backup data. For ease of description, the following example illustrates a backup identifier including a two-level identifier.
[0141] Referring to the schematic diagram of a two-level identification shown in Figure 4, the first-level identification is used to query and record the time version of the backup data, and the first-level identification includes a short identification (also called the first short identification) and a long identification (also called the first long identification). Among them, the short identification in the first-level identification includes the data identification DataID. Furthermore, considering security, the short identification in the first-level identification can also include a user identification, such as the public key DataOwnerPk of the data owner. The format of the short identification in the first-level identification can be: DataOwnerPk|DataID. The long identification in the first-level identification includes a version set (the version in the version set can be represented by time). The second-level identification is used to query and manage the backup metadata of a specified version. Similarly, the second-level identification includes a short identification (the second short identification) and a long identification (the second long identification). The short identification in the second-level identification includes the data identification DataID and the target version in the version set. Furthermore, the short identification in the second-level identification can also include a user identification. The short identifier format in the secondary identifier can be: DataOwnerPk|DataID|Version, where DataOwnerPk represents the user ID of the "user registration domain", DataID represents the data identifier of the "user data domain", and Version represents the version of the backed-up data (for example, the time version). The short identifier is globally unique. The long identifier in the secondary identifier includes the storage address of the target version of the data in the multi-cloud platform. As shown in Figure 2, the long identifier in the secondary identifier is a collection of backup metadata describing the data. The long identifier in the secondary identifier includes: a basic data attribute domain, a backup policy domain, a data addressing information domain, a data verification information domain, and an extensible domain. Among them, the basic data attribute domain describes basic information such as the size and type of the backed-up data, the backup policy domain describes policy information such as the storage period and number of backup copies of the backed-up data, the data addressing information domain describes information such as the storage address of the backed-up data in the multi-cloud platform 200, the data verification information domain describes information such as the hash value of the backed-up data and the stored version, and the extensible domain is other backup information that the user needs to add and can be empty.
[0142] When a user has a query or data recovery need, they can initiate a query request to the blockchain network 300 with the data identifier DataID and version Version as parameters. Specifically, the client 100 receives the data identifier DataID and version Version input by the user and initiates a query request to the blockchain network, carrying the data identifier DataID and version Version. The blockchain network 300 automatically constructs a short identifier based on the parameters carried in the query request, such as the data identifier DataID and version Version. Then, based on the short identifier, the blockchain network 300 obtains the long identifier corresponding to the short identifier from the blockchain ledger, parses the long identifier, and obtains the storage address of each data block (for example, a URL list). It should be noted that the blockchain network 300 can also parse the long identifier to obtain a hash value. When the hash value calculated for the data block is consistent with the hash value returned by the blockchain network 300, the data can be downloaded. Furthermore, the client can return the query results or data recovery results to the user.
[0143] Based on the above description, the data management method provided in the embodiment of the present application designs a cloud-chain fusion mechanism (cloud-chain collaborative mechanism) that encodes the metadata of the data backed up by the multi-cloud platform into a backup identifier and stores the backup identifier in the blockchain network. The backup identifier on the chain records and manages multiple time versions of data, thereby supporting unified positioning and addressing of the data backed up by the multi-cloud platform 200, so as to realize the management of distributed multi-cloud backup data. This method utilizes the security and trustworthiness of the cloud-chain fusion mechanism to improve the security of multi-cloud backup data and eliminate the single-point security bottleneck existing in traditional methods.
[0144] Furthermore, this method designs an adaptive multi-cloud scheduling strategy based on smart contracts. For the backup plan set by the user, the state parameters of all available cloud nodes are extracted and weighted. After sorting, the corresponding number of cloud nodes with weights that meet the requirements are selected. By selecting an appropriate normalization function to process the parameter vector before calculating the weights, the comparability between different parameters can be enhanced. The influence coefficient of each parameter is then determined using appropriate methods (such as the entropy weight method). The influence of each parameter on the weight is processed according to its degree of difference, which can improve the reliability of the multi-cloud scheduling calculation and the flexibility of the scheduling strategy.
[0145] Due to the risk of data consistency being destroyed due to network failures or accidental deletion of data by operation and maintenance personnel, in order to ensure that data can be successfully restored when a disaster occurs, an automatic data consistency maintenance and verification mechanism can also be established to ensure the consistency of the backed up data. To this end, the present application can also introduce the concept of transactions, defining transactions as a series of operations on data in the data management system, including backup, deletion, and incremental updates. Transactions have consistency and atomicity. Consistency means that the data management system must ensure that the execution of transactions causes the cloud chain backup to transfer from one consistent state to another. Atomicity means that the cloud chain operations in the transaction are either all executed or not executed. The present application can achieve consistency and atomicity of backup, update, and deletion transactions through retry and rollback mechanisms.
[0146] Because transaction execution involves three parties: client 100, multi-cloud platform 200, and blockchain network 300, before submitting a transaction, it is possible to first check whether there are any anomalies in the network of client 100, multi-cloud platform 200, and blockchain network 300 before performing the operation task. Since cloud platforms are generally assumed to be semi-trusted, that is, they will correctly execute data operation instructions once they receive them, the inconsistency in this application mainly involves the failure of transaction execution due to an anomaly in the network of a cloud platform during the operation of multi-cloud platform 200.
[0147] The following describes the operation procedures of the backup storage transaction, backup update transaction, and backup deletion transaction provided by this application in conjunction with the embodiments.
[0148] Referring to the operational flow chart of a backup storage transaction, a backup update transaction, and a backup deletion transaction shown in FIG5 , the backup storage transaction includes the following transaction operations: (a) storing the data to be backed up in the multi-cloud platform 200 according to the multi-cloud backup scheduling strategy, and (b) encoding the metadata of the data backed up by the multi-cloud platform 200 as a backup identifier and storing it in the blockchain ledger (also known as metadata on-chain). The client 100 can create a backup storage transaction and then execute the backup storage transaction, thereby executing the transaction operation corresponding to the backup storage transaction. It should be noted that the transaction execution involves multiple participants. When the client executes the backup storage transaction, it executes the transaction operation of distributing and storing c data blocks in n cloud nodes in the multi-cloud platform 200, and provides the blockchain network 300 with the metadata of the data stored in the multi-cloud platform 200, so that the blockchain network 300 executes the transaction operation of encoding the metadata as a backup identifier and storing the backup identifier.
[0149] During the process of storing the data to be backed up on the multi-cloud platform 200, if a network anomaly on a cloud platform causes a transaction to fail, a rollback and retry can be performed. For example, the client 100 can delete the backed-up data on the cloud platform with the network anomaly and retry by selecting a new cloud platform from the multi-cloud platform 200. For another example, the client 100 can delete the backed-up data on all n cloud platforms (or n cloud nodes) participating in the backup and retry by selecting n new cloud platforms (or n cloud nodes) from the multi-cloud platform 200.
[0150] Similarly, a backup update transaction (e.g., a backup incremental update transaction) includes the following transaction operations: (a) storing incremental data in the multi-cloud platform 200 (b) updating the backup identifier in the blockchain ledger. Specifically, the client 100 can obtain incremental data, for example, by receiving incremental data input by the user, creating a backup update transaction, and then executing the backup update transaction to perform the transaction operation of storing the incremental data in the multi-cloud platform and the transaction operation of updating the backup identifier. Among them, the operation of updating the backup identifier can be that the client 100 provides metadata of the incremental data so that the blockchain network 300 can execute the transaction operation of updating the backup identifier. Similar to the backup storage transaction, during the process of storing incremental data in the multi-cloud platform 200, if there is a network anomaly of the cloud platform that causes the transaction execution to fail, a rollback and retry can be performed. For example, the client 100 can delete the incremental data that has been backed up in the cloud platform with the network anomaly and reselect a cloud platform from the multi-cloud platform 200 to retry. For another example, the client 100 can also delete all the backed-up data in the cloud platform participating in the incremental update and reselect a cloud platform from the multi-cloud platform 200 to retry.
[0151] A backup deletion transaction includes the following transaction operations: (a) deleting the data corresponding to the specified data identifier from the multi-cloud platform 200; (b) deleting the backup identifier corresponding to the specified data identifier from the blockchain network (e.g., a blockchain ledger). Specifically, in response to a deletion operation (manually or automatically triggered), the client 100 can create a backup deletion transaction and then execute the backup deletion transaction to perform the transaction operation of deleting the data corresponding to the specified data identifier and the transaction operation of deleting the backup identifier corresponding to the specified data identifier. The operation of deleting the data corresponding to the specified data identifier can be that the client 100 provides the data identifier (or short identifier) to facilitate the multi-cloud platform 200 to delete the corresponding data, and the operation of deleting the backup identifier corresponding to the specified data identifier can be that the client 100 provides the data identifier (or short identifier) to facilitate the blockchain network 300 to delete the corresponding backup identifier. Similar to backup storage transactions and backup update transactions, when deleting the data corresponding to the specified data identifier from the multi-cloud platform 200, if a network anomaly on the cloud platform causes the transaction execution to fail, the deletion can be returned to the location where the deletion failed and the backup identifier in the blockchain network 300 can be updated. The user can manually check the network status of the cloud platform and try again. It should be noted that once the backup deletion transaction is committed, the deleted data cannot usually be restored.
[0152] Taking into account the inconsistencies caused by the accidental deletion of files or directories by the operation and maintenance personnel of the cloud service provider after the data is backed up, this application also designs a consistency verification mechanism that uses the metadata in the blockchain network to verify the consistency of the backed-up data. Among them, the consistency verification mechanism includes verifying the consistency of the storage address of the data on the multi-cloud platform 200 and the storage address declared by the blockchain network 300. Among them, the storage address declared by the blockchain network 300 can be the storage address recorded in the backup identifier stored in the blockchain network 300 (for example, the blockchain ledger maintained by the blockchain network 300).
[0153] In specific implementation, the blockchain network 300 can check the consistency between the actual storage address of the data in the multi-cloud platform 200 and the storage address recorded in the backup identifier stored in the blockchain network 300. If there is inconsistency, the blockchain network 300 restores the data on the multi-cloud platform according to the storage address recorded in the backup identifier. Among them, the blockchain network 300 can periodically check the consistency between the actual storage address of the data in the multi-cloud platform 200 and the storage address recorded in the backup identifier stored in the blockchain network 300, so as to promptly discover the inconsistency caused by the accidental deletion operation and enable timely recovery. When verifying the consistency of the storage address, the blockchain network 300 can obtain the hash value of the storage address of the data in the multi-cloud platform (for example, the hash value of the file directory), call the consistency verification contract, and compare the hash value with the hash value in the backup identifier stored in the blockchain network 300 to verify the consistency of the storage address.
[0154] In some possible implementations, the client 100 may also submit a consistency verification request. For example, in response to a consistency verification operation triggered by a user, the client 100 submits a consistency verification request. The blockchain network 300 responds to the consistency verification request by obtaining the hash value of the storage address of the data uploaded by the multi-cloud platform 200 (for example, the hash value of the file directory), calling the consistency verification contract, comparing the hash value uploaded by the multi-cloud platform 200 with the hash value stored by the blockchain network 300, and returning the verification transaction ID and verification result. When the verification result is inconsistent, the client may also send a backup and restore request to restore data from other cloud platforms.
[0155] Next, the management of incremental update, deletion, and restoration of data after backup is completed will be described in detail with reference to the embodiments.
[0156] After the data backup is complete, the user can submit an incremental update request through client 100. This incremental update request is used to incrementally update the data corresponding to the specified data identifier. Based on the backup update transaction, the data management system 10 performs the incremental update of the cloud data and updates the backup identifier on the blockchain. After the update is completed, the blockchain network 300 can also return a transaction ID. This is explained below with reference to the accompanying figures.
[0157] Referring to the flowchart of a data management method shown in FIG6 , the method includes the following steps:
[0158] ① Client 100 submits an incremental update request.
[0159] An incremental update request is used to incrementally update data. Specifically, an incremental update request carries incremental data (denoted as IncreData) and a specified data identifier (DataID). When the data corresponding to the specified data identifier includes multiple versions, the incremental update request may also carry the version (Version). The specified data identifier and version may be carried via a short identifier. Based on this, an incremental update request may carry incremental data (IncreData) and a short identifier (DataID|Version).
[0160] ② The client 100 executes the incremental update transaction according to the incremental update request.
[0161] First, the client 100 can upload the incremental data for updating to the multi-cloud platform 200 for incremental update, record the storage address of the incremental data (for example, a URL list), and further, the client 100 can also record the hash value of the incremental data. Then, the client 100 sends the transaction parameters, such as a short identifier and metadata of the incremental data, to the blockchain network 300.
[0162] ③ The blockchain network 300 performs identity resolution, obtains the long identity through short identity query, and updates the addressing information field in the long identity.
[0163] Specifically, the blockchain network 300 can call the backup metadata management contract, query the corresponding long identifier based on the short identifier composed of DataOwnerPk|DataID|Version, parse the long identifier to obtain the addressing information field, and update the relevant fields in the addressing information field. For example, based on the metadata of the incremental data, the attribute fields of the incremental data are constructed, including but not limited to the storage address of the incremental data. In some possible implementations, the blockchain network 300 can also parse the long identifier to obtain the data verification information field, and update the relevant fields of the data verification information field. For example, based on the metadata of the incremental data, the attribute fields of the incremental data are constructed, including but not limited to the hash value of the incremental data.
[0164] ④ The blockchain network 300 returns the transaction ID and update success flag to the client 100.
[0165] The update success flag is used to indicate that the incremental update is completed.
[0166] After the data backup is completed, if the user has a deletion requirement, the user can also submit a backup deletion request through the client 100 and provide a short identifier of the data to be deleted. The data management system 10 deletes the backup copy stored in the multi-cloud platform 200 and the backup identifier stored in the blockchain network 300 based on the backup deletion transaction.
[0167] Referring to the flowchart of a data management method shown in FIG7 , the method includes the following steps:
[0168] ① Client 100 submits a backup deletion request.
[0169] A backup delete request is used to delete backed-up data. Specifically, the request carries a data identifier (DataID). If the data corresponding to the DataID includes multiple versions, the request may also carry a version (Version), requesting the deletion of a specific version of the data.
[0170] The data identifier or version can be carried by a short identifier. Based on this, the backup deletion request can carry a short identifier (DataID|Version). If the backup deletion request carries the short identifier in the first-level identifier, such as the first short identifier DataOwnerPk|DataID, it indicates a request to delete all versions of the data and the corresponding backup identifiers. If the backup deletion request carries the short identifier in the second-level identifier, such as the second short identifier DataOwnerPk|DataID|Version, it indicates a request to delete the specified version of the data and the corresponding backup identifier.
[0171] ② The client 100 executes the backup deletion transaction.
[0172] Specifically, the client 100 uses the short identifier in the backup identifier to obtain the long identifier corresponding to the short identifier, and queries the storage address of the backed-up data from the long identifier, for example, obtaining the storage address from the addressing information field of the long identifier. The multi-cloud platform 200 deletes the corresponding data according to the storage address in the addressing information field until the corresponding data is completely deleted. Accordingly, the blockchain network 300 can delete the backup identifier corresponding to the data. If a deletion fails, the client 100 can also record the deletion failure location, and the blockchain network can update the long identifier in the backup identifier based on the deletion failure location.
[0173] In some possible implementations, the backup metadata management contract defines backup metadata deletion rules. The backup metadata deletion rules may include: if the deletion failure record is empty, directly deleting the backup identifier. Furthermore, the backup metadata deletion rules may also include: if the deletion failure record is not empty, updating the long identifier corresponding to the short identifier based on the deletion failure location. The blockchain network 300 may invoke the backup metadata management contract to delete or update the backup identifier.
[0174] ③ When the deletion is successful, the blockchain network 300 returns the transaction ID and the deletion success flag. When the deletion fails, the blockchain network 300 returns the transaction ID and the deletion failure record.
[0175] The deletion success flag is used to indicate that the deletion is complete.
[0176] Considering the possibility that the operation and maintenance personnel of the cloud service provider may accidentally delete data, they can also apply for data recovery. When a user submits a data recovery request and provides a user identifier (such as the user identity identifier DataOwnerPk), the data identifier of the data to be recovered, and the expected recovery time point, the blockchain network 300 can construct a short identifier based on the data identifier of the data to be recovered and the expected recovery time point, perform hierarchical parsing based on the short identifier to obtain the storage address of the backup copy of the data to be recovered, and download the backup copy based on the storage address to restore the data. Furthermore, the method also supports data verification based on the hash value, and returns the data recovery result to the user after the verification is passed.
[0177] Referring to the flowchart of a data management method shown in FIG8 , the method includes the following steps:
[0178] ① Client 100 submits a data recovery request.
[0179] The data recovery request carries the data identifier of the data to be recovered. If the data to be recovered includes multiple versions (time versions), the data recovery request can also carry the expected recovery time point (time version). Considering data security, the data recovery request can also carry the user identifier to facilitate user identity verification.
[0180] In specific implementation, users can submit data recovery requests through the data recovery interface. The request parameters include: user identity, data identifier of the data to be recovered, and expected recovery time point.
[0181] ② The blockchain network 300 queries the backup identifier based on the data identifier to obtain the storage address and hash value of the data.
[0182] Specifically, the blockchain network 300 can construct a short identifier (also referred to as a first-level short identifier, a first short identifier) in the first-level identifier based on the data identifier, query the corresponding first-level long identifier based on the first-level short identifier, and obtain the version closest to the expected recovery time point. According to the data identifier and version, a short identifier (also referred to as a second-level short identifier, a second short identifier) in the second-level identifier can be constructed. The blockchain network 300 can query the corresponding second-level long identifier based on the second-level short identifier. The blockchain network 300 parses the second-level long identifier to obtain the storage address (denoted as DataURLs) and hash value (denoted as DataHashs) of the data backed up by the multi-cloud platform 200. Furthermore, when the version closest to the expected recovery time point is the incrementally updated version, the blockchain network 300 parses the second-level long identifier and can also obtain the storage location and hash value of the incremental data.
[0183] ③ The client 100 downloads data from the multi-cloud platform 200 according to the storage address obtained through the analysis and verifies the hash value of the data.
[0184] Specifically, data is stored in the multi-cloud platform 200 in the form of multiple data blocks. When the first cloud platform in the multi-cloud platform 200 has a data block P i When the data block P is deleted by mistake, the client 100 can obtain the storage address obtained by parsing the blockchain network 300 from the storage address where the data block P is stored. i The second cloud platform downloads data blocks P i , then upload the data block P i To the First Cloud Platform.
[0185] Furthermore, considering the situation that the data block is tampered or the transmission fails, the client 100 can also determine the downloaded data block P i The hash value is compared with the hash value obtained by the block chain network 300 to perform consistency verification. When the consistency verification passes, the data block P is uploaded. i To the First Cloud Platform.
[0186] Based on the data management method of the aforementioned embodiment, the present application also provides a data management system 10 as described above. The data management system 10 is described below with reference to the accompanying drawings.
[0187] Referring to the structural diagram of the data management system 10 shown in FIG1 , the data management system 10 includes a client 100 , a multi-cloud platform 200 , and a blockchain network 300 . The data management system 10 manages the data of the multi-cloud platform 200 .
[0188] The client 100 is configured to receive a backup plan configured by a user for data to be backed up, the backup plan including the number n of cloud nodes used for backup and the number b of backup copies in the multi-cloud platform 200;
[0189] The client 100 is further configured to divide the data to be backed up into c data blocks based on the number of cloud nodes used for backup and the number of backup copies, and to disperse and store the c data blocks on n cloud nodes in the multi-cloud platform 200. For at least one data block among the c data blocks, the multi-cloud platform 200 stores b backup copies of the at least one data block.
[0190] The client 100 is also used to provide the blockchain network 300 with metadata of the data stored in the multi-cloud platform 200, so that the blockchain network 300 encodes the metadata into a backup identifier and stores the backup identifier, which is used to address the data stored in the multi-cloud platform 200.
[0191] In some possible implementations, the backup identifier includes a short identifier and a long identifier, the short identifier includes a data identifier, and the long identifier includes a storage address of the data on the multi-cloud platform 200. The blockchain network 300 is used to:
[0192] Receive a short identifier provided by the user;
[0193] The long identifier corresponding to the short identifier is searched for according to the short identifier, the long identifier is parsed to obtain the storage address of the data on the multi-cloud platform 200 , and the storage address of the data on the multi-cloud platform 200 is returned.
[0194] In some possible implementations, the backup identifier includes a first short identifier, a first long identifier, and a second short identifier and a second long identifier, the first short identifier includes a data identifier, the first long identifier includes a version set, the second short identifier includes the data identifier and a target version in the version set, and the second long identifier includes the storage address of the target version data on the multi-cloud platform 200.
[0195] In some possible implementations, the blockchain network 300 is further configured to:
[0196] Obtaining status parameters of cloud nodes in the multi-cloud platform 200;
[0197] Obtaining weights of cloud nodes in the multi-cloud platform 200 based on the state parameters;
[0198] The node identifiers of n cloud nodes whose weights meet the requirements are returned to the client 100 .
[0199] In some possible implementations, the state parameters include one or more of node bandwidth, node cost, node remaining storage capacity, and node reputation information.
[0200] In some possible implementations, the blockchain network 300 is further configured to:
[0201] Checking the consistency between the actual storage address of the data in the multi-cloud platform 200 and the storage address recorded in the backup identifier stored in the blockchain network 300;
[0202] If there is any inconsistency, the data is restored on the multi-cloud platform 200 according to the storage address recorded in the backup identifier.
[0203] In some possible implementations, the client 100 is further configured to:
[0204] Create a backup storage transaction;
[0205] A backup storage transaction is executed to perform a transaction operation of distributing and storing the c data blocks in n cloud nodes in the multi-cloud platform 200 and a transaction operation of providing metadata of the data stored in the multi-cloud platform 200 to the blockchain network 300.
[0206] In some possible implementations, the client 100 is further configured to:
[0207] Get incremental data;
[0208] A backup update transaction is created and executed to perform a transaction operation of storing incremental data in the multi-cloud platform 200 and a transaction operation of updating a backup identifier.
[0209] In some possible implementations, the client 100 is further configured to:
[0210] In response to the delete operation, creating a backup delete transaction;
[0211] Execute the backup deletion transaction to execute the transaction operation of deleting the data corresponding to the specified data identifier and the transaction operation of deleting the backup identifier corresponding to the specified data identifier.
[0212] In some possible implementations, the number of data blocks of the data to be backed up is equal to C(n,n-q+1), and the client 100 is specifically configured to:
[0213] Generate a scheduling allocation table based on the number of data blocks to be backed up, which records the data blocks to be stored in n cloud nodes;
[0214] According to the scheduling allocation table, the c data blocks are dispersedly stored in the n cloud nodes in the multi-cloud platform 200 .
[0215] The above content describes the data management system 10 provided in the embodiment of the present application from the perspective of hardware. The following describes the data management system 10 from the perspective of functional modularization. The data management system 10 includes:
[0216] The backup storage module 102 is configured to divide the data to be backed up into c data blocks according to the number n of cloud nodes for backup and the number b of backup copies configured by the user, and to disperse and store the c data blocks in n cloud nodes in the multi-cloud platform 200, and to provide the blockchain network 300 with metadata of the data stored in the multi-cloud platform 200, so that the blockchain network 300 encodes the metadata into a backup identifier and stores the backup identifier, which is used to address the data stored in the multi-cloud platform 200.
[0217] The multi-cloud state parameter uploading module 104 is used to upload the state parameters of the cloud nodes in the multi-cloud platform 200 to the blockchain network 300, so that the blockchain network 300 can obtain the weights of the cloud nodes in the multi-cloud platform 200 according to the state parameters, and return the identifiers of n cloud nodes whose weights meet the requirements to the client 100.
[0218] The backup add / delete module 106 is configured to obtain incremental data, create a backup update transaction, and execute the backup update transaction to perform a transaction operation of storing the incremental data in the multi-cloud platform 200 and a transaction operation of updating a backup identifier.
[0219] The backup add / delete module 106 is further configured to create a backup delete transaction in response to a delete operation, and execute the backup delete transaction to perform a transaction operation of deleting the data corresponding to the specified data identifier and a transaction operation of deleting the backup identifier corresponding to the specified data identifier.
[0220] The backup and recovery module 108 is used to query the storage address of the data on the multi-cloud platform 200 from the blockchain network 300, so as to download the backup copy from the corresponding location of the multi-cloud platform 200 to restore the data.
[0221] The backup storage module 102, multi-cloud state parameter upload module 104, backup addition and deletion module 106, and backup recovery module 108 can be implemented as hardware modules or software modules. Specifically, the backup storage module 102, multi-cloud state parameter upload module 104, backup addition and deletion module 106, and backup recovery module 108 can be implemented as a computing device or a computing engine on the computing device. The following description uses the backup storage module 102 as an example.
[0222] When implemented via software, the backup storage module 102 can be an application or application module, such as a computing engine, running on a computing device or computing device cluster. This application can be provided to users as a virtualized service. Virtualized services may include virtual machine (VM) services, bare metal server (BMS) services, and container services. VM services can use virtualization technology to create a virtual machine (VM) resource pool on multiple physical hosts (such as computing devices) to provide users with VMs on demand. BMS services use virtualized BMS resource pools on multiple physical hosts to provide users with BMSs on demand. Container services use virtualized container resource pools on multiple physical hosts to provide users with containers on demand. A VM is a simulated virtual computer, or logically a single computer. BMS is a scalable, high-performance computing service with computing performance comparable to traditional physical machines and secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service and container service in the above-mentioned virtualization services are merely specific examples. In actual applications, the virtualization service may also be other lightweight or heavyweight virtualization services, which are not specifically limited here.
[0223] When implemented through hardware, the backup storage module 102 may include at least one computing device, such as a server. Alternatively, the backup storage module 102 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0224] This application also provides a computing device 900. As shown in Figure 9, computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. Processor 904, memory 906, and communication interface 908 communicate with each other via bus 902. Computing device 900 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 900.
[0225] Bus 902 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG9 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 902 may include a path for transmitting information between various components of computing device 900 (e.g., memory 906, processor 904, and communication interface 908).
[0226] The processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0227] The memory 906 may include a volatile memory, such as a random access memory (RAM). The memory 906 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 906 stores executable program code, and the processor 904 executes the executable program code to implement the aforementioned data management method. Specifically, the memory 906 stores instructions for the data management system 10 to execute the data management method.
[0228] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or a communication network.
[0229] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.
[0230] As shown in Figure 10, the computing device cluster includes at least one computing device 900. The memory 906 of one or more computing devices 900 in the computing device cluster may store the same data management system 10 for executing instructions of the data management method.
[0231] In some possible implementations, one or more computing devices 900 in the computing device cluster may also be used to execute some of the instructions for executing the data management method of the data management system 10. In other words, the combination of one or more computing devices 900 may jointly execute the instructions for executing the data management method of the data management system 10.
[0232] It should be noted that the memories 906 in different computing devices 900 in the computing device cluster may store different instructions for executing partial functions of the data management system 10 .
[0233] Figure 11 illustrates a possible implementation. As shown in Figure 11, two computing devices 900A and 900B are connected via a communication interface 908. The memory in computing device 900A stores instructions for executing the functions of the backup storage module 102 and the multi-cloud state parameter upload module 104. The memory in computing device 900B stores instructions for executing the functions of the backup addition and deletion module 106 and the backup recovery module 108. In other words, the memories 906 of computing devices 900A and 900B jointly store instructions for the data management system 10 to execute the data management method.
[0234] The connection method between the computing device clusters shown in Figure 11 can be considered to take into account the data storage, addition, deletion, and recovery required by the data management method provided in this application. Therefore, it is considered that the functions implemented by the backup storage module 102 and the multi-cloud state parameter upload module 104 are performed by computing device 900A, while the functions implemented by the backup addition and deletion module 106 and the backup recovery module 108 are performed by computing device 900B.
[0235] It should be understood that the functionality of the computing device 900A shown in FIG11 may also be implemented by multiple computing devices 900. Similarly, the functionality of the computing device 900B may also be implemented by multiple computing devices 900.
[0236] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG12 shows a possible implementation. As shown in FIG12 , two computing devices 900C and 900D are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900C stores instructions for executing the functions of the backup storage module 102 and the multi-cloud state parameter upload module 104. At the same time, the memory 906 in the computing device 900D stores instructions for executing the functions of the backup addition and deletion module 106 and the backup recovery module 108.
[0237] The connection method between the computing device clusters shown in Figure 12 can be considered to take into account the need for data storage, data addition, deletion, and recovery of the data management method provided in this application. Therefore, it is considered that the functions implemented by the backup storage module 102 and the multi-cloud state parameter upload module 104 are performed by the computing device 900C, and the functions implemented by the backup addition and deletion module 106 and the backup recovery module 108 are performed by the computing device 900D. It should be understood that the functions of the computing device 900C shown in Figure 12 can also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900D can also be completed by multiple computing devices 900.
[0238] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned data management system for executing the data management method.
[0239] Embodiments of the present application also provide a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the aforementioned data management method.
[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A data management method, characterized in that: Applied to a data management system, the system includes a client, a multi-cloud platform and a blockchain network, the system is used to manage data on the multi-cloud platform, and the method includes: The client receives a backup plan configured by a user for data to be backed up, wherein the backup plan includes the number n of cloud nodes used for backup in the multi-cloud platform and the number b of backup copies; The client divides the data to be backed up into c data blocks according to the number of cloud nodes for backup and the number of backup copies, and stores the c data blocks in n cloud nodes in the multi-cloud platform in a dispersed manner; for at least one data block among the c data blocks, the multi-cloud platform stores b backup copies of the at least one data block; The client provides metadata of the data stored by the multi-cloud platform to the blockchain network, so that the blockchain network encodes the metadata into a backup identifier and stores the backup identifier, where the backup identifier is used to address the data stored by the multi-cloud platform.
2. The method according to claim 1, characterized in that The backup identifier includes a short identifier and a long identifier, the short identifier includes a data identifier, and the long identifier includes a storage address of the data on the multi-cloud platform. The method further includes: The blockchain network receives the short identifier provided by the user; The blockchain network searches for the long identifier corresponding to the short identifier according to the short identifier, parses the long identifier to obtain the storage address of the data on the multi-cloud platform, and returns the storage address of the data on the multi-cloud platform.
3. The method according to claim 2, characterized in that The backup identifier includes a first short identifier, a first long identifier, a second short identifier, and a second long identifier, the first short identifier includes a data identifier, the first long identifier includes a version set, the second short identifier includes a data identifier and a target version in the version set, and the second long identifier includes a storage address of the data of the target version on the multi-cloud platform.
4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The blockchain network obtains status parameters of cloud nodes in the multi-cloud platform; The blockchain network obtains the weight of the cloud node in the multi-cloud platform according to the state parameter; The blockchain network returns the node identifiers of the n cloud nodes whose weights meet the requirements to the client.
5. The method according to claim 4, characterized in that The state parameters include one or more of node bandwidth, node cost, node remaining storage capacity, and node reputation information.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The blockchain network checks the consistency between the actual storage address of the data in the multi-cloud platform and the storage address recorded in the backup identifier stored in the blockchain network; If there is inconsistency, the blockchain network restores the data on the multi-cloud platform according to the storage address recorded in the backup identifier.
7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The client creates a backup storage transaction; The client executes the backup storage transaction to perform the transaction operation of dispersing and storing the c data blocks in n cloud nodes in the multi-cloud platform and the transaction operation of providing metadata of the data stored in the multi-cloud platform to the blockchain network.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The client obtains incremental data; The client creates a backup update transaction and executes the backup update transaction to perform a transaction operation of storing the incremental data in the multi-cloud platform and a transaction operation of updating the backup identifier.
9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: The client creates a backup deletion transaction in response to the deletion operation; The client executes the backup deletion transaction to perform a transaction operation of deleting the data corresponding to the specified data identifier and a transaction operation of deleting the backup identifier corresponding to the specified data identifier.
10. The method according to any one of claims 1 to 9, characterized in that: The number of data blocks of the data to be backed up is equal to C(n,n-q+1), and the client stores the c data blocks in n cloud nodes in the multi-cloud platform in a dispersed manner, including: The client generates a scheduling allocation table according to the number of data blocks cut from the data to be backed up, and the scheduling allocation table records the data blocks to be stored by the n cloud nodes; The client stores the c data blocks in a dispersed manner on n cloud nodes in the multi-cloud platform according to the scheduling allocation table.
11. A data management system, characterized in that: The system includes a client, a multi-cloud platform, and a blockchain network, and the system is used to manage data on the multi-cloud platform; The client is used to receive a backup plan configured by a user for the data to be backed up, wherein the backup plan includes the number n of cloud nodes used for backup in the multi-cloud platform and the number b of backup copies; The client is further configured to divide the data to be backed up into c data blocks according to the number of cloud nodes for backup and the number of backup copies, and store the c data blocks in n cloud nodes in the multi-cloud platform in a dispersed manner; for at least one data block among the c data blocks, the multi-cloud platform stores b backup copies of the at least one data block; The client is also used to provide the blockchain network with metadata of the data stored on the multi-cloud platform, so that the blockchain network encodes the metadata into a backup identifier and stores the backup identifier, and the backup identifier is used to address the data stored on the multi-cloud platform.
12. The system according to claim 11, characterized in that The backup identifier includes a short identifier and a long identifier, the short identifier includes a data identifier, the long identifier includes a storage address of the data on the multi-cloud platform, and the blockchain network is used to: Receiving the short identifier provided by the user; The long identifier corresponding to the short identifier is searched according to the short identifier, the long identifier is parsed to obtain the storage address of the data on the multi-cloud platform, and the storage address of the data on the multi-cloud platform is returned.
13. The system according to claim 12, characterized in that The backup identifier includes a first short identifier, a first long identifier, a second short identifier, and a second long identifier, the first short identifier includes a data identifier, the first long identifier includes a version set, the second short identifier includes a data identifier and a target version in the version set, and the second long identifier includes a storage address of the data of the target version on the multi-cloud platform.
14. The system according to any one of claims 11 to 13, characterized in that: The blockchain network is also used to: Obtaining status parameters of cloud nodes in the multi-cloud platform; Obtaining weights of cloud nodes in the multi-cloud platform according to the state parameters; The node identifiers of the n cloud nodes whose weights meet the requirements are returned to the client.
15. The system according to claim 14, characterized in that The state parameters include one or more of node bandwidth, node cost, node remaining storage capacity, and node reputation information.
16. The system according to any one of claims 11 to 15, characterized in that The blockchain network is also used to: Checking the consistency between the actual storage address of the data in the multi-cloud platform and the storage address recorded in the backup identifier stored in the blockchain network; If they are inconsistent, the data is restored on the multi-cloud platform according to the storage address recorded in the backup identifier.
17. The system according to any one of claims 11 to 16, characterized in that: The client is also used to: Create a backup storage transaction; The backup storage transaction is executed to perform the transaction operation of dispersively storing the c data blocks in the n cloud nodes in the multi-cloud platform and the transaction operation of providing the blockchain network with metadata of the data stored in the multi-cloud platform.
18. The system according to any one of claims 11 to 17, characterized in that The client is also used to: Get incremental data; A backup update transaction is created, and the backup update transaction is executed to perform a transaction operation of storing the incremental data in the multi-cloud platform and a transaction operation of updating the backup identifier.
19. The system according to any one of claims 11 to 18, characterized in that The client is also used to: In response to the delete operation, creating a backup delete transaction; The backup deletion transaction is executed to perform a transaction operation of deleting the data corresponding to the specified data identifier and a transaction operation of deleting the backup identifier corresponding to the specified data identifier.
20. The system according to any one of claims 11 to 19, characterized in that The number of data blocks of the data to be backed up is equal to C(n,n-q+1), and the client is specifically used for: Generate a scheduling allocation table according to the number of data blocks to be backed up, wherein the scheduling allocation table records the data blocks to be stored by the n cloud nodes; According to the scheduling allocation table, the c data blocks are dispersedly stored in the n cloud nodes in the multi-cloud platform.
21. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the method as described in any one of claims 1 to 10.
22. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 10.
23. A computer program product, characterized in that The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Data management method and related equipment
CN118939472A
Blockchain-based distributed storage method
CN108111585A
Data service system and method, server and computer readable storage medium
CN108512935A
Digital archive heterogeneous electronic file cloud disaster recovery system based on block chain
CN110688261A
Distributed node finder in multi-cloud computing environment
CN111131345A
Cited By
Multi-ship anchoring scheduling path planning method based on improved RRT algorithm
CN119294639A
