Data migration method and system

Abstract

The invention provides a data migration method and system, and the method comprises the steps: obtaining a data migration message sent by a management node according to the data type of to-be-migrated data, and determining a target candidate storage node set in a distributed storage cluster according to the data migration message; determining a target storage node in the target candidate storage node set according to the migration type of each candidate storage node in the target candidate storage node set; according to the node information of the target storage node and an identifier of a to-be-migrated storage node, sending a permission voucher to the to-be-migrated storage node; the permission voucher is used for authorizing the to-be-migrated storage node to migrate the to-be-migrated data to the target storage node. According to the invention, decentralization of each storage node is realized to complete data migration, the flexibility of distributed storage cluster management is improved, the security of migration authorization is improved, and the security and legality of the data migration process are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage management, and particularly to a data migration method and system. Background Art

[0002] In terms of application scenarios, distributed storage clusters are widely used in high-performance computing, cloud native, big data analysis, finance, medical care and other fields, and can effectively support complex mixed loads and multi-protocol interconnection. In terms of technical routes, distributed storage clusters are developing in the direction of all-flash, architecture innovation and function expansion. Therefore, how to perform effective data migration on distributed storage clusters to achieve disaster recovery purposes to ensure business continuity and data availability is an important issue that the industry urgently needs to solve at present.

[0003] In the related art, generally, the main process is deployed on the management node, which can centrally obtain the status of all storage nodes in the distributed storage cluster and centrally manage its related services. Once it is predicted that any storage node fails, the management node will select a target storage node from other storage nodes to uniformly migrate the failed storage data to the target storage node. However, such centralized data migration management of the distributed storage cluster through the management node not only has poor flexibility, but also the centralized management node is prone to information leakage, which threatens the service security and stability of the entire distributed storage cluster and cannot meet the requirements of large-scale and complex distributed storage systems for flexible and secure data migration management. Summary of the Invention

[0004] The present invention provides a data migration method and system to solve the defects of poor flexibility and security of data migration caused by centralized management through the management node in the prior art, and to improve the flexibility and security of data migration.

[0005] The present invention provides a data migration method applied to an intelligent contract node, including: Obtaining a data migration message sent by the management node according to the data type of the data to be migrated, and determining a set of target candidate storage nodes in the distributed storage cluster according to the data migration message; Determining a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes; Sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node.

[0006] According to a data migration method provided by the present invention, the obtaining management node sends a data migration message according to the data type of the data to be migrated, and determines a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, including: When the data type of the data to be migrated is the first type, it is determined that the data migration message is a first data migration request; the first data migration request at least includes the required storage space capacity of the data to be migrated and the remaining storage space capacity of each storage node in the distributed storage cluster; Search for the latest health value of each storage node in the storage space of the smart contract node, and adjust the remaining storage space capacity of each storage node according to the latest health value; Compare the adjusted remaining storage space capacity of each storage node with the required storage space capacity of the data to be migrated, and determine the set of target candidate storage nodes in the distributed storage cluster according to the comparison result.

[0007] According to a data migration method provided by the present invention, the latest health value of each storage node is calculated by each storage node according to its own current node status information and then sent to the storage space; The current node status information includes database connection rate, disk wear rate, disk depreciation rate, processor utilization rate, memory usage rate, hard disk usage rate, and node importance.

[0008] According to a data migration method provided by the present invention, the obtaining management node sends a data migration message according to the data type of the data to be migrated, and determines a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, including: When the data type of the data to be migrated is the second type, it is determined that the data migration message is a first set of candidate storage nodes; Determine the set of target candidate storage nodes according to the first set of candidate storage nodes; Wherein, the first set of candidate storage nodes is determined by the management node according to response information returned by at least one storage node in the distributed storage cluster; The response information of any storage node is obtained by parsing a second data migration request sent by the management node by the any storage node, obtaining the index threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the index threshold.

[0009] A data migration method provided by the present invention, the target performance index of any storage node includes a node access index and / or a node real-time index; The node access index is calculated based on the historical access frequency data of the target historical service data of any storage node; the target historical service data is historical service data matching the data type of the data to be migrated; The node real-time index is calculated based on the historical migration time data of the target historical service data.

[0010] A data migration method provided by the present invention, determining a target storage node from the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set includes: In the blockchain, look up the migration types of each candidate storage node; In the target candidate storage node set, determine candidate storage nodes whose migration types match the data type of the data to be migrated, and the number of the matching candidate storage nodes; In the case where the number is multiple, determine the target storage node according to the remaining storage space capacity of the multiple matching candidate storage nodes.

[0011] A data migration method provided by the present invention, sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated includes: In the blockchain, look up the node information of the target storage node; Update the permission certificate template according to the feature information in the node information and the identifier of the storage node to be migrated to obtain the permission certificate, and send the permission certificate to the storage node to be migrated.

[0012] The present invention also provides a data migration method, which is applied to a management node and includes: Obtain the storage space utilization rate of each storage node in the distributed storage cluster; Determine the storage node to be migrated and the migration data information according to the storage space utilization rate; Send a data migration message to the smart contract node according to the data type of the data to be migrated in the migration data information; Among them, the data migration message is used to instruct the smart contract node to determine a target candidate storage node set in the distributed storage cluster according to the data migration message, determine a target storage node in the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set, and send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

[0013] According to a data migration method provided by the present invention, the sending a data migration message to a smart contract node according to the data type of the data to be migrated in the migration data information includes: When the data type of the data to be migrated is the first type, generate a first data migration request according to the migration request generation mode corresponding to the first type, and send the first data migration request to the smart contract node as the data migration message; When the data type of the data to be migrated is the second type, generate a second data migration request according to the migration request generation mode corresponding to the second type, broadcast the second data migration request to each of the storage nodes, determine a first candidate storage node set according to the identifiers in the response information returned by at least one of the storage nodes, and send the first candidate storage node set to the smart contract node as the data migration message; Among them, the response information of any storage node is obtained by the any storage node parsing the second data migration request, obtaining the exponential threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the exponential threshold.

[0014] The present invention also provides a data migration system, including: a distributed storage cluster, a smart contract node, and a blockchain, where the distributed storage cluster includes a plurality of storage nodes and a management node; The smart contract node is used to execute the data migration method described in any one of the above; The management node is used to execute the data migration method described in any one of the above.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the data migration method described in any one of the above when executing the program.

[0016] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data migration method described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the data migration method described in any one of the above is implemented.

[0018] The data migration method and system provided by the present invention, when the management node triggers the migration process, can obtain in real time, through the smart contract node, the data migration message sent by the management node according to the data type of the data to be migrated, and determine the set of target candidate storage nodes according to the node selection path corresponding to different data migration messages. And according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, dynamically determine the target storage node that can receive the data to be migrated. Finally, combining the node information of the target storage node and the identifier of the storage node to be migrated, send a permission certificate to the storage node to be migrated, thereby realizing effective data migration. During the data migration process, on the one hand, by limiting the role of the management node to the trigger of the migration process, and dispersing other data migration tasks to the smart contract node for processing, a non-centralized storage data migration is realized, enabling each storage node to complete data migration in a decentralized manner, improving the flexibility of distributed storage cluster management, and at the same time enhancing the overall utilization rate of storage resources. On the other hand, by combining smart contract technology, the security of migration authorization is effectively enhanced, ensuring the security and legality of the data migration process, preventing unauthorized data access and tampering, and further enhancing the credibility and stability of data migration. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0020] Figure 1 It is one of the flow diagrams of the data migration method provided by the present invention.

[0021] Figure 2 It is the structural schematic diagram of the data migration system provided by the present invention.

[0022] Figure 3 It is the second flow diagram of the data migration method provided by the present invention.

[0023] Figure 4 It is the third flow diagram of the data migration method provided by the present invention.

[0024] Figure 5 It is a schematic structural diagram of the electronic device provided by the present invention. Specific embodiments

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0026] With the rapid development of the digital economy, the amount of data has shown an explosive growth, especially the proportion of unstructured data has been rising continuously. In terms of application scenarios, distributed storage clusters are widely used in high-performance computing, cloud native, big data analysis, finance, medical care and other fields, and can effectively support complex mixed loads and multi-protocol interconnection. In terms of technical routes, distributed storage clusters are developing in the direction of all-flash, architecture innovation and function expansion. Therefore, how to perform effective data migration on distributed storage clusters to achieve the purpose of disaster tolerance to ensure business continuity and data availability is an important issue that the industry urgently needs to solve at present.

[0027] Currently, many distributed storage solutions provide data redundancy and cross-node data replication functions to achieve the purpose of disaster tolerance and ensure business continuity and data availability. A distributed storage cluster generally includes multiple storage nodes, which form a cluster to provide data storage services externally. Usually, the main process in a distributed storage cluster is deployed on a management node, which can centrally obtain the status of multiple storage nodes and manage their related services. Once it is predicted that any storage node fails, the management node will select a target storage node from other storage nodes to uniformly migrate the stored data of the failed storage node to the target storage node.

[0028] Although this disaster tolerance migration technology for stored data has the advantages of high availability and high reliability, in current practical applications, with the increase in the scale and complexity of distributed storage clusters, this method of centrally managing the disaster tolerance migration of distributed storage clusters through a management node is difficult to meet the needs of large-scale storage services, difficult to achieve flexible storage management and resource utilization, and prone to management rigidity. In addition, if information leakage occurs in the management node, the service security and stability of the entire distributed storage cluster will be threatened, resulting in poor flexibility and security of data migration.

[0029] In response to this, the present application provides a data migration method. In this method, each storage node in the distributed storage cluster pre-uploads its node information and migration type to the blockchain for storage, and periodically calculates the health value and sends it to the smart contract node. The management node triggers the migration process, and routes the migration request to the smart contract and the cluster broadcast respectively according to the migration data type. Candidate storage nodes are obtained through the two paths respectively. The smart contract determines the target storage node based on the migration type by obtaining the candidate storage nodes of one path and generates a permission certificate. The target storage node verifies the permission certificate and performs data migration, so that the management node only triggers the migration process, and transfers other data migration tasks to the smart contract node for decentralized processing, thereby realizing a non-centralized storage data migration, that is, each storage node completes data migration in a decentralized manner, effectively improving the flexibility of the management of the distributed storage cluster, and combining blockchain and smart contracts for data migration management, effectively enhancing the security of migration authorization. Even if the management node has a data leak, it does not affect the security authentication of the data migration process.

[0030] Figure 1 FIG. 4 is one of the schematic flowcharts of the data migration method provided by the present invention. This method is a non-centralized data migration method proposed for the disaster recovery problem of a distributed storage cluster. This method can be applied to a data migration system. The data migration system can be a computing center or other types of cloud service provider platforms, etc., and the present embodiment does not make specific limitations on this.

[0031] Figure 2 FIG. 5 is a schematic structural diagram of the data migration system provided by the present invention. As Figure 2 shown, the system architecture of the data migration system includes a distributed storage cluster, a smart contract node, and a blockchain; among them, the distributed storage cluster includes a management node and multiple storage nodes (such as storage node a, storage node b, storage node c, storage node d, etc.).

[0032] Among them, each storage node is used to store various types of data. The data here includes data in various fields, including but not limited to data in fields such as finance, medical care, and news.

[0033] The management node is only used to trigger the data migration operation, and assemble the corresponding data migration request according to the migration data type and route it to different processing directions respectively. Different from the existing centralized management node, it does not directly manage the storage data migration.

[0034] The smart contract node is used to store the health values calculated and sent by each storage node , according to the latest health value Adjust the remaining storage space capacity of each storage node and compare it with the space capacity requirement of the migrating data to determine candidate storage nodes, and query the blockchain to obtain the migration type of the candidate storage nodes, determine the target storage node, and send a corresponding permission certificate to the storage node to be migrated to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node.

[0035] The blockchain is used to store each storage node 's node identifier , node information and migration type and other information.

[0036] The execution entity of the data migration method provided in this embodiment is a smart contract node. As Figure 1 shown, the method includes step 110, step 120, and step 130.

[0037] Step 110: Obtain a data migration message sent by the management node according to the data type of the data to be migrated, and determine a set of target candidate storage nodes in the distributed storage cluster according to the data migration message.

[0038] Figure 3 is the second flowchart of the data migration method provided by the present invention. As Figure 3 shown, during the data migration process, each storage node in the distributed storage cluster queries its own total disk capacity and used capacity of each disk, calculates the average value of its own total disk capacity and used capacity of each disk, and then obtains its own storage space utilization rate. Then, each storage node sends the calculated storage space utilization rate, remaining storage space capacity, and node identifier of itself to the management node according to a preset period through the network protocol.

[0039] After the management node receives the storage space utilization rate of each storage node in the distributed storage cluster sent by each of them, it may compare the storage space utilization rate of each storage node with the utilization threshold to determine the storage nodes in the distributed storage cluster whose storage space utilization rate is greater than the utilization threshold as the storage nodes to be migrated (such as storage node a, with the identifier , for the sake of simplicity of description, it will be described by storage node a in the subsequent steps), and obtain the migration data information of the data to be migrated in storage node a. The migration data information at least includes the data type and the required storage space capacity. The utilization threshold here can be set by experience, or can be dynamically generated according to the actual utilization rate of each storage node, etc. This embodiment does not make specific limitations on this.

[0040] In addition, after obtaining the migration data information, the management node can determine based on the data types in the migration data information, and determine different node selection modes according to different data types, so as to determine the data migration messages to be sent to the smart contract nodes according to different node selection modes. The data migration messages here at least include the corresponding data migration requests generated by the management node according to the data types, or the candidate storage node set (also called the first candidate storage node set) formed by integrating the response information returned by the management node after screening the data migration requests corresponding to each storage node according to the data types by itself.

[0041] Optionally, after obtaining the data migration message sent by the management node, the smart contract node can select the corresponding node screening mode according to the data content included in the data migration message, and dynamically determine the target candidate storage node set in the distributed storage cluster according to the corresponding node screening mode, such as , where respectively represent the node identifiers corresponding to storage node b, storage node c, storage node f, storage node g, storage node l itself.

[0042] For example, when the data migration request corresponding to the data type of the data to be migrated generated by the management node is included in the data migration message, the performance parameters of each storage node in the distributed storage cluster can be calculated or adjusted according to the information included in the data migration request (such as adjusting the remaining storage space capacity, etc.), and the candidate storage nodes whose calculated or adjusted performance parameters meet the migration conditions are determined in the distributed storage cluster to form a target candidate storage node set; for another example, when the first candidate storage node set integrated by the management node is included in the data migration message, the first candidate storage node set can be directly determined as the target candidate storage node set.

[0043] Step 120: Determine the target storage node in the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set.

[0044] It should be noted that each storage node is configured with a corresponding blockchain node to store the relevant information of the storage node on the chain. Therefore, before executing step 110, each storage node in the distributed storage cluster will pre-store its own key information on the chain to ensure the security, integrity and immutability of the data, and at the same time improve the transparency and traceability of the system.

[0045] For example, each storage node can store its own node identifier , node information And the migration type Perform chain storage. Among them, the node information Is the summary data of the main information of the storage node Specifically, it is obtained by performing a hash calculation on the main information of the storage node And is subsequently used for the smart contract to generate a license certificate. The migration type Is the type of migration data that the storage node Predeterminedly prefers to receive. Subsequently, the smart contract can determine the target storage node according to the migration type. The node identifier Is the identifier that marks the unique identity of the storage node

[0046] As shown in Table 1, the migration type of the storage node Specifically may include at least one of basic type, security type, and high-access type, etc. It should be noted that the migration type here is only an example. In actual application scenarios, there may be more migration types, and each type may also include other sub-data types.

[0047] Table 1 Distribution table of migration types of storage nodes

[0048] For example, storage node a prefers to receive basic type and high-access type migration data, storage node b prefers to receive basic type migration data, and storage node c prefers to receive security type and high-access type migration data. Among them, the basic type of data is data without configured data storage security and priority access level, such as log data or file system data, etc.; the security type of data is data with configured data storage security, such as enterprise critical data such as customer information, financial records, and inventory data, as well as system configuration, network configuration, security settings, etc.; the high-access data is data that needs to be frequently accessed and quickly responded to, such as data generated by specific applications, etc. It should be noted that if the hardware of a certain storage node is updated, the corresponding changed key information, such as node identifier, node information, and migration type, etc. also needs to be stored on the chain according to the timestamp, so as to obtain the latest data of each storage node according to the timestamp subsequently.

[0049] As can be seen from the above, the key information of each storage node is pre-stored in the blockchain. Therefore, after the smart contract node obtains the target candidate storage node set, it can load the migration type of each candidate storage node in the blockchain, so as to further screen out the target storage node that can store the data to be migrated in the target candidate storage node set (such as storage node b, the identifier is , for the sake of simplicity of description, it is described by storage node b in the subsequent steps). ​

[0050] During the process of screening the target storage node, the smart contract node can directly match and select the target storage node based on the migration type of each candidate storage node and the data type of the data to be migrated; or on the basis of matching and selecting the migration type of each candidate storage node and the data type of the data to be migrated, further assisted screening can be carried out in combination with other performance indicators of each candidate storage node, such as the remaining storage space capacity, etc., to obtain the target storage node, etc. This embodiment does not make specific limitations on this.

[0051] Step 130, according to the node information of the target storage node and the identifier of the storage node to be migrated, send a permission certificate to the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node.

[0052] Optionally, after obtaining the target storage node, the smart contract node can obtain the node information of the target storage node in the blockchain, so as to generate a corresponding permission certificate in combination with the node information of the target storage node and the identifier of the storage node to be migrated. Here, the generation of the permission certificate can use the node information of the target storage node and the identifier of the storage node to be migrated as input, and automatically generate it through a pre-trained language model, or it can be generated by template filling after feature extraction of the node information of the target storage node and the identifier of the storage node to be migrated, etc. This embodiment does not make specific limitations on this. At least the node information of the target storage node and the identifier of the storage node to be migrated are included in the generated permission certificate, so as to verify the permission certificate during data migration, thereby ensuring the security, accuracy and legality of the data migration process, preventing unauthorized data access and tampering, while improving the efficiency and flexibility of data migration, reducing management complexity, and enhancing the overall security and credibility of the distributed storage cluster.

[0053] After generating the permission certificate, the smart contract node can send the permission certificate to the storage node to be migrated.

[0054] After receiving the permission certificate sent by the smart contract node by the storage node to be migrated, it can assemble the permission certificate into a data migration instruction, so as to send the data migration instruction carrying the permission certificate to the target storage node. The target storage node verifies the permission certificate in the data migration instruction, and after the verification passes, establishes a data channel to receive the data to be migrated sent by the storage node to be migrated. In addition, after successfully receiving the data to be migrated, the target storage node can also send the migration result to the management node, so that the management node can record the information related to the overall migration operation.

[0055] For the method provided in this embodiment, when the management node triggers the migration process, the data migration message sent by the management node according to the data type of the data to be migrated can be obtained in real time through the smart contract node, and the set of target candidate storage nodes can be determined according to the node selection path corresponding to different data migration messages. Then, according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, the target storage node that can receive the data to be migrated is dynamically determined. Finally, combining the node information of the target storage node and the identifier of the storage node to be migrated, a permission certificate is sent to the storage node to be migrated, thereby realizing effective data migration. During the data migration process, on the one hand, by defining the role of the management node as the trigger of the migration process and dispersing other data migration tasks to the smart contract node for processing, a non-centralized storage data migration is realized, enabling each storage node to complete data migration in a decentralized manner, improving the flexibility of distributed storage cluster management, and at the same time enhancing the utilization rate of the overall storage resources. On the other hand, by combining smart contract technology, the security of migration authorization is effectively improved, ensuring the security and legality of the data migration process, preventing unauthorized data access and tampering, and further enhancing the credibility and stability of data migration.

[0056] In some embodiments, step 110 specifically includes: When the data type of the data to be migrated is the first type, determining the data migration message as the first data migration request; the first data migration request at least includes the required storage space capacity of the data to be migrated and the remaining storage space capacities of the storage nodes in the distributed storage cluster; Searching for the latest health values of the storage nodes in the storage space of the smart contract node, and adjusting the remaining storage space capacities of the storage nodes according to the latest health values; Comparing the adjusted remaining storage space capacities of the storage nodes with the required storage space capacity of the data to be migrated, and determining the set of target candidate storage nodes in the distributed storage cluster according to the comparison result.

[0057] It should be noted that during the data migration process, different types of data to be migrated can be processed through different processing paths. This process specifically takes into account the different requirements of different types of data to be migrated. The type setting mainly considers the impact of subsequent operations. For example, basic data is mostly log data, etc., with a low access frequency and relatively low requirements for storage nodes. Subsequently, the smart contract can directly select the target storage node to receive the data. High-access type data has high requirements for storage nodes, and the situation of each storage node is changing in real time. It is necessary to judge according to the performance, load, etc. of each storage node. Therefore, when the data migration type is advanced, the migration request needs to be broadcast among all storage nodes, and each storage node judges according to its own situation and then processes it.

[0058] Therefore, as Figure 3 shown, during the process of triggering the migration process by the management node, it can be judged based on the data type of the data to be migrated. When the data type of the data to be migrated is the first type (that is, the data level is relatively low, such as basic type), since basic data is mostly log data, etc., with a low access frequency and relatively low requirements for storage nodes, the smart contract can directly select the target storage node to receive the data subsequently. Therefore, when it is determined that the data type of the data to be migrated is the first type, a first data migration request is formed according to the request assembly mode corresponding to the first type. And this request is sent to the smart contract node as a data migration message.

[0059] It should be noted that under the request assembly mode corresponding to the first type, the first data migration request includes at least the required storage space capacity of the data to be migrated and the remaining storage space capacity of each storage node in the distributed storage cluster. It can also include other information, such as the data type of the data to be migrated and the identifier of the storage node to be migrated, etc. This embodiment does not make specific limitations on this.

[0060] Optionally, when the data type of the data to be migrated is the first type, the management node sends the first data migration request to the smart contract node as a data migration message. Correspondingly, the smart contract node can determine that the received data migration message is the first data migration request.

[0061] When the smart contract node determines that the received data migration message is the first data migration request, it can look up the latest health values of each storage node in the latest records in its local storage space (Storage) (i.e., the smart contract node side), and adjust the remaining storage space capacity of each storage node based on the latest health values to ensure the accuracy of the judgment process and improve the accuracy of the smart contract node in determining candidate storage nodes. Subsequently, compare the adjusted remaining storage space capacity of each storage node with the required storage space capacity of the data to be migrated to form a target candidate storage node set. The specific implementation steps are as follows: Since the health values uploaded by each storage node are pre-stored in the Storage of the smart contract node . Therefore, when the smart contract node receives the first data migration request sent by the management node through the data migration message, it can query the latest health values of each storage node in the latest records in the Storage, and use the latest health values of each storage node as adjustment factors to adjust the remaining storage space capacity of each storage node in the first data migration request to obtain the adjusted remaining storage space capacity. For example, it can be multiplying the latest health value of each storage node by the remaining storage space capacity of each storage node to obtain the adjusted remaining storage space capacity.

[0062] The health value therein is used to represent the storage resource utilization situation of the storage node.

[0063] Taking storage node b as an example, its remaining storage space capacity before adjustment is , and its latest health value is , then the adjusted remaining storage space capacity can be specifically calculated through the following formula:[[]] .

[0064] Subsequently, the smart contract node can compare the adjusted remaining storage space capacity of each storage node with the required storage space capacity of the data to be migrated, and use the storage nodes whose adjusted remaining storage space capacity is greater than the required storage space capacity of the data to be migrated as candidate storage nodes, thereby forming a target candidate storage node set. For example, after the selection in the above steps, the target candidate storage node set corresponding to the first data migration request obtained by the smart contract node can be .

[0065] ​For the method provided in this embodiment, when the data type of the data to be migrated is basic data, the intelligent contract node can screen out a set of target candidate storage nodes that meet the conditions in the distributed storage cluster according to the first data migration request sent by the management node, realizing efficient, flexible and reliable data migration management for basic data, effectively improving the flexibility and resource utilization rate of the distributed storage system, and ensuring the security and accuracy of data migration at the same time.

[0066] In some embodiments, the latest health value of each storage node is sent to the storage space by each storage node after calculating the health value according to its own current node status information; The current node status information includes database connection rate, disk wear rate, disk depreciation rate, processor utilization rate, memory usage rate, hard disk usage rate and node importance.

[0067] Optionally, before executing step 110, each storage node can also perform preprocessing according to the following steps: Each storage node can calculate the health value in real time according to its own current node status information and send it to the intelligent contract node for storage in the Storage of the intelligent contract node according to the time stamp. Compared with the prior art where the management node usually manages the node status information of each storage node and calculates the health value, in this embodiment, considering that the management node also has the risk of downtime and from the perspective of data security, each storage node calculates this value according to its own data situation and only sends this value to the intelligent contract instead of sending the original data to the management node, which can effectively improve the security and reliability of the data.

[0068] It should be noted that the health value of the storage node can be calculated regularly after storing its node identifier , node information and migration type on the chain and sent to the intelligent contract for subsequent use by the intelligent contract to select the target storage node for receiving the migrated data in combination with the remaining resources of the real-time disk space.

[0069] The following describes the calculation and storage process of the health value with a specific example.

[0070] In the calculation process of the health value of the storage node , the utilization factor of the storage node can be calculated based on the processor (CPU) utilization rate , memory usage rate , and hard disk usage rate of the storage node ; of the storage node​ Calculations are performed, and the specific calculation formula is: ; Among them, , , are the CPU utilization rate , the memory usage rate , and the hard disk usage rate respectively. Their weight coefficients can be obtained by fitting historical data. , , are the coefficients of the interaction terms, indicating the interaction effects between different parameters, and can also be obtained by fitting historical data.

[0071] In addition, based on the database connection rate of the storage node , the disk wear rate , and the disk depreciation rate , the attenuation factor of the storage node is calculated. The specific calculation formula is: ; Among them, , and are the weight coefficients of the database connection rate , the disk wear rate , and the disk depreciation rate respectively; they can be obtained by fitting historical data. , , are the coefficients of the interaction terms, indicating the interaction effects between different parameters, and can also be obtained by fitting historical data.

[0072] Subsequently, based on the utilization factor of the storage node and the attenuation factor , as well as the node importance, the health value of the storage node is calculated. The specific calculation formula is: ; Among them, is the importance of the storage node in the distributed storage cluster to which it belongs. For example, when the importance of a certain storage node is the lowest among all storage nodes, then ; is the total number of all storage nodes in the distributed storage cluster; is the utilization factor; is the attenuation factor; is a coefficient, and its data range can be set according to actual requirements, such as any value in [1, 2].

[0073] After the operations of the pre - process, the intelligent contract node stores the latest health values calculated by each storage node according to the CPU utilization rate, memory usage rate, database connection rate, disk wear rate, etc.

[0074] The method provided in this embodiment realizes the decentralized management and real - time update of node status information by each storage node independently calculating the health value and sending it to the intelligent contract node, effectively reducing the downtime risk of the management node and potential data security hazards. At the same time, the calculation of the health value comprehensively considers the utilization factor and attenuation factor of the storage node, as well as the node importance, and can more comprehensively and accurately reflect the current state and performance of the storage node, providing a reliable basis for the intelligent contract node to select a suitable target storage node during the data migration process, thereby improving the efficiency and reliability of data migration and enhancing the overall performance and stability of the distributed storage system.

[0075] In some embodiments, step 110 further includes: When the data type of the data to be migrated is the second type, determining the data migration message as the first candidate storage node set; Determining the target candidate storage node set according to the first candidate storage node set; Among them, the first candidate storage node set is determined by the management node according to the response information returned by at least one storage node in the distributed storage cluster; The response information of any storage node is obtained by the any storage node parsing the second data migration request sent by the management node, obtaining the index threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the index threshold.

[0076] Such as Figure 3As shown, during the process of triggering the migration process, the management node can make a judgment based on the data type of the data to be migrated. When the data type of the data to be migrated is the second type (that is, the data level is relatively high, such as security type or high access), due to the high requirements for storage nodes and the real-time changes in the conditions of each storage node, it is necessary to make a judgment according to the various performances, loads, etc. of the storage nodes. Therefore, when the migration data type is the second type, the management node needs to broadcast the data migration request corresponding to the second type among each storage node, and each storage node makes a judgment according to its own situation and then processes it. Therefore, when it is determined that the data type of the data to be migrated is the second type, the management node forms a second data migration request according to the request assembly mode corresponding to the second type. And broadcast this request in the distributed storage cluster.

[0077] It should be noted that in the request assembly mode corresponding to the second type, the second data migration request includes at least the index threshold corresponding to the target performance index of each storage node and the data type of the data to be migrated, and may also include other information, such as the required storage space capacity of the data to be migrated and the identifier of the storage node to be migrated, etc. This embodiment does not make specific limitations on this.

[0078] After each storage node receives the second data migration request broadcast by the management node it can compare its own conditions according to the migration requirements. For data of the second type, it is required that the storage node has a high-performance processor, fast memory, high input / output (I / O) throughput, low-latency network connection, etc. Therefore, for each storage node, after this storage node receives the second data migration request broadcast by the management node it can calculate its own target performance index according to the data type of the data to be migrated in the second data migration request and compare its own target performance index with its own index threshold to determine whether it meets the migration requirements (that is, whether its own target performance index is consistent with its own index threshold). If it meets the migration requirements (that is, its own target performance index is consistent with its own index threshold), then this storage node returns its own node identifier to the management node as a response message. The target performance index here is an index characterizing the performance of the storage node, including but not limited to the node access index and / or the node real-time index.

[0079] After the management node receives the node identifiers returned by at least one storage node that meets the migration conditions, it can integrate the returned node identifiers to form a second data migration request The corresponding set of candidate storage nodes, that is, the first set of candidate storage nodes, and send the first set of candidate storage nodes to the smart contract node in a data migration message. Correspondingly, the smart contract node can determine that the received data migration message is the first set of candidate storage nodes.

[0080] When the smart contract node determines that the received data migration message is the first set of candidate storage nodes, it can directly determine the first set of candidate storage nodes as the target set of candidate storage nodes.

[0081] In the method provided in this embodiment, when the data type of the data to be migrated is the second type, the management node broadcasts a second data migration request, so that each storage node makes an independent judgment based on its own performance and returns response information, thereby forming a target set of candidate storage nodes. This not only improves the flexibility and accuracy of data migration, but also reduces the burden on the management node, enhances the security and stability of the system, and effectively solves the deficiencies of traditional centralized management in handling high-performance data migration.

[0082] In some embodiments, the target performance index of any storage node includes a node access index and / or a node real-time index; The node access index is calculated based on the historical access frequency data of the target historical business data of any storage node; the target historical business data is historical business data that matches the data type of the data to be migrated; The node real-time index is calculated based on the historical migration time data of the target historical business data.

[0083] Optionally, for a storage node , after it receives the second data migration request broadcast by the management node , it can specifically execute the following steps to calculate the target performance index: The storage node In its own historical data record, obtain the historical business data that matches the data type of the data to be migrated in the second data migration request as the target historical business data. For example, if the data type in the second data migration request is high access, then among the target historical business data obtained from the historical data record of the storage node , it should at least include: the access frequency data and migration time data of high-access data within a preset time range.

[0084] Immediately afterwards, the storage node calculates the node access index of the storage node according to the historical access frequency data (such as historical average access frequency and historical maximum access frequency) of the target historical business data , which reflects the access capability of the storage node to process specific types of data. The specific calculation formula is: ; Among them, is the historical average access frequency of historical business data matching the data type of the data to be migrated, is the historical maximum access frequency of historical business data matching the data type of the data to be migrated.

[0085] In addition, the storage node can also calculate the node real-time index of the storage node based on the historical migration time data of the target historical business data (such as the most recent migration rate, historical maximum migration rate, historical minimum migration rate, and real-time measurement value) , to reflect the data migration efficiency of the storage node. The specific calculation formula is: ; Among them, is the most recent migration rate of the target historical business data, is the historical maximum migration rate of the target historical business data, is the historical minimum migration rate of the target historical business data. is the real-time measurement value of the target historical business data.

[0086] When obtaining the node access index and the real-time index , the storage node can then determine whether it meets the current migration requirements based on this, thereby achieving autonomous screening of the node.

[0087] The method provided in this embodiment realizes precise evaluation and autonomous screening of the performance of the storage node by the storage node autonomously calculating the node access index and the node real-time index, not only improving the accuracy and efficiency of data migration, but also reducing the burden on the management node and enhancing the flexibility and scalability of the system.

[0088] In some embodiments, step 120 specifically includes: In the blockchain, search for the migration types of each of the candidate storage nodes; In the set of target candidate storage nodes, determine the candidate storage nodes whose migration types match the data type of the data to be migrated, and the number of the matching candidate storage nodes; In the case where the number is multiple, determine the target storage node according to the remaining storage space capacity of the multiple matching candidate storage nodes.

[0089] For example Figure 3As shown, after the smart contract node obtains the target candidate storage node set through step 110, it can access the blockchain to find the migration types of each candidate storage node, such as querying the migration type information of storage node b and storage node c. To match the migration types of each candidate storage node with the data type of the data to be migrated, so as to determine the candidate storage nodes that match the data type of the data to be migrated, and count the number of matching candidate storage nodes. If the number of matching candidate storage nodes is one, then this one matching candidate storage node is determined as the target storage node. If the number of matching candidate storage nodes is multiple, then the candidate storage node with the largest remaining storage space capacity among the multiple matching candidate storage nodes is determined as the target storage node (such as storage node )

[0090] The method provided in this embodiment, by the smart contract node finding the migration types of candidate storage nodes in the blockchain and making matches according to the data type of the data to be migrated, ensures the accuracy and pertinence of data migration. When the number of matching candidate storage nodes is multiple, the target storage node is determined by comparing the remaining storage space capacity, further optimizing the utilization of storage resources, improving the efficiency and reliability of data migration. This process makes full use of the decentralized and immutable characteristics of the blockchain, enhances the security and transparency of the system, and effectively solves the flexibility and security problems of traditional centralized management in data migration.

[0091] In some embodiments, step 130 specifically includes: In the blockchain, find the node information of the target storage node; According to the characteristic information in the node information and the identifier of the storage node to be migrated, update the license credential template to obtain the license credential, and send the license credential to the storage node to be migrated.

[0092] As Figure 3 shown, during the license credential generation process, the smart contract node can access the blockchain to find the node information of the target storage node, and extract the characteristic information from the node information, so as to update the license credential template pre-stored in the smart contract node based on the extracted characteristic information and the identifier of the storage node to be migrated, to obtain the license credential.

[0093] The method provided in this embodiment accesses the blockchain through a smart contract node to obtain the node information of the target storage node, extracts the feature information, and updates the pre-stored voucher template in combination with the identifier of the storage node to be migrated to generate a permission voucher, ensuring the security and credibility of data migration. By utilizing the immutable and decentralized characteristics of the blockchain, the authenticity and integrity of the node information are guaranteed, preventing unauthorized data access and tampering. At the same time, this process realizes the automated generation and management of permission vouchers, improving the efficiency and flexibility of data migration, reducing the burden on the management node, and enhancing the overall performance and scalability of the distributed storage system.

[0094] In some embodiments, the present application further provides a data migration method, and the execution subject of this method is a management node.

[0095] Figure 4 It is the third schematic diagram of the process of the data migration method provided by the present invention; as Figure 4 shown, this method includes step 410, step 420, and step 430.

[0096] Step 410, obtain the storage space utilization rate of each storage node in the distributed storage cluster; Step 420, determine the storage node to be migrated and the migration data information according to the storage space utilization rate; Step 430, send a data migration message to the smart contract node according to the data type of the data to be migrated in the migration data information; wherein, the data migration message is used to instruct the smart contract node to determine a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, determine a target storage node in the set of target candidate storage nodes according to the migration type of each candidate storage node in the set of target candidate storage nodes, and send a permission voucher to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

[0097] Optionally, during the data migration process, each storage node in the distributed storage cluster obtains its own total disk capacity and used capacity of each disk by querying, calculates the average value of its own total disk capacity and used capacity of each disk, and then obtains its own storage space utilization rate. Then, each storage node sends the calculated storage space utilization rate, remaining storage space capacity, and node identifier of itself to the management node through the network protocol.

[0098] The management node receives each storage node in the distributed storage cluster After sending their respective storage space utilization rates, it can be to compare the storage space utilization rates of each storage node with a utilization rate threshold to determine the storage nodes in the distributed storage cluster with a storage space utilization rate greater than the utilization rate threshold as storage nodes to be migrated (such as storage node a, identified as , for the sake of simplicity of description, it will be described by storage node a in the subsequent steps), and obtain the migration data information of the data to be migrated in storage node a. This migration data information at least includes the data type and the required storage space capacity.

[0099] After obtaining the migration data information, the management node can judge based on the data type in the migration data information to determine different node selection modes according to different data types, and thus determine the data migration messages to be sent to the smart contract nodes according to different node selection modes. Here, the data migration messages at least include the corresponding data migration requests generated by the management node according to this data type, or the candidate storage node set (also called the first candidate storage node set) formed by the management node integrating the response information returned after screening the data migration requests corresponding to this data type by each storage node on its own node.

[0100] After receiving the data migration message, the smart contract node can select the corresponding node screening mode according to the data content contained in the data migration message, and dynamically determine the target candidate storage node set in the distributed storage cluster according to the corresponding node screening mode; and after obtaining the target candidate storage node set, it can be to load the migration types of each candidate storage node in the blockchain, and further screen out the target storage nodes that can store the data to be migrated in the target candidate storage node set according to the migration types of each candidate storage node; and after obtaining the target storage node, the smart contract node can obtain the node information of the target storage node in the blockchain, and generate the corresponding permission certificate in combination with the node information of the target storage node and the identifier of the storage node to be migrated, so that the storage node to be migrated can migrate the data to be migrated to the target storage node according to the permission certificate. The specific implementation steps can be seen in steps 110-130 and will not be elaborated here.

[0101] The method provided in this embodiment can, when the management node triggers the migration process, obtain in real time through the smart contract node the data migration message sent by the management node according to the data type of the data to be migrated, determine the set of target candidate storage nodes according to the node selection path corresponding to different data migration messages, and dynamically determine the target storage node that can receive the data to be migrated according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, so as to send a permission certificate to the storage node to be migrated in combination with the node information of the target storage node and the identifier of the storage node to be migrated, thereby realizing effective data migration. During the data migration process, on the one hand, by defining the role of the management node as the trigger of the migration process and dispersing other data migration tasks to the smart contract node for processing, a non-centralized storage data migration is realized, enabling each storage node to complete the data migration in a decentralized manner, improving the flexibility of distributed storage cluster management, and at the same time enhancing the overall utilization rate of storage resources. On the other hand, by combining smart contract technology, the security of migration authorization is effectively enhanced, ensuring the security and legality of the data migration process, preventing unauthorized data access and tampering, and further enhancing the credibility and stability of data migration.

[0102] In some embodiments, step 430 specifically includes: When the data type of the data to be migrated is the first type, generate a first data migration request according to the migration request generation mode corresponding to the first type, and send the first data migration request as the data migration message to the smart contract node; When the data type of the data to be migrated is the second type, generate a second data migration request according to the migration request generation mode corresponding to the second type, broadcast the second data migration request to each of the storage nodes, determine a first candidate storage node set according to the identifiers in the response information returned by at least one of the storage nodes, and send the first candidate storage node set as the data migration message to the smart contract node; Among them, the response information of any storage node is obtained by the any storage node parsing the second data migration request, obtaining the index threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the index threshold; the data level corresponding to the first type is lower than the data level corresponding to the second type.

[0103] Such as Figure 3As shown, during the process of triggering the migration process, the management node can make a judgment based on the data type of the data to be migrated. When the data type of the data to be migrated is the first type (that is, the data level is relatively low, such as basic type), the first data migration request is formed according to the request assembly mode corresponding to the first type. And this request is sent to the smart contract node as a data migration message, so that when the smart contract node determines that the received data migration message is the first data migration request, it can find the latest health values of each storage node in the latest record in its local Storage, and adjust the remaining storage space capacity of each storage node according to the latest health values, and compare the adjusted remaining storage space capacity of each storage node with the required storage space capacity of the data to be migrated to form a set of target candidate storage nodes.

[0104] When the data type of the data to be migrated is the second type (that is, the data level is relatively high, such as security type or high access), the management node forms the second data migration request according to the request assembly mode corresponding to the second type. And this request is broadcast in the distributed storage cluster.

[0105] After each storage node receives the second data migration request broadcast by the management node It can compare its own conditions according to the migration requirements. For data of the second type, the storage node needs to have a high-performance processor, fast memory, high input / output (I / O) throughput, low-latency network connection, etc. Therefore, for each storage node, after this storage node receives the second data migration request broadcast by the management node It can calculate its own target performance index according to the data type of the data to be migrated in the second data migration request And compare its own target performance index with its own index threshold to determine whether it meets the migration requirements (that is, whether its own target performance index is consistent with its own index threshold). If it meets the migration requirements (that is, its own target performance index is consistent with its own index threshold), then this storage node returns its own node identifier to the management node as a response message.

[0106] After the management node receives the node identifiers returned by at least one storage node that meets the migration conditions, it can integrate the returned node identifiers to form a set of candidate storage nodes corresponding to the second data migration request That is, the first candidate storage node set, and send the first candidate storage node set to the smart contract node as a data migration message, so that the smart contract node can directly determine the first candidate storage node set as the target candidate storage node set.

[0107] The method provided in this embodiment, when performing migration management on a distributed storage cluster, the management node assembles migration requests according to the types of migrated data and routes them to different processing directions respectively to screen candidate storage nodes for data migration, which not only improves the flexibility of data migration, but also effectively improves resource utilization.

[0108] The data migration system provided by the present invention will be described below. The data migration system described below can be correspondingly referred to the data migration method described above.

[0109] As Figure 2 shown, the data migration system includes: a distributed storage cluster, a smart contract node, and a blockchain. The distributed storage cluster includes multiple storage nodes and a management node. Among them, the smart contract node is used to execute the data migration method provided in the above embodiments. For details, refer to steps 110-130, which will not be elaborated here. The management node is used to execute the data migration method provided in the above embodiments. For details, refer to steps 410-430, which will not be elaborated here.

[0110] The system provided in this embodiment, on the one hand, by defining the role of the management node as the trigger of the migration process and dispersing other data migration tasks to the smart contract node for processing, realizes non-centralized storage data migration, enables each storage node to complete data migration in a decentralized manner, improves the flexibility of distributed storage cluster management, and at the same time improves the utilization rate of the overall storage resources. On the other hand, by combining smart contract technology, it effectively improves the security of migration authorization, ensures the security and legality of the data migration process, prevents unauthorized data access and tampering, and further improves the credibility and stability of data migration.

[0111] The system provided by the present invention is used to execute the above method embodiments. For the specific process and detailed content, please refer to the above embodiments, which will not be elaborated here.

[0112] Figure 5 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 5As shown in the figure, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communications interface 520, and the memory 530 complete communication with each other through the communication bus 540. The processor 510 may call the logical instructions in the memory 530 to execute a data migration method, which includes: obtaining a data migration message sent by a management node according to the data type of the data to be migrated, and determining a set of target candidate storage nodes in the distributed storage cluster according to the data migration message; determining a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes; sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node, or, includes: obtaining the storage space utilization rate of each storage node in the distributed storage cluster; determining the storage node to be migrated and migration data information according to the storage space utilization rate; sending a data migration message to a smart contract node according to the data type of the data to be migrated in the migration data information; wherein, the data migration message is used to instruct the smart contract node to determine a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, determine a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, and send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

[0113] In addition, when the logical instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0114] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data migration method provided by each of the above methods. The method includes: obtaining a data migration message sent by a management node according to the data type of the data to be migrated, and determining a set of target candidate storage nodes in the distributed storage cluster according to the data migration message; determining a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes; sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node, or, it includes: obtaining the storage space utilization rate of each storage node in the distributed storage cluster; determining the storage node to be migrated and migration data information according to the storage space utilization rate; sending a data migration message to a smart contract node according to the data type of the data to be migrated in the migration data information; wherein, the data migration message is used to instruct the smart contract node to determine a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, determine a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, and send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

[0115] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a data migration method provided by the above-mentioned various methods. The method includes: obtaining a data migration message sent by a management node according to the data type of data to be migrated, and determining a set of target candidate storage nodes in a distributed storage cluster according to the data migration message; determining a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes; sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node, or, includes: obtaining the storage space utilization rate of each storage node in the distributed storage cluster; determining the storage node to be migrated and migration data information according to the storage space utilization rate; sending a data migration message to a smart contract node according to the data type of the data to be migrated in the migration data information; wherein, the data migration message is used to instruct the smart contract node to determine a set of target candidate storage nodes in the distributed storage cluster according to the data migration message, determine a target storage node in the set of target candidate storage nodes according to the migration types of the candidate storage nodes in the set of target candidate storage nodes, and send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, also by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data migration method, characterized in that, Applied to smart contract nodes, including: Obtain a data migration message sent by a management node according to the data type of data to be migrated, and determine a target candidate storage node set in the distributed storage cluster according to the data migration message; Determine a target storage node in the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set; Send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated; the permission certificate is used to authorize the storage node to be migrated to migrate the data to be migrated to the target storage node.

2. The data migration method according to claim 1, wherein The obtaining a data migration message sent by a management node according to the data type of data to be migrated, and determining a target candidate storage node set in the distributed storage cluster according to the data migration message, includes: When the data type of the data to be migrated is the first type, determine that the data migration message is a first data migration request; the first data migration request at least includes the required storage space capacity of the data to be migrated, and the remaining storage space capacities of the storage nodes in the distributed storage cluster; Search for the latest health values of the storage nodes in the storage space of the smart contract node, and adjust the remaining storage space capacities of the storage nodes according to the latest health values; Compare the adjusted remaining storage space capacities of the storage nodes with the required storage space capacity of the data to be migrated, and determine the target candidate storage node set in the distributed storage cluster according to the comparison result.

3. The data migration method according to claim 2, wherein The latest health value of each storage node is sent to the storage space by each storage node after calculating the health value according to its own current node status information; The current node status information includes database connection rate, disk wear rate, disk depreciation rate, processor utilization rate, memory usage rate, hard disk usage rate, and node importance.

4. The data migration method according to claim 1, wherein The obtaining a data migration message sent by a management node according to the data type of data to be migrated, and determining a target candidate storage node set in the distributed storage cluster according to the data migration message, includes: When the data type of the data to be migrated is the second type, determine that the data migration message is a first candidate storage node set; Determine the target candidate storage node set according to the first candidate storage node set; Wherein, the first candidate storage node set is determined by the management node according to the response information returned by at least one storage node in the distributed storage cluster; The response information of any storage node is obtained by the any storage node parsing a second data migration request sent by the management node, obtaining the index threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the index threshold.

5. The data migration method according to claim 4, wherein The target performance index of any of the storage nodes includes a node access index and / or a node real-time index; The node access index is calculated based on the historical access frequency data of the target historical service data of any of the storage nodes; the target historical service data is historical service data matching the data type of the data to be migrated; The node real-time index is calculated based on the historical migration time data of the target historical service data.

6. The data migration method according to any one of claims 1-5, characterized in that Determining a target storage node in the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set includes: In the blockchain, querying the migration types of the candidate storage nodes; In the target candidate storage node set, determining candidate storage nodes whose migration types match the data type of the data to be migrated, and the number of the matching candidate storage nodes; When the number is multiple, determining the target storage node according to the remaining storage space capacities of the multiple matching candidate storage nodes.

7. The data migration method according to any one of claims 1-5, characterized in that Sending a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated includes: In the blockchain, querying the node information of the target storage node; Updating a permission certificate template according to the feature information in the node information and the identifier of the storage node to be migrated to obtain the permission certificate, and sending the permission certificate to the storage node to be migrated.

8. A data migration method, characterized in that Applied to a management node, it includes: Obtaining the storage space utilization rates of the storage nodes in the distributed storage cluster; Determining the storage node to be migrated and the migration data information according to the storage space utilization rates; Sending a data migration message to the smart contract node according to the data type of the data to be migrated in the migration data information; Wherein, the data migration message is used to instruct the smart contract node to determine a target candidate storage node set in the distributed storage cluster according to the data migration message, determine a target storage node in the target candidate storage node set according to the migration types of the candidate storage nodes in the target candidate storage node set, and send a permission certificate to the storage node to be migrated according to the node information of the target storage node and the identifier of the storage node to be migrated.

9. The data migration method according to claim 8, wherein Sending a data migration message to the smart contract node according to the data type of the data to be migrated in the migration data information includes: When the data type of the data to be migrated is the first type, generating a first data migration request according to the migration request generation mode corresponding to the first type, and sending the first data migration request as the data migration message to the smart contract node; When the data type of the data to be migrated is the second type, generate a second data migration request according to the migration request generation mode corresponding to the second type, broadcast the second data migration request to each of the storage nodes, determine a first candidate storage node set according to the identifiers in the response information returned by at least one of the storage nodes, and send the first candidate storage node set to the smart contract node as the data migration message; Among them, the response information of any storage node is obtained by the any storage node parsing the second data migration request, obtaining the exponential threshold corresponding to the target performance index of the any storage node and the data type of the data to be migrated, calculating the target performance index of the any storage node according to the data type of the data to be migrated, and returning according to the identifier of the any storage node when it is confirmed by comparison that the target performance index of the any storage node is consistent with the exponential threshold.

10. A data migration system, characterized in that, Including: A distributed storage cluster, a smart contract node, and a blockchain, where the distributed storage cluster includes multiple storage nodes and a management node; The smart contract node is used to execute the data migration method according to any one of claims 1 to 7; The management node is used to execute the data migration method according to any one of claims 8 to 9.

11. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data migration method according to any one of claims 1 to 7, or the data migration method according to any one of claims 8 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data migration method according to any one of claims 1 to 7, or the data migration method according to any one of claims 8 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data migration method according to any one of claims 1 to 7, or the data migration method according to any one of claims 8 to 9.

Images