Data migration method, device and system

Abstract

The invention discloses a data migration method, device and system, and relates to the field of storage. The data migration method is executed by the data migration system, and the data migration system comprises a source storage cluster, a public link cluster and a target storage cluster. And the data migration system determines a target logic cluster for migrating the to-be-migrated data according to a target storage cluster corresponding to the to-be-migrated data, and migrating the to-be-migrated data to the target storage cluster by adopting the target logic cluster. The target logic cluster migrates the to-be-migrated data corresponding to the target storage cluster, and other to-be-migrated data does not need to be migrated to other storage clusters. According to the data migration method and device, the explosion radius is reduced, the number of storage clusters needing to be correspondingly migrated by the target logic cluster is reduced, the complexity of the target logic cluster is reduced, resources capable of being used for migrating the to-be-migrated data are increased, the time for migrating the to-be-migrated data is shortened, and the data migration efficiency is improved.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 22, 2023, with application number 202311572698.X and application name “A method, device and other equipment for data processing”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of storage, and in particular to a data migration method, device and system. Background Art

[0003] Data replication refers to the process of replicating data in a source storage node to a destination storage node. The source storage node and the destination storage node may refer to a storage cluster composed of multiple storage nodes. The computing node determines the transmission path between the source storage node and the destination storage node, obtains the data in the source storage node, and transmits the data to the destination storage node through the transmission path, thereby replicating the data from the source storage node to the destination storage node. The computing node completes the entire process of data replication, which is complex to implement and has low data replication efficiency. Summary of the invention

[0004] The present application provides a data migration method, device and system for solving the problems of the existing data migration system, that the entire process of completing data replication is complex to implement and the data replication efficiency is low.

[0005] In a first aspect, the present application provides a data migration system. The data migration system includes a source storage cluster, a public link cluster and a target storage cluster. The public link cluster includes multiple logical clusters. The source storage cluster is used to receive a data migration request sent by a user. The data migration request is used to indicate: the identifier of the data to be migrated and the target storage cluster. The source storage cluster is also used to: determine the target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the identifier of the target storage cluster. The source storage cluster is also used to: establish a communication connection with the target logical cluster, and send the data to be migrated to the target logical cluster. The public link cluster is used to: receive the data to be migrated based on the target logical cluster. The public link cluster is also used to: migrate the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster. The target storage cluster is used to: store the data to be migrated.

[0006] Compared with the data migration system, the computing node is used to migrate the data to be migrated to the target storage cluster. In the present application, the data migration system determines the target logical cluster for migrating the data to be migrated according to the target storage cluster corresponding to the data to be migrated, and uses the target logical cluster to migrate the data to be migrated to the target storage cluster. The target logical cluster migrates the data to be migrated corresponding to the target storage cluster, and there is no need to migrate other data to be migrated to other storage clusters. In the case that the working state of the target logical cluster is abnormal, it will not affect the data migration of other storage clusters outside the target storage cluster, reducing the explosion radius (i.e., the influence range of the target logical cluster). And the number of storage clusters that need to be migrated corresponding to the target logical cluster is reduced, the complexity of the target logical cluster is reduced, so that the resources available for migrating the data to be migrated are increased, the time for migrating the data to be migrated is shortened, and the data migration efficiency is improved.

[0007] In a possible implementation, the public link cluster also includes a control cluster. The control cluster stores a transmission path relationship graph, and the transmission path relationship graph includes: a correspondence between a logical cluster and a target storage cluster. The source storage cluster is specifically used to: obtain the transmission path relationship graph from the control cluster, and determine a target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the transmission path relationship graph. In this way, the public link cluster can determine the target logical cluster corresponding to the target storage cluster according to the correspondence indicated by the transmission path relationship graph. The method for determining the target logical cluster is simple, which shortens the time required to determine the target logical cluster and improves the efficiency of data migration.

[0008] In another possible implementation, before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster is also used to: adjust the data transmission capacity of the transmission path for migrating the data to be migrated according to the business type of the data to be migrated, so that the data transmission capacity matches the business type of the data to be migrated. The data transmission capacity includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay. In this way, the public link cluster can migrate the data to be migrated using transmission paths with different data transmission capacities according to the business type of the data to be migrated, so that the data transmission capacity of the transmission path for migrating the data to be migrated matches the business type of the data to be migrated, thereby reducing costs.

[0009] In another possible implementation, before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster is also used to: determine the data migration capacity of the target logical cluster based on the control cluster. If the data migration capacity of the target logical cluster is less than a preset threshold, the public link cluster adjusts the data migration capacity of the target logical cluster. In this way, the public link cluster adjusts the data migration capacity of the target logical cluster according to the capacity threshold, so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the capacity threshold, so that the target logical cluster can use more resources to migrate the data to be migrated, shorten the time to migrate the data to be migrated, and improve the efficiency of migrating the data to be migrated.

[0010] In another possible implementation, the public link cluster is specifically used to: add at least one logical node in the target logical cluster so that the data migration capacity of the adjusted target logical cluster is greater than or equal to a preset threshold. In this way, the logical nodes included in each logical cluster in the data migration system can be flexibly adjusted according to actual migration requirements to meet different migration requirements.

[0011] In another possible implementation, the control cluster further stores a node view. The node view is used to indicate the logical node information included in the target logical cluster. After at least one logical node is added to the target logical cluster, the public link cluster is further used to: update the node view according to the logical node information after the at least one logical node is added. In this way, after adjusting the logical nodes included in the target logical cluster, the public link updates the node view according to the logical nodes included in the adjusted target logical cluster to obtain an updated node view. The accuracy of the data migration capability of the target logical cluster detected by the public link cluster according to the node view can be improved, and the efficiency of data migration can be improved.

[0012] In a second aspect, the present application provides a data migration method. The method is applied to a public link cluster. The public link cluster includes multiple logical clusters. The method includes: the public link cluster receives the data to be migrated sent by the source storage cluster based on the target logical cluster. The target logical cluster is determined by the source storage cluster from multiple logical clusters according to the identifier of the target storage cluster. The public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

[0013] In a possible implementation, before the public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster further adjusts the data transmission capacity of the transmission path for migrating the data to be migrated according to the service type of the data to be migrated, so that the data transmission capacity matches the service type of the data to be migrated. The data transmission capacity includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay.

[0014] In another possible implementation, the public link cluster further includes a control cluster. Before the public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster further determines the data migration capability of the target logical cluster based on the control cluster. If the data migration capability of the target logical cluster is less than a preset threshold, the public link cluster adjusts the data migration capability of the target logical cluster.

[0015] In another possible implementation, the public link cluster adjusts the data migration capability of the target logical cluster, including: adding at least one logical node in the target logical cluster, so that the adjusted data migration capability of the target logical cluster is greater than or equal to a preset threshold.

[0016] In another possible implementation, the control cluster stores a node view. The node view is used to indicate the logical node information included in the target logical cluster. After at least one logical node is added to the target logical cluster, the control cluster updates the node view according to the logical node information after the at least one logical node is added.

[0017] In a third aspect, the present application provides a data migration method. The method is applied to a source storage cluster, and the method includes: the source storage cluster receives a data migration request sent by a user. The data migration request is used to indicate the identifier of the data to be migrated and the target storage cluster. The source storage cluster determines the target logical cluster corresponding to the target storage cluster from the multiple logical clusters included in the public link cluster according to the identifier of the target storage cluster. And the source storage cluster establishes a communication connection with the target logical cluster, and sends the data to be migrated to the target logical cluster, so that the target logical cluster migrates the data to be migrated to the target storage cluster.

[0018] In a possible implementation, the public link cluster further includes a control cluster. The control cluster stores a transmission path relationship graph, which includes a correspondence between logical clusters and target storage clusters. The source storage cluster obtains the transmission path relationship graph from the control cluster, and determines a target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the transmission path relationship graph.

[0019] In a fourth aspect, the present application provides a data migration method. The method is applied to a data migration system. The data migration system includes a source storage cluster, a public link cluster and a target storage cluster, and the public link cluster includes multiple logical clusters. The method includes: the source storage cluster receives a data migration request sent by a user. The data migration request is used to indicate the identifier of the data to be migrated and the target storage cluster. The source storage cluster determines the target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the identifier of the target storage cluster. The source storage cluster establishes a communication connection with the target logical cluster and sends the data to be migrated to the target logical cluster. The public link cluster receives the data to be migrated based on the target logical cluster. The public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster. The target storage cluster stores the data to be migrated.

[0020] In a fifth aspect, the present application provides a data migration device. The device includes: a transceiver module and a migration module. The transceiver module is used to: receive the data to be migrated sent by the source storage cluster based on the target logical cluster. The target logical cluster is determined by the source storage cluster from multiple logical clusters according to the identifier of the target storage cluster. The migration module is used to: migrate the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

[0021] In a sixth aspect, the present application provides a data migration device. The device includes: a transceiver module and a processing module. The transceiver module is used to: receive a data migration request sent by a user. The data migration request is used to indicate the identifier of the data to be migrated and the target storage cluster. The processing module is used to: determine, according to the identifier of the target storage cluster, a target logical cluster corresponding to the target storage cluster from multiple logical clusters included in the public link cluster. The transceiver module is also used to: establish a communication connection with the target logical cluster, and send the data to be migrated to the target logical cluster, so that the target logical cluster migrates the data to be migrated to the target storage cluster.

[0022] In a seventh aspect, the present application provides a chip. The chip includes: a processor and a power supply circuit; the power supply circuit is used to supply power to the processor, the processor is used to execute the method in the second aspect or any possible implementation of the second aspect, or the processor is used to execute the method in any possible implementation of the third aspect, or the processor is used to execute the method in the fourth aspect.

[0023] In an eighth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, the computing device includes a memory and a processor, the memory is used to store computer instructions; when the processor executes the computer instructions, the method in the second aspect or any possible implementation of the second aspect is implemented, or, when the processor executes the computer instructions, the method in the third aspect is implemented, or, when the processor executes the computer instructions, the method in the fourth aspect is implemented.

[0024] In a ninth aspect, the present application provides a computer-readable storage medium. The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a processing device, the method in the second aspect or any possible implementation of the second aspect is implemented, or when the computer program or instruction is executed by a processing device, the method in the third aspect is implemented, or when the computer program or instruction is executed by a processing device, the method in the fourth aspect is implemented.

[0025] In a tenth aspect, the present application provides a computer program product. The computer program product includes a computer program or an instruction, and when the computer program or the instruction is executed by a processing device, the method in the second aspect or any possible implementation of the second aspect is implemented, or, when the computer program or the instruction is executed by a processing device, the method in the third aspect is implemented, or, when the computer program or the instruction is executed by a processing device, the method in the fourth aspect is implemented.

[0026] The beneficial effects of the second to tenth aspects above can be referred to the description of the first aspect or any implementation method of the first aspect, and will not be elaborated here.

[0027] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The figure is a flowchart of data migration;

[0029] Figure 2 A schematic diagram of the architecture of a data migration system provided for this application;

[0030] Figure 3 A flowchart of a data migration method provided for this application;

[0031] Figure 4 An example diagram of data migration provided for this application;

[0032] Figure 5 A structural example diagram of a data migration system provided in this application;

[0033] Figure 6 A schematic diagram of the structure of the first data migration device provided in this application;

[0034] Figure 7 A schematic diagram of the structure of a second data migration device provided by the present application;

[0035] Figure 8 A schematic diagram of the structure of a computing device provided for this application;

[0036] Fig. 9 A schematic diagram of the structure of a computer cluster provided for this application;

[0037] Fig.10 A schematic diagram of the connection between computing devices provided in this application. DETAILED DESCRIPTION

[0038] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given.

[0039] A cloud computing platform refers to a platform that provides computing resources and services through the Internet. Based on virtualization technology, the cloud computing platform abstracts and aggregates the computing, storage, network and other resources provided by the underlying physical hardware and infrastructure, allowing users to use these resources on demand without having to worry about the underlying physical hardware and infrastructure. The cloud computing platform may include multiple nodes, which may be computing nodes for implementing computing functions, storage nodes for storing data, and network nodes for implementing data interaction functions. In the embodiments of this application, the cloud computing platform is referred to as the cloud platform, and the processing operations performed using the cloud computing platform may be referred to as cloud computing.

[0040] In cloud computing, a region refers to the geographical area where a cloud service provider divides its cloud platform. Each region is usually composed of one or more data centers (DCs), which can include multiple nodes and can provide various cloud services, such as computing, storage, database, etc. Different data centers can provide different cloud services (or cloud platform services), or they can provide the same cloud services without restriction.

[0041] As more and more businesses are deployed on the cloud computing platform, the corresponding storage data is increasing. In order to avoid the storage performance degradation or data explosion of the storage nodes under the cloud computing platform due to excessive storage of data, it is necessary to migrate the storage data in the storage nodes, such as migrating the storage data from storage node a (or source storage node) to storage node b (or target storage node). The source storage node and the target storage node can refer to a storage cluster composed of multiple storage nodes.

[0042] Figure 1 A flowchart of data migration is shown in FIG. Figure 1 As shown, the computing node 110 transmits the data stored in the storage cluster 120 (or source storage cluster) to the storage cluster 130 (or target storage cluster) via a transmission path.

[0043] The following provides a specific example of data migration. The data migration process includes the following ① to ⑤.

[0044] ① The storage cluster 120 receives a data migration instruction initiated by a user.

[0045] The data migration instruction includes the data to be migrated and an identifier of a target storage cluster. The identifier may be a name of the target storage cluster or an ID of the target storage cluster.

[0046] ② The computing node 110 obtains a data migration instruction, and determines a transmission path for migrating the data to be migrated according to the data migration instruction.

[0047] ③ The computing node 110 obtains the data to be migrated from the storage cluster 120.

[0048] ④ The computing node 110 sends the data to be migrated to the target storage cluster 130 through the transmission path.

[0049] ⑤ The target storage cluster 130 receives and stores the data to be migrated.

[0050] The computing node determines the transmission path between the source storage cluster and the target storage cluster for the data to be migrated, obtains the data in the source storage cluster, and transmits the data to the target storage cluster through the transmission path. The computing node completes the entire process of data replication, which is complex to implement and has low data replication efficiency.

[0051] To this end, the present application provides a data migration system, which is used to migrate data in a source storage cluster to a target storage cluster to meet disaster recovery, backup, migration, replication and other requirements for the data stored in the source storage cluster.

[0052] Figure 2 A schematic diagram of the architecture of a data migration system provided for this application is shown in FIG. Figure 2 As shown, the data migration system 200 includes: a source storage cluster 210, a public link cluster 220 and a target storage cluster 230. The public link cluster 220 includes multiple logical clusters. The source storage cluster 210, the public link cluster 220 and the target storage cluster 230 can communicate in a wired manner or in a wireless manner. In some possible examples, the public link cluster can also be called a data link cluster (databus).

[0053] It should be understood that the functions of the cluster in the present application can be implemented by the nodes included in the cluster. For example, the source storage cluster receiving the data migration request sent by the user can be replaced by the description that the node in the cluster receives the data migration request sent by the user, etc., which is not limited and is uniformly described here.

[0054] The source storage cluster 210 and the target storage cluster 230 can be deployed in different data centers in the same area, in the same data center in the same area, or in different data centers in different areas. The present application does not limit the location of the storage cluster deployment. The source storage cluster 210 and the target storage cluster 230 can respectively include a variety of storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, a memory card, a magnetic tape, etc. According to the needs of actual applications, the source storage cluster 210 and the target storage cluster 230 can be used to store different data, such as the source storage cluster 210 and the target storage cluster 230 can be used to store data generated or required by cloud services.

[0055] The public link cluster 220 can use the logical cluster to migrate data in the source storage cluster 210 to the target storage cluster 230. The logical cluster can include at least one logical node. The logical node can be a device with data forwarding function, such as a router, a switch, etc.

[0056] In a possible example, the public link cluster 220 may further include a control cluster, which may include at least one control node. The public link cluster 220 may use the control cluster to perform at least one of compression, encryption, and other operations on the data to be migrated.

[0057] The wired communication may be: Ethernet, optical fiber, and various peripheral component interconnect express (PCIe) buses provided inside the data migration system for connecting the control node 210 and the working node cluster 220.

[0058] The wireless communication may be: Internet, wireless fidelity (WIFI) and ultra wide band (UWB) technology.

[0059] In a possible example, the source storage cluster 210 may be mounted on a virtual machine corresponding to the cloud service, and the input / output (I / O) load of the cloud service is on the source storage cluster 210 .

[0060] In a possible embodiment, the source storage cluster 210 and the target storage cluster 230 may be set in the data migration system 200, or may be set independently from the data migration system 200 and communicate only with the data migration system 200 to migrate data. The present application does not limit the setting method between the data migration system 200 and the source storage cluster 210 and the target storage cluster 230.

[0061] The above embodiments are merely examples and should not be construed as limiting the present application.

[0062] In a possible example, the data migration system 200 may further include a terminal 240 , a terminal 250 , and a terminal 260 .

[0063] Terminal 240, terminal 250, and terminal 260 can be a mobile phone, a tablet computer, a handheld computer, a personal computer (PC), a cellular phone, a personal digital assistant (PDA), a wearable device (such as a smart watch), a smart home device (such as a television), a car computer, a game console, and an augmented reality (AR) / virtual reality (VR) device, etc. This application does not impose any special restrictions on the specific device form of terminal 240, terminal 250, and terminal 260.

[0064] In another possible example, the terminal 240 , the terminal 250 , and the terminal 260 are independently configured and are communicatively connected to the data migration system 200 .

[0065] It should be noted that Figure 2 The illustrated data migration system architecture is only an example, and the device type or number of devices in the system can be configured according to actual needs, and the embodiments of the present application are not limited to this. For example, the data migration system may also include more terminals, source storage clusters, public link clusters, and target storage clusters.

[0066] The present application also provides a data migration method, which can be applied to Figure 2 The data migration system shown. According to the needs of actual applications, the data migration system can migrate the data in the source storage cluster to the target storage cluster according to the data disaster recovery, data replication, data backup and other requirements of the source storage cluster. The implementation method of the data migration method provided in the embodiment of the present application will be described in detail below in conjunction with the accompanying drawings.

[0067] Figure 3 A data migration method provided in this application is a flowchart, which can be applied to Figure 2The data migration system shown in the figure. The method includes the following steps S310 to S360.

[0068] Step S310: The source storage cluster receives a data migration request sent by a user.

[0069] The data migration request is used to indicate the data to be migrated and the identifier of the target storage cluster. The identifier of the target storage cluster may be the name of the target storage cluster or the ID of the target storage cluster.

[0070] In a possible scenario, the source storage cluster may receive a trigger operation initiated by a user through a user display interface, and in response to the trigger operation, obtain the user's data migration request. In a possible scenario, the data migration request includes an identifier of the data to be migrated. The identifier of the data to be migrated may refer to the name of the data to be migrated, the storage location of the data to be migrated, and the like. The name of the data to be migrated may refer to the name of an object bucket storing the data to be migrated, the name of a storage block, the name of a file, and the like. The storage location of the data to be migrated may include, but is not limited to: a sector, a storage block, an object bucket, a file, and the like where the data to be migrated is located.

[0071] In a possible scenario, the source storage cluster may further include a control cluster. The control cluster may include a control node. In this scenario, the control node in the source storage cluster may receive a data migration request sent by a user. For example, the control node in the source storage cluster receives a trigger operation from a user, and in response to the trigger operation, obtains the data migration request sent by the user.

[0072] Step S320: The source storage cluster determines a target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the identifier of the target storage cluster.

[0073] Each of the multiple logical clusters included in the public link cluster can be used to migrate the data to be migrated to one or more target storage clusters, and at least one of the multiple logical clusters included in the public link cluster can also be used to migrate the data to be migrated to a target storage cluster. This application is not limited to this. According to the needs of actual applications, each of the multiple logical clusters included in the public link cluster can be used to migrate the data to be migrated to a target storage cluster. The following takes the example that each of the multiple logical clusters included in the public link cluster can be used to migrate the data to be migrated to a target storage cluster to illustrate the data migration method provided by the application.

[0074] In a possible scenario, the public link cluster also includes a control cluster, the control cluster stores a transmission path relationship graph, and the transmission path relationship graph includes a correspondence between a logical cluster and a target storage cluster. Before step S330, the source storage cluster may also perform step S330A to determine a target logical cluster for migrating the data to be migrated. Step S330A may include the following steps S31 and S32.

[0075] Step S31: The source storage cluster obtains a transmission path relationship diagram from the control cluster.

[0076] In some possible scenarios, the common link cluster stores a transmission path relationship graph. The transmission path relationship graph includes the correspondence between the logical cluster, the target storage cluster, and the transmission path. The transmission path relationship graph may store the correspondence between the identifier of the logical cluster, the identifier of the target storage cluster, and the identifier of the transmission path, and reflect the correspondence between the logical cluster, the target storage cluster, and the transmission path according to the correspondence between the above identifiers. In some possible examples, the transmission path relationship graph may also be referred to as a replication path relationship graph, a path relationship graph, a migration path relationship graph, and the like. The transmission path may also be referred to as a replication path, a migration path, and the like. Table 1 is a possible example of a transmission path relationship graph.

[0077] Table 1 Transmission path relationship diagram Possible examples

[0078] Logical cluster Target storage cluster Transmission Path The ID of the logical cluster The ID of the target storage cluster Identification of the transmission path

[0079] In this case, after obtaining the data migration request, the source storage cluster sends a transmission path relationship graph acquisition request to the public link cluster, and receives the transmission path relationship graph sent by the public link cluster based on the transmission path relationship graph acquisition request.

[0080] In some possible situations, the public link cluster may include a control cluster, and the control cluster includes a control node. In this situation, the control node uses its own storage resources to store the transmission path relationship graph. And when the control node receives a transmission path relationship graph acquisition request from a source storage cluster, it sends the transmission path relationship graph to the source storage cluster.

[0081] It is given above that the control cluster included in the public link cluster stores a transmission path relationship graph. In some possible examples, the source storage cluster may store a transmission path relationship graph. The transmission path relationship graph may be sent to the source storage cluster by the user when initiating a data migration request, or it may be pre-stored in the source storage cluster, and this application is not limited to this. In the case where the source storage cluster stores the transmission path relationship graph, the source storage cluster determines the target logical cluster for receiving the data to be migrated and the transmission path for transmitting the data to be migrated according to the target storage cluster indicated by the data migration request.

[0082] Step S32: The source storage cluster determines a target logical cluster corresponding to the target storage cluster from the multiple logical clusters according to the transmission path relationship diagram.

[0083] The source storage cluster determines the target storage cluster for receiving the data to be migrated according to the identifier of the target storage cluster indicated by the migration request. The source storage cluster determines the target logical cluster for migrating the data to be migrated and the transmission path for transmitting the data to be migrated according to the corresponding relationship between the target storage cluster and the logical cluster reflected in the transmission path relationship diagram and the target storage cluster for receiving the data to be migrated.

[0084] In some possible situations, the source storage cluster may use a control node included in the source storage cluster to execute the above process to determine a target logical cluster for migrating the data to be migrated and a transmission path for transmitting the data to be migrated.

[0085] Step S330: the source storage cluster establishes a communication connection with the target logical cluster, and sends the data to be migrated to the target logical cluster.

[0086] In one possible scenario, the source storage cluster can establish a communication connection with the target logical cluster using a private network (Virtual Private Cloud, VPC), and send the data to be migrated to the target logical cluster based on the VPC. The VPC can be wired or wireless. For details about the wired and wireless methods, please refer to the relevant description above, which will not be repeated here.

[0087] Step S340: the public link cluster receives the data to be migrated based on the target logical cluster.

[0088] The public link cluster uses the target logical cluster to receive the data to be migrated. The logical cluster may include at least one logical node. In this case, the public link cluster uses one or more logical nodes in the target logical cluster to receive the data to be migrated.

[0089] In a possible scenario, after the logical nodes included in the logical cluster receive the data to be migrated, the logical nodes included in the logical cluster may also use their own computing resources, storage resources, etc. to perform at least one of compression, encryption, etc. on the migrated data to obtain the processed data to be migrated. According to the needs of the actual application, the logical cluster may use the logical node that receives the data to be migrated to perform at least one of compression, encryption, etc. on the migrated data, or may use other logical nodes to perform at least one of compression, encryption, etc. on the migrated data, which is not limited in this application.

[0090] Step S350: The public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

[0091] In one possible scenario, before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster may also perform step S350A to adjust the data transmission capacity of the transmission path for migrating the data to be migrated. Step S350A specifically includes: the public link cluster adjusts the data transmission capacity of the transmission path for migrating the data to be migrated according to the business type of the data to be migrated, so that the data transmission capacity matches the business type of the data to be migrated. Among them, the data transmission capacity includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay. In this way, the public link cluster can use transmission paths with different data transmission capacities to migrate the data to be migrated according to the business type of the data to be migrated, so that the data transmission capacity of the transmission path for migrating the data to be migrated matches the business type of the data to be migrated, thereby reducing costs.

[0092] The public link cluster may use the network type of the transmission path to indicate the data transmission capacity of the transmission path. In this case, the public link cluster may adjust the data transmission capacity of the transmission path by adjusting the network type of the transmission path. The network type of the transmission path may include, but is not limited to: platinum, gold, silver, etc. The data transmission capacity of a transmission path with a network type of platinum is higher than the data transmission capacity of a transmission path with a network type of gold, and the data transmission capacity of a transmission path with a network type of gold is higher than the data transmission capacity of a transmission path with a network type of silver.

[0093] The public link cluster can determine the network type of the transmission path for migrating the data to be migrated according to the business type of the data to be migrated, and determine the data transmission capacity of the transmission path by determining the network type of the transmission path. The public link cluster can determine the business type of the data to be migrated according to the service level agreement (SLA) signed between the user who initiated the data migration request and the cloud service provider. If the level of the SLA signed by the user is higher, the public link cluster uses a transmission path with a platinum network type indicating a higher data transmission capacity to migrate the data to be migrated.

[0094] In a possible scenario, before the public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, it can also perform step S350B to adjust the data migration capacity of the target logical cluster so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the threshold. In this way, the public link cluster adjusts the data migration capacity of the target logical cluster according to the capacity threshold so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the capacity threshold, so that the target logical cluster can use more resources to migrate the data to be migrated, shorten the time to migrate the data to be migrated, and improve the efficiency of migrating the data to be migrated. Step S350B may include the following steps S51 and S52.

[0095] Step S51 : before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the public link cluster determines the data migration capability of the target logical cluster based on the control cluster.

[0096] The data migration capability of the target logical cluster may refer to the available bandwidth of the target logical cluster, the number of input / output operations per second (iops), the amount of data per atomic operation, etc. The capability threshold may refer to: a preset fixed value, a percentage of the data migration capability of the target logical cluster, etc.

[0097] The public link cluster can determine the data migration capability of the target logical cluster by the situation of the logical nodes included in the target logical cluster. The control cluster included in the public link cluster also stores a node view. The node view is used to indicate the logical node information included in the target logical cluster. In this case, the public link cluster can determine the data migration capability of the target logical cluster by the node view stored in the control cluster.

[0098] The node view includes the working status of each logical node in all logical nodes in the target logical cluster and the data migration capability information of each logical node in all logical nodes. Table 2 is an example of a node view.

[0099] Table 2

[0100]

[0101] The public link cluster can determine the data migration capability of the target logical cluster based on the logical nodes in the target logical cluster that are in normal working status. For example, the public link cluster detects the working status of each logical node in the target logical cluster, obtains logical node 1 and logical node 2 that are in normal working status, and determines the data migration capability of the target logical cluster based on the data migration capability information of logical node 1 and logical node 2. In some possible examples, the public link cluster can use the control cluster to obtain the data migration capability of the target logical cluster. In this case, the control cluster can use the method used by the public link cluster to obtain the data migration capability of the target logical cluster. For related content, please refer to the description above, which will not be repeated here.

[0102] Step S52: If the data migration capability of the target logical cluster is less than a preset threshold, the public link cluster adjusts the data migration capability of the target logical cluster.

[0103] Exemplarily, the total bandwidth of the target logical cluster is 300 megabytes per second (MB / S), and the available bandwidth of the target logical cluster is 100 MB / S. The capacity threshold is 80% of the data migration capacity of the target logical cluster. In this case, the capacity threshold of the target logical cluster is 240 MB / S (i.e., 300 MB / S*80%=240 MB / S). Since the available bandwidth of the target logical cluster of 100 MB / S is less than the capacity threshold of 240 MB / S, in this case, the public link cluster adds at least one logical node in the target logical cluster to adjust the available bandwidth of the target logical cluster so that the adjusted available bandwidth of the target logical cluster is greater than or equal to 240 MB / S.

[0104] The public link cluster may use multiple methods to adjust the data migration capability of the target logical cluster so that the adjusted data migration capability of the target logical cluster is greater than or equal to a preset threshold. Two possible examples are given below.

[0105] Example 1: The public link cluster may add at least one logical node in the target logical cluster, so that the adjusted data migration capability of the target logical cluster is greater than or equal to a preset threshold.

[0106] The at least one logical node may all come from other logical clusters included in the common link cluster, or all may be newly added, or partly come from other logical clusters included in the common link cluster and partly are newly added, which is not limited in this application. In the case where at least one logical node all comes from other logical clusters included in the common link cluster, the at least one logical node may come from the same logical cluster in the common link cluster, or from different logical clusters in the common link cluster, which is not limited in this application. In this way, the logical nodes included in each logical cluster in the data migration system can be flexibly adjusted according to the actual migration requirements to meet different migration requirements.

[0107] Example 2: The public link cluster adjusts the working state of each logical node in the target logical cluster to adjust the data migration capability of the target logical cluster, so that the adjusted data migration capability of the target logical cluster is greater than or equal to a preset threshold.

[0108] The public link cluster detects the working status of all logical nodes in the target logical cluster, suspends the migration of data with a lower priority than the data to be migrated, and releases the resources occupied by other data with a lower priority than the data to be migrated, so as to improve the data migration capacity of the target logical cluster to migrate the data to be migrated. The public link cluster detects the data migration capacity of the target logical cluster after releasing the resources occupied by other data. If the data migration capacity of the target logical cluster after releasing the resources occupied by other data is greater than or equal to the capacity threshold, the target logical cluster is used to migrate the data to be migrated. If the data migration capacity of the target logical cluster after releasing the resources occupied by other data is less than the capacity threshold, the public link cluster can use the method given in Example 1 to adjust the data migration capacity of the target logical cluster so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the capacity threshold.

[0109] In a possible scenario, after at least one logical node is added to the target logical cluster, the public link cluster updates the node view according to the logical node information after the at least one logical node is added. The updated node view indicates that the target logical cluster includes the at least one logical node. In this way, after adjusting the logical nodes included in the target logical cluster, the public link updates the node view according to the logical nodes included in the adjusted target logical cluster to obtain an updated node view. The accuracy of the data migration capability of the target logical cluster detected by the public link cluster according to the node view can be improved, and the efficiency of data migration can be improved.

[0110] Step S360: The target storage cluster stores the data to be migrated.

[0111] According to actual application needs, the public link cluster may use one logical cluster to migrate the data to be migrated from the source storage cluster to the target storage cluster, or may use multiple logical clusters to migrate the data to be migrated from the source storage cluster to the target storage cluster.

[0112] For example, when the source storage cluster and the target storage cluster are deployed in the same data center, the public link cluster can use the same logical cluster to migrate the data to be migrated from the source storage cluster to the target storage cluster. For another example, when the source storage cluster and the target storage cluster are deployed in different data centers in the same area, the public link cluster can use the same logical cluster to migrate the data to be migrated from the source storage cluster to the target storage cluster, or use different logical clusters to migrate the data to be migrated from the source storage cluster to the target storage cluster. For another example, when the source storage cluster and the target storage cluster are deployed in different data centers in different areas, the public link cluster can use different logical clusters to migrate the data to be migrated from the source storage cluster to the target storage cluster.

[0113] When the public link cluster uses different logical clusters to migrate the data to be migrated from the source storage cluster to the target storage cluster, the target storage cluster can receive the data to be migrated through the logical cluster corresponding to the source storage cluster that sends the data to be migrated, and the logical cluster sends the data to be migrated to the target storage cluster. And the target storage cluster receives and stores the data to be migrated. The target storage cluster can use object buckets, storage blocks, files, etc. to store the data to be migrated, which is not limited in this application, and according to the needs of actual applications, the target storage cluster can store the data to be migrated to one or more storage nodes among the multiple storage nodes included in the target storage cluster.

[0114] The above text takes the target logical cluster of the public link cluster migrating the data to be migrated from the source storage cluster to the target storage cluster as an example to illustrate the data migration method provided by the present application. In some possible examples, the data migration system can migrate the data in the source storage cluster to multiple target storage clusters according to the data migration method described above. In this case, the source storage cluster receives multiple data migration requests sent by the user. Each data migration request includes the identifier of the data to be migrated and the target storage cluster. And the data migration system uses the data migration method described above to process each data migration request in the multiple data migration requests.

[0115] The following takes the case where the source storage cluster receives two data migration requests sent by the user as an example to illustrate the process of the data migration system migrating data in the source storage cluster to the target storage cluster. The source storage cluster receives data migration request 1 and data migration request 1. The data migration request includes the data to be migrated 1 and the identifier of the target storage cluster 1. The data migration request 2 includes the data to be migrated 2 and the identifier of the target storage cluster 2. The source storage cluster determines the first logical cluster 1 corresponding to the target storage cluster 1 from the multiple logical clusters included in the public link cluster according to the identifier of the target storage cluster 1, and determines the first logical cluster 2 corresponding to the target storage cluster 2 from the multiple logical clusters included in the public link cluster according to the identifier of the target storage cluster 2. The source storage cluster sends the data to be migrated 1 to the logical cluster 1, and sends the data to be migrated 2 to the logical cluster 2. The public link cluster uses the logical cluster 1 to send the data to be migrated 1 to the target storage cluster 1, and uses the logical cluster 2 to send the data to be migrated 2 to the target storage cluster 2.

[0116] Combination of the above Figures 1 to 3 The data migration method provided by the present application is described. The following takes the data migration system completing data migration between three storage clusters deployed in different areas as an example to illustrate the process of data migration between different storage clusters. Figure 4 An example diagram of data migration provided for this application, such as Figure 4 As shown, storage cluster 1 is deployed in region 1, storage cluster 2 is deployed in region 2, and storage cluster 3 is deployed in region 3. Storage cluster 1 stores data 1 to data 3 to be migrated. Data 1 to be migrated can be referred to as data 1, data 2 to be migrated can be referred to as data 2, and data 3 to be migrated can be referred to as data 3. Among them, data 1 needs to be migrated to storage cluster 2, and after data 1 is migrated to storage cluster 2, it also needs to be migrated from storage cluster 2 to storage cluster 3. Data 2 and data 3 need to be migrated to storage cluster 3. The data migration system can use the following process to complete the above migration.

[0117] The public link cluster includes a logical cluster 11, a logical cluster 12, a logical cluster 21, a logical cluster 22, a logical cluster 31, and a logical cluster 32. The logical cluster 11 and the logical cluster 21 are used to implement the data migration process between the storage cluster 1 and the storage cluster 2, the logical cluster 12 and the logical cluster 32 are used to implement the data migration process between the storage cluster 1 and the storage cluster 3, and the logical cluster 22 and the logical cluster 31 are used to implement the data migration process between the storage cluster 1 and the storage cluster 3.

[0118] (1) Migrate data 1 in storage cluster 1 to storage cluster 2.

[0119] Storage cluster 1 establishes a connection with logical cluster 11, and sends data 1 to logical cluster 11. Logical cluster 11 receives data 1, and transmits data 1 to logical cluster 21 via a transmission path whose network type is gold. Logical cluster 21 receives data 1, and sends data 1 to storage cluster 2. Storage cluster 2 receives and stores data 1.

[0120] (2) Data 1 in storage cluster 2 is transferred to storage cluster 3.

[0121] Storage cluster 2 establishes a connection with logical cluster 22, and sends data 1 to logical cluster 22. Logical cluster 22 receives data 1, and transmits data 1 to logical cluster 31 via a transmission path whose network type is platinum. Logical cluster 31 receives data 1, and sends data 1 to storage cluster 3. Storage cluster 3 receives and stores data 1.

[0122] (3) Migrate data 2 and data 3 in storage cluster 1 to storage cluster 3.

[0123] Storage cluster 1 establishes a connection with logical cluster 12, and sends data 2 and data 3 to logical cluster 12. Logical cluster 12 receives data 2 and data 3, and transmits data 2 and data 3 to logical cluster 32 through a transmission path whose network type is silver. Logical cluster 32 receives data 2 and data 3, and sends data 2 and data 3 to storage cluster 3. Storage cluster 3 receives and stores data 2 and data 3.

[0124] Compared with the data migration system, the computing node is used to migrate the data to be migrated to the target storage cluster. In the present application, the data migration system determines the target logical cluster for migrating the data to be migrated according to the target storage cluster corresponding to the data to be migrated, and uses the target logical cluster to migrate the data to be migrated to the target storage cluster. The target logical cluster migrates the data to be migrated corresponding to the target storage cluster, and there is no need to migrate other data to be migrated to other storage clusters. In the case that the working state of the target logical cluster is abnormal, it will not affect the data migration of other storage clusters outside the target storage cluster, reducing the explosion radius (i.e., the influence range of the target logical cluster). And the number of storage clusters that need to be migrated corresponding to the target logical cluster is reduced, the complexity of the target logical cluster is reduced, so that the resources available for migrating the data to be migrated are increased, the time for migrating the data to be migrated is shortened, and the data migration efficiency is improved.

[0125] Combined with the above Figures 1 to 4 , describes in detail the data migration method provided by the present application. In the following embodiments, the present application also provides a data migration system. Figure 5 As shown, Figure 5This is a structural example diagram of a data migration system provided in the present application. The data migration system 500 includes: a source storage cluster 510, a public link cluster 520 and a target storage cluster 530. Among them, the public link cluster 520 includes multiple logical clusters. The data migration system is used to migrate the data to be migrated from the source storage cluster to the target storage cluster. Exemplarily, Figure 5 Only the situation in which the data migration system 500 migrates the data to be migrated in a source storage cluster 510 to the target storage cluster 530 is shown.

[0126] The source storage cluster 510 is used to receive a data migration request sent by a user. The data migration request is used to indicate the data to be migrated and the identifier of the target storage cluster 530. The source storage cluster 510 is also used to determine a target logical cluster corresponding to the target storage cluster 530 from multiple logical clusters according to the identifier of the target storage cluster 530, establish a communication connection with the target logical cluster, and send the data to be migrated to the target logical cluster. The public link cluster 520 is used to receive the data to be migrated based on the target logical cluster, and migrate the data to be migrated to the target storage cluster 530 according to the transmission path corresponding to the target logical cluster. The target storage cluster 530 is used to store the data to be migrated.

[0127] In a possible scenario, the public link cluster 520 also includes a control cluster. The control cluster stores a transmission path relationship graph, which includes a correspondence between logical clusters and a target storage cluster 530. The source storage cluster 510 is specifically configured to: obtain the transmission path relationship graph from the control cluster, and determine a target logical cluster corresponding to the target storage cluster 530 from multiple logical clusters according to the transmission path relationship graph.

[0128] In another possible scenario, before migrating the data to be migrated to the target storage cluster 530 according to the transmission path corresponding to the target logical cluster, the public link cluster 520 is further used to: adjust the data transmission capacity of the transmission path for migrating the data to be migrated according to the service type of the data to be migrated, so that the data transmission capacity matches the service type of the data to be migrated. The data transmission capacity includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay.

[0129] In another possible scenario, before migrating the data to be migrated to the target storage cluster 530 according to the transmission path corresponding to the target logical cluster, the public link cluster 520 is further used to: determine the data migration capability of the target logical cluster based on the control cluster, and adjust the data migration capability of the target logical cluster if the data migration capability of the target logical cluster is less than a preset threshold.

[0130] In another possible scenario, the public link cluster 520 is specifically used to: add at least one logical node in the target logical cluster, so that the adjusted data migration capability of the target logical cluster is greater than or equal to a preset threshold.

[0131] In another possible scenario, the control cluster further stores a node view. The node view is used to indicate the logical node information included in the target logical cluster. After at least one logical node is added to the target logical cluster, the public link cluster 520 is further used to: update the node view according to the logical node information after the at least one logical node is added.

[0132] The source storage cluster 510 and the target storage cluster 530 may include multiple storage nodes, and the public link cluster 520 may include multiple control nodes and logical nodes. For a more detailed description of more functions implemented by the source storage cluster 510, the public link cluster 520, and the target storage cluster 530, please refer to the above Figure 3 or Figure 4 The contents shown will not be repeated here.

[0133] As an example of a software functional unit, a storage node, a control node, and a logical node may include code running on a computing instance. Among them, a computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the above-mentioned computing device may be one or more. For example, a storage node, a control node, and a logical node may include code running on multiple hosts / virtual machines / containers. It should be noted that multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. Multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0134] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway must be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0135] As an example of a hardware functional unit, a storage node, a control node, and a logic node may include at least one computing device, such as a memory, a server, etc. Alternatively, the control node may also be a device implemented using ASIC or PLD, etc. The PLD may be implemented using CPLD, FPGA, GAL, or any combination thereof.

[0136] The multiple computing devices included in each storage node, control node, and logical node can be distributed in the same region or in different regions. The multiple computing devices included in each storage node, control node, and logical node can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in each storage node, control node, and logical node can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0137] It is worth noting that the source storage cluster of the aforementioned method embodiment may correspond to the source storage cluster 510, the public link cluster may correspond to the public link cluster 520, the target storage cluster may correspond to the target storage cluster 530, and may correspond to the method of executing the embodiment of the present application. Figure 3 or Figure 4 The corresponding corresponding subjects, and the operations and / or functions of each node in the data migration system 500 are respectively to achieve Figure 3 or Figure 4 For the sake of brevity, the corresponding processes of each method in the corresponding embodiment are not repeated here.

[0138] in addition, Figure 5 The data migration system 500 shown can also be implemented through a communication device, where the communication device may refer to the source storage cluster, public link cluster or target storage cluster in the aforementioned method embodiment, or, when the communication device is a chip or chip system applied to a processing device, the data migration system can also be implemented through the chip or chip system.

[0139] The embodiment of the present application also provides a chip system, which includes a control circuit and an interface circuit, wherein the interface circuit is used to obtain a data migration request and the control circuit is used to implement the function of the data migration system in the above method according to the data migration request.

[0140] In a possible design, the chip system further includes a memory for storing program instructions and / or data. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0141] The present application also provides a data migration device. Figure 6The structural diagram of the first data migration device provided by this application is as follows: Figure 6 As shown, the data migration device 600 includes a transceiver module 610 and a migration module 620 .

[0142] The transceiver module 610 is used to receive the data to be migrated sent by the source storage cluster based on the target logical cluster. The target logical cluster is determined by the source storage cluster from multiple logical clusters according to the identifier of the target storage cluster. The migration module 620 is used to migrate the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

[0143] In a possible scenario, the data migration device 600 further includes: an adjustment module 630. The adjustment module 630 is used to: adjust the data transmission capacity of the transmission path for migrating the data to be migrated according to the service type of the data to be migrated, so that the data transmission capacity matches the service type of the data to be migrated. The data transmission capacity includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay.

[0144] In a possible scenario, the public link cluster also includes a control cluster. Before the migration module 620 migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the adjustment module 630 determines the data migration capability of the target logical cluster based on the control cluster. If the data migration capability of the target logical cluster is less than a preset threshold, the adjustment module 630 adjusts the data migration capability of the target logical cluster.

[0145] In a possible scenario, the adjustment module 630 is specifically configured to add at least one logical node in the target logical cluster, so that the data migration capability of the adjusted target logical cluster is greater than or equal to a preset threshold.

[0146] In a possible scenario, the control cluster stores a node view. The node view is used to indicate the logical node information included in the target logical cluster. After adding at least one logical node in the target logical cluster, the adjustment module 630 updates the node view according to the logical node information after adding at least one logical node.

[0147] For a more detailed description of the functions implemented by the transceiver module 610, the migration module 620 and the adjustment module 630, please refer to the above Figures 3 to 5 The contents shown will not be repeated here.

[0148] The present application also provides another data migration device. Figure 7 A schematic diagram of the structure of the second data migration device provided in this application is as follows: Figure 7 As shown, the data migration device 700 includes a transceiver module 710 and a processing module 720 .

[0149] The transceiver module 710 is used to receive a data migration request sent by a user. The data migration request is used to indicate the identifier of the data to be migrated and the target storage cluster. The processing module 720 is used to determine, according to the identifier of the target storage cluster, a target logical cluster corresponding to the target storage cluster from the multiple logical clusters included in the public link cluster. The transceiver module 710 is also used to establish a communication connection with the target logical cluster and send the data to be migrated to the target logical cluster, so that the target logical cluster migrates the data to be migrated to the target storage cluster.

[0150] In a possible scenario, the public link cluster also includes a control cluster. The control cluster stores a transmission path relationship graph, which includes a correspondence between logical clusters and target storage clusters. The processing module 720 obtains the transmission path relationship graph from the control cluster, and determines a target logical cluster corresponding to the target storage cluster from multiple logical clusters according to the transmission path relationship graph.

[0151] For a more detailed description of the functions implemented by the transceiver module 710 and the processing module 720, please refer to the above Figures 3 to 5 The contents shown will not be repeated here.

[0152] The present application also provides a computing device. Figure 8 As shown, Figure 8 A schematic diagram of the structure of a computing device provided in the present application, the computing device 800 includes: a bus 802, a processor 804, a memory 806 and a communication interface 808. The processor 804, the memory 806 and the communication interface 808 communicate with each other through the bus 802. The computing device 800 can be a server or a terminal device, and the computing device 800 can be the above-mentioned source storage cluster, public link cluster or target link cluster. It is worth noting that the present application does not limit the number of processors and memories in the computing device 800.

[0153] The bus 802 may be, but is not limited to: a PCIe bus, a USB, or an inter-integrated circuit (I2C), an EISA (extended industry standard architecture) bus, UB, CXL, CCIX, etc. The bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 802 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 802 may include a path for transmitting information between various components of the computing device 800 (eg, the memory 806, the processor 804, and the communication interface 808).

[0154] The processor 804 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0155] The memory 806 may include a volatile memory, such as a random access memory (RAM). The memory 806 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0156] The memory 806 stores executable program codes, and the processor 804 executes the executable program codes to respectively implement the functions of the aforementioned source storage cluster, public link cluster or target storage cluster, thereby implementing the aforementioned data migration method. That is, the memory 806 stores instructions for executing the data migration method.

[0157] The communication interface 808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 800 and other devices or communication networks.

[0158] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device, which can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0159] like Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of a computer cluster provided in the present application. The computing device cluster includes at least one computing device 800. The memory 806 in one or more computing devices 800 in the computing device cluster may store the same instructions for executing the data migration method. The computer cluster may be the above-mentioned data migration system.

[0160] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also store partial instructions for executing the data migration method. In other words, the combination of one or more computing devices 800 may jointly execute instructions for executing the data migration method.

[0161] It should be noted that the memory 806 in different computing devices 800 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the data migration system. That is, the instructions stored in the memory 806 in different computing devices 800 can implement one or more functions of the source storage cluster, the public link cluster, and the target storage cluster to migrate the data to be migrated in the source storage cluster to the target storage cluster.

[0162] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.10 A possible implementation is shown. Fig.10 As shown, Fig.10 A schematic diagram of a connection between computing devices provided in the present application, wherein two computing devices 800A and 800B are connected via a network. Specifically, the connection is made to the network via a communication interface in each computing device. In this type of possible implementation, the memory 806 in the computing device 800A stores instructions for executing the functions of the source storage cluster and the target storage cluster. At the same time, the memory 806 in the computing device 800B stores instructions for executing the functions of the above-mentioned public link cluster.

[0163] Fig. 9 The connection method between the computing device clusters shown may be based on the fact that the public link cluster of the present application needs to complete the process of migrating the data to be migrated in the source storage cluster to multiple target storage clusters, so it is considered to hand over the functions implemented by the public link cluster to the computing device 800B.

[0164] It should be understood that Fig. 9 The functions of the computing device 800A shown in FIG. 8 may also be completed by multiple computing devices 800. Similarly, the functions of the computing device 800B may also be completed by multiple computing devices 800.

[0165] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the data migration method.

[0166] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the data migration method.

[0167] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data migration system, characterized in that: The data migration system includes a source storage cluster, a public link cluster and a target storage cluster, wherein the public link cluster includes multiple logical clusters; wherein: The source storage cluster is used to receive a data migration request sent by a user, where the data migration request is used to indicate the data to be migrated and an identifier of the target storage cluster; The source storage cluster is further used to determine, from the multiple logical clusters, a target logical cluster corresponding to the target storage cluster according to the identifier of the target storage cluster; The source storage cluster is further configured to establish a communication connection with the target logical cluster and send the data to be migrated to the target logical cluster; The public link cluster is used to receive the data to be migrated based on the target logical cluster; The public link cluster is further used to migrate the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster; The target storage cluster is used to store the data to be migrated.

2. The data migration system according to claim 1, characterized in that: The public link cluster further includes a control cluster, the control cluster stores a transmission path relationship diagram, the transmission path relationship diagram includes a correspondence between a logical cluster and a target storage cluster; the source storage cluster is specifically used for: Acquire the transmission path relationship graph from the control cluster; According to the transmission path relationship graph, a target logical cluster corresponding to the target storage cluster is determined from the multiple logical clusters.

3. The data migration system according to claim 1 or 2, characterized in that: The public link cluster is also used for: Before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, adjusting the data transmission capacity of the transmission path for migrating the data to be migrated according to the service type of the data to be migrated, so that the data transmission capacity matches the service type of the data to be migrated; The data transmission capability includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay.

4. The data migration system according to claim 1 or 2, characterized in that: The public link cluster is also used for: Before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, determining the data migration capability of the target logical cluster based on the control cluster; If the data migration capability of the target logical cluster is less than a preset threshold, the data migration capability of the target logical cluster is adjusted.

5. The data migration system according to claim 4, characterized in that: The public link cluster is specifically used for: At least one logical node is added to the target logical cluster so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the preset threshold.

6. The data migration system according to claim 5, characterized in that: The control cluster further stores a node view, where the node view is used to indicate the logical node information included in the target logical cluster, and the public link cluster is further used to: After at least one logical node is added to the target logical cluster, the node view is updated according to the logical node information after the at least one logical node is added.

7. A data migration method, characterized in that: The method is applied to a common link cluster, the common link cluster includes a plurality of logical clusters, and the method includes: receiving the data to be migrated sent by the source storage cluster based on the target logical cluster, wherein the target logical cluster is determined by the source storage cluster from the multiple logical clusters according to the identifier of the target storage cluster; The data to be migrated is migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

8. The method according to claim 7, characterized in that Before migrating the to-be-migrated data to the target storage cluster according to the transmission path corresponding to the target logical cluster, the method includes: According to the service type of the data to be migrated, adjusting the data transmission capacity of the transmission path for migrating the data to be migrated, so that the data transmission capacity matches the service type of the data to be migrated; The data transmission capability includes one or more of the following: transmission rate, transmission bandwidth, and transmission delay.

9. The method according to claim 7, characterized in that: The public link cluster further includes a control cluster. Before migrating the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster, the method further includes: Determining data migration capability of a target logical cluster based on the control cluster; If the data migration capability of the target logical cluster is less than a preset threshold, the data migration capability of the target logical cluster is adjusted.

10. The method according to claim 9, characterized in that The adjusting the data migration capability of the target logical cluster includes: At least one logical node is added to the target logical cluster so that the adjusted data migration capacity of the target logical cluster is greater than or equal to the preset threshold.

11. The method according to claim 10, characterized in that The control cluster stores a node view, where the node view is used to indicate logical node information included in the target logical cluster. After adding at least one logical node in the target logical cluster, the method further includes: The node view is updated according to the logical node information after at least one logical node is added.

12. A data migration method, characterized in that: The method is applied to a source storage cluster, and the method includes: Receiving a data migration request sent by a user, where the data migration request is used to indicate the data to be migrated and an identifier of a target storage cluster; According to the identifier of the target storage cluster, determining a target logical cluster corresponding to the target storage cluster from a plurality of logical clusters included in the public link cluster; A communication connection is established with the target logical cluster, and the data to be migrated is sent to the target logical cluster, so that the target logical cluster migrates the data to be migrated to the target storage cluster.

13. The method according to claim 12, characterized in that The public link cluster further includes a control cluster, the control cluster stores a transmission path relationship graph, the transmission path relationship graph includes a correspondence between a logical cluster and a target storage cluster, and the method further includes: Acquire the transmission path relationship graph from the control cluster; According to the transmission path relationship graph, a target logical cluster corresponding to the target storage cluster is determined from the multiple logical clusters.

14. A data migration method, characterized in that: The method is applied to a data migration system, the data migration system includes a source storage cluster, a public link cluster and a target storage cluster, the public link cluster includes a plurality of logical clusters, and the method includes: The source storage cluster receives a data migration request sent by a user, where the data migration request is used to indicate the data to be migrated and an identifier of the target storage cluster; The source storage cluster determines, according to the identifier of the target storage cluster, a target logical cluster corresponding to the target storage cluster from the multiple logical clusters; The source storage cluster establishes a communication connection with the target logical cluster, and sends the data to be migrated to the target logical cluster; The public link cluster receives the data to be migrated based on the target logical cluster; The public link cluster migrates the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster; The target storage cluster stores the data to be migrated.

15. A data migration device, characterized in that: The device comprises: The transceiver module is used to: receive the data to be migrated sent by the source storage cluster based on the target logical cluster, where the target logical cluster is determined by the source storage cluster from multiple logical clusters according to the identifier of the target storage cluster; The transceiver module is further used to migrate the data to be migrated to the target storage cluster according to the transmission path corresponding to the target logical cluster.

16. A data migration device, characterized in that: The device comprises: The transceiver module is used to: receive a data migration request sent by a user, where the data migration request is used to indicate the identifier of the data to be migrated and the target storage cluster; A processing module, configured to: determine, according to the identifier of the target storage cluster, a target logical cluster corresponding to the target storage cluster from a plurality of logical clusters included in the public link cluster; The transceiver module is further configured to establish a communication connection with the target logical cluster and send the data to be migrated to the target logical cluster, so that the target logical cluster migrates the data to be migrated to the target storage cluster.

17. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method according to any one of claims 7 to 11; Or, the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method according to claim 12 or 13; Or, the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as claimed in claim 14.

18. A computer-readable storage medium, characterized in that: comprising computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 7 to 11; or, when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to claim 12 or 13; Or, when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method of claim 14.

19. A computer program product comprising instructions, characterized in that When the instruction is executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 7 to 11; Or, when the instructions are executed by a computing device cluster, the computing device cluster executes the method according to claim 12 or 13; Or, when the instructions are executed by a computing device cluster, the computing device cluster executes the method as claimed in claim 14.

Images