Data migration method and system

Abstract

The present application provides a data migration method and system. The method comprises: a target node receives a first migration command, the first migration command instructing to migrate data of a first slot on a source node to the target node, wherein the source node and the target node are among a plurality of nodes; the source node receives a second migration command, the second migration command instructing to migrate the data of the first slot on the source node to the target node; the source node copies the data of the first slot to a shared memory pool; and the target node acquires the data of the first slot from the shared memory pool. The solution of embodiments of the present application is helpful to improve the slot migration speed.

Description

Data migration methods and systems

[0001] This application claims priority to Chinese Patent Application No. 202510031698.1, filed with the State Intellectual Property Office of China on January 8, 2025, entitled “Method and System for Data Migration”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and more specifically, to a method and system for data migration. Background Technology

[0003] Redis (Remote Dictionary Service) is a high-performance in-memory database that provides storage for various data structures, including key-value pairs. It is widely used for storing and retrieving cached data in various business applications. As the scale of Redis usage continues to expand, a single Redis instance can no longer meet the demands of large-scale data storage. Redis clusters have become one of the main methods for distributed storage. Redis clusters achieve high availability and load balancing by distributing data across multiple nodes.

[0004] In a Redis cluster, each node is allocated a certain number of slots, and each slot can be used to store data. Slot migration is a crucial task in Redis clusters. Taking a serverless scenario as an example, rapid scaling of Redis is a key technology. For instance, when capacity is insufficient, or when facing concentrated loads due to hot access, dynamically adjusting the size of Redis nodes ensures stable system operation, thereby providing consistent performance and reliability. Redis's rapid scaling is achieved through slot migration.

[0005] However, current slot migration solutions are slow, and it usually takes several hours to double the capacity of a cluster with hundreds of gigabytes of data.

[0006] Therefore, improving the migration speed of slots has become an urgent problem to be solved. Summary of the Invention

[0007] This application provides a data migration method and system, which helps to improve the migration speed of slots.

[0008] In a first aspect, a data migration method is provided, applied to a distributed storage system comprising multiple nodes, at least some of which have one or more slots distributed on them. The method includes: a target node receiving a first migration command, the first migration command instructing the migration of data from a first slot on a source node to the target node, the source node and the target node belonging to the multiple nodes; the source node receiving a second migration command, the second migration command instructing the migration of data from the first slot on the source node to the target node; the source node copying the data from the first slot to a shared memory pool; and the target node retrieving the data from the shared memory pool.

[0009] According to the scheme of the embodiments of this application, slot migration is realized by using a shared memory pool, which replaces the traditional network transmission, which is beneficial to improve the transmission rate and thus improve the migration speed.

[0010] The source node and the target node share this shared memory pool.

[0011] For example, the number of first slots can be one or more.

[0012] For example, the number of source nodes can be one or more.

[0013] For example, the number of target nodes can be one or more.

[0014] For example, the first migration command can be used to set the first slot to an imported state on the target node.

[0015] For example, the second migration command can be used to set the first slot to a migration state on the source node.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the source node generating the full data of the first slot; and the source node copying the data of the first slot to the shared memory pool, including: the source node copying the full data of the first slot to the shared memory pool.

[0017] Optionally, the target node retrieves the data for the first slot from the shared memory pool, including: the target node retrieves the full data for the first slot from the shared memory pool.

[0018] According to the scheme of this application embodiment, migration is performed at the slot level, which reduces the number of transmissions during the migration process, thereby helping to reduce network overhead. At the same time, migration at the slot level avoids separate key processing procedures (such as serialization and deserialization), which helps to reduce migration time and improve migration speed.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the source node receiving a first operation request, the first operation request indicating a first operation related to first data, the first data belonging to a first slot; and performing the first operation in a shared memory pool.

[0020] According to the scheme of the embodiments of this application, during slot migration, the shared memory pool performs the operations related to the slot and stores the generated incremental data, thereby avoiding the incremental data from occupying a large amount of storage space of the source node. At the same time, it reduces the blocking time of the source node, which is beneficial to avoid blocking access to the source node.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the target node obtaining the data obtained from the first operation from the shared memory pool.

[0022] For example, the first operation can be a write operation.

[0023] According to the scheme of the embodiments of this application, incremental data (such as the data obtained from the first operation) can be synchronized from the shared memory pool to the target node, which helps to ensure the accuracy of the migrated data in the target node.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, before the target node obtains the data obtained from the first operation from the shared memory pool, the method further includes: the source node and / or the target node sending first information indicating that the first slot is allocated to the target node.

[0025] According to the solution of this application embodiment, after the migration of all data is completed, the owner node of the slot can be set to the target node. Subsequent client operation requests can be sent to the target node, and access to the data in the slot can be achieved through the target node. For the client, it is equivalent to the migration being completed, which improves the migration speed.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the target node receiving a second operation request, the second operation request indicating a second operation related to the second data, the second data belonging to the first slot; if the incremental data in the shared memory pool includes the second data, the second operation is performed in the shared memory pool, or if the incremental data in the shared memory pool does not include the second data, the second operation is performed in the target node.

[0027] Secondly, a data migration system is provided, comprising: the system being a distributed storage system, the distributed storage system including multiple nodes, at least some of the multiple nodes having one or more slots distributed on them, the multiple nodes including a target node and a source node, the target node being used to receive a first migration command, the first migration command instructing to migrate data of a first slot on the source node to the target node; the source node being used to: receive a second migration command, the second migration command instructing to migrate data of the first slot on the source node to the target node; and copy the data of the first slot to a shared memory pool; the target node is also used to retrieve the data of the first slot from the shared memory pool.

[0028] According to the scheme of the embodiments of this application, slot migration is realized by using a shared memory pool, which replaces the traditional network transmission, which is beneficial to improve the transmission rate and thus improve the migration speed.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the source node is also used to: generate the full data of the first slot; and specifically to: copy the full data of the first slot to the shared memory pool.

[0030] In conjunction with the second aspect, in some implementations of the second aspect, the source node is also used to: receive a first operation request, the first operation request indicating a first operation related to the first data, the first data belonging to the first slot, and the first operation being executed in a shared memory pool.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, the target node is also used to: retrieve the data obtained from the first operation from the shared memory pool.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, before the target node is further configured to retrieve the data obtained from the first operation from the shared memory pool, the source node and / or the target node is further configured to: send a first message indicating that the first slot has been allocated to the target node.

[0033] In conjunction with the second aspect, in some implementations of the second aspect, the target node is further configured to: receive a second operation request, the second operation request indicating a second operation related to second data, the second data belonging to the first slot; if the second data is included in the incremental data of the shared memory pool, the second operation is executed in the shared memory pool, or if the second data is not included in the incremental data of the shared memory pool, the second operation is executed in the target node.

[0034] It should be understood that the extensions, limitations, explanations and descriptions of the relevant content in the first aspect above also apply to the same content in the second aspect.

[0035] Thirdly, a computing device cluster is provided, comprising at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, causing the computing device cluster to perform the methods of the first aspect or any implementation thereof.

[0036] Fourthly, a computer-readable medium is provided, including computer program instructions that, when executed by a cluster of computing devices, perform the method of the first aspect or any implementation thereof.

[0037] Fifthly, a computer program product containing instructions is provided, which, when executed by a cluster of computing devices, causes the cluster of computing devices to perform the method described in the first aspect or any implementation thereof. Attached Figure Description

[0038] Figure 1 is a schematic diagram of a Redis cluster.

[0039] Figure 2 is a schematic diagram of the expansion scenario.

[0040] Figure 3 is a schematic diagram of the slot migration process.

[0041] Figure 4 is a schematic flowchart of a data migration method according to an embodiment of this application.

[0042] Figure 5 is a schematic diagram of a migration process according to an embodiment of this application.

[0043] Figure 6 is a schematic diagram of a migration process according to an embodiment of this application.

[0044] Figure 7 is a schematic block diagram of a data migration system according to an embodiment of this application.

[0045] Figure 8 is a schematic block diagram of a computing device according to an embodiment of this application.

[0046] Figure 9 is a schematic block diagram of a computing device cluster according to an embodiment of this application.

[0047] Figure 10 is a schematic block diagram of another computing device cluster according to an embodiment of this application. Detailed Implementation

[0048] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0049] The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” and “the” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one,” “at least one,” and “one or more” refer to one, two, or more than two. “First,” “second,” and various numerical designations are merely distinctions for descriptive convenience and are not intended to limit the scope of the embodiments of this application. “And / or” is used to describe the correspondence between corresponding objects, indicating that three relationships can exist. For example, “A and / or B” can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship. The order of the process numbers below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic and should not constitute any limitation on the implementation process of the embodiments of this application. For example, in the embodiments of this application, the words "301", "401", "501" etc. are merely identifiers made for the convenience of description and do not limit the order of execution steps.

[0050] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. In this application, the words "exemplary" or "for example" are used to indicate that something is illustrative, exemplary, or descriptive. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized. In the embodiments of this application, descriptions such as "when," "in the case of," "if," and "if" all refer to the fact that the device will perform a corresponding processing under certain objective circumstances, and are not a limitation on time, nor do they require the device to perform a judgment action during implementation, nor do they imply any other limitations.

[0051] In this application, "for indicating" can include both direct and indirect indication. When describing an indication message as indicating A, it can include whether the indication message directly indicates A or indirectly indicates A, but does not necessarily mean that the indication message carries A.

[0052] To facilitate understanding of the solutions in the embodiments of this application, the terms that may be involved in the embodiments of this application will be explained below.

[0053] (1) Redis;

[0054] Redis is a high-performance in-memory database that provides storage functionality for various data structures, including key-value pairs.

[0055] As the use of Redis continues to expand, a single Redis instance can no longer meet the needs of large-scale data storage. Redis Cluster has become one of the main methods of distributed storage. Redis Cluster is a distributed cluster architecture supported by Redis. In Redis Cluster mode, data is distributed across multiple nodes to achieve high availability and load balancing.

[0056] For example, a node can be a Redis instance participating in a Redis cluster, and a Redis instance can be a standalone running Redis server process.

[0057] (2) slot;

[0058] In Redis cluster mode, slots are a mechanism for distributed data storage. Redis clusters use slots to distribute data across different nodes.

[0059] In a Redis cluster, each node is assigned a certain number of slots, and each slot has a unique owner node. Each slot stores multiple key-value pairs, with each key assigned to a specific slot. The key-value pair data is stored in key-value format.

[0060] A key is a string identifier used to uniquely identify a value stored in Redis. Each key is associated with a specific value, and various operations can be performed on the corresponding value using this key.

[0061] A Redis cluster typically divides the entire data space into 16384 slots, numbered from 0 to 16383. That is, the data in a Redis cluster is distributed across these 16384 slots.

[0062] Figure 1 illustrates a Redis cluster. As shown in Figure 1, the Redis cluster consists of four nodes: cache node 1 through cache node 4. The 16384 slots in the Redis cluster are numbered from 0 to 16383. Each slot belongs to one node, and each node has multiple slots. Taking cache node 1 as an example, slots 0 to 7120, a total of 7121 slots, are assigned to cache node 1, and so on.

[0063] For data partitioning, Redis clusters can employ a consistent hashing scheme, which distributes all keys across 16384 slots using a consistent hashing algorithm. Specifically, each slot corresponds to a hash range, and a hash value for each key is calculated using a hash function. Each key is then mapped to its corresponding slot based on its hash value.

[0064] For example, the slot number to which the key belongs can be calculated by performing a cyclic redundancy check (CRC) 16 hash on the key and then taking the modulo 16384. The key can then be mapped to that slot, that is, the key and the corresponding value can be stored on the node to which the slot belongs.

[0065] For example, taking Figure 1 as an example, when storing a key-value pair of data, the CRC16 algorithm can be performed on the key and the modulus can be taken as 16384. Assuming the result is 7890, the key can be assigned to slot 7890. Since slot 7890 belongs to cache node 2, the key-value pair data is stored on cache node 2.

[0066] (3) Slot migration;

[0067] In Redis cluster mode, slot migration refers to the process of migrating one or more slots from one node to another.

[0068] The main purpose of slot migration is to achieve horizontal scaling or load balancing, ensuring that data can be evenly distributed among cluster nodes and avoiding overload of some nodes.

[0069] For example, when it is necessary to add a new node to the Redis cluster, remove a node from the Redis cluster, or perform load balancing, the Redis cluster can perform slot migration, transferring some slots from one node to another to achieve horizontal scaling, load balancing, or fault recovery.

[0070] In serverless scenarios, rapid scaling of Redis is a key technology. For example, when capacity is insufficient, or when facing concentrated loads due to hot access, dynamically adjusting the size of Redis nodes ensures stable system operation, providing consistent performance and reliability. Redis's rapid scaling flexibly responds to traffic fluctuations, reduces resource waste, and optimizes costs. In modern applications, especially in high-concurrency and distributed systems, Redis's rapid scaling is particularly important. Redis's rapid scaling is achieved through slot migration.

[0071] Figure 2 illustrates a scaling scenario. For example, as shown in Figure 2, a Redis cluster consists of two nodes (e.g., node 1 and node 2). If node 1 experiences hotspot access, or when the node's capacity reaches its limit, the Redis cluster can be scaled up, increasing the number of nodes to three. This scaling operation is achieved through slot migration.

[0072] Figure 3 shows a schematic diagram of a slot migration process. The slot migration process will be explained below with reference to Figure 3.

[0073] The source node (src node) is the node that currently holds the slot to be migrated, and the destination node (dst node) is the node that receives the slot.

[0074] As shown in Figure 3, slot migration may include the following steps.

[0075] 1) Set the slot to be migrated to the import state on the target node.

[0076] As shown in Figure 3, the client sets the slot to the import state on the target node.

[0077] The dashed arrows in Figure 3 represent return values, command execution results, or responses. For example, a client sends a command to a target node to set a slot to an "import" state, and the target node can send a return value back to the client. The target node is ready to receive data from the new slot.

[0078] 2) Set the slot to be migrated to the migrating state on the source node.

[0079] As shown in Figure 3, the client sets the slot to migrating state on the source node.

[0080] In this way, the source node has prepared the data to be migrated from the slot and can start accepting the migrate command.

[0081] 3) Query the key in the slot.

[0082] As shown in Figure 3, the client can obtain all the keys belonging to this slot.

[0083] 4) The client sends a migrate command to the source node to instruct the source node to migrate the key to the target node.

[0084] 5) The source node executes the migrate command to migrate the key to the target node.

[0085] Repeat steps 4)-5) until all keys in the slot have been migrated to the target node. During this loop, if a client needs to access a key that has been migrated to the target node, the source node will return an `ask` to instruct the client to access the key from the target node.

[0086] During and after the loop, the source node can still accept new operation requests for the slot. For any modification operation, the source node will synchronize these changes to the target node after execution to maintain data consistency.

[0087] If a failure or interruption occurs during the migration process, it may be necessary to re-initiate the data migration to ensure that the data on the target node is up-to-date.

[0088] 6) Notify all nodes in the cluster that the slot has been assigned to the target node.

[0089] Once all modified data in this slot has been synchronized, the cluster will update its internal state, i.e., update the cluster configuration to reflect the new slot allocation. Subsequent client requests for this slot will be directly routed to the target node.

[0090] The above-mentioned migration method is relatively slow. To double the capacity of a cluster with hundreds of gigabytes of data, it usually takes several hours to migrate.

[0091] In view of this, embodiments of this application provide a data migration method that helps to improve migration speed.

[0092] Figure 4 is a schematic flowchart of a data migration method provided in an embodiment of this application.

[0093] For example, the method 400 shown in Figure 4 can be applied to a cloud management platform. The cloud management platform is used to manage infrastructure, which includes at least one cloud data center, each of which has multiple servers.

[0094] For example, method 400 can be applied to services or software that provide distributed storage functionality. Distributed storage can also be replaced by distributed caching, etc.

[0095] For example, method 400 can be applied to a distributed caching system that is compatible with the Redis protocol and Redis data structures and has a shared memory pool component.

[0096] The solution in this application embodiment can be applied to a distributed storage system, which may include multiple nodes. At least some of the multiple nodes have one or more slots distributed on them. The multiple nodes may include source nodes and target nodes. Exemplarily, the distributed storage system can also be replaced with a distributed caching system, and the multiple nodes can also be replaced with multiple cache nodes. For example, the distributed storage system can be a Redis cluster. For ease of description, method 400 is mainly described using a Redis cluster as an example.

[0097] As shown in Figure 4, method 400 may include the following steps.

[0098] 410. The source node copies the data of the slot to be migrated (an example of the first slot) to the shared memory pool.

[0099] 420, The target node retrieves the data for the slot to be migrated from the shared memory pool.

[0100] The "first" in "first slot" is for descriptive convenience only and has no limiting effect. Any slot to be migrated can be called the first slot.

[0101] Furthermore, prior to step 410, the source node and / or the target node may be notified to prepare for slot migration.

[0102] Optionally, before step 410, method 400 may also include steps 401 and 402.

[0103] 401, The target node receives the first migration command, which instructs the data of the slot to be migrated on the source node to be migrated to the target node.

[0104] 402, the source node receives a second migration command, which instructs the data of the slot to be migrated on the source node to be migrated to the target node.

[0105] The terms "first" and "second" in "first migration command" and "second migration command" are for descriptive convenience only and do not have any limiting effect. The migration command received by the target node is the first migration command, and the migration command received by the source node is the second migration command.

[0106] The specific format and content of the migration command can be set as needed.

[0107] As an example, migration commands can be used to set the status of a slot to be migrated on both the target and source nodes, informing both nodes to prepare for the slot migration.

[0108] For example, the first migration command can be used to set the slot to be migrated to state #1 on the target node, so that the target node is ready to receive the data of the slot.

[0109] For example, status #1 can be an import status. The client can send a command to the target node to set the slot to be migrated to the import status on the target node.

[0110] It should be understood that setting a slot to the import state is only an example. In other implementations, it can be set to other states or other forms of migration commands can be used, as long as the target node is ready to receive data from the new slot.

[0111] For example, the second migration command can be used to set the slot to be migrated to state #2 on the source node, so that the source node is ready to migrate the data of that slot.

[0112] For example, status #2 is the migrating status. The client can send a command to the source node to set the slot to be migrated to the migrating status on the source node.

[0113] It should be understood that setting a slot to the migrating state is merely an example. In other implementations, it can be set to other states or other forms of migration commands can be used, as long as the source node is ready to migrate the slot's data.

[0114] In this embodiment, the step numbers are for ease of description only and do not limit the execution order of the steps. For example, taking steps 401 and 402 as examples, step 401 can be executed first and then step 402, or step 402 can be executed first and then step 401.

[0115] The number of slots to be migrated can be one or more. This application does not limit this. For ease of description, method 400 is mainly illustrated using one slot as an example, and does not constitute a limitation on the solution of this application.

[0116] The source node is the node that currently holds the slot to be migrated, and the target node is the node that receives the slot to be migrated.

[0117] The number of source nodes can be one or more. For example, there may be multiple slots to be migrated, which may be distributed across one node or multiple nodes before migration. For ease of description, method 400 is mainly illustrated using a single source node as an example, and does not constitute a limitation on the solution of the embodiments of this application.

[0118] The number of target nodes can be one or more. For example, there may be multiple slots to be migrated, which may need to be migrated to one node or multiple nodes. For ease of description, method 400 is mainly described using a single target node as an example, and does not constitute a limitation on the solution of the embodiments of this application.

[0119] Multiple nodes in the cluster, such as the source node and the target node, share this shared memory pool.

[0120] A shared memory pool may or may not belong to the distributed system.

[0121] A shared memory pool can also be replaced with shared memory.

[0122] A shared memory pool can be understood as one or more memory nodes or storage nodes. Alternatively, a shared memory pool can be understood as a remote service that provides memory semantics or access speeds close to those of memory.

[0123] Interaction between individual nodes and the shared memory pool can be achieved through efficient network protocols, such as remote direct memory access (RDMA).

[0124] The network overhead of a shared memory pool is lower than that of traditional network communication. In the scheme of this application embodiment, the shared memory pool is used to realize slot migration, which replaces the traditional network transmission. This is beneficial to improve the transmission rate and thus improve the migration speed, so as to achieve rapid rebalancing.

[0125] In the relevant migration schemes, such as the one shown in Figure 3, if a failure or interruption occurs during the migration process, it may be necessary to re-initiate a full data migration to ensure that the data on the target node is up-to-date.

[0126] In the scheme of this application embodiment, the slot data is copied to the shared memory pool. Even if the target node fails during the migration process, there is no need for the source node to resend the data, which helps to reduce the amount of data transmission.

[0127] For example, steps 410 and / or 420 can be performed on a key-by-key basis. For instance, the source node copies the keys one by one to the shared memory pool, and the target node can retrieve the keys one by one from the shared memory pool until all the original keys in the source node have been migrated.

[0128] This method requires a large number of transmissions. Therefore, this application proposes a scheme for slot migration at the slot level.

[0129] As one possible implementation, method 400 may also include: the source node generating full data of the slot to be migrated.

[0130] That is, the full data of a slot can be generated at the slot level.

[0131] If the slot to be migrated consists of multiple slots, full data can be generated for each slot separately.

[0132] The full data of a slot refers to all the data in that slot held by the source node at the start of the migration.

[0133] During slot migration, the entire data for a slot can be transferred at the slot-by-slot level. This allows multiple keys within a slot to be transferred in a single transaction, rather than being transferred individually.

[0134] Optionally, step 410 may include: the source node copying the full data of the slot to be migrated into a shared memory pool.

[0135] Optionally, step 420 may include: the target node obtaining the full data of the slot to be migrated from the shared memory pool.

[0136] It should be understood that migrating at the slot level does not mean that the entire data of that slot must be transferred in one go, as long as the number of transfers of the entire data of that slot is less than the number of keys in that slot.

[0137] Steps 410 and 420 can be viewed as a full data migration process.

[0138] In relevant migration schemes, such as the one shown in Figure 3, slot migration is performed at the key level; that is, the migration of the next key only proceeds after the migration of the previous key is successful. If the slot contains n keys, at least n network transmissions are required, which incur significant network overhead. Furthermore, because migration is performed at the key level, each key needs to be serialized and deserialized individually before being migrated to the target node, which is time-consuming.

[0139] In the scheme of this application embodiment, migration is performed at the slot level, which reduces the number of transmissions during the migration process, thereby helping to reduce network overhead. At the same time, migration at the slot level avoids separate key processing procedures (such as serialization and deserialization), which helps to reduce migration time and improve migration speed.

[0140] During slot migration, operations on that slot, specifically operations on the keys belonging to that slot, can still be performed. The slot migration process refers to the period from the start of the migration until its formal completion. Formal completion of the migration can be understood as the moment when both full and incremental data are synchronized. For example, in Method 1 described later, the formal completion time can be the moment the target node marks the completion of the slot migration. Similarly, in Method 2 described later, the formal completion time can be the moment a broadcast message notifies the change of the slot's owner node.

[0141] The data changes caused by operations on this slot during this period, such as added, updated, or deleted key-value pairs, can be called incremental data. Alternatively, incremental data can also be understood as data that differs from the full dataset during this period.

[0142] Incremental data may be generated during and / or after the full data transfer is completed. In other words, operations on the slot can be performed during and / or after the full data transfer is completed, and the corresponding data changes can be considered as incremental data.

[0143] As one possible implementation, operations on a slot can be performed in a shared memory pool during slot migration. Accordingly, the resulting incremental data resides in the shared memory pool.

[0144] Optionally, method 400 may also include steps 430 and 440 (not shown in the figure).

[0145] 430, Received an operation request. This operation request indicates the operation associated with the key. This key belongs to this slot.

[0146] 440. Perform this operation in the shared memory pool.

[0147] For example, the operation request can come from the client.

[0148] For example, step 430 can be performed by the source node.

[0149] Alternatively, step 430 can be performed by the target node.

[0150] For example, the slot to which a key belongs can be determined by calculating the hash value, as described above, and will not be repeated here.

[0151] The key in step 430 can be any key belonging to that slot.

[0152] Operations related to a key can also be replaced with operations on the key-value pairs corresponding to that key, or operations on the value corresponding to that key.

[0153] For example, key-related operations may include read operations on key-value pairs and / or write operations on key-value pairs. For instance, write operations on key-value pairs may include one or more of adding, updating, or deleting key-value pairs.

[0154] The execution object of the operation request in the shared memory pool is unrelated to the full data.

[0155] For example, a key-related operation could be adding a key-value pair, in which case the key is not included in the full data. Step 440 can be understood as writing the key-value pair into the shared memory pool.

[0156] For example, a key-related operation could be updating the key-value pair corresponding to that key, where the key already exists in the full dataset. In this case, the operation could be performed on a key obtained through copying, specifically copying the key-value pair within the shared memory pool. Step 440 can be understood as updating the copied key-value pair within the shared memory pool.

[0157] The key-value pair can be copied by the shared memory pool itself. Alternatively, it can be copied from other nodes. For example, in step 430, the source node may receive the operation request, and the key-value pair can be copied from the source node to the shared memory pool.

[0158] For example, the return value of an operation in the shared memory pool, i.e., the response to the operation request, can be returned to the initiator of the operation request, such as a client. For instance, in step 430, the operation request can be received by the source node, and the return value can be returned to the client through the source node. Similarly, in step 440, the operation request can be received by the target node, and the return value can be returned to the client through the target node.

[0159] It should be understood that the above is only one possible implementation and does not constitute a limitation on the solutions of the embodiments of this application. During slot migration, operations on the slot can also be handled in other ways. For example, during slot migration, operations on the slot can also be performed on the source node, in which case the generated incremental data is located in the source node.

[0160] Furthermore, method 400 may also include step 450 (not shown in the figure).

[0161] 450, Send a message (an example of the first message) indicating that the slot has been assigned to the target node.

[0162] In other words, this information indicates that the owner node of the slot has been changed to the target node. Afterwards, operation requests for that slot can be sent to the target node.

[0163] The word "first" in "first information" is used for descriptive convenience only and has no limiting effect.

[0164] For example, before step 450 is executed, that is, before the change of the owner node of the slot is notified, the source node can receive the operation request related to the slot. After step 450 is executed, that is, after the change of the owner node of the slot is notified, the target node can receive the operation request related to the slot.

[0165] Step 450 can be performed by the target node and / or the source node.

[0166] For example, step 450 may include: the target node and / or the source node broadcasting the information to notify all nodes that the slot has been assigned to the target node.

[0167] For example, step 450 may include: the target node and / or the source node sending the information to the client to notify the client that the slot has been assigned to the target node.

[0168] Furthermore, method 400 may include step 460 (not shown in the figure).

[0169] 460, the target node can obtain incremental data.

[0170] Step 460 can be performed after step 420 is completed. For example, after the full data transmission is completed, incremental data can be synchronized.

[0171] As previously mentioned, the incremental data may reside in a shared memory pool. In this case, step 460 may include: the target node retrieving the incremental data from the shared memory pool.

[0172] For example, the target node can obtain incremental data at the key level until the transmission of all incremental data is completed, that is, the synchronization of all incremental data is completed.

[0173] Figure 5 illustrates a migration process according to an embodiment of this application. For example, as shown in Figure 5, slot migration is achieved through full data at the slot level and incremental data at the key level. The transmission of full data at the slot level means transmitting all data at the slot level. The transmission of incremental data at the key level means transmitting incremental data at the key level.

[0174] As an example, during the full data transmission process, i.e., during the execution of steps 410 and 420, the client may send an operation request for the slot. For example, the operation request in step 430 may be received during the full data transmission process. The full data transmission process and the operation on the slot during this period can be executed in parallel, i.e., steps 410 and 420, as well as steps 430 and 440, can be executed in parallel.

[0175] As another example, during the transmission of incremental data, i.e., during the execution of step 460, the client may send an operation request for the slot. For example, the operation request in step 430 may be received during the transmission of incremental data. The transmission of incremental data and the operation on the slot during this period may be executed in parallel; that is, steps 460, 430, and 440 may be executed in parallel.

[0176] In relevant migration schemes, such as the one shown in Figure 3, the Redis cluster uses the `migrate` command to migrate keys. This command is synchronous and blocking, which will block all key reads and writes during the migration process. `migrate` is time-consuming; for example, poor network conditions or migrating large keys can cause server-side blocking, making it unable to respond to business commands.

[0177] In the scheme of this application embodiment, during slot migration, the shared memory pool performs the operations related to the slot and stores the generated incremental data, avoiding the incremental data from occupying a large amount of storage space of the source node. At the same time, it reduces the blocking time of the source node, which is beneficial to avoid blocking access to the source node, especially avoiding the blocking of business commands in the case of large keys.

[0178] The following describes steps 430 to 460 using two possible implementation methods (Method 1 and Method 2) as examples. For ease of description, Method 1 and Method 2 mainly use the example of performing operations related to the slot in a shared memory pool.

[0179] Method 1;

[0180] In Method 1, step 450 can be performed before the incremental data transmission is complete (i.e., step 460 is completed).

[0181] Furthermore, step 450 can be performed before incremental data transmission begins.

[0182] Optionally, step 450 can be performed between steps 420 and 460. That is, after the full data transmission is completed and before the incremental data transmission begins, the owner node of the slot is notified of the change. The following explanation mainly uses this as an example.

[0183] As one possible implementation, steps 430 and 440 can be performed before step 450. In this case, the operation request related to the slot can be received by the source node.

[0184] Steps 430 and 440 can be achieved through steps 1A) and 1B) as described below.

[0185] 1A) The source node receives operation request #1 (an example of a first operation request), which indicates operation #1 (an example of a first operation) related to key #1 (an example of first data). Key #1 belongs to this slot.

[0186] 1B) Perform operation #1 in the shared memory pool.

[0187] The numbers #1, #1, and #1 are for descriptive convenience only and have no limiting effect. Any key belonging to this slot can be considered as key #1, any operation request for key #1 can be considered as operation request #1, and any operation related to key #1 can be considered as operation #1.

[0188] The word "first" in "first operation request," "first data," and "first operation" is used to describe the scheme and has no limiting function.

[0189] For example, during the transmission of all data, the source node can continue to receive operation requests for that slot. Taking operation #1 as a write operation as an example, the source node can copy key #1 to the shared memory pool, perform the write operation in the shared memory pool, and the data obtained from the write operation is the incremental data corresponding to key #1. If the source node subsequently receives another operation related to key #1, it can perform that operation in the shared memory pool based on the incremental data corresponding to key #1.

[0190] Optionally, step 460 may include: the target node retrieving the data obtained from operation #1 from the shared memory pool.

[0191] Operation #1 can be a write operation. Incremental data can include the data obtained from this write operation.

[0192] As another possible implementation, steps 430 and 440 can be executed after step 450. In this case, after the full data transmission is complete, the target node can receive the operation request related to the slot.

[0193] Step 430 can be achieved through step 2A).

[0194] 2A) The target node receives operation request #2 (an example of a second operation request), which indicates operation #2 (an example of a second operation) related to key #2 (an example of second data). Key #2 belongs to this slot.

[0195] As an example, after the target node retrieves the incremental data corresponding to a certain key from the shared memory pool, it can delete the incremental data corresponding to that key from the shared memory pool. In this case, if the target node still has that key in the shared memory pool when it receives an operation request related to that key, it means that the target node has not yet retrieved the incremental data corresponding to that key.

[0196] Step 440 can be achieved through step 2B).

[0197] 2B) If the incremental data in the shared memory pool includes key #2, perform operation #2 in the shared memory pool.

[0198] If the incremental data in the shared memory pool does not include key #2, perform operation #2 on the target node.

[0199] In other words, when a target node receives an operation request, it can determine whether the key related to the operation request exists in the incremental data of the shared memory pool. If it exists, it accesses the key in the shared memory; if it does not exist, it accesses the key in the target node.

[0200] In this case, step 440 can also be understood as follows: Before receiving operation request #2, key #2 may have been modified, meaning that incremental data corresponding to key #2 has already been generated during slot migration. In this case, if key #2 in the incremental data has not yet been retrieved from the shared memory pool to the target node, then operation #2 is executed in the shared memory pool; if key #2 in the incremental data has already been retrieved from the shared memory pool to the target node, then operation #2 is executed on the target node. Alternatively, before receiving operation request #2, key #2 may not have been modified, meaning that incremental data corresponding to key #2 has not yet been generated during slot migration. In this case, operation #2 can be executed on the target node.

[0201] As another example, after the target node retrieves the incremental data corresponding to a certain key from the shared memory pool, it can also continue to retain the incremental data corresponding to that key in the shared memory pool.

[0202] Step 440 can be achieved through step 2C).

[0203] 2C) If the incremental data in the shared memory pool includes key #2, perform operation #2 in the shared memory pool. Otherwise, perform operation #2 in the target node.

[0204] In this scenario, during incremental data synchronization, all access to the modified key can be performed within the shared memory pool.

[0205] Further, step 2C) may include: if the incremental data in the shared memory pool includes key #2, and the incremental data corresponding to key #2 has not yet been obtained from the target node, then perform operation #2 in the shared memory pool. Otherwise, perform operation #2 on the target node.

[0206] In this scenario, during incremental data synchronization, all access to the modified key can be performed either in the shared memory pool or on the target node. Specifically, if the modified key has been retrieved from the target node (i.e., it has been synchronized), the corresponding operation can be performed on the target node. If the modified key has not yet been retrieved from the target node and has not yet been synchronized, the corresponding operation can be performed in the shared memory pool.

[0207] Optionally, step 460 may include: the target node retrieving the data obtained from operation #2 from the shared memory pool.

[0208] Operation #2 can be a write operation. Incremental data can include the data obtained from this write operation.

[0209] If operation #2 is performed in the shared memory pool and is a write operation, the target node can obtain the corresponding incremental data from the shared memory pool.

[0210] The numbers #2, #2, and #2 are for descriptive convenience only and have no limiting effect. Any key belonging to this slot can be considered as key #2, any operation request for key #2 can be considered as operation request #2, and any operation related to key #2 can be considered as operation #2.

[0211] The word "second" in "second operation request," "second data," and "second operation" is used to describe the scheme and has no limiting function.

[0212] Furthermore, different states can be set for the key, such as a normal state and a migrating state.

[0213] The normal state indicates that operations related to the key should be performed in local memory. Local memory can be the local memory of the source node or the local memory of the target node.

[0214] The migrating status indicates that an operation related to that key will be performed in the shared memory pool.

[0215] Below is an example of how to set the status of a key.

[0216] (1) When slot migration is not performed, the key is in the normal state on the source node, and read and write operations can be performed on the key in the local memory of the source node.

[0217] (2) After slot migration begins, for example, after the slot on the source node is set to the migrating state, before the slot's owner node changes from the source node to the target node, the source node receives a write operation request for the key, the key's state is changed from the normal state to the migrating state, the source node copies the key to the shared memory pool, and performs a write operation on the key in the shared memory pool.

[0218] During this period, if the source node receives another operation request related to that key, it can execute the corresponding operation in the shared memory pool. In this way, both incremental data read and write operations can be completed in the shared memory pool.

[0219] (3) If the target node receives an operation request related to the key before it obtains the key from the shared memory pool, it can perform the corresponding operation in the shared memory pool.

[0220] After the target node obtains the key from the shared memory pool, the key's status is changed from migrating to normal, and then read and write operations can be performed on the key in the target node's local memory.

[0221] It should be understood that the above are only possible implementations of operations on incremental data, and do not constitute a limitation on the solutions of the embodiments of this application.

[0222] Furthermore, once it is confirmed that all incremental data has been synchronized, the target node can mark the slot as having completed the migration. After that, all data on the slot can be accessed in local memory.

[0223] In the solution of this application embodiment, after the migration of all data is completed, the owner node of the slot can be set to the target node. Subsequent client operation requests can be sent to the target node, and access to the data in the slot can be achieved through the target node. For the client, it is equivalent to the migration being completed, which improves the migration speed.

[0224] Method 2;

[0225] In Method 2, step 450 can be executed after the incremental data transmission is complete (i.e., step 460 is complete). In this case, the operation request related to the slot can be received by the source node. Steps 430 and 440 can be implemented through steps 3A) and 3B) as described below.

[0226] 3A) The source node receives operation request #3, which indicates operation #3 related to key #3. Key #3 belongs to this slot.

[0227] 3B) Perform operation #3 in the shared memory pool.

[0228] The numbers #3, #3, and #3 are for descriptive convenience only and have no limiting effect. Any key belonging to this slot can be considered as key #3, any operation request for key #3 can be considered as operation request #3, and any operation related to key #3 can be considered as operation #3.

[0229] For example, during slot migration, the source node can continue to receive operation requests for that slot. Taking operation #3 as a write operation as an example, the source node can copy key #3 to the shared memory pool, perform the write operation in the shared memory pool, and the data obtained from the write operation is the incremental data corresponding to key #3. If the source node subsequently receives another operation related to key #3, it can perform that operation in the shared memory pool based on the incremental data corresponding to key #3.

[0230] Optionally, step 460 may include: the target node retrieving the data obtained from operation #3 from the shared memory pool.

[0231] Operation #3 can be a write operation. Incremental data can include the data obtained from this write operation.

[0232] Once it is confirmed that all incremental data has been synchronized, step 450 can be executed. After that, all data on the slot can be accessed in the local memory of the target node.

[0233] Figure 6 illustrates a migration process according to an embodiment of this application. The scheme shown in Figure 6 can be considered as a specific implementation of the scheme shown in Figure 4. To avoid repetition, some descriptions are omitted when describing the scheme shown in Figure 6.

[0234] As shown in Figure 6, the process includes the following steps.

[0235] S1, set the slot to be migrated to the import state on the target node.

[0236] For example, as shown in Figure 6, there are 100 slots to be migrated (slot 1-slot 100). The client sets slots 1-slot 100 to the import state on the target node.

[0237] It should be understood that the number of slots to be migrated, the slot numbers to be migrated, the number of source nodes, and the number of target nodes in Figure 6 are all examples and do not constitute a limitation on the scheme of the embodiments of this application.

[0238] S2, the target node sends a return value to the client.

[0239] The client sends a command to the target node to set slots 1-100 to the import state. The target node can then send a return value to the client. The target node is ready to receive data from slots 1-100.

[0240] S3, set the slot to be migrated to the migrating state on the source node.

[0241] As shown in Figure 6, the client sets slots 1-100 to migrating state on the source node.

[0242] S4, the source node sends a return value to the client.

[0243] The client sends a command to the source node to set slots 1-100 to the migrating state, and the source node can send a return value to the client. The source node is ready to migrate the data from slots 1-100.

[0244] Steps S5 to S24 can be executed in a loop until all slots have been migrated.

[0245] S5, the source node copies the full data of the slot to the shared memory pool.

[0246] The child process is forked to copy the entire slot's data to the shared memory pool.

[0247] S6, the shared memory pool sends a return value to the source node to indicate that the replication was successful.

[0248] As shown in Figure 6, the shared memory pool notifies the source node that replication was successful.

[0249] S7, the source node receives an operation request for the key belonging to this slot.

[0250] For example, a client may need to perform operations on an existing key belonging to that slot, and can send an operation request related to that key to the source node.

[0251] S8, the source node writes the key to the shared memory pool.

[0252] The source node can copy this key to the shared memory pool, which will then execute the operation indicated in the operation request. All subsequent operations on this key will be performed by the shared memory pool.

[0253] S9, the shared memory pool sends a return value to the source node to indicate the operation result.

[0254] S10, the source node sends a return value to the client to indicate the operation result.

[0255] As shown in Figure 6, "par" indicates parallel execution. Steps S5 and S6, and steps S7 to S10 are executed in parallel, that is, the operation request in step S7 is received during the execution of steps S5 to S6.

[0256] S11, the source node notifies the target node to obtain the full data from the shared memory pool.

[0257] S12, the target node sends a return value to the source node.

[0258] S13, the target node retrieves the full data from the shared memory pool.

[0259] S14, the shared memory pool sends a return value to the target node.

[0260] S15, the target node reconstructs the data within the node.

[0261] The target node reconstructs the data in the slot based on the acquired full data within the node.

[0262] S16, the source node broadcasts information to notify the slot's owner node as the target node.

[0263] The source node assigns a slot to the target node and broadcasts the information.

[0264] After that, all the data in the slot to be migrated exists in the shared memory pool and the target node, and all the data in the slot can be accessed on the target node.

[0265] The following steps S17 to S23 can be executed repeatedly until all incremental data for the slot is synchronized.

[0266] S17, the target node retrieves the incrementally modified key from the shared memory pool.

[0267] That is, the target node retrieves incremental data from the shared memory pool at the key level.

[0268] S18, the shared memory pool sends a return value to the target node.

[0269] S19, the target node receives an operation request for the key belonging to the slot.

[0270] For example, a client may need to perform an operation on a key belonging to that slot, and can send an operation request related to that key to the target node.

[0271] S20, the target node can determine whether the key is in the shared memory pool.

[0272] If the key belongs to the incremental data in the shared memory pool, then the operation requested by the operation is performed in the shared memory pool, and step S21 is executed.

[0273] If the key is not in the incremental data of the shared memory pool, then the operation requested by the operation is executed on the target node, and step S22 is executed.

[0274] S21, the target node retrieves data from the shared memory pool.

[0275] For example, a shared memory pool sends a return value to the target node to indicate the result of the operation.

[0276] S22, the target node sends a return value to the client to indicate the operation result.

[0277] "alt" indicates a choice.

[0278] As shown in Figure 6, steps S17 and S18, and steps S19 to S22 are executed in parallel, that is, the operation request in step S19 is received during the execution of steps S17 and S18.

[0279] S23, the target node marks the slot as having completed migration.

[0280] After this, all data in the slot can be accessed in the local memory of the target node.

[0281] To minimize the impact on performance, incremental data is typically synchronized asynchronously. For example, the target node can mark the slot to complete an asynchronous migration.

[0282] The dashed lines in Figure 6 represent the sent return values.

[0283] The apparatus of the present application embodiment will now be described with reference to Figures 7 to 10. It should be understood that the apparatus described below is capable of performing the methods of the foregoing embodiments of the present application. To avoid unnecessary repetition, repeated descriptions will be appropriately omitted when introducing the apparatus of the present application embodiment.

[0284] Figure 7 is a schematic block diagram of a data migration system according to an embodiment of this application. The system 2000 shown in Figure 7 can be used to perform the methods shown in Figure 4 and / or Figure 6.

[0285] Taking the system 2000 for executing the method 400 shown in Figure 4 as an example, as shown in Figure 7, the system 2000 may include a target node 2010 and a source node 2020.

[0286] The target node 2010 is used to receive the first migration command, which instructs to migrate the data of the first slot on the source node to the target node.

[0287] Source node 2020 is used for:

[0288] Receive a second migration command, which instructs that the data in the first slot on the source node be migrated to the target node;

[0289] Copy the data from the first slot to the shared memory pool.

[0290] The target node 2010 is also used to retrieve data from the first slot in the shared memory pool.

[0291] Optionally, the source node 2020 is also used to: generate the full data of the first slot; and specifically, the source node 2020 is used to: copy the full data of the first slot to the shared memory pool.

[0292] Optionally, the source node 2020 is also configured to: receive a first operation request, the first operation request indicating a first operation related to first data, the first data belonging to a first slot, and the first operation being executed in a shared memory pool.

[0293] Optionally, the target node 2010 is also used to: retrieve the data obtained from the first operation from the shared memory pool.

[0294] Optionally, before the target node 2010 is further configured to obtain the data obtained from the first operation from the shared memory pool, the source node 2020 and / or the target node 2010 are further configured to: send first information indicating that the first slot is allocated to the target node 2010.

[0295] Optionally, the target node 2010 is further configured to: receive a second operation request, the second operation request indicating a second operation related to second data, the second data belonging to the first slot; if the second data is included in the incremental data of the shared memory pool, the second operation is executed in the shared memory pool, or if the second data is not included in the incremental data of the shared memory pool, the second operation is executed in the target node 2010.

[0296] For a detailed description, please refer to Method 400 above; it will not be repeated here.

[0297] Each device in system 2000 can be implemented in software or hardware. For example, the implementation of target node 2010 will be described below. Similarly, the implementation of other devices can follow the same method as target node 2010.

[0298] As an example of a software functional unit, the target node 2010 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, the aforementioned computing instance may be one or more. For example, the target node 2010 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ comprising one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0299] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0300] As an example of a hardware functional unit, the target node 2010 may include at least one computing device, such as a server. Alternatively, the target node 2010 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0301] The target node 2010 includes multiple computing devices that can be distributed within the same region or in different regions. These computing devices can be distributed within the same Availability Zone (AZ), within the same Virtual Private Cloud (VPC), or across multiple VPCs. Furthermore, these multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0302] It should be noted that in other embodiments, the target node 2010 can be used to execute any step in the method described above, and other devices can be used to implement any step in the method described above. The steps to be implemented by each device can be specified as needed, and all functions of the system 2000 can be realized by implementing different steps in the method described above by each device.

[0303] This application also provides a computing device 1000. As shown in FIG8, the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other via the bus 1002. The computing device 1000 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1000.

[0304] Bus 1002 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 8, but this does not imply that there is only one bus or one type of bus. Bus 1002 can include pathways for transmitting information between various components of computing device 1000 (e.g., memory 1006, processor 1004, communication interface 1008).

[0305] The processor 1004 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0306] The memory 1006 may include volatile memory, such as random access memory (RAM). The processor 1004 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0307] The memory 1006 stores executable program code, and the processor 1004 executes the executable program code to implement the functions of the aforementioned devices, thereby implementing the method described above. That is, the memory 1006 stores instructions for executing the method described above.

[0308] The communication interface 1008 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1000 and other devices or communication networks.

[0309] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0310] As shown in Figure 9, the computing device cluster includes at least one computing device 1000. The memory 1006 of one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the methods described above.

[0311] In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the methods described above. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions for performing the methods described above.

[0312] It should be noted that the memory 1006 in different computing devices 1000 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the system described above. That is, the instructions stored in the memory 1006 of different computing devices 1000 can implement the functions of one or more devices in the system 2000.

[0313] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.

[0314] Figure 10 illustrates one possible implementation. As shown in Figure 10, two computing devices 1000A and 1000B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1006 in computing device 1000A stores instructions for executing the functions of target node 2010. Simultaneously, the memory 1006 in computing device 1000B stores instructions for executing the functions of source node 2020.

[0315] The connection method between the computing device clusters shown in Figure 10 can be considered as taking into account that the method described above in this application may need to store data, so the function implemented by the source node 2020 is handed over to the computing device 1000B for execution.

[0316] It should be understood that the functions of computing device 1000A shown in Figure 10 can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices 1000.

[0317] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the methods described above.

[0318] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the methods described above.

[0319] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0320] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0321] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0322] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0323] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0324] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the processing methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0325] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data migration method, characterized in that, The method is applied to a distributed storage system, the distributed storage system comprising multiple nodes, at least some of the nodes having one or more slots distributed on them, and the method comprising: The target node receives a first migration command, which instructs the migration of data in the first slot on the source node to the target node, wherein the source node and the target node belong to the plurality of nodes; The source node receives a second migration command, which instructs the data in the first slot on the source node to be migrated to the target node. The source node copies the data from the first slot to the shared memory pool; The target node obtains the data from the first slot from the shared memory pool.

2. The method according to claim 1, characterized in that, The method further includes: The source node generates all the data for the first slot; and the source node copies the data for the first slot to the shared memory pool, including: The source node copies all the data from the first slot to the shared memory pool.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The source node receives a first operation request, which indicates a first operation related to first data, and the first data belongs to the first slot. The first operation is performed in the shared memory pool.

4. The method according to claim 3, characterized in that, The method further includes: The target node obtains the data obtained from the first operation from the shared memory pool.

5. The method according to claim 4, characterized in that, Before the target node retrieves the data obtained from the first operation from the shared memory pool, the method further includes: The source node and / or the target node send a first message indicating that the first slot is allocated to the target node.

6. The method according to claim 5, characterized in that, The method further includes: The target node receives a second operation request, which indicates a second operation related to second data, the second data belonging to the first slot; If the incremental data in the shared memory pool includes the second data, then the second operation is performed in the shared memory pool, or... If the second data is not included in the incremental data of the shared memory pool, the second operation is performed on the target node.

7. A data migration system, characterized in that, The system is a distributed storage system, comprising multiple nodes, at least some of which have one or more slots distributed across them. The multiple nodes include a target node and a source node. The target node is used to receive a first migration command, which instructs to migrate the data of the first slot on the source node to the target node. The source node is used for: Receive a second migration command, which instructs that the data in the first slot on the source node be migrated to the target node; Copy the data from the first slot to the shared memory pool; The target node is also used to obtain data from the first slot from the shared memory pool.

8. The system according to claim 7, characterized in that, The source node is also used for: Generate the full data for the first slot; and the source node is specifically used for: Copy all data from the first slot to the shared memory pool.

9. The system according to claim 7 or 8, characterized in that, The source node is also used for: A first operation request is received, the first operation request indicating a first operation related to first data, the first data belonging to the first slot, and the first operation being performed in the shared memory pool.

10. The system according to claim 9, characterized in that, The target node is also used for: Retrieve the data obtained from the first operation from the shared memory pool.

11. The system according to claim 10, characterized in that, Before the target node is further configured to retrieve the data obtained from the first operation from the shared memory pool, the source node and / or the target node are further configured to: Send a first message indicating that the first slot is assigned to the target node.

12. The system according to claim 11, characterized in that, The target node is also used for: Receive a second operation request, the second operation request indicating a second operation related to second data, the second data belonging to the first slot; If the incremental data in the shared memory pool includes the second data, the second operation is performed in the shared memory pool, or... If the second data is not included in the incremental data of the shared memory pool, the second operation is performed on the target node.

13. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 6.

15. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1 to 6.

Images