Data migration method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a data migration method and device, computer equipment and a storage medium, the data migration method comprises the steps of firstly migrating an original key value pair of an old storage pool to a new storage pool, migrating stock data, during the period, creating a backup for a first target key value pair targeted by a user write request, and writing the backup key value pair into a new storage pool. And then, assigning the value of the backup key value pair in the new storage pool to the corresponding original key value pair, migrating the incremental data, and during the period, writing the backup key value pair of the second target key value pair and the second target key value pair which are aimed at by the user write request into the new storage pool. And the values of the original key value pair and the backup key value pair in the same group are consistent and are the latest values through assignment operation. And finally, redundant data are cleared by clearing the backup key value pairs of the new storage pool. Therefore, the write request of the user is supported during the migration of the stock data and the incremental data, the problem of concurrent write conflict does not exist, and the accuracy and the real-time performance of read-write data are ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data migration method, device, computer equipment and storage medium. Background Art

[0002] Key-Value (KV) storage is a data storage method based on key-value pairs and is widely used in various scenarios. As key-value storage services expand and systems are upgraded, it may be necessary to upgrade the existing KV storage system to one with better performance or lower storage costs. This involves migrating data from one KV storage system to another.

[0003] During data migration, to avoid concurrent write conflicts and data loss caused by repeated write requests for the same key, all user write requests are currently prohibited during data migration. However, this approach sacrifices service availability. If the data migration time is long, user requests may be unavailable for a long time. Summary of the Invention

[0004] In view of this, the present invention provides a data migration method, apparatus, computer device and storage medium to solve the problem that the existing method stops user services during data migration, affecting the user experience.

[0005] In a first aspect, the present invention provides a data migration method, the method comprising:

[0006] In response to the existing data migration instruction, all original key-value pairs in the old storage pool are written into the new storage pool;

[0007] During the migration of existing data, in response to a user's write request for a first target key-value pair, creating a backup of the first target key-value pair, and writing the obtained backup key-value pair into a new storage pool;

[0008] In response to the incremental data migration instruction, assigning the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair;

[0009] During the incremental data migration, in response to a user's write request for the second target key-value pair, writing a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair in the new storage pool;

[0010] After the incremental data migration is complete, clear the backup key-value pairs stored in the new storage pool.

[0011] Beneficial effect: The present application first migrates all the original key-value pairs in the old storage pool to the new storage pool to migrate the existing data. During the migration of existing data, a backup is created for the first target key-value pair targeted by the user's write request, and the corresponding backup key-value pair is written into the new storage pool to save the latest value written by the user during the migration of existing data. Then, the value of the backup key-value pair in the new storage pool is assigned to the corresponding original key-value pair to migrate the incremental data. And during the incremental data migration, the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair targeted by the user's write request are written into the new storage pool to save the latest value written by the user during the incremental data migration, and through the assignment operation during the incremental data migration, it is ensured that the values ​​of the same group of original key-value pairs and backup key-value pairs are consistent and the latest values. Finally, after the incremental data migration is completed, the backup key-value pairs in the new storage pool are cleared to clean up the redundant data and complete the data migration from the old storage pool to the new storage pool. The present invention supports user write requests all the time during the migration of stock data and incremental data, which is beneficial to user experience, does not cause concurrent write conflicts, and can ensure the accuracy and real-time performance of stored data.

[0012] In an optional implementation, creating a backup of the first target key-value pair includes:

[0013] Rename the key in the first target key-value pair to obtain a backup key; wherein the first target key-value pair includes the original key-value pair;

[0014] The backup key and the value in the first target key-value pair are concatenated to obtain a backup key-value pair corresponding to the first target key value.

[0015] Beneficial effect: During the migration of existing data, the present application renames the key of the first target key-value pair targeted by the user's write request to form a backup key-value pair to save the latest value written by the user, thereby avoiding concurrent write conflicts between the existing data migration operation and the user write operation on the value of the same key.

[0016] In an optional embodiment, during the migration of existing data, the method further includes:

[0017] In response to a user's read request for a target key, query the backup key-value pairs in the new storage pool according to the backup key corresponding to the target key, and return the first target value obtained from the query to the user;

[0018] Alternatively, based on the target key, a query is performed on the original key-value pairs in the new storage pool, and the first target value obtained by the query is returned to the user;

[0019] Alternatively, based on the target key, a query is performed in the original key-value pairs of the old storage pool, and the first target value obtained by the query is returned to the user.

[0020] Beneficial Effects: During the migration of existing data, this application first queries the new storage pool for the backup key corresponding to the target key in the user's read request, and returns the latest value of the target key to the user. If the backup key corresponding to the target key does not exist in the new storage pool, the target key is queried in the new storage pool and the old storage pool, and the corresponding value is returned to the user. This allows the user to read the latest and correct value corresponding to the target key, ensuring the real-time and accuracy of the returned data.

[0021] In an optional implementation, writing the backup key-value pair corresponding to the second target key-value pair and the second target key-value pair into the new storage pool includes:

[0022] Writing a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair into the new storage pool in sequence; wherein the second target key-value pair includes the first target key-value pair or the original key-value pair;

[0023] Assign the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair.

[0024] Beneficial Effects: During incremental data migration, the present application saves the latest value written by the user for the second target key-value pair requested by the user by writing the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair to the new storage pool. Furthermore, by assigning the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair, the values ​​of the same set of backup key-value pairs and the original key-value pairs in the new storage pool are consistent and the latest values, thus avoiding concurrent write conflicts between the migration operation and the user write operation on the same key value during incremental data migration.

[0025] In an optional embodiment, during the incremental data migration, the method further includes:

[0026] In response to a user's read request for the target key, query the backup key-value pairs in the new storage pool according to the backup key corresponding to the target key, and return the second target value obtained from the query to the user;

[0027] Alternatively, based on the target key, a query is performed in the original key-value pairs in the new storage pool, and the second target value obtained by the query is returned to the user.

[0028] Beneficial Effects: During incremental data migration, this application first queries the new storage pool for the backup key corresponding to the target key in a user read request, returning the latest value of the target key to the user. If the backup key corresponding to the target key does not exist in the new storage pool, the new storage pool is queried for the target key and the corresponding value is returned to the user. This allows the user to read the latest and correct value corresponding to the target key, ensuring the real-time and accuracy of the returned data.

[0029] In an optional embodiment, after clearing the backup key-value pairs stored in the new storage pool, the method further includes:

[0030] In response to a user's write request for a third target key-value pair, the third target key-value pair is written into the new storage pool; wherein the third target key-value pair includes the second target key-value pair, the first target key-value pair, or the original key-value pair.

[0031] Beneficial effect: After the application completes the clearing of all backup key-value pairs in the new storage pool, the data migration is completed. If a third target key-value pair is received for a user write request, the third target key-value pair can be directly written into the new storage pool to realize the writing of new data.

[0032] In an optional embodiment, after clearing the backup key-value pairs stored in the new storage pool, the method further includes:

[0033] In response to a read request from the user for the target key, a query is performed in the original key-value pairs in the new storage pool according to the target key, and a third target value obtained by the query is returned to the user.

[0034] Beneficial effect: After this application completes the clearing of all backup key-value pairs in the new storage pool, it means that the data migration is completed. If a user read request is received for a target key, the target key is directly queried in the new storage pool, and the corresponding value is returned to the user to realize data reading.

[0035] In a second aspect, the present invention provides a data migration device, the device comprising:

[0036] A first processing module is configured to write all original key-value pairs in the old storage pool into the new storage pool in response to an existing data migration instruction;

[0037] a second processing module, configured to, during the migration of the existing data, create a backup of the first target key-value pair in response to a user's write request for the first target key-value pair, and write the obtained backup key-value pair into a new storage pool;

[0038] a third processing module, configured to assign, in response to the incremental data migration instruction, the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair;

[0039] a fourth processing module, configured to, during the incremental data migration, write a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair in the new storage pool in response to a user's write request for the second target key-value pair;

[0040] The fifth processing module is used to clear the backup key-value pairs stored in the new storage pool after the incremental data migration is completed.

[0041] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data migration method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data migration method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 2 is a schematic diagram of a concurrent write conflict scenario according to an embodiment of the present invention;

[0045] Figure 2 is a flowchart of a data migration method according to an embodiment of the present invention;

[0046] Figure 3 is a flow chart of another data migration method according to an embodiment of the present invention;

[0047] Figure 4 is a schematic diagram of the architecture of a data migration method according to an embodiment of the present invention;

[0048] Figure 5 1 is a schematic diagram of an architecture for existing data migration according to an embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of a process of existing data migration according to an embodiment of the present invention;

[0050] Figure 7 is a schematic diagram of an architecture for incremental data migration according to an embodiment of the present invention;

[0051] Figure 8 is a schematic diagram of a process of incremental data migration according to an embodiment of the present invention;

[0052] Figure 9 is a schematic diagram of an architecture for redundant data cleaning according to an embodiment of the present invention;

[0053] Figure 10is a schematic diagram of a process for clearing redundant data according to an embodiment of the present invention;

[0054] Figure 11 is a structural block diagram of a data migration device according to an embodiment of the present invention;

[0055] Figure 12 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0057] In a key-value store, each data item consists of a unique key and an associated value. The key is unique, while the value can be any arbitrary data. Key-value stores play a vital role in internet services due to their simple, flexible data model and scalability. They are commonly used for fast data access and retrieval, and are widely used in various scenarios, such as caching and image and video storage, due to their high scalability and flexibility.

[0058] With the expansion of key-value storage business and system upgrade iterations, it may be necessary to upgrade the existing KV storage system to a KV storage system with better performance and lower storage cost, which will involve data migration. Data migration is a complex but crucial task that involves the integrity, availability, and correctness of data. Data migration tasks usually face the following technical challenges: 1) The huge challenge of data scale: As the amount of data continues to grow, data migration tasks become more complex and time-consuming; 2) Service availability: During the migration process, how to ensure business availability and reduce downtime is an important challenge. The migration process needs to be carefully planned to reduce the service unavailability time during the migration process to reduce the impact on the business; 3) Data correctness: Ensure the correctness and integrity of the data before and after migration, especially in concurrent scenarios. It is necessary to ensure that there are no concurrency conflicts and that the data is correct and not missed.

[0059] For example, under the premise of ensuring the availability of KV storage services during data migration, there will be a concurrent conflict between the migration tool migrating key values ​​from the old storage system to the new storage system and the user writing the same key values. Figure 1As shown in the example, the old storage system has a key-value pair key:value1. During data migration, the user rewrites a new value for this key, key:value2. Before the migration, the migration tool checks to see if the new storage system has the key and performs the migration. Then, after the user writes the new value, the old value key:value1 is written to the new storage system. When the user reads the key again, the old value value1 is returned. Consequently, repeated write requests for the same key may cause concurrent writes with the migration tool, resulting in the user's newly written data being overwritten by the old data being migrated. The user can only read the old data, resulting in the loss of the new data.

[0060] According to an embodiment of the present invention, an embodiment of a data migration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] In this embodiment, a data migration method is provided, which can be used for devices that perform data migration, such as computers, tablet computers, etc. Figure 2 is a flow chart of a data migration method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0062] Step S201 : In response to an existing data migration instruction, all original key-value pairs in the old storage pool are written into the new storage pool.

[0063] Specifically, after receiving a stock data migration instruction instructing the migration of existing data, in response to the stock data migration instruction, all existing data in the old storage pool must first be migrated to the new storage pool, where the existing data includes the original key-value pairs stored in the old storage pool, such as key:value1. It should be noted that the old storage pool refers to the source storage pool of the data, and the new storage pool refers to the target storage pool to which the data is migrated.

[0064] In step S201, all original key-value pairs in the old storage pool are written to the new storage pool as is. After the original key-value pairs are migrated to the new storage pool, their key values ​​will not change. For example, the original key-value pair key:value1 in the old storage pool remains key:value1 after being migrated to the new storage pool.

[0065] Step S202 : During the migration of existing data, in response to a user's write request for a first target key-value pair, a backup of the first target key-value pair is created, and the obtained backup key-value pair is written into a new storage pool.

[0066] Specifically, during the migration of existing data, users will continue to issue read and write requests. For example, a user might issue a write request for the first target key-value pair, such as key:value2, to update the value of key to value2. To ensure user service availability, during the migration of existing data, a backup of the first target key-value pair targeted by the user's write request is created, and the backup key-value pair of the first target key-value pair, such as key_backup:value2, is written to the new storage pool.

[0067] In this embodiment, the original key-value pairs are not backups of other key-value pairs, and the backup key-value pairs are backups of their corresponding original key-value pairs. The keys of the same set of original key-value pairs and backup key-value pairs have the same meaning and both store the value of the same parameter. However, in order to distinguish and facilitate searching, a backup key-value pair of the original key-value pair can be formed by adding a backup identifier to the key of the original key-value pair, renaming the key of the original key-value pair, and performing other operations, such as key_backup. That is, during data migration, the new storage pool stores both the original key-value pairs and the backup key-value pairs corresponding to the original key-value pairs.

[0068] In step S202, the backup key-value pair corresponding to the first target key-value pair targeted by the user's write request is written to the new storage pool, continuing to provide write request services to the user and saving the latest value of the key in the new storage. As a result, when the user issues a read request for a key during the migration of existing data, the latest value written by the user can be obtained from the backup key-value pair stored in the new storage pool, thereby ensuring the accuracy of the read data and not affecting the migration of the original key-value pair.

[0069] Step S203 : In response to the incremental data migration instruction, assign the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair.

[0070] Specifically, stock data refers to all data stored in the old storage pool before the start of the stock data migration. Incremental data refers to data newly written by users after that time, namely, the backup key-value pairs stored in the new storage pool during the stock data migration. After the stock data migration is completed, the incremental data migration instruction instructing the migration of incremental data is received, and the incremental data generated during the stock data migration is migrated.

[0071] In step S203, all backup key-value pairs in the new storage pool are scanned, and the original key-value pair corresponding to each backup key-value pair is determined, and the value of the backup key-value pair is assigned to the corresponding original key-value pair.

[0072] In this embodiment, during the migration of existing data, the user requests to update the value of the original key-value pair through a write operation. Step S202 only creates a backup key-value pair that stores the latest value to ensure that it does not affect user services and the accuracy of user data reading. Step S203 formally updates the value of the original key-value pair to the latest value through an assignment operation.

[0073] In some embodiments, the first target key-value pair includes the original key-value pair in the old storage pool (the key of the original key-value pair already exists in the old storage pool). Step S202 creates a backup key-value pair corresponding to the first target key-value pair, such as key:value2, such as key_backup:value2. Step 203 assigns the value value2 of the backup key-value pair, such as key_backup:value2, to its corresponding original key-value pair to form key:value2, thereby realizing the numerical update of the original key-value pair.

[0074] In some embodiments, the first target key-value pair also includes a new original key-value pair (the key of the original key-value pair does not exist in the old storage pool and is newly created). Step 202 creates a backup key-value pair corresponding to the first target key-value, such as the newkey:value pair, such as newkey_backup:value. Step 203 writes the value of the above-mentioned backup key-value pair, such as newkey_backup:value, into the corresponding original key-value pair to form newkey:value, thereby realizing new data writing and avoiding clearing new data when subsequent data is cleared.

[0075] Step S204 : During the incremental data migration, in response to a user's write request for the second target key-value pair, a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair are written into the new storage pool.

[0076] Specifically, during the migration of the incremental data generated in step S202, the user may continue to issue write requests. The user may issue a write request for the second target key-value pair, such as key:value3, to update the value of key to value3. To ensure the availability of user services, during the incremental data migration, a backup of the second target key-value pair targeted by the user's write request is created, and the backup key-value pair corresponding to the second target key-value pair and the second target key-value pair are written to the new storage pool.

[0077] In some embodiments, the second target key-value pair includes the first target key-value pair, that is, the user may update the value of the same key in step S202 and step S204. If the assignment operation of step S203 has been completed before step S204, for example, the value of key has been updated to value2 in step S203, and key_backup:value2 and key:value2 are stored in the new storage pool, then when step S204 is executed, the backup key-value pair corresponding to the second target key-value pair, such as key_backup:value3, and the second target key-value pair, such as key:value3, will be written to the new storage pool. Since key_backup and key already exist and are unique in the new storage pool, this write operation will directly overwrite the values ​​of key_backup and key with value3.

[0078] Furthermore, if the assignment operation in step S203 has not yet been performed, and the user has already issued a write request during the incremental data migration, then, following the above step example, the new storage pool currently contains key_backup:value2 and key:value1 (which have not yet been assigned). The user first issues a write request for the second target key-value pair, such as key:value3. Then, step S204 is executed to write the backup key-value pair, such as key_backup:value3, and the second target key-value pair, such as key:value3, corresponding to the second target key-value pair, such as key:value3, into the new storage pool. Since key_backup and key already exist and are unique in the new storage pool, the write operation in step S204 will overwrite the values ​​of key_backup and key with value3. When it is the turn of key_backup to assign a value to key in step S203, the values ​​of key_backup and key are still the latest value, value3.

[0079] In this embodiment, when performing incremental data migration, the order of the assignment operation in step S203 and the user's write operation for the same key in step S204 does not affect the accuracy and real-time performance of the data. That is, during the incremental data migration, user write requests are still supported and the accuracy of user read data can be ensured.

[0080] In some embodiments, the second target key-value pair includes the original key-value pair (the key of the original key-value pair already exists in the old storage pool and the new storage pool). Step S204 creates a backup key-value pair corresponding to the second target key-value pair, such as key:value3, such as key_backup:value3, and writes key_backup:value3 and key:value3 to the new storage pool. Due to the uniqueness of the key, the values ​​of key_backup and key in the new storage pool will be updated to the latest value value3 to achieve numerical update.

[0081] In some embodiments, the first target key-value pair also includes a new original key-value pair (the key of the original key-value pair does not exist in the new storage pool and is newly created). For example, if the user requests to write newkey:value, step S204 will write newkey_backup:value and newkey:value in the new storage pool to realize the writing of new data and avoid clearing the new data when the data is cleared subsequently.

[0082] Step S205: After the incremental data migration is completed, the backup key-value pairs stored in the new storage pool are cleared.

[0083] Specifically, during incremental data migration, the value of the backup key-value pair is assigned to the corresponding original key-value pair. After receiving the second target key-value pair targeted by the user's write request, the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair are written into the new storage pool. This ensures that each original key-value pair in the new storage pool has a corresponding backup key-value pair stored, and the values ​​of the same set of backup key-value pairs and the original key-value pairs are the same. In other words, at this point, two copies of the latest value for the same key are stored in the new storage pool. The backup key-value pairs in the new storage pool are cleared to eliminate redundant data and complete the data migration.

[0084] Compared with the related art method of prohibiting all new write requests or repeated write requests for existing keys during data migration in order to avoid concurrency issues, this application does not sacrifice the availability of the service. Even if the data migration time is long, it supports long-term availability of user requests.

[0085] Some related technologies also introduce distributed lock components to coordinate the execution order of migration and user writes through distributed locks. The lock is first locked before executing the query logic of the new storage system, and the lock is released after the migration is completed. Similarly, the lock is first locked before executing the user write request, and the lock is released after the write is completed. The related technologies avoid concurrency through the lock mechanism, but it also involves the interaction of data migration, read requests, write requests and distributed lock components, which has the problem of time consumption. And each key data needs to be locked, but the inventory of data to be migrated is generally much larger than the user's new write request, so the cost of locking is very high. The present application does not need to restrict the execution order of user read requests, write requests and data migration, and can complete the migration of existing data and the new write operation of incremental data, which is time-saving and low-cost.

[0086] Some related technologies also rely on the Compare-And-Swap (CAS) operation of the storage system. The CAS operation is an atomic operation that modifies the value of a variable, which turns the two steps of querying and writing to the new storage system into one atomic operation, thereby avoiding concurrency problems. Before modifying the variable, the CAS operation will first check whether the current value in the memory is equal to the expected original value. If equal, the value in the memory is updated to the new value. The entire comparison and exchange process is atomic, and the operation will not be interrupted by the operations of other threads, thereby ensuring that the update of shared data in a concurrent environment is safe. However, not all existing KV storage systems support CAS operations, and the KV storage systems that support CAS operations are relatively expensive; in the commonly used distributed key-value database HBase, one key corresponds to multiple versions of value, but only the latest version of the value supports CAS operations, and the migration of other versions of data cannot be guaranteed to be correct in concurrent scenarios through CAS operations. The present application can realize the migration of all versions of data, and supports users to perform write operations during the migration of existing data and incremental data, and ensure the accuracy of the data.

[0087] The data migration method provided by this embodiment first migrates all the original key-value pairs in the old storage pool to the new storage pool to migrate the existing data. During the migration of the existing data, a backup is created by creating a backup of the first target key-value pair targeted by the user's write request, and the corresponding backup key-value pair is written into the new storage pool to save the latest value written by the user during the migration of the existing data. Then, the value of the backup key-value pair in the new storage pool is assigned to the corresponding original key-value pair to migrate the incremental data. During the incremental data migration, the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair targeted by the user's write request are written into the new storage pool to save the latest value written by the user during the incremental data migration, and through the assignment operation during the incremental data migration, it is ensured that the values ​​of the same group of original key-value pairs and backup key-value pairs are consistent and the latest values. Finally, after the incremental data migration is completed, the backup key-value pairs in the new storage pool are cleared to clean up the redundant data and complete the data migration from the old storage pool to the new storage pool. The present invention supports user write requests all the time during the migration of stock data and incremental data, which is beneficial to user experience, does not cause concurrent write conflicts, and can ensure the accuracy and real-time performance of stored data.

[0088] In this embodiment, a data migration method is provided, which can be used for devices that perform data migration, such as computers, tablet computers, etc. Figure 3 is a flow chart of a data migration method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0089] Step S301: In response to an existing data migration instruction, all original key-value pairs in the old storage pool are written into the new storage pool.

[0090] Specifically, Figure 4 The figure shows the architecture diagram of the data migration method of the present application. The storage service is responsible for receiving the key-value pairs of the user's write request and writing them to the new storage pool. It can also receive user read requests and query the corresponding values ​​from the new storage pool and the old storage pool. The storage pool can be a built storage database responsible for storing user data. In the data migration scenario, it can be divided into the old storage pool and the new storage pool. The distributed coordinator is responsible for synchronizing information with the storage service. The storage service can obtain change information such as various migration instructions in a timely manner by monitoring the distributed coordinator. The migration tool is responsible for copying user data such as key:value from the old storage pool to the new storage pool.

[0091] Specifically, if Figure 5 and Figure 6As shown, the distributed coordinator issues an existing data migration instruction, which contains information about the current existing data migration phase. The storage service listens to the distributed coordinator's existing data migration instruction and enters the read and write logic of the existing data migration phase. The operations and maintenance personnel then launch the migration tool, which scans all original key-value pairs (such as key:value1) in the old storage pool and writes them to the new storage pool to migrate the existing data.

[0092] Step S302 : During the migration of existing data, in response to a user's write request for a first target key-value pair, a backup of the first target key-value pair is created, and the obtained backup key-value pair is written into a new storage pool.

[0093] Specifically, the above step S302 includes:

[0094] Step S3021, during the migration of existing data, in response to the user's write request for the first target key-value pair, the key in the first target key-value pair is renamed to obtain a backup key, and the backup key and the value in the first target key-value pair are concatenated to obtain a backup key-value pair corresponding to the first target key value.

[0095] It should be noted that the first target key-value pair includes the original key-value pair.

[0096] In some embodiments, see again Figure 5 and Figure 6 During the migration of existing data, the storage service receives the first target key-value pair key:value2 from the user's write request, renames the key in the first target key-value pair key:value2 to key_backup, concatenates the backup key key_backup with the value value2 of the first target key-value pair to obtain the backup key-value pair key_backup:value2 corresponding to the first target key, and writes it to the new storage pool.

[0097] During the migration of existing data, an embodiment of the present application renames the key of the first target key-value pair targeted by the user's write request to form a backup key-value pair to save the latest value written by the user, thereby avoiding concurrent write conflicts between the existing data migration operation and the user write operation on the value of the same key.

[0098] Step S3022: During the migration of existing data, in response to a user's read request for a target key, a query is performed in the new storage pool or the old storage pool to obtain a first target value and return it to the user.

[0099] Specifically, based on the backup key corresponding to the target key, a query is performed on the backup key-value pairs in the new storage pool, and the first target value obtained by the query is returned to the user. Alternatively, based on the target key, a query is performed on the original key-value pairs in the new storage pool, and the first target value obtained by the query is returned to the user. Alternatively, based on the target key, a query is performed on the original key-value pairs in the old storage pool, and the first target value obtained by the query is returned to the user.

[0100] In some embodiments, see again Figure 5 and Figure 6 During the migration of existing data, the storage service receives the target key of the user's read request and first attempts to query key_backup in the new storage pool. If key_backup exists, the value of key_backup is returned to the user as the first target value. If not, the storage service continues to try to query key in the new storage pool. If key exists in the new storage pool, the value of the new storage pool key is returned to the user as the first target value. If key does not exist in the new storage pool, the storage service attempts to query key in the old storage pool. If key exists in the old storage pool, the value of the old storage pool key is returned to the user as the first target value. If key does not exist in the old storage pool, the storage service prompts the user that the value does not exist.

[0101] During the migration of existing data, this embodiment of the application first queries the new storage pool for the backup key corresponding to the target key in a user's read request, returning the latest value of the target key to the user. If the backup key corresponding to the target key does not exist in the new storage pool, the target key is queried in both the new and old storage pools, and the corresponding value is returned to the user. This allows the user to read the latest and correct value corresponding to the target key, ensuring the real-time and accuracy of the returned data.

[0102] In this embodiment of the present application, during the existing data migration phase, the storage service renames the write request key to key_backup to avoid concurrent write conflicts with the migration tool for the same key data. Furthermore, in read requests, the storage service prioritizes reading the renamed key_backup data, ensuring that newly written data is correctly returned.

[0103] Step S303 : In response to the incremental data migration instruction, assign the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair.

[0104] Specifically, if Figure 7 and Figure 8As shown, incremental data migration instructions are issued through the distributed coordinator, and the incremental data migration instructions contain information about the current incremental data migration stage. The storage service listens to the incremental data migration instructions of the distributed coordinator and enters the read-write logic of the incremental migration stage. The operation and maintenance personnel can then start the migration tool again. The migration tool logic scans all backup key-value pairs in the new storage pool. If there is a key-value pair with backup, such as key_backup:value2, it will be written to the corresponding original key-value pair in the new storage pool, such as key:value2, to migrate the incremental data. After the write is completed, it is determined again whether the value of the new storage pool key_backup is the same as the key. If not, the key_backup value will continue to be written to the key. This process is repeated until the value of key_backup is the same as the key.

[0105] Step S304 : During the incremental data migration, in response to a user's write request for the second target key-value pair, a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair are written into the new storage pool.

[0106] Specifically, the above step S304 includes:

[0107] Step S3041, during incremental data migration, in response to the user's write request for the second target key-value pair, write the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair in the new storage pool in sequence, and assign the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair.

[0108] It should be noted that the second target key-value pair includes the first target key-value pair or the original key-value pair;

[0109] In some embodiments, see again Figure 7 and Figure 8 During incremental data migration, the storage service receives a user's write request for a second target key-value pair, such as key:value3. It renames the key in the second target key-value pair to key_backup and simultaneously writes two data items, key_backup:value3 and key:value3, to the new storage pool. key_backup serves as backup data, always storing the most recent value requested by the user. After the write is complete, the storage service checks again to see if the value of key_backup in the new storage pool is identical to key. If not, the key_backup value is written to key again.

[0110] During incremental data migration, the embodiment of the present application saves the latest value written by the user for the second target key-value pair requested by the user by writing the backup key-value pair corresponding to the second target key-value pair and the second target key-value pair to the new storage pool. Furthermore, by assigning the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair, the values ​​of the same set of backup key-value pairs in the new storage pool are consistent with and the latest values ​​of the original key-value pairs, thereby avoiding concurrent write conflicts between the migration operation and the user write operation on the same key value during incremental data migration.

[0111] Step S3042: During the incremental data migration, in response to a user's read request for the target key, a query is performed in the new storage pool to obtain a second target value and return it to the user.

[0112] Specifically, based on the backup key corresponding to the target key, a query is performed on the backup key-value pairs in the new storage pool, and the second target value obtained by the query is returned to the user. Alternatively, based on the target key, a query is performed on the original key-value pairs in the new storage pool, and the second target value obtained by the query is returned to the user.

[0113] In some embodiments, see again Figure 7 and Figure 8 During incremental data migration, the storage service receives a read request for the target key. It first attempts to query the new storage pool for key_backup. If key_backup exists in the new storage pool, the value of key_backup is returned to the user as the second target value. If key_backup does not exist in the new storage pool, the storage service continues to query the new storage pool for key. If key exists in the new storage pool, the value of the new storage pool key is returned to the user as the second target value.

[0114] During incremental data migration, this embodiment of the present application first queries the new storage pool for the backup key corresponding to the target key in a user's read request, returning the latest value of the target key to the user. If the backup key corresponding to the target key does not exist in the new storage pool, the new storage pool is queried for the target key and the corresponding value is returned to the user. This allows the user to read the latest and correct value corresponding to the target key, ensuring the real-time and accuracy of the returned data.

[0115] In an embodiment of the present application, during the incremental data migration phase, the storage service will write the backup key-value pair key_backup:value3 and the original key-value pair key:value3 in the new storage pool for the second target key-value pair of the write request, such as key:value3, and the migration tool will be responsible for writing the value of the backup key-value pair to the original key-value pair. Since the storage service and the migration tool may generate concurrent write conflicts for the same key data, the migration tool needs to check the values ​​of key_backup and key again after writing. If the two are not equal, it means that a concurrent write conflict has occurred, and the migration tool performs the assignment operation again until the values ​​of the two are the same, thereby resolving the concurrent write conflict. In read requests, the storage service still gives priority to reading the key_backup data, which can ensure that the newly written data of the user is correctly returned.

[0116] Step S305: After the incremental data migration is completed, the backup key-value pairs stored in the new storage pool are cleared.

[0117] In some embodiments, as Figure 9 and Figure 10 As shown, the distributed coordinator publishes information about the current redundant data cleanup phase. The storage service listens to the distributed coordinator's information and enters the redundant data cleanup read and write logic. The operations and maintenance personnel can then re-launch the migration tool, which scans all key-value pairs in the new storage pool and deletes any backup key-value pairs containing the word "backup."

[0118] Step S306 : In response to the user's write request for the third target key-value pair, write the third target key-value pair into the new storage pool.

[0119] It should be noted that the third target key-value pair includes the second target key-value pair or the first target key-value pair or the original key-value pair.

[0120] In some embodiments, see again Figure 9 and Figure 10 As shown, after the redundant data is cleaned up, the storage service receives the third target key-value pair of the user's write request, such as key:value4, and writes data key:value4 into the new storage pool.

[0121] In the embodiment of the present application, after all backup key-value pairs in the new storage pool are cleared, the data migration is completed. If a third target key-value pair targeted by a user write request is received, the third target key-value pair can be directly written into the new storage pool to implement new data writing.

[0122] Step S307 : In response to the user's read request for the target key, a query is performed in the original key-value pairs in the new storage pool according to the target key, and the third target value obtained by the query is returned to the user.

[0123] In some embodiments, see again Figure 9 and Figure 10 As shown, after the redundant data is cleaned up, the storage service receives the target key key targeted by the user's read request, directly queries the key in the new storage pool, and returns the value corresponding to the key as the third target value to the user.

[0124] In the embodiment of the present application, after all backup key-value pairs in the new storage pool are cleared, the data migration is completed. If a target key targeted by a user read request is received, the target key is directly queried in the new storage pool, and the corresponding value is returned to the user to realize data reading.

[0125] This application divides the entire data migration process into three phases: existing data migration, incremental data migration, and redundant data cleanup. The storage service read and write logic and the migration tool migration logic will be modified during these three phases. By coordinating the storage service read and write logic with the migration tool migration logic, write conflicts can be avoided in concurrent write scenarios, ensuring that user read and write requests are always correct.

[0126] The data migration method provided in this embodiment ensures the correct reading and writing of user data during the migration process while ensuring service availability. By adjusting the storage service's data read and write logic during the migration process and the migration tool's data migration logic, this method resolves the concurrent write conflicts that may arise during the data migration process, ensuring the accuracy of user data read and write.

[0127] This invention addresses concurrent write conflicts during data migration by modifying how users read and write data during the migration phase, as well as how the migration tool migrates data. This ensures that users can read and write data correctly. This invention resolves concurrency conflicts through a lockless approach, without using distributed locks, which does not affect data migration efficiency. Furthermore, this invention does not rely on the CAS operation of the storage system, offering greater versatility.

[0128] The data migration solution of the present invention is described in detail below with reference to a specific application example.

[0129] See again Figure 4 In this application example, the storage pool can be a distributed key-value database, HBase, responsible for storing user data. In data migration scenarios, it can be divided into an old storage pool and a new storage pool. Zookeeper can be used as the distributed coordinator to synchronize information with the storage service.

[0130] The migration tool is responsible for copying user data from the old storage pool to the new storage pool. The storage service is responsible for accepting user read and write requests and can obtain timely information about changes in the migration stage by monitoring the / migrateStage node information in Zookeeper.

[0131] The entire data migration process is divided into three phases: existing data migration, incremental data migration, and redundant data cleanup. The specific steps are as follows:

[0132] Step 1: Existing data migration phase.

[0133] In step 1.1, the operator updates the Zookeeper / migrateStage node information to ExistingDataMigration using the command line. The storage service monitors the Zookeeper node information change and enters the read and write logic of the ExistingDataMigration stage. The operator can then start the migration tool.

[0134] Step 1.2, migration tool logic: Scan all key-value pairs (such as key:value1) in the old HBase storage pool and write them to the new storage pool. Because an HBase key may correspond to multiple versions of values, each version can be copied to the new storage pool.

[0135] Step 1.3, storage service write data logic: The storage service receives the key-value pair key:value2 and its corresponding version number in the user's write request, renames the key to key_backup, and writes it to the new storage pool key_backup:value2 (corresponding version).

[0136] Step 1.4, storage service read data logic: The storage service receives a user read request and queries the value of a specific key version. It first attempts to query the corresponding key_backup version in the new storage pool. If the new storage pool contains the key value for that version, it returns the result to the user. If the new storage pool does not exist, it continues to query the corresponding key version in the new storage pool. If the key value exists, it returns the result to the user. If the key does not exist in the new storage pool, it attempts to query the corresponding key version in the old storage pool and returns the result to the user.

[0137] Step 2: Incremental data migration phase.

[0138] In step 2.1, the operator updates the Zookeeper / migrateStage node information to IncrementalDataMigration using the command line. The storage service monitors the Zookeeper node information change and enters the read and write logic of the incremental migration phase. The operator can then start the migration tool.

[0139] Step 2.2, migration tool logic: Scan all key-value pairs in the new storage pool. If key_backup:value2 exists, write it to key:value2 (the corresponding version) in the new storage pool. After writing, check again to see if the value of key_backup in the new storage pool matches the value of the corresponding version of key. If not, continue writing the value corresponding to key_backup to key. Repeat this process until the value of key_backup matches the value of key.

[0140] Step 2.3, storage service write data logic: The storage service receives the key-value pair key:value3 and its corresponding version number in the user's write request, renames the key to key_backup, and writes two data key_backup:value3 and key:value3 (corresponding versions) to the new storage pool.

[0141] Step 2.4, storage service read data logic: The storage service accepts a user's read request to query the value of a specific version of a key. It first attempts to query key_backup (the corresponding version) in the new storage pool. If the key value exists, it returns it to the user. If not, it continues to query key (the corresponding version) in the new storage pool. If the key value exists, it returns it to the user.

[0142] Step 3: Redundant data cleaning phase.

[0143] In step 3.1, the operator updates the Zookeeper / migrateStage node information to IncrementalDataMigration using the command line. The storage service monitors the Zookeeper node information change and enters the read and write logic of the incremental migration phase. The operator can then start the migration tool.

[0144] Step 3.2, Migration Tool Logic: Scan all key-value pairs in the new storage pool. If the key key_backup exists, delete it.

[0145] Step 3.3, storage service write data logic: The storage service receives the key-value pair key:value4 and its corresponding version number in the user's write request, and writes the data key:value4 (corresponding version) in the new storage pool.

[0146] Step 3.4, storage service read data logic: The storage service accepts the user's read request key, queries the data corresponding to the key in the new storage pool, and returns it to the user.

[0147] The present invention can ensure the correctness of data during data migration under the condition of concurrent user writing, ensure that read and write requests are correct during user data migration, and prevent user data from being overwritten by errors.

[0148] This invention boasts a simple overall architecture, eliminating the need for third-party components to implement distributed locks to coordinate the read and write logic of the storage service and the migration tool. Without relying on distributed locks, the migration tool and storage service don't need to frequently interact with distributed lock components, improving migration efficiency. Furthermore, the lock-free approach resolves concurrency conflicts without blocking user read and write requests.

[0149] The solution of the present invention for concurrent writing of storage services and migration tools does not rely on the read and write characteristics unique to any storage engine (such as CAS operations). Therefore, it has high versatility and can be applied to data migration scenarios of many KV storage systems under concurrent user writing conditions.

[0150] This embodiment also provides a data migration device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0151] This embodiment provides a data migration device, such as Figure 11 Shown, including:

[0152] The first processing module 1101 is configured to write all original key-value pairs in the old storage pool into the new storage pool in response to an existing data migration instruction;

[0153] The second processing module 1102 is configured to, during the stock data migration period, create a backup of the first target key-value pair in response to a user's write request for the first target key-value pair, and write the obtained backup key-value pair into a new storage pool;

[0154] A third processing module 1103 is configured to assign the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair in response to the incremental data migration instruction;

[0155] A fourth processing module 1104 is configured to, during the incremental data migration, write a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair into the new storage pool in response to a user's write request for the second target key-value pair;

[0156] The fifth processing module 1105 is configured to clear the backup key-value pairs stored in the new storage pool after the incremental data migration is completed.

[0157] In some optional implementations, the second processing module 1102 is further configured to:

[0158] Rename the key in the first target key-value pair to obtain a backup key; wherein the first target key-value pair includes the original key-value pair;

[0159] The backup key and the value in the first target key-value pair are concatenated to obtain a backup key-value pair corresponding to the first target key value.

[0160] In some optional implementations, during the stock data migration, the second processing module 1102 is further configured to:

[0161] In response to a user's read request for a target key, query the backup key-value pairs in the new storage pool according to the backup key corresponding to the target key, and return the first target value obtained from the query to the user;

[0162] Alternatively, based on the target key, a query is performed on the original key-value pairs in the new storage pool, and the first target value obtained by the query is returned to the user;

[0163] Alternatively, based on the target key, a query is performed in the original key-value pairs of the old storage pool, and the first target value obtained by the query is returned to the user.

[0164] In some optional implementations, the fourth processing module 1104 is further configured to:

[0165] Writing a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair into the new storage pool in sequence; wherein the second target key-value pair includes the first target key-value pair or the original key-value pair;

[0166] Assign the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair.

[0167] In some optional implementations, during the incremental data migration, the fourth processing module 1104 is further configured to:

[0168] In response to a user's read request for the target key, query the backup key-value pairs in the new storage pool according to the backup key corresponding to the target key, and return the second target value obtained from the query to the user;

[0169] Alternatively, based on the target key, a query is performed in the original key-value pairs in the new storage pool, and the second target value obtained by the query is returned to the user.

[0170] In some optional implementations, after clearing the backup key-value pairs stored in the new storage pool, the apparatus is further configured to:

[0171] In response to a user's write request for a third target key-value pair, the third target key-value pair is written into the new storage pool; wherein the third target key-value pair includes the second target key-value pair, the first target key-value pair, or the original key-value pair.

[0172] In some optional implementations, after clearing the backup key-value pairs stored in the new storage pool, the apparatus is further configured to:

[0173] In response to a read request from the user for the target key, a query is performed in the original key-value pairs in the new storage pool according to the target key, and a third target value obtained by the query is returned to the user.

[0174] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0175] The data migration device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0176] The embodiment of the present invention also provides a computer device having the above Figure 11 The data migration device shown.

[0177] See also Figure 12 , Figure 12 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 12 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 12 A processor 10 is taken as an example.

[0178] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0179] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0180] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0182] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 12 The bus connection is taken as an example.

[0183] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0184] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0185] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0186] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data migration method, characterized in that: The method comprises: In response to the existing data migration instruction, all original key-value pairs in the old storage pool are written into the new storage pool; During the migration of existing data, in response to a user's write request for a first target key-value pair, creating a backup of the first target key-value pair, and writing the obtained backup key-value pair into a new storage pool; In response to the incremental data migration instruction, assigning the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair; During the incremental data migration, in response to a user's write request for the second target key-value pair, writing a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair in the new storage pool; After the incremental data migration is complete, clear the backup key-value pairs stored in the new storage pool.

2. The method according to claim 1, characterized in that The step of creating a backup of the first target key-value pair includes: Rename the key in the first target key-value pair to obtain a backup key; wherein the first target key-value pair includes the original key-value pair; The backup key and the value in the first target key-value pair are concatenated to obtain a backup key-value pair corresponding to the first target key value.

3. The method according to claim 2, characterized in that During the migration of the existing data, the method further includes: In response to a user's read request for a target key, querying the backup key-value pairs in the new storage pool according to the backup key corresponding to the target key, and returning a first target value obtained from the query to the user; Alternatively, searching the original key-value pairs in the new storage pool according to the target key, and returning the first target value obtained from the search to the user; Alternatively, according to the target key, a query is performed in the original key-value pairs of the old storage pool, and the first target value obtained by the query is returned to the user.

4. The method according to claim 2, characterized in that Writing the backup key-value pair and the second target key-value pair corresponding to the second target key-value pair into the new storage pool includes: Writing a backup key-value pair and the second target key-value pair corresponding to the second target key-value pair into the new storage pool in sequence; wherein the second target key-value pair includes the first target key-value pair or the original key-value pair; Assign the value in the backup key-value pair corresponding to the second target key-value pair to the second target key-value pair.

5. The method according to claim 4, characterized in that During the incremental data migration, the method further includes: In response to a user's read request for a target key, querying backup key-value pairs in the new storage pool according to a backup key corresponding to the target key, and returning a second target value obtained from the query to the user; Alternatively, based on the target key, a query is performed in the original key-value pairs in the new storage pool, and the second target value obtained by the query is returned to the user.

6. The method according to any one of claims 1 to 5, characterized in that After clearing the backup key-value pairs stored in the new storage pool, the method further includes: In response to a user's write request for a third target key-value pair, the third target key-value pair is written into the new storage pool; wherein the third target key-value pair includes the second target key-value pair, the first target key-value pair, or the original key-value pair.

7. The method according to any one of claims 1 to 5, characterized in that After clearing the backup key-value pairs stored in the new storage pool, the method further includes: In response to a read request from a user for a target key, a query is performed in the original key-value pairs in the new storage pool according to the target key, and a third target value obtained by the query is returned to the user.

8. A data migration device, characterized in that: The device comprises: A first processing module is configured to write all original key-value pairs in the old storage pool into the new storage pool in response to an existing data migration instruction; a second processing module, configured to, during the migration of the existing data, create a backup of the first target key-value pair in response to a user's write request for the first target key-value pair, and write the obtained backup key-value pair into a new storage pool; a third processing module, configured to assign, in response to the incremental data migration instruction, the value of the backup key-value pair stored in the new storage pool to the original key-value pair corresponding to the backup key-value pair; a fourth processing module, configured to, during the incremental data migration, write a backup key-value pair and a second target key-value pair corresponding to the second target key-value pair in the new storage pool in response to a user's write request for the second target key-value pair; The fifth processing module is used to clear the backup key-value pairs stored in the new storage pool after the incremental data migration is completed.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data migration method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data migration method according to any one of claims 1 to 7.

Images