Data migration methods, apparatus and computer equipment

Abstract

This application relates to a data migration method, apparatus, and computer device. The method includes, in response to a data migration task, migrating data to be migrated from a source data block to a target data block, and redirecting a first pointer pointing to the source data block to the target data block; protecting the source data block through a preset protection mechanism; obtaining a concurrent access count of the source data block, and deleting the source data block when the concurrent access count is zero. The data migration method of this application can avoid lock contention and memory consumption between data migration and concurrent access, and will not affect the performance of the storage system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data migration method, apparatus and computer equipment. Background Technology

[0002] In modern storage systems, numerous scenarios trigger data migration, such as cold and hot data migration in tiered storage, and defragmentation and garbage collection in file, block, and object storage systems. During data migration, various front-end and back-end services may concurrently access the data being migrated. Ensuring the correctness of the accessed data while avoiding significant impact on the overall performance of the storage system is arguably the most critical challenge faced by various data migration algorithms and processes.

[0003] To avoid data inconsistencies caused by data migration and concurrent operations with other businesses, existing technologies typically rely on metadata locks to synchronize concurrent access to data block mappings and strictly track the concurrent business counts of the migrated region to eliminate concurrent access to the migrated data blocks.

[0004] However, in existing technologies, data migration and concurrent access involve intense lock contention and memory consumption, which can affect the performance of the storage system. Summary of the Invention

[0005] Therefore, it is necessary to provide a data migration method, apparatus, and computer equipment that can avoid lock contention and memory consumption between data migration and concurrent access, and will not affect the performance of the storage system, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a data migration method, which includes:

[0007] In response to the data migration task, the data to be migrated is moved from the source data block to the target data block, and the first pointer pointing to the source data block is redirected to the target data block;

[0008] The source data block is protected by a preset protection mechanism, which supports reading, copying, or updating operations on the data in the source data block.

[0009] Get the concurrent access count of the source data block, and delete the source data block when the concurrent access count is zero.

[0010] In the above embodiments, the data migration process employs a preset protection mechanism to protect the source data blocks, eliminating the need for locks and thus avoiding lock contention during concurrent access. Furthermore, by acquiring the concurrent access count of a set of multiple source data blocks, the decision to delete a source data block is made. Compared to existing technologies that independently count source data blocks within each interval of the source data block set, this method consumes less memory, allowing the saved memory to be used to improve the performance of the storage system. Additionally, the data migration scheme provided in this application is simple and easy to implement.

[0011] In one embodiment, the data migration method further includes:

[0012] During the data migration process, if a business access is received, a second pointer is loaded to determine the data block that the second pointer points to; the data block pointed to by the second pointer is either the source data block or the target data block.

[0013] Access the data in the data block pointed to by the second pointer.

[0014] In the above embodiments, when performing business access, the data in the accessed data block is protected by a preset protection mechanism, which ensures that the data in the accessed data block will not be corrupted.

[0015] In one embodiment, the data migration method further includes, before deleting the source data block:

[0016] Determine the target data segment where the source data block is located; each data segment has a corresponding version number;

[0017] The version number of the target data segment is incremented.

[0018] The above embodiments, by determining the target data segment where the source data block is located before deleting the source data block and incrementing the version number of the target data segment, facilitate subsequent determination of whether the accessed data is accurate.

[0019] In one embodiment, the data migration method further includes:

[0020] During the data migration process, if it is determined that the priority of business access is lower than the priority of data migration, then the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs is obtained;

[0021] Access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is either an incremented version number or a non-incremented version number;

[0022] The access result of the business access is determined based on the first version number and the second version number.

[0023] In the above embodiments, during data migration, if the priority of a received business access is lower than the priority of the data migration, the access result of the business access can be determined by using the first version number and second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs. This facilitates timely completion of the business access even if the access result is incomplete. Furthermore, the data migration process provided in this embodiment is applicable to low-priority concurrent access processes (such as the source data reading process of local or remote asynchronous replication tasks), ensuring the relatively high priority of the data migration process while avoiding lock contention. Additionally, in existing asynchronous replication and similar services, the source data blocks are typically large and contiguous, with relatively few active source data segments. The introduction of a data segment version number mechanism in this implementation reduces the memory resources required to maintain the source data segment version numbers, thereby reducing memory consumption.

[0024] In one embodiment, the access result of the service access is determined based on the first version number and the second version number, including:

[0025] Compare the first version number and the second version number to obtain the comparison results;

[0026] If the comparison result shows that the first version number and the second version number are different, then the access result of the business access is incomplete.

[0027] If the comparison result shows that the first version number and the second version number are the same, then the access result of the business access is "access completed".

[0028] The above embodiment uses a simple, fast, and easy-to-implement method to determine the access result of a business access by comparing whether the first version number and the second version number are the same.

[0029] In one embodiment, the data migration method further includes:

[0030] If the access result of the business access is incomplete, a new second pointer is reloaded, and the steps corresponding to the first version number of the data segment to which the data block pointed to by the second pointer belongs are re-executed according to the new second pointer.

[0031] When the result of the service access is incomplete, the above embodiments can ensure the completion of the service access by loading a new second pointer and re-executing the steps in the corresponding embodiments.

[0032] In one embodiment, if a service access is received, a second pointer is loaded, and the data block pointed to by the second pointer is determined, including:

[0033] If a business request is received, the second pointer is loaded atomically to determine the data block that the second pointer points to.

[0034] In the above embodiments, by loading the second pointer in an atomic manner, the integrity of the second pointer can be guaranteed without using a locking mechanism, thereby ensuring the efficient and correct execution of data migration and business access tasks.

[0035] In one embodiment, redirecting a first pointer to a source data block to a target data block includes:

[0036] Atomically redirect the first pointer to the source data block to the target data block.

[0037] In the above embodiments, by redirecting the first pointer in an atomic manner, the integrity of the first pointer can be guaranteed without using a locking mechanism, thereby ensuring the efficient and correct execution of data migration and business access tasks.

[0038] Secondly, one embodiment of this application provides a data migration apparatus, the apparatus comprising:

[0039] The migration module is used to respond to data migration tasks, migrate the data to be migrated from the source data block to the target data block, and redirect the first pointer pointing to the source data block to the target data block;

[0040] The protection module is used to protect the source data block through a preset protection mechanism. The protection mechanism supports reading, copying or updating the data in the source data block.

[0041] The deletion module is used to obtain the concurrent service access count of the source data block and delete the source data block when the concurrent service access count is zero.

[0042] Thirdly, one embodiment of this application provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect above. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the data migration process and concurrent read operations in one embodiment;

[0044] Figure 2 This is a schematic diagram of the internal structure of a terminal in one embodiment;

[0045] Figure 3 This is a flowchart illustrating the steps of a data migration method in one embodiment;

[0046] Figure 4 This is a flowchart illustrating the steps of a data migration method in another embodiment;

[0047] Figure 5This is a flowchart illustrating the data migration process and concurrent read operations in another embodiment;

[0048] Figure 6 This is a flowchart illustrating the steps of a data migration method in another embodiment;

[0049] Figure 7 This is a flowchart illustrating the steps of a data migration method in another embodiment;

[0050] Figure 8 This is a flowchart illustrating the steps of a data migration method in another embodiment;

[0051] Figure 9 This is a flowchart illustrating the data migration process and concurrent read operations in another embodiment;

[0052] Figure 10 A flowchart illustrating the steps of the data migration method in another embodiment;

[0053] Figure 11 A flowchart illustrating the steps of the data migration method in another embodiment;

[0054] Figure 12 This is a schematic diagram of the data migration device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] Before detailing the technical solutions of the embodiments of this disclosure, the technical background or evolution of the embodiments of this disclosure is first introduced. In modern storage systems, numerous scenarios trigger data migration, such as cold and hot data migration in tiered storage, and defragmentation and garbage collection in file, block, and object storage systems. After determining the source and destination addresses for data block migration, the data migration begins. During the data migration process, various front-end and back-end services are likely to concurrently access the data being migrated. Ensuring the correctness of the accessed data while avoiding a significant impact on the overall performance of the storage system is arguably the most critical challenge faced by various data migration algorithms and processes.

[0057] To avoid data inconsistencies caused by concurrent data migration with other business processes, existing technologies typically rely on metadata locks to synchronize concurrent access (read, copy, and update) to data block mappings and strictly track concurrent business counts in the migrated region to eliminate concurrent access to the migrated data blocks. Existing technologies for data migration typically avoid concurrent read operations reading incorrect data as follows: Figure 1 As shown.

[0058] The data migration process includes:

[0059] Step 100: Copy the contents of a set of data blocks from the source data blocks to the target data blocks;

[0060] Step 110: Add a write lock (exclusive lock) to protect against modification of the data block pointer;

[0061] A source data block may be pointed to by multiple pointers, and all pointers pointing to this set of source data blocks may be scattered in different metadata areas, so multiple write locks are required.

[0062] Step 120: Redirect all pointers pointing to the source data block to the target data block;

[0063] Step 130: Release the write lock;

[0064] Step 140: Wait for all data blocks from the source to be read to complete;

[0065] After the data migration process releases the lock, it must wait for all incomplete read operations on the source data block, that is, empty all the "old" reads, before proceeding to the next step (deleting the source data block).

[0066] To empty the "old" reads, a counting and tracking scheme is used, which tracks the number of read operations in the source data block area being moved, and the emptying ends when the number drops to 0.

[0067] During the data migration process, the data is divided into source data block intervals according to the granularity of the migration. For example, if a migration task processes 1024 consecutive data blocks, the source data can be divided into intervals of 1024 data blocks or even smaller. Concurrent read operations are counted independently in each interval.

[0068] Step 150: Delete all source data blocks involved in this migration.

[0069] Once a source data block is deleted, its contents may be immediately overwritten, for example, by being reallocated and written with new data by other processes.

[0070] The concurrent data reading process includes:

[0071] Step 160: Acquire the pointer read lock (shared lock);

[0072] By acquiring a pointer read lock, the loading process of the data block pointer is protected.

[0073] Step 170: Load a pointer to a data block, which points to either the source data block or the target data block;

[0074] Step 180: Release the read lock;

[0075] Step 190: Read data from the data block pointed to by the pointer.

[0076] If a pointer that has not yet been redirected is loaded, the data read will access the corresponding source data block in the data migration process.

[0077] However, in the aforementioned prior art, while the data migration process actively waits for conflicting concurrent accesses to be cleared, metadata locking conflicts are still significant. Furthermore, the fine-grained counting and tracking scheme used for data migration and clearing concurrent access operations—using a single count for each small interval within the source data block set—may lead to substantial memory consumption or frequent memory allocation and deallocation operations, impacting storage system performance. To address this, this application provides a data migration method.

[0078] The data migration method provided in this application embodiment can be applied to a terminal, and the internal structure diagram of the terminal can be as follows: Figure 2 As shown, the terminal includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data transfer method. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the terminal casing, or an external keyboard, touchpad, or mouse. The terminal can be, but is not limited to, a personal computer, laptop, smartphone, tablet, or portable wearable device. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc.

[0079] In one embodiment, such as Figure 3 As shown, a data migration method is provided. This embodiment applies this method to applications such as... Figure 2The terminal shown is used as an example for illustration. In this embodiment, the method includes the following steps:

[0080] Step 300: In response to the data migration task, migrate the data to be migrated from the source data block to the target data block, and redirect the first pointer pointing to the source data block to the target data block.

[0081] Data migration tasks can be triggered in the foreground or background of the terminal. Specifically, if the data migration task is triggered in the foreground, it can be input by the user through the terminal's input device.

[0082] A data migration task includes the data to be migrated, the source data block to which the data belongs, and the target data block. Upon receiving the data migration task, the terminal responds by migrating the data from the source data block to the target data block; that is, it copies the data to be migrated from the source data block to the target data block. After migrating the data from the source data block to the target data block, the terminal redirects the first pointer that originally pointed to the source data block to the target data block. A source data block may be indexed by one or more pointers; after the data migration is complete, all pointers (the first pointer) are redirected to the corresponding target data block.

[0083] Step 310: Protect the source data block using a preset protection mechanism.

[0084] The protection mechanism supports reading, copying, or updating data in the source data block.

[0085] During the data migration process, the terminal will protect the source data block through a preset protection mechanism. In other words, during the data migration process, when accessing the data in the source data block (e.g., reading, copying, and updating operations), the data in the source data block will not be deleted.

[0086] In one optional embodiment, the preset protection mechanism is the RCU (Read-Copy-Update) mechanism. During data migration, the data in the source data block is protected by the RCU read critical section.

[0087] Step 320: Obtain the concurrent service access count of the source data block, and delete the source data block when the concurrent service access count is zero.

[0088] During data migration, the terminal acquires the concurrent service access count of the source data block in real time; that is, the number of times the foreground and backend perform access operations on the data in the source data block. The preset protection mechanism includes a counting module that counts the number of times the foreground and backend perform access operations on the data in the source data block. In other words, when the foreground or backend accesses the data in the source data block, the count module increments by 1; when the foreground or backend finishes accessing the data in the source data block, the count module decrements by 1.

[0089] If the terminal determines that the acquired concurrent access count is zero, the source data block is deleted. If the terminal determines that the concurrent access count is zero, it means that there are currently no operations accessing the data in the source data block, and deleting the source data block at this time will not cause data errors due to concurrent access. If the terminal determines that the acquired concurrent access count is not zero, it means that there are currently operations accessing the data in the source data block, and in this case, the source data block must be deleted only after the concurrent access count reaches zero.

[0090] The data migration method provided in this application includes, in response to a data migration task, migrating the data to be migrated from a source data block to a target data block, and redirecting a first pointer pointing to the source data block to the target data block; protecting the source data block through a preset protection mechanism; obtaining the concurrent access count of the source data block, and deleting the source data block when the concurrent access count is zero. In this embodiment, during the data migration process, the data migration process uses a preset protection mechanism to protect the source data block, eliminating the need for locks and thus avoiding lock contention during concurrent access. Furthermore, by obtaining the concurrent access count of a set of multiple source data blocks to determine whether to delete the source data block, compared to the prior art of independently counting source data blocks in each interval of the source data block set, memory usage is reduced, and the saved memory can be used to improve the performance of the storage system storing the data. In addition, the data migration scheme provided in this application is simple and easy to implement.

[0091] When the protection mechanism is RCU, using RCU to count concurrent accesses to the source data block set enables batch emptying. A data migration process typically requires emptying more concurrent access traffic. For a typical SRCU, SRCU internally performs a maximum of two rounds of counting and emptying. Therefore, the average upper limit of the time for concurrent access emptying using the SRCU interface, i.e., the grace period, is twice the average latency of concurrent accesses. Foreground concurrent accesses are usually low-latency concurrent accesses with small granularity, ranging from milliseconds for hybrid flash systems to microseconds for all-flash systems. As background concurrent access tasks, a single data migration process typically moves hundreds or thousands of data blocks. In this embodiment, the time cost required to empty concurrent accesses is completely acceptable and will not affect the performance of the storage system.

[0092] In one embodiment, such as Figure 4 As shown, the data migration process involves concurrent business access flows. The data migration method also includes the following steps:

[0093] Step 400: During the data migration process, if a business access is received, the second pointer is loaded to determine the data block pointed to by the second pointer; the data block pointed to by the second pointer is either the source data block or the target data block.

[0094] Business access can be either front-end or back-end access to data in the storage system. This business access is concurrent, meaning that the data migration process and the business access process are executed simultaneously. During data migration, the concurrent access process includes, if the terminal receives a business access request, loading a second pointer and determining the data block pointed to by the second pointer.

[0095] The second pointer cannot determine whether it points to the source data block or the target data block. In other words, the data block pointed to by the second pointer could be either the source data block or the target data block.

[0096] Step 410: Access the data in the data block pointed to by the second pointer.

[0097] After determining the data block pointed to by the second pointer, the terminal accesses the data in that data block. Specifically, it performs read, copy, or update operations on the data in the data block pointed to by the second pointer.

[0098] If the terminal determines that the data block pointed to by the second pointer is the target data block, the terminal directly accesses the data in the target data block. If the terminal determines that the data block pointed to by the second pointer is the source data block, the data in the source data block is protected by a preset protection mechanism, ensuring that the data in the accessed source data block will not be deleted.

[0099] In this embodiment, when accessing a service, the data in the accessed data block is protected by a preset protection mechanism, which ensures that the data in the accessed data block will not be corrupted.

[0100] In an optional embodiment, the preset protection mechanism is the RCU mechanism. The read critical section of the RCU mechanism starts from SRCU read lock and ends from SRCU read unlock. That is, within this critical section, the data in the source data block will not be deleted, and the accuracy of business access will not be affected.

[0101] In an optional embodiment, when the preset protection mechanism is the RCU mechanism, the process for preventing concurrent read operations from reading erroneous data during data migration is as follows: Figure 5 As shown.

[0102] The data migration process includes:

[0103] Step 500: Move the data to be migrated from the source data block to the target data block.

[0104] Step 510: Redirect the first pointer pointing to the source data block to the target data block.

[0105] Step 520: Access the Sleepable RCU (SRCU) interface to obtain the concurrent service access count of the source data block.

[0106] Step 530: Delete the source data block when the concurrent service access count is zero.

[0107] The concurrent data read process includes: within the read critical section of the RCU mechanism (from SRCU read lock to SRCU read unlock),

[0108] Step 540: Load the second pointer;

[0109] Step 550: Determine the data block pointed to by the second pointer and access the data in the data block pointed to by the second pointer.

[0110] In one embodiment, an implementation involving redirecting a first pointer to a source data block to a target data block includes:

[0111] Atomically redirect the first pointer to the source data block to the target data block.

[0112] When the terminal redirects the first pointer to the source data block to the target data block, the redirection is performed atomically. For example, `atomic64_set` redirects the first pointer in a 64-bit atomic manner.

[0113] In one embodiment, an implementation involves loading a second pointer and determining the data block pointed to by the second pointer upon receiving a service access. This implementation includes:

[0114] If a business request is received, the second pointer is loaded atomically to determine the data block that the second pointer points to.

[0115] After receiving a service request, the terminal loads the second pointer atomically. For example, `atomic64_read` loads the second pointer in a 64-bit atomic manner.

[0116] In this embodiment, by redirecting the first pointer in an atomic manner and loading the second pointer in an atomic manner, the integrity of the first and second pointers can be guaranteed, thereby ensuring the normal execution of data migration tasks and business access tasks without pointer lock protection.

[0117] Please see Figure 6 The data migration process involves several steps. Before deleting the source data blocks, the data migration method includes the following steps:

[0118] Step 600: Determine the target data segment where the source data block is located; each data segment has a corresponding version number.

[0119] A data segment includes multiple source data blocks, and each data segment is assigned a corresponding version number. The version number for each data segment can be preset by the user; this embodiment does not impose such restrictions.

[0120] Before deleting the source data block after determining that the concurrent service access count is zero, the terminal determines the data segment to which the source data block belongs, i.e., the target data segment.

[0121] Step 610: Increment the version number of the target data segment.

[0122] After determining the target data segment where the source data block is located, the terminal obtains the version number of the target data segment and increments the version number. Specifically, the terminal adds a preset value to the version number of the target data segment. The preset value can be 1 or other integer values.

[0123] In this embodiment, during the data migration process, before deleting the source data block, the target data segment where the source data block is located is determined, and the version number of the target data segment is incremented to facilitate concurrent services in determining whether the accessed data is accurate.

[0124] In one embodiment, such as Figure 7 As shown, the data migration process involves concurrent business access flows. The data migration method also includes the following steps:

[0125] Step 700: During the data migration process, if it is determined that the priority of business access is lower than that of data migration, then obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs.

[0126] During data migration, after receiving a service access request, the process for this concurrent service access includes determining that the priority of the received service access is lower than the priority of the data migration. Then, after loading the second pointer, the version number corresponding to the data segment to which the data block pointed to by the second pointer belongs is obtained and recorded as the first version number. The data block pointed to by the second pointer can be either a source data block or a target data block; that is, the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs may be the version number of the data segment to which the source data block belongs, or it may be the version number of the data segment to which the target data block belongs. In this embodiment, the process of obtaining the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs is within the protection mechanism; that is, when obtaining the first version number, the data block pointed to by the second pointer is protected by the protection mechanism.

[0127] Step 710: Access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is either an incremented version number or a non-incremented version number.

[0128] During concurrent business access, after loading the second pointer, the terminal accesses the data in the data block pointed to by the second pointer and obtains the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs. The second version number is either an incremented version number or a version number that has not been incremented during the data migration process. In other words, within the preset protection mechanism, the terminal loads the second pointer and obtains the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; outside the preset protection mechanism, the data migration process continues to execute, that is, the version number of the target data segment where the source data block is located is incremented. In this embodiment, the process of accessing the data in the data block pointed to by the second pointer and obtaining the second version number is outside the protection mechanism, that is, the data block pointed to by the second pointer is not protected by the protection mechanism.

[0129] If the data block pointed to by the second pointer is the source data block, and the version number of the data segment to which the source data block belongs was incremented during the data migration process, then the second version number will be the version number after the increment process; that is, the first version number and the second version number will be different. If the data block pointed to by the second pointer is the target data block, and the version number of the data segment to which the target data block belongs was not incremented during the data migration process, then the second version number will be the version number without the increment process; that is, the first version number and the second version number will be the same.

[0130] Step 720: Determine the access result of the business access based on the first version number and the second version number.

[0131] After determining the first version number and the second version number, the terminal determines the access result of the service access based on the first version number and the second version number. The access result of the service access includes whether the service access is completed or not.

[0132] If the business access is to read data, the access result can be either successful data reading or unsuccessful data reading.

[0133] In this embodiment, during the data migration process, if the priority of the received business access is lower than the priority of the data migration, the access result of the business access can be determined by using the first version number and the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs. This avoids reading erroneous data and ensures that the business access is completed in a timely manner when the access result is incomplete. Furthermore, the data migration process provided in this embodiment is applicable to low-priority concurrent access processes (such as the source data reading process of local or remote asynchronous replication tasks, whose priority is usually lower than data migration services such as defragmentation), ensuring the relatively high priority of the data migration process while avoiding lock contention.

[0134] In addition, existing asynchronous replication and similar services typically use large, contiguous source data blocks with relatively few active source data segments. This implementation introduces a data segment version number mechanism, which reduces the memory resources required to maintain the source data segment version number, thereby reducing memory consumption.

[0135] In one embodiment, such as Figure 8 As shown, this involves an implementation method for determining the access result of a business access based on a first version number and a second version number. The steps of this implementation method include:

[0136] Step 800: Compare the first version number and the second version number to obtain the comparison result.

[0137] After obtaining the first version number and the second version number, the terminal compares the first version number and the second version number to obtain a comparison result. The comparison result includes whether the first version number and the second version number are the same or not.

[0138] Step 810: If the comparison result shows that the first version number and the second version number are different, then the access result of the business access is incomplete access.

[0139] If the terminal compares the first version number and the second version number and determines that the first version number and the second version number are different, it means that the data on the data block pointed to by the second pointer may have changed, and the access result of the service access is incomplete.

[0140] Step 820: If the comparison result shows that the first version number and the second version number are the same, then the access result of the business access is "access completed".

[0141] If the terminal compares the first version number and the second version number and determines that the first version number and the second version number are the same, it means that the data on the data block pointed to by the second pointer has not changed, and the access result of the service access is completed.

[0142] In this embodiment, the method of determining the access result of the service access by comparing whether the first version number and the second version number are the same is simple, fast, and easy to implement.

[0143] In one embodiment, the data migration method further includes the following steps:

[0144] If the access result of the business access is incomplete, a new second pointer is reloaded, and the steps corresponding to the first version number of the data segment to which the data block pointed to by the second pointer belongs are re-executed according to the new second pointer.

[0145] If the terminal determines that the access result of the service access is incomplete, the service access needs to be performed again. That is, the new second pointer is reloaded, the new second pointer is used as the second pointer, and the process returns to steps 700-720.

[0146] Due to the data migration process, the source data block was deleted. The new data block pointed to by the second pointer after the terminal is reloaded is the target data block. The first version number and the second version number obtained by the terminal are the same, which are both the version numbers corresponding to the target data block. Therefore, the access result of the business access is "access completed".

[0147] In this embodiment, when the result of the service access is incomplete, steps 700-720 can be re-executed by loading a new second pointer, thereby ensuring that the service access is completed.

[0148] In an optional embodiment, when the preset protection mechanism is the RCU mechanism and the priority of service access is lower than the priority of data migration, the process for preventing concurrent read operations from reading erroneous data during data migration is as follows: Figure 9 As shown.

[0149] The data migration process includes:

[0150] Step 900: Move the data to be migrated from the source data block to the target data block.

[0151] Step 910: Redirect the first pointer pointing to the source data block to the target data block.

[0152] Step 920: Access the Sleepable RCU (SRCU) interface, obtain the concurrent service access count of the source data block, and wait for the concurrent service access count to return to zero.

[0153] Step 930: Increment the version number of the data segment to which the source data block belongs.

[0154] Step 940: Delete the source data block.

[0155] The concurrent data reading process includes:

[0156] Within the read critical section of the RCU mechanism (from the start of SRCU read lock to the end of SRCU read unlock),

[0157] Step 950: Load the second pointer;

[0158] Step 960: Obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs;

[0159] Step 970: After the SRCU read unlock is completed, access the data in the data block pointed to by the second pointer;

[0160] Step 980: Obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs;

[0161] Step 990: Determine whether the first version number and the second version number are the same;

[0162] Step 991: If they are different, reload the new second pointer and return to execute steps 950-991.

[0163] Step 992: If they are the same, the access ends.

[0164] In the data migration process described above, only the version of the data block pointed to by the second pointer needs to be obtained within the read critical section; there is no need to access the data in the data block. This reduces the time spent waiting for the counter to be cleared during the data migration process. Furthermore, using the RCU mechanism can avoid lock contention and excessive memory consumption.

[0165] In one embodiment, such as Figure 10 As shown, if the priority of business access is higher than the priority of data migration, the steps of the data migration method include:

[0166] Step 101: In response to the data migration task, migrate the data to be migrated from the source data block to the target data block, and redirect the first pointer pointing to the source data block to the target data block in an atomic manner;

[0167] Step 102: Protect the source data block through a preset protection mechanism. The protection mechanism supports reading, copying, or updating the data in the source data block.

[0168] Step 103: Obtain the concurrent service access count of the source data block, and delete the source data block when the concurrent service access count is zero;

[0169] Step 104: During the data migration process, if a business access is received, the second pointer is loaded atomically to determine the data block pointed to by the second pointer; the data block pointed to by the second pointer is either the source data block or the target data block.

[0170] Step 105: Access the data in the data block pointed to by the second pointer.

[0171] In one embodiment, such as Figure 11 As shown, if the priority of business access is lower than the priority of data migration, the steps of the data migration method include:

[0172] Step 111: In response to the data migration task, migrate the data to be migrated from the source data block to the target data block, and redirect the first pointer pointing to the source data block to the target data block in an atomic manner;

[0173] Step 112: Protect the source data block through a preset protection mechanism. The protection mechanism supports reading, copying, or updating the data in the source data block.

[0174] Step 113: Obtain the concurrent service access count of the source data block and wait for the count to return to zero;

[0175] Step 114: Determine the target data segment to which the source data block belongs. Each data segment has a corresponding version number.

[0176] Step 115: Increment the version number of the target data segment;

[0177] Step 116: During the data migration process, if a business access is received, the second pointer is loaded atomically to obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs;

[0178] Step 117: Access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is either an incremented version number or a non-incremented version number;

[0179] Step 118: Compare the first version number and the second version number to obtain the comparison result;

[0180] Step 119: If the comparison result shows that the first version number and the second version number are different, then the access result of the business access is incomplete access;

[0181] Step 120: If the comparison result shows that the first version number and the second version number are the same, then the access result of the business access is "access completed".

[0182] Step 121: If the access result of the business access is incomplete, reload the new second pointer and return to execute steps 116-121 based on the new second pointer.

[0183] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0184] Based on the same inventive concept, this application also provides a data migration apparatus for implementing the data migration method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data migration apparatus embodiments provided below can be found in the limitations of the data migration method described above, and will not be repeated here.

[0185] In one embodiment, such as Figure 12 As shown, a data migration device 10 is provided, including: a migration module 11, a protection module 12, and a deletion module 13, wherein:

[0186] The migration module 11 is used to respond to a data migration task by migrating the data to be migrated from the source data block to the target data block and redirecting the first pointer pointing to the source data block to the target data block;

[0187] The protection module 12 is used to protect the source data block through a preset protection mechanism. The protection mechanism supports reading, copying or updating the data in the source data block.

[0188] The deletion module 13 is used to obtain the concurrent service access count of the source data block and delete the source data block when the concurrent service access count is zero.

[0189] In one embodiment, the data migration apparatus 10 further includes a data block determination module and an access module. The data block determination module, during the data migration process, if a business access is received, loads a second pointer to determine the data block pointed to by the second pointer; the data block pointed to by the second pointer is either a source data block or a target data block; the access module is used to access the data in the data block pointed to by the second pointer.

[0190] In one embodiment, the data migration apparatus 10 further includes a target data segment determination module and a processing module. The target data segment determination module is used to determine the target data segment where the source data block is located; each data segment has a corresponding version number; the processing module is used to increment the version number of the target data segment.

[0191] In one embodiment, the data migration apparatus 10 further includes a first acquisition module, a second acquisition module, and an access result determination module. The first acquisition module is used to, during the data migration process, if it is determined that the priority of the business access is lower than the priority of the data migration, acquire the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second acquisition module is used to access the data in the data block pointed to by the second pointer and acquire the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is either an incremented version number or a non-incremented version number; the access result determination module is used to determine the access result of the business access based on the first version number and the second version number.

[0192] In one embodiment, the access result determination module is specifically used to compare the first version number and the second version number to obtain a comparison result; if the comparison result shows that the first version number and the second version number are different, then the business access...

[0193] If the access result for the question is "access incomplete", then if the comparison result shows that the first version number and the second version number are the same, then the access result for service 5 is "access completed".

[0194] In one embodiment, the data migration apparatus 10 further includes a re-execution module. The re-execution module is used to reload a new second pointer if the access result of the business access is incomplete, and re-execute the steps of the first acquisition module, the second acquisition module, and the access result determination module according to the new second pointer.

[0195] In one embodiment, the data block determination module is specifically used to, if a service access is received, atomically load a second pointer and determine the data block pointed to by the second pointer.

[0196] In one embodiment, the relocation module 11 is specifically used to atomically redirect a first pointer to a source data block to a target data block.

[0197] Each module in the aforementioned data migration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0198] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 2 As shown.

[0199] 0. As those skilled in the art will understand, Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0200] 5. In one embodiment, a computer device is provided, including a memory and a processor, the memory...

[0201] The processor stores a computer program, and when it executes the computer program, it performs the following steps:

[0202] In response to the data migration task, the data to be migrated is moved from the source data block to the target data block, and the first pointer pointing to the source data block is redirected to the target data block;

[0203] The source data block is protected by a preset protection mechanism, which supports reading, copying, or updating operations on the data in the source data block.

[0204] Get the concurrent access count of the source data block, and delete the source data block when the concurrent access count is zero.

[0205] In one embodiment, when the processor executes the computer program, it further performs the following steps: during the data migration process, if a business access is received, a second pointer is loaded, and the data block pointed to by the second pointer is determined; the data block pointed to by the second pointer is either a source data block or a target data block; and the data in the data block pointed to by the second pointer is accessed.

[0206] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the target data segment where the source data block is located; each data segment has a corresponding version number; and incrementing the version number of the target data segment.

[0207] In one embodiment, when the processor executes the computer program, it further performs the following steps: during the data migration process, if it is determined that the priority of the business access is lower than the priority of the data migration, then obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is a version number that has been incremented or a version number that has not been incremented; and determine the access result of the business access based on the first version number and the second version number.

[0208] In one embodiment, when the processor executes the computer program, it further performs the following steps: comparing a first version number and a second version number to obtain a comparison result; if the comparison result is that the first version number and the second version number are different, the access result of the service access is incomplete access; if the comparison result is that the first version number and the second version number are the same, the access result of the service access is complete access.

[0209] In one embodiment, when the processor executes the computer program, it further implements the following steps: if the access result of the business access is incomplete, a new second pointer is reloaded, and according to the new second pointer, the step of re-executing the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs is re-executed.

[0210] In one embodiment, when the processor executes the computer program, it also performs the following steps: if a business access is received, it atomically loads a second pointer and determines the data block pointed to by the second pointer.

[0211] In one embodiment, when the processor executes a computer program, it also performs the following steps: atomically redirecting a first pointer to a source data block to a target data block.

[0212] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0213] In response to the data migration task, the data to be migrated is moved from the source data block to the target data block, and the first pointer pointing to the source data block is redirected to the target data block;

[0214] The source data block is protected by a preset protection mechanism, which supports reading, copying, or updating operations on the data in the source data block.

[0215] Get the concurrent access count of the source data block, and delete the source data block when the concurrent access count is zero.

[0216] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: during the data migration process, if a business access is received, a second pointer is loaded, and the data block pointed to by the second pointer is determined; the data block pointed to by the second pointer is either a source data block or a target data block; and the data in the data block pointed to by the second pointer is accessed.

[0217] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target data segment where the source data block is located; each data segment has a corresponding version number; and incrementing the version number of the target data segment.

[0218] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: during the data migration process, if it is determined that the priority of the business access is lower than the priority of the data migration, then obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is a version number that has been incremented or a version number that has not been incremented; determine the access result of the business access based on the first version number and the second version number.

[0219] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: comparing a first version number and a second version number to obtain a comparison result; if the comparison result is that the first version number and the second version number are different, the access result of the business access is incomplete access; if the comparison result is that the first version number and the second version number are the same, the access result of the business access is complete access.

[0220] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: if the access result of the business access is incomplete, a new second pointer is reloaded, and according to the new second pointer, the steps corresponding to the first version number of the data segment to which the data block pointed to by the second pointer belongs are re-executed.

[0221] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if a business access is received, a second pointer is loaded atomically to determine the data block pointed to by the second pointer.

[0222] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: atomically redirecting a first pointer to the source data block to the target data block.

[0223] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0224] In response to the data migration task, the data to be migrated is moved from the source data block to the target data block, and the first pointer pointing to the source data block is redirected to the target data block;

[0225] The source data block is protected by a preset protection mechanism, which supports reading, copying, or updating operations on the data in the source data block.

[0226] Get the concurrent access count of the source data block, and delete the source data block when the concurrent access count is zero.

[0227] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: during the data migration process, if a business access is received, a second pointer is loaded, and the data block pointed to by the second pointer is determined; the data block pointed to by the second pointer is either a source data block or a target data block; and the data in the data block pointed to by the second pointer is accessed.

[0228] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target data segment where the source data block is located; each data segment has a corresponding version number; and incrementing the version number of the target data segment.

[0229] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: during the data migration process, if it is determined that the priority of the business access is lower than the priority of the data migration, then obtain the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is a version number that has been incremented or a version number that has not been incremented; determine the access result of the business access based on the first version number and the second version number.

[0230] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: comparing a first version number and a second version number to obtain a comparison result; if the comparison result is that the first version number and the second version number are different, the access result of the business access is incomplete access; if the comparison result is that the first version number and the second version number are the same, the access result of the business access is complete access.

[0231] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: if the access result of the business access is incomplete, a new second pointer is reloaded, and according to the new second pointer, the steps corresponding to the first version number of the data segment to which the data block pointed to by the second pointer belongs are re-executed.

[0232] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if a business access is received, a second pointer is loaded atomically to determine the data block pointed to by the second pointer.

[0233] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: atomically redirecting a first pointer to the source data block to the target data block.

[0234] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0235] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0236] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data migration method, characterized in that, The method includes: In response to the data migration task, the data to be migrated is moved from the source data block to the target data block, and the first pointer pointing to the source data block is redirected to the target data block; The source data block is protected by a preset protection mechanism, which supports reading, copying, or updating operations on the data in the source data block. Obtain the concurrent service access count of the source data block, and delete the source data block when the concurrent service access count is zero.

2. The method according to claim 1, characterized in that, The method further includes: During the data migration process, if a business access is received, a second pointer is loaded to determine the data block pointed to by the second pointer; the data block pointed to by the second pointer is either the source data block or the target data block. Access the data in the data block pointed to by the second pointer.

3. The method according to claim 2, characterized in that, Before deleting the source data block, the method further includes: The target data segment containing the source data block is determined; each data segment has a corresponding version number; The version number of the target data segment is incremented.

4. The method according to claim 3, characterized in that, The method further includes: During the data migration process, if it is determined that the priority of the service access is lower than the priority of the data migration, then the first version number corresponding to the data segment to which the data block pointed to by the second pointer belongs is obtained; Access the data in the data block pointed to by the second pointer, and obtain the second version number corresponding to the data segment to which the data block pointed to by the second pointer belongs; the second version number is either a version number that has undergone the incrementing process or a version number that has not undergone the incrementing process. The access result of the service access is determined based on the first version number and the second version number.

5. The method according to claim 4, characterized in that, Determining the access result of the service access based on the first version number and the second version number includes: Compare the first version number and the second version number to obtain the comparison result; If the comparison result shows that the first version number and the second version number are different, then the access result of the service access is incomplete. If the comparison result shows that the first version number and the second version number are the same, then the access result of the service access is "access completed".

6. The method according to claim 5, characterized in that, The method further includes: If the access result of the service access is incomplete, a new second pointer is reloaded, and the steps corresponding to the first version number of the data segment to which the data block pointed to by the second pointer belongs are re-executed according to the new second pointer.

7. The method according to any one of claims 2-6, characterized in that, If a service access is received, loading the second pointer and determining the data block pointed to by the second pointer includes: If a business access request is received, the second pointer is loaded atomically to determine the data block that the second pointer points to.

8. The method according to any one of claims 1-6, characterized in that, The step of redirecting the first pointer pointing to the source data block to the target data block includes: The first pointer pointing to the source data block is redirected atomically to the target data block.

9. A data migration device, characterized in that, The device includes: The migration module is used to respond to a data migration task by migrating the data to be migrated from the source data block to the target data block, and redirecting the first pointer pointing to the source data block to the target data block; The protection module is used to protect the source data block through a preset protection mechanism, which supports reading, copying or updating operations on the data in the source data block; The deletion module is used to obtain the concurrent service access count of the source data block and delete the source data block when the concurrent service access count is zero.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images