Method and apparatus for migrating virtual functions

Abstract

The application discloses a virtual function migration method and device, and relates to the technical field of computer storage, and comprises the following steps: in the case that a migration instruction for a target virtual function is acquired, performing migration initialization, wherein the migration initialization comprises resource checking on a destination namespace; enabling a double-write buffer mechanism, wherein the double-write buffer mechanism refers to that in the migration process, when a write request is received, write data is written into a source namespace and the destination namespace at the same time, and when a read request is received, the read request is routed to the source namespace; the source namespace is a namespace where the target virtual function currently locates, and the destination namespace is a namespace where the target virtual function is to be migrated to; migrating historical data in the source namespace to the destination namespace; and switching namespace binding of the target virtual function from the source namespace to the destination namespace, so as to solve the core pain point of interrupting I / O services in the related art, and realize real hot migration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer storage, and particularly relates to a virtual function migration method and device. BACKGROUND

[0002] At present, for a solid state disk (SSD) of a Non Volatile Memory Express (NVMe) protocol under a Single Root I / O Virtualization (SRIOV) architecture, the SSD usually allocates an independent virtual function (VF) to each virtual machine and binds the VF to a specific Name Space (NS). In actual application, due to load balancing, wear leveling or fault recovery, etc., it is necessary to migrate the VF from one NS to another NS of the same SSD.

[0003] However, the VF migration usually needs to rely on the cooperation of a host system, and is completed through steps such as suspending virtual machine I / O, copying data, and reconfiguring mapping relationship, so that the I / O service needs to be interrupted in the VF migration process, and real hot migration cannot be realized. SUMMARY

[0004] The present application provides a virtual function migration method and device to at least solve the problem in the related art that the I / O service needs to be interrupted in the VF migration process, and real hot migration cannot be realized.

[0005] The present application provides a virtual function migration method, comprising: in the case that a migration instruction for a target virtual function is acquired, performing migration initialization, wherein the migration initialization comprises resource verification on a destination Name Space;

[0006] enabling a double write buffer mechanism, wherein the double write buffer mechanism refers to that, in the migration process, when a write request is received, the write data is written into a source Name Space and a destination Name Space at the same time, and when a read request is received, the read request is routed to the source Name Space, the source Name Space is a Name Space where the target virtual function currently locates, and the destination Name Space is a Name Space where the target virtual function is to be migrated to;

[0007] migrating historical data in the source Name Space to the destination Name Space;

[0008] switching Name Space binding of the target virtual function from the source Name Space to the destination Name Space.

[0009] The application further provides a virtual function migration device, comprising: a first initialization module, configured to perform migration initialization when a migration instruction for a target virtual function is acquired, wherein the migration initialization comprises resource checking on a destination namespace;

[0010] a first double-write module, configured to enable a double-write buffer mechanism, wherein the double-write buffer mechanism refers to writing data into a source namespace and a destination namespace simultaneously when a write request is received during migration, and routing a read request to the source namespace when a read request is received, the source namespace being a namespace where the target virtual function currently locates, and the destination namespace being a namespace where the target virtual function is to be migrated to;

[0011] a first migration module, configured to migrate historical data in the source namespace to the destination namespace;

[0012] a first switching module, configured to switch namespace binding of the target virtual function from the source namespace to the destination namespace.

[0013] The application further provides an electronic device, comprising: a memory, configured to store a computer program; and a processor, configured to implement the steps of the virtual function migration method when executing the computer program.

[0014] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program implements the steps of the virtual function migration method when executed by a processor.

[0015] The application further provides a computer program product, comprising a computer program, and the computer program implements the steps of the virtual function migration method when executed by a processor.

[0016] According to the application, during migration of a target virtual function, write data can be written into a source namespace and a destination namespace simultaneously when a write request is received, and a read request can be routed to the source namespace when a read request is received, so that I / O service continuity is not affected, the core pain point of interrupting I / O service in the related art is solved, and real hot migration is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0018] Figure 1A flowchart of a virtual function migration method provided by an embodiment of the present application;

[0019] Figure 2 A flowchart of a virtual function migration example provided by an embodiment of the present application;

[0020] Figure 3 A flowchart of a breakpoint resume example provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative work fall within the protection scope of the present application.

[0022] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0023] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0024] At present, for a solid state disk (SSD) under a single root I / O virtualization (SRIOV) architecture and a non volatile memory express (NVMe) protocol of a non volatile memory host controller interface protocol, the SSD usually allocates an independent virtual function for each virtual machine and binds to a specific name space (NS). In actual application, due to load balancing, wear leveling or fault recovery, etc., it is necessary to migrate the VF from one NS to another NS of the same SSD.

[0025] However, VF migration typically requires the cooperation of the host system, involving steps such as pausing virtual machine I / O, copying data, and reconfiguring mapping relationships. This has the following drawbacks: I / O services need to be interrupted during the VF migration process, making true hot migration impossible; it relies on external host coordination, increasing complexity; and data consistency is difficult to guarantee, especially when write operations occur during the migration process. Therefore, a hot migration method for VFs that is entirely implemented in software within the SSD, does not rely on an external host, and can guarantee data consistency is needed. In view of this, this application provides a method and device for migrating virtual functions. The method for migrating virtual functions will be described in detail below.

[0026] Figure 1 This is a flowchart illustrating a virtual function migration method provided in an embodiment of this application. This method can be executed by an SSD, etc. Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0027] S110. Upon receiving a migration instruction for the target virtual function, perform migration initialization, which includes resource verification of the target namespace.

[0028] Specifically, the target virtual function refers to the virtual function to be migrated.

[0029] Specifically, there are multiple ways to obtain migration instructions. The following are typical examples, but they do not constitute a limitation of this disclosure. Figure 2 A flowchart illustrating a virtual function migration example provided in an embodiment of this application. Figure 2 As shown, in some examples, the SSD receives migration instructions sent by the host through the SSD management interface (such as a special NVMe management command 0xC0). In other examples, the SSD automatically generates migration instructions when it detects that the flash block containing the source namespace has been erased ≥ a preset erase count threshold (such as 3000 times) using a wear leveling algorithm.

[0030] Specifically, the migration command may include a target virtual function identifier, a source namespace identifier, and a destination namespace identifier. The source namespace is the namespace where the target virtual function currently resides, and the destination namespace is the namespace to which the target virtual function will be migrated.

[0031] Specifically, the core of migration initialization is to complete resource verification before migration to avoid migration failure due to invalid resources or resource conflicts.

[0032] Optionally, migration initialization includes: S111, performing resource verification on the destination namespace.

[0033] Specifically, such as Figure 2As shown, at least one resource check is performed on the target namespace as follows: 1) existence check: query the namespace list of the SSD to confirm that the target namespace has been created; 2) capacity check: confirm that the available physical capacity of the target namespace is greater than the physical capacity of the source namespace (to avoid data overflow after migration); 3) health check: confirm that the bad block rate of the flash block where the target namespace is located is less than a preset bad block rate threshold (such as 0.1% or the like); 4) occupancy state check: query the virtual function-namespace binding table to confirm that the target namespace is not allocated to any virtual function; and 5) format compatibility check: the sector size of the target namespace is consistent with that of the source namespace (to avoid I / O address mapping misplacement). If any of the checks fails, an error log (event code such as 0x01, describing the specific failure reason) is sent to the host; the migration initialization is terminated, and the occupied resources are released.

[0034] S112, in the case where the target namespace passes the resource check, a migration control block is created and initialized.

[0035] Specifically, the migration control block (MCB) is a state hub of the SSD during execution of virtual function migration, and the state of the entire migration process is controlled and recorded by the migration control block to ensure data integrity and recoverability of the source namespace and the target namespace.

[0036] Optionally, the migration control block is created and initialized, including: S1121, allocating storage space for the migration control block in the temporary memory.

[0037] Specifically, the size of the storage space occupied by the migration control block can be set by a person skilled in the art according to actual conditions, which is not limited here. For example, a block of 1024 bytes of continuous space is allocated in the migration management area of the internal DRAM of the SSD to store the migration control block, but the present application is not limited thereto.

[0038] S1122, initializing the key fields of the migration control block, wherein the key fields of the migration control block include at least one of the following: target virtual function identifier, source namespace identifier, target namespace identifier, migration state, data synchronization byte offset, double-write enabling flag, migration block size, total migration block number, migrated block number, migration log write offset, reserved field, and check information.

[0039] Specifically, the target virtual function identifier refers to the identifier of the target virtual function, and this field indicates which virtual function initiates the migration.

[0040] Specifically, the source namespace identifier refers to the identifier of the source namespace, and the target namespace identifier refers to the identifier of the target namespace, and these two fields indicate that the data is migrated from which namespace to which namespace.

[0041] Specifically, the migration state field indicates the current migration stage. For example, 0x01 indicates migration initialization, 0x02 indicates double writing, 0x03 indicates synchronization completion, 0x04 indicates switching completion, and 0x05 indicates migration end (migration is completely completed).

[0042] Specifically, the data synchronization byte offset indicates how many bytes of data have been synchronized, which is used for breakpoint continuation.

[0043] Specifically, the double writing enable flag indicates whether the double writing buffer mechanism is enabled. After the double writing buffer mechanism is enabled, all write data is written to the source namespace and the destination namespace at the same time, preventing data inconsistency during migration.

[0044] Specifically, the migration block size indicates the basic unit of each migration. It needs to be aligned with the SSD page size (such as 4KB, 16KB, or 64KB, etc.), which improves efficiency.

[0045] Specifically, the total migration block number indicates how many data blocks are migrated from the source namespace to the destination namespace, which is equal to the used physical capacity of the source namespace divided by the migration block size and then rounded up.

[0046] Specifically, the migrated block number indicates how many data blocks have been migrated from the source namespace to the destination namespace. Multi-threaded parallel migration is supported, and this field records the migrated block number of each thread at this time.

[0047] Specifically, the migration log write offset field indicates where the migration log is currently written.

[0048] Specifically, the reserved field is reserved for future expansion functions (such as timestamp, priority, etc.)

[0049] Specifically, the verification information is the verification value obtained by verifying all the previous fields (such as CRC32), which prevents data damage.

[0050] For example, the structure definition of the migration control block is as follows:

[0051] typedef struct {

[0052] uint32_t vf_id; / / Target virtual function identifier (such as 3)

[0053] uint32_t src_ns; / / Source namespace identifier (such as 0x01)

[0054] uint32_t dst_ns; / / Destination namespace identifier (such as 0x02)

[0055] uint8_t status; / / migration status

[0056] uint64_t sync_offset; / / data sync offset (initially 0)

[0057] bool double_write_en; / / double write enable flag (initially false)

[0058] uint32_t block_size; / / migration block size

[0059] uint32_t total_blocks; / / total migration block number

[0060] uint32_t completed_blocks[4]; / / migrated block number (initially 0)

[0061] uint32_t log_offset; / / migration log write offset (initially 0)

[0062] uint8_t reserved

[512] ; / / reserved field (for extension)

[0063] uint32_t crc32; / / check information

[0064] } MigrationControlBlock

[0065] S1123, creating a migration control block backup in the non-volatile storage area, wherein the migration control block is periodically synchronized to the migration control block backup.

[0066] Specifically, a migration control block backup is created in the SSD internal power failure protection area (such as a non-volatile storage area), and is synchronized once every period of time (such as 100 ms) during the migration process to prevent power failure loss.

[0067] It can be understood that by creating and initializing the migration control block, a unified control center for the migration process of the virtual function is established, and the target virtual function, the source and destination namespace, the migration status, the progress information, etc. are recorded. This way of creating and initializing the migration control block not only provides a clear management account for the migration process, but also supports breakpoint resume after abnormal power-off, and provides a reliable basis for data consistency verification. In this way, it is beneficial to improve the reliability and maintainability of the migration process.

[0068] S113, resource locking of the source namespace and the destination namespace.

[0069] Optionally, the source namespace and the destination namespace are resource-locked, including: S1131, disabling deletion and formatting of the source namespace, and shielding the hardware queue associated with the target virtual function.

[0070] Specifically, any host, management command, firmware, etc. is not allowed to delete or format the source namespace. It can be understood that there is historical data in the source namespace, and if the source namespace is deleted or formatted, the historical data will be lost, resulting in migration failure or even business interruption.

[0071] Specifically, the I / O channel corresponding to the target virtual function is temporarily closed, the hardware queue (i.e. the submission queue SQ and the completion queue CQ of NVMe) associated with the target virtual function is shielded, and no new read / write request is received. In this way, it can be prevented that new write requests enter the hardware queue when the migration just starts and is not ready, but the read / write requests cannot be discarded directly, so the shielding is performed first, and the new write requests are released after the double-write buffering mechanism is enabled, and it is ensured that the new write requests can be written to the source namespace and the destination namespace at the same time.

[0072] S1132, disabling deletion and formatting of the destination namespace, and disabling binding of the destination namespace to other virtual functions.

[0073] Specifically, any host, management command, firmware, etc. is not allowed to delete or format the destination namespace. It can be understood that the historical data in the source namespace will be synchronized to the destination namespace later, and if the destination namespace is deleted or formatted, the migration will fail.

[0074] Specifically, the destination namespace is not allowed to be used by other virtual functions or other hosts at the same time. In this way, it can be prevented that two hosts write data to the destination namespace at the same time, resulting in that the historical data migrated from the source namespace is overwritten or conflicted, and the data consistency is destroyed.

[0075] S1133, waiting for the completion of the unfinished read / write request.

[0076] Specifically, the hardware queue associated with the target virtual function is shielded, and new read / write requests can be blocked, but there can be some unfinished read / write requests (i.e. submitted but not completed). The state of the hardware queue is queried in a loop (e.g. every 1ms), and all unfinished read / write requests are waited to be completed. Of course, a timeout time (e.g. 10s) can also be set, and if all unfinished read / write requests are not completed within the timeout time, the migration is terminated, and a log (e.g. event code 0x04, description of timeout) is recorded.

[0077] Of course, the target virtual function can also be labeled as being migrated to prevent multiple migration instructions from being initiated at the same time for the same virtual function (e.g. the user shakes the migration button twice).

[0078] S114, initialize the migration log.

[0079] Specifically, a log namespace (or a block of storage space of the non-volatile storage area) dedicated to the SSD is opened, and the initialized migration log is written. Optionally, the key fields of the migration log include at least one of the following: timestamp, event type, target virtual function identifier, source namespace identifier, destination namespace identifier, migration status, and migration control block check information.

[0080] Specifically, the timestamp indicates the start time of the migration task.

[0081] Specifically, the event type identifies the classification of the migration log. For example, START indicates that the migration task has started, SYNC_PROGRESS indicates the synchronization progress, COMPLETE indicates that the migration task has ended, ERROR indicates that the migration task has an error, and the like.

[0082] Specifically, as shown in Figure 2 , after the resource lock of the source namespace and the destination namespace is performed, the START migration log is written. For example, the format of the START migration log is as follows:

[0083] [timestamp=1620000000000 (i.e., timestamp), event=START (i.e., event type is that the migration task has started), vf=3 (i.e., target virtual function identifier), src_ns=0x01 (1T) (i.e., source namespace identifier), dst_ns=0x02 (2T) (i.e., destination namespace identifier), status=INIT (i.e., migration status), crc=0x12345678 (i.e., migration control block check information)]

[0084] Specifically, as shown in Figure 2 , the migration log write offset (i.e., uint32_t log_offset) in the migration control block is updated to the current migration log at the current start write position (such as 0x100, etc.), ensuring that the subsequent migration log can be continuously written.

[0085] Specifically, the migration control block is also backed up to the log namespace to obtain a backup of the migration control block. In this way, each time the migration control block is updated, two backups are updated, one migration control block backup is in the non-volatile storage area, and the other backup of the migration control block is in the log namespace, preventing an exception in one of the backups.

[0086] It can be understood that the resource check before migration and the state initialization can be completed by performing resource check on the target namespace, creating and initializing the migration control block, performing resource locking on the source namespace and the target namespace, and initializing the migration log, so as to avoid migration failure caused by invalid and / or conflicting resources.

[0087] S120, enabling a double write buffer mechanism, wherein the double write buffer mechanism refers to writing write data into the source namespace and the target namespace at the same time when receiving a write request in the migration process, and routing a read request to the source namespace when receiving a read request, the source namespace being a namespace where the target virtual function currently locates, and the target namespace being a namespace where the target virtual function is to be migrated to.

[0088] Specifically, the double write buffer mechanism is a mechanism for writing write data into the source namespace and the target namespace at the same time by using a double write buffer area.

[0089] Optionally, enabling the double write buffer mechanism comprises: S121, initializing the double write buffer area.

[0090] Specifically, the double write buffer area is a special buffer area for temporarily storing write data written by the host during the migration process, and supports writing the write data into the source namespace and the target namespace at the same time.

[0091] Optionally, initializing the double write buffer area comprises: S1211, allocating storage space for the double write buffer area in the temporary memory.

[0092] Specifically, the specific size of the storage space of the double write buffer area can be set by a person skilled in the art according to actual conditions, which is not limited here. For example, a special circular buffer area with a fixed size (for example, 1MB) is allocated in the internal DRAM of the SSD as the double write buffer area, but it is not limited to this.

[0093] S1212, initializing a cache management structure corresponding to the double write buffer area, wherein the cache management structure comprises: a to-be-double-written data cache area, an index of the earliest unreleased data cache, an index to be written into the next data cache, a write metadata area, a current index of the metadata cache, and a logical block address of the last successful double writing.

[0094] Specifically, the cache management structure is a core control block for managing the running state of the double write buffer area.

[0095] Specifically, the to-be-double-written data cache area is a circular buffer area of the double write buffer area, which is used to temporarily store write data in a write request from the host, and its type can be an array, but it is not limited to this.

[0096] Specifically, once the write data of a write request is successfully double-written and returned to the host, the write data can be released from the data buffer to be double-written. The index of the earliest unreleased data buffer is used to point to the starting position of the earliest unreleased data block in the data buffer to be double-written.

[0097] Specifically, the index to be written to the next data cache is used to point to the location in the data cache to be double-written where the write data of the next write request will be written.

[0098] Specifically, the metadata area is used to record the metadata for each write request.

[0099] Specifically, the current index of the metadata cache is used to point to the slot index where the next metadata to be written in the metadata area will be written.

[0100] Specifically, the address of the logical block that was most recently successfully double-written is used to record the address of the logical block that was most recently successfully double-written.

[0101] For example, the cache management structure is as follows:

[0102] typedef struct {

[0103] uint8_t data_buf[data_size]; / / Data buffer to be double-written

[0104] uint32_t head; / / Index of the earliest unreleased cached data (initially 0)

[0105] uint32_t tail; / / The index at which the next data cache will be written (initially 0)

[0106] WriteMeta meta_buf[meta_size]; / / Writes metadata to the metadata area (20 bytes per entry).

[0107] uint32_t meta_buf_idx; / / Metadata cache current index (initially 0)

[0108] uint64_t last_lba; / / Address of the logical block last double-written

[0109] DoubleWriteBuf

[0110] Understandably, this method of initializing the dual-write cache not only provides centralized management of the critical control information of the dual-write cache, but also supports state recovery after abnormal power failure, and provides a reliable basis for data consistency verification. In this way, it helps to improve the reliability and maintainability of the migration process.

[0111] S122, restore the hardware queue associated with the target virtual function.

[0112] Specifically, the shielding of the hardware queue associated with the target virtual function is removed, so that the SSD can receive read and write requests of the target virtual function again.

[0113] S123, for the write request in the hardware queue, intercept the write request, and write the write data into the source namespace and the destination namespace at the same time by using the double-write cache area.

[0114] S124, for the read request in the hardware queue, route the read request to the source namespace.

[0115] Specifically, all write requests of the target virtual function are intercepted, and the write request processing flow is modified to enter the double-write processing flow. The read request is still normally routed to the source namespace according to the original path and does not participate in double-write.

[0116] Optionally, for the write request in the hardware queue, the write request is intercepted, and the write data is written into the source namespace and the destination namespace at the same time by using the double-write cache area, including: S1231, copy the write data to the double-write cache area, and update the index of the next data cache to be written.

[0117] Specifically, when a write request (including logical block address, data length, and data content) is received, the data content (i.e. write data) is copied to the double-write data cache area in the double-write cache area, and the starting position is indicated by the index of the next data cache to be written (i.e. uint32_t tail), and the index of the next data cache to be written is updated to prepare for the next write position. Of course, if the double-write data cache area is full (i.e. (uint32_t tail +1) modulo the size of the double-write data cache area is equal to head, indicating that the double-write data cache area is full), the write request is blocked, and the write request is continued after the double-write data cache area is released.

[0118] S1232, calculate a first check value of the write data; write the write data in the double-write cache area to the source namespace as a data source, calculate a second check value of the write data written in the source namespace, and write the write data to the destination namespace, calculate a third check value of the write data written in the destination namespace; if the first check value is consistent with the second check value, the write data is written successfully, if the first check value is inconsistent with the second check value, the write data is written to the source namespace again until the write data is written successfully or reaches a first preset rewrite upper limit; if the first check value is consistent with the third check value, the write data is written successfully, if the first check value is inconsistent with the third check value, the write data is written to the destination namespace again until the write data is written successfully or reaches a first preset rewrite upper limit.

[0119] Specifically, a first check value (such as a CRC check value, etc.) of the current write data is calculated. Two write requests are initiated in parallel: write the write data to the source namespace based on the logical address in the write request, and write the write data to the same logical address offset of the destination namespace, and wait for both write requests to return a "success" response. Then read the written data from the source namespace, recalculate the second check value (such as the CRC check value), read the written data from the destination namespace, recalculate the third check value (such as the CRC check value), compare the second check value with the original first check value, and compare the third check value with the original first check value. If the check fails on either side, retry the write to the namespace, up to a first preset upper limit of retries (such as 3 times, etc.), otherwise mark an error and terminate the migration.

[0120] S1236, record the metadata of the write request in the metadata area.

[0121] Specifically, the metadata of the write request can include at least one of the following: logical address, data length, timestamp, first check value.

[0122] For example, the metadata of the current write request is recorded in meta_buf[meta_buf_idx], and the format is as follows:

[0123] {LBA: 0x123456 (i.e. logical address), data_len: 4096 (i.e. data length), timestamp: 1620000001000 (i.e. timestamp), crc: 0xABCDEF (i.e. first check value)}

[0124] Specifically, if the metadata area is full, all the metadata is batched and written to the log namespace (or a block of storage space in the non-volatile storage area), and then the cache is emptied and the current index of the metadata cache is reset.

[0125] S1237, after the write data is written successfully, the write data is released in the double-write cache area, and the index of the earliest unreleased data cache is updated.

[0126] Specifically, after the write data is written successfully, an NVMe write success response is returned to the host, the space occupied by the write data in the double-write data cache area is released, and the index of the earliest unreleased data cache is updated for subsequent reuse.

[0127] It can be understood that by first temporarily storing the write data written by the host in the double-write cache area, and then copying the write data in the double-write cache area to the same position in the source namespace and the destination namespace, it can be ensured that the two sides are always consistent. And through the check value calculation and metadata recording, data corruption or metadata loss can be prevented.

[0128] Specifically, as shown in Figure 2 enabling the double write cache mechanism further includes setting the migration state in the migration control block to 0x02 (i.e., in double write), setting the double write enable flag to true, synchronously backing up the migration control block to the non-volatile storage area and the log namespace, implementing the double backup mechanism, and ensuring that the state can be restored after power failure.

[0129] It can be understood that enabling the double write cache mechanism can ensure that the new write data is synchronized to the source namespace and the destination namespace, and avoid data loss during migration. In this way, the I / O service of the host can be normally performed without interruption, and the user experience is improved.

[0130] S130, migrating the historical data in the source namespace to the destination namespace.

[0131] Specifically, the historical data refers to the data written to the source namespace.

[0132] Optionally, migrating the historical data in the source namespace to the destination namespace includes: S131, determining the data range of the historical data, and determining the total number of migration blocks according to the data range.

[0133] Specifically, the usage of the logical blocks of the source namespace is read, and the used logical block range is filtered out, excluding idle blocks, bad blocks, and logical blocks that have been trimmed, to obtain an effective data area that needs to be synchronized. If the effective data is not continuous, it is divided into multiple continuous data ranges, and each continuous data range is recorded by a start logical address and an end logical address, denoted as: [lba_start1, lba_end1], [lba_start2, lba_end2], …, [lba_startn, lba_endn].

[0134] Specifically, for each continuous data range, the number of data blocks corresponding to the continuous data range is calculated: data block number = (end logical address - start logical address + 1) x sector size / migration block size. The number of data blocks of each continuous data range is summed to obtain the total number of migration blocks, and the total number of migration blocks is assigned to the uint32_t total_blocks variable in the migration control block.

[0135] S132, using a multi-threaded concurrent mode to migrate the data blocks of the total number of migration blocks from the source namespace to the destination namespace.

[0136] Specifically, there are multiple cores on the SSD, and several of the cores are used for multi-threaded concurrent migration. A thread is created on each core, and each thread migrates a part of the data blocks. For example, there are 8 cores on the SSD, and 4 of the cores are used. A thread is created on each core, and each thread migrates a part of the data blocks, but it is not limited thereto.

[0137] Optionally, the data blocks of the total migration block number are migrated from the source namespace to the destination namespace in a multi-thread concurrent manner, including: S1321, dividing the data range of the historical data into multiple sub data ranges.

[0138] Specifically, the data range of the historical data is equally divided by the number of data blocks (may not be completely divided, and the number of data blocks processed by each thread is ensured to differ by no more than 1, and load balancing is ensured), to obtain multiple sub data ranges, and each thread is responsible for a sub data range.

[0139] S1322, each thread migrates the data blocks in its allocated sub data range in sequence, wherein the migration steps of each data block are as follows: reading the data block from the source namespace, calculating the fourth check value of the data block, if the data block is not written into the destination namespace by the double-write buffer mechanism, writing the data block into the destination namespace, calculating the fifth check value of the data block written in the destination namespace, if the fourth check value and the fifth check value are consistent, the data block migration is successful, if the fourth check value and the fifth check value are inconsistent, the data block is re-written into the destination namespace until the data block migration is successful or the second preset re-write upper limit is reached.

[0140] Specifically, each thread processes the data blocks in its allocated sub data range in sequence, and processes one data block of a migration block size (such as 64KB) each time. The specific steps are as follows: reading a data block from the source namespace and calculating the fourth check value (such as CRC check) of the data block; querying the write metadata area and the log namespace to determine whether the logical address of the current data block is within the double-write range to determine whether the current data block has been written into the destination namespace by the double-write buffer mechanism; if it is within the double-write range, skip the data block and migrate the next data block; if it is not within the double-write range, write the current data block into the destination namespace, read the data block just written in the destination namespace, calculate the fifth check value of the read data block, if the fourth check value and the fifth check value are consistent, the data block migration is successful, if the fourth check value and the fifth check value are inconsistent, re-write the data block into the destination namespace until the data block migration is successful or the second preset re-write upper limit (such as 3 times) is reached, if the final migration is successful, continue to migrate the next data block, and increment the migrated block number corresponding to the thread in the migration control block, if the final migration fails, mark the logical block as a bad block, and enable the spare block update.

[0141] Specifically, as Figure 2As shown, a progress record can also be written to the log namespace whenever a certain thread accumulates a preset number (such as 1024, etc.) of data block migrations, in the following format: [timestamp=1620000010000 (i.e. timestamp), event=SYNC_PROGRESS (i.e. event type is synchronization progress), vf=3 (i.e. destination virtual function identifier), thread=0 (i.e. thread identifier), completed=1024 (i.e. the number of migrated blocks of the thread), total=100000 (i.e. the total number of migration blocks of the thread)], for fault recovery to determine the synchronization progress completed by each thread.

[0142] Specifically, the sum of the migrated block numbers of all threads in the migration control block can also be checked once a thread completes its migration task, to determine whether the sum is equal to the total number of migration blocks. For example, as shown in the following table, the sum of the migrated block numbers of all threads in the migration control block is 1024, which is equal to the total number of migration blocks, indicating that all threads have completed their migration tasks. Figure 2 As shown, when all threads complete the migration of the data blocks in their allocated sub-data ranges, the migration state in the migration control block is updated to 0x03 (i.e. synchronization complete), the synchronization backup migration control block is updated to the non-volatile storage area and the log namespace, and the double backup is performed.

[0143] It can be understood that, by using the multi-thread concurrent mode, each thread migrates data blocks in the order of the divided sub-data ranges and skips the double-written data blocks, which can avoid repeated operations and improve migration efficiency. Moreover, data reading verification is performed during the migration process, and progress information is persistently recorded, which can ensure that the data migration can be resumed after abnormal power-off, thereby improving the reliability of data migration.

[0144] By using the multi-thread concurrent mode, data in the source namespace that has not been synchronized through the double-write cache mechanism is copied to the destination namespace, each thread divides tasks according to data ranges, skips the double-written area, performs data verification and progress persistence, and high-efficiency and reliable data synchronization is achieved.

[0145] It can be understood that, by using the multi-thread concurrent mode to migrate historical data, the migration efficiency can be improved.

[0146] S140, switching the namespace binding of the target virtual function from the source namespace to the destination namespace.

[0147] Optionally, switching the namespace binding of the target virtual function from the source namespace to the destination namespace includes: S141, shielding the hardware queue associated with the target virtual function, waiting for the completion of the unfinished read-write request, and stopping the double-write cache mechanism.

[0148] Specifically, shielding the hardware queue associated with the target virtual function can block new read and write requests, but there can still be some unfinished read and write requests (i.e., submitted but not completed). The hardware queue state is queried in a loop (e.g., every 1 ms), and all unfinished read and write requests are waited for to be completed. Of course, a timeout period (e.g., 10 s) can also be set, and if all unfinished read and write requests are not completed within the timeout period, the read and write requests are forcibly terminated, and a log is recorded.

[0149] Specifically, the double-write buffering mechanism can also be stopped, the double-write enable flag in the migration control block is set to false, the double-write buffer is released, and the remaining metadata is flushed to the log namespace.

[0150] S142, the binding between the target virtual function and the source namespace is released, the binding between the target virtual function and the destination namespace is established, and the host is notified that the namespace binding relationship has changed.

[0151] Specifically, the binding between the target virtual function and the source namespace is released, the target virtual function is bound to the source namespace, and the virtual function-namespace binding table inside the SSD is updated. An asynchronous event is sent to the host to notify the host that the namespace binding relationship has changed, triggering the host to query and obtain the latest namespace binding relationship.

[0152] S143, the hardware queue associated with the target virtual function is restored.

[0153] Specifically, the shielding of the hardware queue associated with the target virtual function is released, so that the SSD can receive read and write requests of the target virtual function again.

[0154] S144, within a preset time period, for read and write requests in the hardware queue, if the read and write request is directed to the destination namespace, the read and write request is routed to the destination namespace, and if the read and write request is directed to the source namespace, the read and write request is forwarded to the destination namespace.

[0155] Specifically, within a preset time period (e.g., 3 s), all read and write requests of the target virtual function are intercepted, the namespace identifier carried in the read and write request is checked, if the namespace identifier carried in the read and write request is equal to the source namespace identifier, the read and write request is routed to the destination namespace according to the mapping relationship between the source namespace identifier and the destination namespace identifier recorded in the migration control block, and if the namespace identifier carried in the read and write request is equal to the destination namespace identifier, the read and write request is normally routed to the destination namespace.

[0156] Specifically, as shown in Figure 2 The migration state in the migration control block can also be set to 0x04 (i.e., switching is completed), and the migration control block is synchronously backed up to the non-volatile storage area and the log namespace for double backup.

[0157] It can be understood that, by setting the short shielding hardware queue + forwarding the read and write requests within the preset time length, even if the host does not update the namespace binding information in time, the read and write requests pointing to the source namespace can still be forwarded to the destination namespace, avoiding the failure of read and write request response.

[0158] In the embodiments of the present disclosure, during the migration of the target virtual function, the write data can be written into the source namespace and the destination namespace simultaneously when the write request is received, and the read request is routed to the source namespace when the read request is received, without suspending the response to the read and write requests, so that the I / O service continuity is not affected, the core pain point of interrupting the I / O service in the related art is solved, and the real hot migration is realized.

[0159] In another embodiment of the present disclosure, after switching the namespace binding of the target virtual function from the source namespace to the destination namespace, the method further comprises: S150, checking the data consistency of the source namespace and the destination namespace; if the data consistency is passed, releasing the resource lock of the source namespace and the destination namespace; recording the migration information of this migration in the migration log; synchronizing the migration control block to the migration control block backup in the non-volatile storage memory and the standby migration control block backup in the migration log.

[0160] Specifically, the data consistency check includes random block check and / or double-write metadata check.

[0161] Specifically, the random block check comprises the following steps: randomly extracting a preset percentage (such as 5%) of the total number of migration blocks as check data blocks from the data synchronization coverage range of the source namespace and the destination namespace, wherein the sampling follows the uniform distribution principle to ensure that a corresponding proportion of data blocks in the sub-data range responsible by each thread are selected, avoiding sampling bias; for each check data block, reading the data block corresponding to the check data block from the source namespace and calculating the sixth check value (such as CRC check) of the read data block, and reading the data block corresponding to the check data block from the destination namespace and calculating the seventh check value (such as CRC check) of the read data block; if the sixth check value and the seventh check value are consistent, the check data block data is consistent; if the sixth check value and the seventh check value are inconsistent, re-synchronize the check data block and check (up to 3 times), and if the check still fails, mark it as an inconsistent block and send a warning log (such as event code 0x06, describing data inconsistency) to the host.

[0162] The specific steps of the double-write metadata verification are as follows: reading the last double-write logical block address in the cache management structure; reading the data corresponding to the last double-write logical block address from the source namespace and calculating the eighth check value of the read data, querying the first check value corresponding to the last double-write logical block address recorded in the write metadata area, and if the eighth check value is consistent with the first check value, it is determined that the double-write data is consistent, otherwise it is determined that the double-write data is inconsistent.

[0163] Specifically, the multiple threads started in the closing S130 are closed, the locks on the source namespace and the destination namespace are released, the migrating label on the destination virtual function is cleared, and the subsequent migration request of the destination namespace is allowed.

[0164] Specifically, the migration information of the current migration is recorded in the migration log, and the migration information includes at least one of the following: a timestamp, an event type, a target virtual function identifier, a source namespace identifier, a destination namespace identifier, a total time consumed from the start to the end of the entire migration task, a total amount of data actually synchronized in the migration process, a migration task result, and whether an error occurs in the migration process.

[0165] Specifically, the migration information has the following format: [timestamp=1620000045000 (i.e., the timestamp), event=COMPLETE (i.e., the event type is that the migration task has ended), vf=3 (i.e., the target virtual function identifier), src_ns=0x01 (i.e., the source namespace identifier), dst_ns=0x02 (i.e., the destination namespace identifier), duration=45000ms (i.e., the total time consumed from the start to the end of the entire migration task), data=180GB (i.e., the total amount of data actually synchronized in the migration process), success=true (i.e., the migration task result is successful completion), and error_count=0 (i.e., no error occurs in the migration process)].

[0166] Specifically, as shown in Figure 2 The migration state in the migration control block can also be set to 0x05 (i.e., the migration ends), the migration control block is synchronized and backed up to the non-volatile storage area and the log namespace, and double backup is performed.

[0167] It can be understood that, by verifying the data consistency and releasing the resources after the verification, the system state consistency after the migration is completed, the resource recycling in time, and the smooth switching of the business can be ensured.

[0168] In another embodiment of the present disclosure, the method further includes: in the case of restarting after abnormal power-off, reading the migration control block backup from the non-volatile storage area;

[0169] Based on the migration status in the migration control block backup, determine the migration interruption location and jump to the step corresponding to the migration interruption location to continue execution.

[0170] Specifically, Figure 3 A flowchart illustrating an example of resuming interrupted downloads provided in this application embodiment. For example... Figure 3 As shown, in the event of an abnormal power-down followed by a restart, after the SSD restarts, it first reads the migration control block backup from the non-volatile storage area to verify the data integrity of the migration control block backup. If the verification fails, it reads the standby migration control block backup stored in the log namespace. Then, it identifies the migration status and selects a recovery path based on the migration status.

[0171] Specifically, such as Figure 3 As shown, if the migration status is 0x01 (i.e., migration initialization), a rollback operation is performed. Specifically, the locked source and destination namespace resources are released, the current migration control block and its backup are deleted, an exception log is recorded (event code 0x07, describing a power outage during migration initialization), and the system waits to receive migration instructions again. If the migration status is 0x02 (i.e., dual-write in progress), a recovery operation is performed. Specifically, the hardware queue of the target virtual function is disabled, the dual-write buffer is reinitialized, the write metadata area is restored (metadata is read from the log namespace), the hardware queue of the target virtual function is restored, the dual-write buffer process continues, the range of logical blocks and the number of data blocks to be synchronized are recalculated, and the subsequent data block migration continues according to the number of migrated blocks completed by each thread recorded in the migration control block. If the migration status is 0x03 (i.e., synchronization completed), S140 and S150 are re-executed. If the migration status is 0x04 (i.e., switchover completed), S150 is re-executed. If the migration status is 0x05 (i.e., migration completed), no action is taken.

[0172] Understandably, after an abnormal power outage, the migration interruption point is identified through the migration status in the persistent migration control block backup, and the process automatically resumes to the corresponding stage to ensure the continuity of the migration process and data consistency.

[0173] In summary, the virtual function migration method provided by the embodiments of the present application has the following advantages: (1) true hot migration without interruption: the host does not need to pause I / O services during the migration process, and the service continuity is not affected, which solves the core pain point of interrupting services in related technologies; high data consistency guarantee: the double-write buffer mechanism ensures synchronization of new data, CRC verification and differential synchronization guarantee the integrity of historical data, avoiding data loss or misplacement during migration, differential synchronization (skipping double-write coverage data blocks) reduces invalid data transmission, and multi-thread concurrent synchronization further improves migration speed, thus guaranteeing data consistency and greatly reducing the amount of synchronized data, balancing data consistency and migration efficiency; seamless address mapping switching: a mechanism of temporarily shielding the hardware queue + preset time read-write request forwarding mechanism is designed to solve the recognition delay problem of the host to the namespace change, and even if the host does not update the namespace binding relationship in time, the read-write request pointing to the source namespace can still be forwarded to the destination namespace; strong device independence: completely realized by software inside the SSD, without relying on external devices such as the host and the network card, reducing complexity and host resource occupation; easy to implement and maintain: the migration logic is reusable, and the subsequent iteration or troubleshooting is difficult; high migration reliability: covering power failure exception handling, supporting breakpoint resuming and data repair, and ensuring migration stability; migration efficiency optimization: excellent compatibility: transparent to the upper virtual machine, the host and the existing driver, without the need to modify the existing architecture for deployment.

[0174] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0175] The embodiments of the present application also provide a virtual function migration device, comprising: a first initialization module configured to perform migration initialization when a migration instruction for a target virtual function is acquired, wherein the migration initialization comprises performing resource verification on a destination namespace;

[0176] A first double-write module is configured to enable a double-write buffer mechanism, wherein the double-write buffer mechanism means that when a write request is received during the migration process, the write data is written into the source namespace and the destination namespace at the same time, and when a read request is received, the read request is routed to the source namespace, the source namespace is a namespace where the target virtual function currently resides, and the destination namespace is a namespace where the target virtual function is to be migrated to;

[0177] A first migration module is configured to migrate historical data in the source namespace to the destination namespace;

[0178] A first switching module is configured to switch the namespace binding of the target virtual function from the source namespace to the destination namespace.

[0179] The features of the embodiments of the virtual function migration device can be understood in conjunction with the features of the embodiments of the virtual function migration method, which will not be repeated here.

[0180] Embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the embodiments of the virtual function migration method described above.

[0181] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the embodiments of the virtual function migration method described above when executed.

[0182] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0183] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the embodiments of the virtual function migration method described above.

[0184] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the embodiments of the virtual function migration method described above.

[0185] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0186] The above describes in detail the virtual function migration method and device provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and the core idea thereof. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method of migrating a virtual function, the method comprising: Comprise: In the case of obtaining the migration instruction for the target virtual function, migration initialization is performed, wherein the migration initialization comprises resource checking on the destination namespace; A double write buffer mechanism is enabled, wherein the double write buffer mechanism refers to that, during the migration process, when a write request is received, the write data is written into the source namespace and the destination namespace at the same time, and when a read request is received, the read request is routed to the source namespace, the source namespace being a namespace where the target virtual function currently locates, and the destination namespace being a namespace where the target virtual function is to be migrated to; Historical data in the source namespace is migrated to the destination namespace; Namespace binding of the target virtual function is switched from the source namespace to the destination namespace; The double write buffer mechanism is enabled, comprising: A double write buffer area is initialized; Hardware queues associated with the target virtual function are recovered; For the write request in the hardware queue, the write request is intercepted, and the write data is written into the source namespace and the destination namespace at the same time by using the double write buffer area; For the read request in the hardware queue, the read request is routed to the source namespace.

2. The virtual function migration method according to claim 1, wherein The migration initialization in the case of obtaining the migration instruction for the target virtual function comprises: Resource checking is performed on the destination namespace; In the case that the destination namespace passes the resource checking, a migration control block is created and initialized; Resource locking is performed on the source namespace and the destination namespace; Migration log is initialized.

3. The virtual function migration method according to claim 2, wherein, The creation and initialization of the migration control block comprise: Storage space is allocated for the migration control block in temporary memory; Key fields of the migration control block are initialized, wherein the key fields of the migration control block comprise at least one of the following: target virtual function identifier, source namespace identifier, destination namespace identifier, migration state, data synchronization byte offset, double write enable flag, migration block size, total migration block number, migrated block number, migration log write offset, reserved field and checking information; A migration control block backup is created in a non-volatile storage area, wherein the migration control block is periodically synchronized to the migration control block backup; The resource locking on the source namespace and the destination namespace comprises: The source namespace is prohibited from being deleted and formatted, and hardware queues associated with the target virtual function are shielded; The destination namespace is prohibited from being deleted and formatted, and the destination namespace is prohibited from being bound with other virtual functions; Waiting for the end of unfinished read and write requests.

4. The virtual function migration method according to claim 1, wherein, The initialization of the double write buffer area comprises: Storage space is allocated for the double write buffer area in temporary memory; A cache management structure corresponding to the double write buffer area is initialized, wherein the cache management structure comprises: a double write data cache area, an index of the earliest unreleased data cache, an index where the next data cache is to be written, a write metadata area, a current index of the metadata cache, and a logical block address of the last successful double write. The intercepting the write request in the hardware queue and writing the write data corresponding to the write request into the source namespace and the destination namespace simultaneously by using the double-write buffer area, comprises: copying the write data to the double-write buffer area and updating the index of the next data buffer to be written; calculating a first check value of the write data; writing the write data in the double-write buffer area into the source namespace, calculating a second check value of the write data written in the source namespace, and writing the write data into the destination namespace, calculating a third check value of the write data written in the destination namespace; if the first check value is consistent with the second check value, the write data is written successfully, and if the first check value is inconsistent with the second check value, the write data is re-written into the source namespace until the write data is written successfully or a first preset re-write upper limit is reached; if the first check value is consistent with the third check value, the write data is written successfully, and if the first check value is inconsistent with the third check value, the write data is re-written into the destination namespace until the write data is written successfully or the first preset re-write upper limit is reached; recording metadata of the write request in the write metadata area; after the write data is written successfully, releasing the write data in the double-write buffer area and updating the index of the earliest unreleased data buffer.

5. The virtual function migration method according to claim 1, wherein, The migration of the historical data in the source namespace to the destination namespace comprises: determining a data range of the historical data and determining a total number of migration blocks according to the data range; migrating data blocks of the total number of migration blocks from the source namespace to the destination namespace in a multi-thread concurrent mode.

6. The virtual function migration method according to claim 5, wherein, The migration of data blocks of the total number of migration blocks from the source namespace to the destination namespace in a multi-thread concurrent mode comprises: dividing the data range of the historical data into a plurality of sub-data ranges to obtain a plurality of sub-data ranges; each thread migrates data blocks in the allocated sub-data range in sequence, wherein the migration steps of each data block are as follows: reading the data block from the source namespace, calculating a fourth check value of the data block, if the data block is not written into the destination namespace by the double-write buffer mechanism, writing the data block into the destination namespace, calculating a fifth check value of the data block written in the destination namespace, if the fourth check value and the fifth check value are consistent, the data block is migrated successfully, and if the fourth check value and the fifth check value are inconsistent, the data block is re-written into the destination namespace until the data block is migrated successfully or a second preset re-write upper limit is reached.

7. The virtual function migration method according to claim 1, wherein, The switching of the namespace binding of the target virtual function from the source namespace to the destination namespace comprises: masking the hardware queue associated with the target virtual function, waiting for the completion of the unfinished read-write request, and stopping the double-write buffer mechanism; unbinding the target virtual function from the source namespace, binding the target virtual function to the destination namespace, and notifying a host of the change in the namespace binding relationship; restoring the hardware queue associated with the target virtual function; for a read-write request in the hardware queue within a preset time length, if the read-write request is directed to the destination namespace, routing the read-write request to the destination namespace, and if the read-write request is directed to the source namespace, forwarding the read-write request to the destination namespace.

8. The virtual function migration method according to claim 2, wherein, After the switching of the namespace binding of the target virtual function from the source namespace to the destination namespace, the method further comprises: checking data consistency of the source namespace and the destination namespace; if the data consistency is passed, releasing resource locks of the source namespace and the destination namespace; recording migration information of the present migration in the migration log; synchronizing the migration control block to a migration control block backup in a non-volatile storage memory and a backup migration control block backup in the migration log; wherein the method further comprises: in the case of restarting after abnormal power-off, reading the migration control block backup from the non-volatile storage area; determining a migration interruption position according to the migration state in the migration control block backup, and jumping to the step corresponding to the migration interruption position for continuous execution.

9. An electronic device, comprising: comprise: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the migration method of the virtual function according to any one of claims 1 to 8.

Images