pre-populated command at a storage device for live migration of data

Abstract

A method for managing data migration operations is provided, including: creating, by a storage device, a read commit queue entry indicating a location of data at a source storage of a storage device to be copied from the source storage to a target storage, the read commit queue entry including a field containing metadata including information for reading the data from the source storage.

Description

TECHNICAL FIELD

[0001] One or more aspects in accordance with one or more embodiments of the present disclosure relate to systems and methods for data storage. BACKGROUND

[0002] This Background section is intended to provide only context and is not construed as an admission that the embodiments or concepts described in this section are prior art.

[0003] In the field of data storage, a live migration operation (or live migration process) can refer to a migration operation or migration process that copies data (e.g., storage data) from one storage device to another storage device (e.g., from a source storage device to a target storage device) when the source storage device can potentially receive user data reads or user data writes from a source server (e.g., the source storage device is live). As used herein, a “live migration” component (e.g., a “live migration server” or a “live migration storage device,” etc.) refers to a component that can be involved in the transfer of data from a source storage device to a target storage device and can have higher / additional privileges (e.g., higher / additional privileges to access data within a system) than other components of the system.

[0004] A data structure can be used to track locations of data within a source storage device that is to be copied to a target storage device. Although the metadata associated with the data structure is relatively small compared to the storage capacity of the storage device, operations to process the data structure can consume a relatively large amount of host processing resources. For example, a host can process metadata of the data structure to create commands (e.g., submission queue entries (SQEs)) for extracting data out of the source storage device for live migration.

[0005] Accordingly, there can be a need for methods, devices, and systems suitable for improving communication of metadata for managing or performing data migration (e.g., live migration). SUMMARY

[0006] Aspects of one or more embodiments of the present disclosure relate to computer storage systems and provide improvements to systems and methods for processing submission queue entries to manage data migration.

[0007] According to one or more embodiments of the present disclosure, a method for managing a data migration operation is provided, including creating, by a storage device, a read submission queue entry indicating a location of data to be copied from a source storage device of the storage device to a target storage device at the source storage device, the read submission queue entry including a field containing metadata, the metadata including information for reading the data from the source storage device.

[0008] The method can further include sending, by the storage device, the read commit queue entry to the host, receiving, by the storage device, a command based on the read commit queue entry, and reading, by the storage device, the data from the source storage based on the command.

[0009] The method can further include reading the data from the source storage based on the read commit queue entry being stored on the storage device.

[0010] The field of the read commit queue entry can be one of a plurality of entries of the read commit queue entry, and the plurality of entries can include information for reading the data from the source storage.

[0011] The metadata can include a data pointer or a command identifier.

[0012] The read commit queue entry can be created using information from a write commit queue entry corresponding to a user data write.

[0013] The method can further include generating a scatter-gather list or bitmap based on the user data writes received in the source storage, wherein the read commit queue entry is created based on the scatter-gather list or bitmap.

[0014] The method can further include copying, by the storage device, the data from the source storage to a cache of the storage device, or causing the data to be stored on the host.

[0015] The method can further include creating, by the storage device, a write commit queue entry for processing by a target live migration server or a target live migration storage device.

[0016] According to one or more other embodiments of the present disclosure, a storage device for managing data migration operations is provided, the storage device configured to create a read commit queue entry indicating a location of data to be copied from a source storage of the storage device to a target storage, the read commit queue entry including a field containing metadata including information for reading the data from the source storage.

[0017] The field of the read commit queue entry can be one of a plurality of entries of the read commit queue entry, and the plurality of entries can include information for reading the data from the source storage.

[0018] The metadata can include a data pointer or a command identifier.

[0019] The read commit queue entry can be created using information from a write commit queue entry corresponding to a user data write.

[0020] The storage device can be configured to generate a scatter-gather list or bitmap based on the user data write received in the source storage and create a read commit queue entry based on the scatter-gather list or bitmap.

[0021] The storage device can be configured to create a write commit queue entry for processing by the target live migration server or the target live migration storage device.

[0022] According to one or more other embodiments of the present disclosure, a system for managing data migration operations is provided, including a host and a storage device, wherein the storage device is configured to create a read commit queue entry indicating a location of data to be copied from a source storage of the storage device to a target storage, the read commit queue entry including a field containing metadata including information for reading the data from the source storage.

[0023] The storage device can be configured to send the read commit queue entry to the host, receive a command based on the read commit queue entry, and read the data from the source storage based on the command.

[0024] The storage device can be configured to read the data from the source storage based on the read commit queue entry stored on the storage device.

[0025] The read commit queue entry can be created using information from a write commit queue entry corresponding to the user data write.

[0026] The storage device can be configured to create a write commit queue entry for processing by the target live migration server or the target live migration storage device. BRIEF DESCRIPTION OF DRAWINGS

[0027] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, wherein the same reference numerals are used to denote similar components throughout the views.

[0028] Figure 1 FIG. 1 is a system diagram depicting a system for managing data migration using a live migration server creating commit queue entries, according to one or more embodiments of the present disclosure.

[0029] Figure 2 FIG. 2 is a system diagram depicting a system for managing data migration using a live migration storage device creating commit queue entries, according to one or more embodiments of the present disclosure.

[0030] Figure 3A FIG. 3 is a diagram depicting a data structure as a bitmap, according to one or more embodiments of the present disclosure.

[0031] Figure 3Bis a diagram depicting a data structure as a scatter-gather list according to one or more embodiments of the present disclosure.

[0032] Figure 3C is a diagram depicting fields of a read submission queue entry according to one or more embodiments of the present disclosure.

[0033] Figure 3D is a diagram depicting fields of a write submission queue entry according to one or more embodiments of the present disclosure.

[0034] Figure 4 is an example data queue according to one or more embodiments of the present disclosure.

[0035] Figure 5 is a flow diagram depicting example operations of a method for managing data migration using live migration storage device creation submission queue entries according to one or more embodiments of the present disclosure.

[0036] Throughout the drawings, like reference numerals will be understood to refer to like elements, features and structures. The technical person will appreciate that the elements in the drawings are shown for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements and areas in the drawings can have been exaggerated relative to other elements and areas to help to improve understanding of various embodiments. Also, common but well-known elements and components have not been described in order to avoid obscuring the disclosure. Furthermore, the description can omit certain features in order to not unnecessarily elongate the description. DETAILED DESCRIPTION

[0037] Aspects of the disclosure and methods of implementing the same can be understood more readily by reference to the following detailed description and the accompanying drawings. In the following description, for the purposes of explanation, numerous specific details of the embodiments are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in order to avoid obscuring the understanding of the present disclosure.

[0038] Throughout the drawings and the written description, the same reference numerals and characters will be understood to refer to like elements, features and structures. The description can omit certain features in order to not unnecessarily elongate the description. In the drawings, the relative sizes of elements and regions can be exaggerated for clarity.

[0039] In the detailed description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of various embodiments. It will be apparent, however, that various embodiments can be practiced without these specific details.

[0040] It will be understood that, although the terms “first,” “second,” “third,” etc. can be used herein to describe various elements, components, regions, and / or sections, these elements, components, regions, and / or sections should not be limited by these terms. These terms are used to distinguish one element, component, region, or section from another element, component, region, or section. Thus, a first element, component, region, or section described below could be termed a second element, component, region, or section without departing from the spirit and scope of the present disclosure.

[0041] It will be understood that when an element or component is referred to as being “on” another element or component, “connected to” another element or component, or “coupled to” another element or component, it can be directly on, connected, or coupled to the other element or component or one or more intervening elements or components can also be present. In contrast, when an element or component is referred to as being “directly on,” “directly connected to,” or “directly coupled to” another element or component, it is to be understood that there are no intervening elements or components present. While the term “directly” is used herein to describe a relationship between elements or components, other terms such as “between,” “directly between,” or “adjacent to” and “directly adjacent to” can be similarly interpreted. In addition, it will also be understood that when an element or component is referred to as being “between” two elements or components, it can be the only element or component between the two elements or components or one or more intervening elements or components can also be present.

[0042] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” “including,” “has,” “have,” “contains” and “containing,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, each of the terms “and / or” and “and / or” includes any and all combinations of one or more of the associated listed items.

[0043] For purposes of this disclosure, expressions such as “at least one of’ when preceding a list of elements, modifies the entire list of elements and does not modify the individual elements of the list. For example, “at least one of X, Y, and Z” and “at least one selected from the group consisting of X, Y, and Z” can be interpreted as X alone, Y alone, Z alone, or any combination of two or more of X, Y, and Z, such as XYZ, XYY, YZ, and ZZ.

[0044] As used herein, the terms “substantially,” “approximately,” “about,” and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in measured or calculated values that would be recognized by those of ordinary skill in the art. As used herein, “about” or “approximately” includes the stated value and means the particular value is within an acceptable range of deviation determined by one of ordinary skill in the art to be acceptable in light of the purpose of a particular measurement and the associated measurement error (i.e., limitations of the measurement system). For example, “about” can mean within one or more standard deviations, or within ±30%, 20%, 10%, 5% of the stated value. Furthermore, the use of “may” when describing embodiments of the disclosure means “one or more embodiments of the disclosure.”

[0045] When one or more embodiments can be practiced differently, a particular process sequence can be performed in an order different from the described order without departing from the scope of the described embodiments. For example, two processes described in succession can be executed substantially concurrently, or in the reverse order as described.

[0046] Any component described or any combination of components described (e.g., in any of the system diagrams included herein) can be used to perform one or more operations of any of the flow diagrams included herein. Moreover, (i) operations are merely examples and various additional operations can be devised that are not explicitly covered, and (ii) the order of the operations can be varied.

[0047] Electronic or electrical devices and / or any other related devices or components according to embodiments of the disclosure described herein can be implemented using any suitable hardware, firmware (e.g., dedicated integrated circuit), software, or combinations of software, firmware, and hardware. For example, various components of the devices can be formed on one integrated circuit (IC) chip or on separate IC chips. Moreover, various components of the devices can be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on one substrate.

[0048] Furthermore, various components of these devices can be processes or threads running on one or more processors in one or more computing devices, executing computer program instructions and interacting with other system components as described herein to perform the various functionalities described herein. The computer program instructions are stored in memory, for example, memory can be embodied in a computing device using standard memory devices such as random access memory (RAM). For example, the computer program instructions can also be stored in other non-transitory computer-readable media such as, for example, a CD-ROM, a flash drive, etc. Also, a person of ordinary skill in the art will recognize that the functionality of various components can be combined or integrated into a single computing device, or the functionality of particular components can be distributed across one or more other computing devices without departing from the spirit and scope of embodiments of the present disclosure.

[0049] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this inventive concept belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and / or the

[0050] As described above, in the field of data storage, a live migration operation or live migration process can refer to an operation or process (which can be referred to as a migration process) of copying data (e.g., storage data) from a source storage device to a target storage device while the source storage device can potentially receive user data reads or user data writes from a source server (e.g., from a host that can be assigned lower supervisory privileges than a hypervisor (or live migration server)) (e.g., while the source storage device is live).

[0051] For example, a data center owner, operator, and / or a software platform operator that sells computing resources can implement a live migration policy in which, when data at a source storage device can be changed (e.g., when data at a source storage device can be updated based on receipt of user data writes), the data can be moved from the source storage device to a new target storage device. The live migration policy can involve moving a namespace (e.g., a set of objects in a storage device) from a source controller / storage device to a target controller / storage device.

[0052] Data structures such as bitmaps and / or scatter-gather lists (e.g., vector lists where each vector gives the location and length of one segment in an overall read or write request) can be used to track locations of data within a source storage device to be copied to a target storage device. For example, an entire source storage drive that can have a large storage capacity (e.g., 8 terabytes (TB)) can have each logical block address / LBA (e.g., 4 kilobytes (KB)) represented by tracking metadata of a significantly smaller size in one or more data structures to indicate whether the corresponding LBA includes data that should be copied from the source storage device to the target storage device.

[0053] As will be discussed in further detail below with reference to the appended drawings, a live migration controller (e.g., a source live migration controller) can assist a live migration server (e.g., a source live migration server) in managing a live migration operation by creating tracking metadata. For example, creating the tracking metadata can include creating a data structure (e.g., a bitmap or a scatter-gather list) as a temporary communication mechanism to identify locations of user data writes written within the source storage device during a time period when a first iteration of an instance of copying data from the source storage device to the target storage device occurs. That is, user data writes received at locations of the source storage device that contain data previously copied from the source storage device to the target storage device can cause the source storage device data to change.

[0054] For example, locations (e.g., LBAs) in the source storage device that correspond to user data writes that cause the source storage device data to change can be referred to as “dirty pages” that contain “dirty data.” The live migration controller can track locations of the dirty pages with the data structure and can send the data structure to the live migration server to copy the dirty data from the source storage device to the target storage device (e.g., update data in the target storage device with current / more recent data in the source storage device) in the next iteration of copying.

[0055] Although the metadata associated with the data structure is relatively small compared to the storage capacity of the storage device, processing the data structure can consume a large amount of host processing resources. For example, the host can process the metadata of the data structure to create commands (e.g., SQEs). The SQEs can include read SQEs associated with read commands to be sent to the live migration storage device to extract data from the source storage device for live migration. The SQEs can also include write SQEs associated with write commands to be sent to the target storage device for writing the extracted data from the source storage device to the target storage device. In processing the data structure to create the SQEs, the live migration server can perform redundant steps of parsing into the data structure and then can exit the data structure.

[0056] A system for managing live migration according to the present disclosure can use a source live migration storage device to create and pre-populate SQEs (e.g., read commands, write commands, etc.), thereby improving overall system performance by removing the workload associated with creating and populating SQEs from the source live migration server and by offloading the workload to the source live migration storage device. By creating and pre-populating one or more fields of the SQEs at the source live migration storage device, the storage system can achieve faster read performance of live migration data, thereby resulting in faster overall data migration.

[0057] In one or more embodiments, the source live migration storage device can read data from the source storage based on receiving, from the source live migration server, SQEs that were created and pre-populated by the source live migration storage device. For example, the source live migration server can (i) complete populating fields of the SQEs that were not populated by the source live migration storage device when creating the SQEs, and / or (ii) overwrite some fields of the SQEs that were pre-populated by the source live migration storage device when creating the SQEs.

[0058] In one or more other embodiments, instead of the source live migration storage device creating SQEs and sending them to the source live migration server and then waiting to receive commands from the source live migration server, the source live migration storage device can have enough information from the SQEs to automatically retrieve data from the source storage (e.g., without needing to wait to receive commands from the source live migration server).

[0059] In one or more embodiments, instead of the source live migration storage device creating a data structure in the form of a bitmap or in the form of a scatter-gather list, the source live migration storage device can use write SQEs (and corresponding user data writes) corresponding to the source server submission queue to generate SQEs. For example, the source live migration storage device can be configured to convert write SQEs corresponding to dirty pages into read SQEs. For example, the source live migration storage device can be configured to extract metadata from one or more fields of the write SQEs to pre-populate one or more fields of the corresponding read SQEs.

[0060] In one or more embodiments, the source live migration storage device can store dirty data copied from the source storage in a cache of the source live migration storage device. For example, the source live migration storage device can create a data queue on the storage device, and can later (e.g., asynchronously) send the data queue to the source live migration server.

[0061] In one or more embodiments, the source live migration storage device can store dirty data copied from the source storage to a buffer on the source live migration server. For example, the source live migration storage device can create a data queue on the source live migration server. In one or more embodiments, the location (or locations) of the data queue can be used (e.g., identified) in both the read SQEs from the source and the write SQEs to the target. If the write SQEs are filled (e.g., pre-filled), the target live migration server can establish a matching memory address range for copying the data queue between the source live migration server and the target live migration server.

[0062] In one or more embodiments, the source live migration storage device can also create write SQEs for the target live migration server or the target live migration storage device to process when writing data to the target storage.

[0063] Figure 1 is a system diagram depicting a system 1 for managing data migration using source live migration server 100b creating submission queue entries (SQEs) in accordance with one or more embodiments of the present disclosure.

[0064] Referring to Figure 1 , the system 1 can include a source host system 100S and a target host system 100T. The source host system 100S can be connected to a source storage 200. The target host system 100T can be connected to a target storage 300. The source host system 100S and the target host system 100T can be associated with, or can include, a central processing unit (CPU). The source storage 200 and the target storage 300 can include logical block addresses (LBAs) (e.g., LBA0 to LBA 15, as depicted within the source storage 200 as a simplified example in Figure 1 ). The LBAs can be associated with physical locations in the source storage 200 and the target storage 300 for storing data (e.g., user data).

[0065] Target host system 100T can include a target server 100c (e.g., a target virtual machine (VM)). Target server 100c can be connected to target storage device 250c via communication link 20. Communication link 20 can include or be implemented by various technologies (e.g., peripheral component interconnect express (PCIe), PCIe-based non-volatile memory express (NVMe), NVMe over Fabrics (NVMe-oF), etc.).

[0066] Target storage device 250c can include a target storage controller 210c and a target controller memory 220c. Target controller memory 220c can include or be a RAM (e.g., static random-access memory (SRAM) or dynamic random-access memory (DRAM)). Target storage controller 210c can include or be implemented by embedded logic. The embedded logic can be embedded within target storage device 250c. The embedded logic can enable target storage controller 210c to process requests from a server (e.g., target server 100c) to copy / write data to target storage 300.

[0067] Target storage device 250c can include target storage 300 (or a portion of target storage 300). For example, target storage 300 can include target storage 330. Target storage 330 can include or be long-term storage. For example, target storage 330 can include non-volatile storage and / or a storage tier (including volatile and non-volatile storage), and can correspond to long-term storage devices of one or more target storage devices 250c. For example, target storage 300 can refer to long-term storage that is made up of one or more target storage devices 250c of a distributed storage system (e.g., a virtualized distributed storage system). Target storage device 250c can be a solid-state drive (SSD) that includes one or more underlying target storage devices 250c (which can be virtual or physical). Target storage device 250c can be connected to target storage 300 via storage interface 22. Storage interface 22 can include or be a flash interface (e.g., a NAND flash interface) or a remote interface (e.g., an interface implemented over Ethernet).

[0068] Source host system 100S can include source server 100a. Source server 100a can be a host or can be a source VM. Source server 100a can service read and write requests from a user application. For example, a user can use an application (e.g., a word processing application) to send a user data write UDW (e.g., a user application data write request) or a user data read UDR (e.g., a user application data read request) to source storage 200. The read / write request can be sent to source storage 200 via source storage device 250a. Write requests from source server 100a can be queued at source server 100a before being sent to source storage 200 to include one or more write submission queue entries WSQEs in a source server submission queue SSSQ. Similarly, in one or more embodiments, read requests from source server 100a can be queued at source server 100a before being sent to source storage 200 to include one or more read submission queue entries RSQEs in a source server submission queue SSSQ.

[0069] Source server 100a can be connected to source storage device 250a via communication link 20. Communication link 20 can include or be implemented by various technologies (e.g., PCIe, PCIe-based NVMe, NVMe-oF, etc.).

[0070] Source storage device 250a can include source storage controller 210a and source controller memory 220a. Source controller memory 220a can include or be a RAM (e.g., SRAM or DRAM). Source storage controller 210a can include or be implemented by embedded logic (e.g., software or firmware embedded within source storage device 250a) for processing requests from a server (e.g., source server 100a) to replicate / write data to source storage 200. Source storage device 250a can include source storage 200 or a portion thereof.

[0071] For example, source storage 200 can include source memory 230. Source memory 230 can include or be long-term memory. For example, source memory 230 can include non-volatile memory and / or a memory tier (including volatile and non-volatile memory). Source memory 230 can correspond to long-term memory devices of one or more source storage devices 250a. For example, source storage 200 can refer to long-term memory that is made up of one or more source storage devices 250a of a distributed storage system, such as a virtualized distributed storage system. Source storage devices 250a can be solid state drives (SSDs) that include one or more underlying source storage devices 250a. The underlying source storage devices can be virtual or physical. Source storage devices 250a can be connected to source storage 200 via storage interface 22. Storage interface 22 can include or be a flash interface (e.g., a NAND flash interface) or a remote interface (which can be implemented over Ethernet).

[0072] Source host system 100S can include source live migration server 100b. Source live migration server 100b can be a server that is assigned higher / additional privileges compared to source server 100a or target server 100c. The privileges can correspond to data access within system 1. Source live migration server 100b can be assigned to manage live migration operations (e.g., live migration processes) for copying data from source storage 200 to target storage 300. Source live migration server 100b can include source live migration server memory 120b.

[0073] Source live migration server memory 120b can include or be RAM (e.g., SRAM or DRAM). Source live migration server memory 120b can be used to store metadata for managing (e.g., monitoring and / or directing) live migration operations. Source live migration server 100b can be connected to source live migration storage device 250b via communication link 20. Communication link 20 can include or be implemented by various technologies (e.g., PCIe, PCIe-based NVMe, NVMe-oF, etc.).

[0074] Source live migration storage device 250b can be a storage device that is assigned higher / additional privileges compared to source storage device 250a or target storage device 250c. The assigned privileges can correspond to data access within system 1. Source live migration storage device 250b can be assigned to assist live migration operations by reading data from source storage 200.

[0075] Source live migration storage device 250b can include source live migration controller memory 220b and source live migration controller 210b.

[0076] The source live migration controller memory 220b can include or be a RAM (e.g., SRAM or DRAM). The source live migration controller memory 220b can be used to store metadata for managing (e.g., monitoring and / or directing) a live migration operation. The source live migration controller memory 220b can also include a live migration controller cache 222b for storing (e.g., temporarily storing) data from the source storage 200.

[0077] The source live migration controller 210b can include or be implemented by embedded logic that can be part of the source live migration storage device 250b for assisting the source live migration server 100b in copying data from the source storage 200 to the target storage 300.

[0078] The source live migration storage device 250b can be connected to the source storage 200 via a storage interface 22. The storage interface 22 can include or be a flash interface (e.g., a NAND flash interface) or a remote interface (e.g., implemented over Ethernet).

[0079] The source live migration server 100b can be connected or linked to the target host system 100T via a“live migration server to target” link 10. Depending on whether the system 1 is implemented via hardware (HW), software (SW), or a combination of HW and SW, the“live migration server to target” link 10 can be implemented by various technologies.

[0080] In addition to the target server 100c described above, the target host system 100T can include a target live migration server 100d. The target live migration server 100d can be a server that is assigned higher / additional privileges compared to the source server 100a or the target server 100c. The privileges can correspond to data access within the system 1. The target live migration server 100d can be assigned to assist the source live migration server 100b in managing a live migration operation. The target live migration server 100d can include a target live migration server memory 120d.

[0081] The target live migration server memory 120d can include or be a RAM (e.g., SRAM or DRAM). The target live migration server memory 120d can be used to store metadata for managing (e.g., monitoring and / or directing) a live migration operation. The target live migration server 100d can be connected to a target live migration storage device 250d via a communication link 20. The communication link 20 can include or be implemented by various technologies (e.g., PCIe, PCIe-based NVMe, NVMe-oF, etc.).

[0082] Target live migration storage device 250d can be a storage device assigned higher / additional privileges compared to source storage device 250a or target storage device 250c. The assigned privileges can correspond to data access within system 1. Target live migration storage device 250d can be assigned to assist in a live migration operation by writing data (e.g., data originating from source storage 200) to target storage 300.

[0083] Target live migration storage device 250d can include target live migration controller memory 220d. Target live migration controller memory 220d can include or be a RAM (e.g., SRAM or DRAM). Target live migration controller memory 220d can be used to store metadata for managing (e.g., monitoring and / or directing) a live migration operation.

[0084] Target live migration storage device 250d can also include target live migration controller 210d. Target live migration controller 210d can include or be implemented by embedded logic that can be part of target live migration storage device 250d for assisting target live migration server 100d in writing data to target storage 300. Target live migration storage device 250d can be connected to target storage 300 via storage interface 22. Storage interface 22 can include or be a flash interface (e.g., a NAND flash interface) or a remote interface (e.g., implemented over Ethernet).

[0085] Servers and storage devices of system 1 can be implemented via HW, SW, or a combination of HW and SW. For example, in some embodiments, "live migration server to target" link 10 can be a physical network connection (e.g., Ethernet). In some embodiments, one or more servers (e.g., source server 100a, source live migration server 100b, target live migration server 100d, or target server 100c) can be a software entity. For example, one or more servers can be a virtual machine (VM) managed by a hypervisor associated with one or more central processing units (CPUs).

[0086] Likewise, one or more of storage devices 250a, 250b, 250c, 250d and / or source storage 200 and / or target storage 300 can be virtualized and implemented by HW and / or SW technology. For example, one or more of storage devices 250a, 250b, 250c, 250d and / or source storage 200 and / or target storage 300 can be provided by any combination of physical storage devices. In one or more embodiments, as Figure 1The source storage device 250a, the source live migration storage device 250b, and the source storage appliance 200 can be components of a source SSD, as indicated by the dashed lines. Similarly, the target storage device 250c, the target live migration storage device 250d, and the target storage appliance 300 can be components of a target SSD. In one or more other embodiments, the migration can be from a first virtual machine to a second virtual machine, noting that both virtual machines can be supported by one physical storage device. In one or more embodiments, the source live migration server 100b and the target live migration server 100d can be the same. For example, within the same physical storage device, the migration can be from a source live migration controller to a target live migration controller, or from a source namespace to a target namespace. It should be appreciated that various other combinations, for example, between physical devices and virtual devices, can be implemented without departing from the spirit and scope of the present disclosure.

[0087] The source live migration server 100b and the source live migration storage device 250b, along with the target live migration server 100d and the target live migration storage device 250d, can coordinate management of the live migration process by communicating metadata indicative of a state of data corresponding to LBAs in the source storage appliance 200.

[0088] For example, during the live migration process, during a first copy iteration, a region of the source storage appliance can be copied from the source storage appliance 200 to the target storage appliance 300. The region of the source storage appliance 200 can correspond to one or more storage locations (e.g., LBAs or “pages”) having data to be copied during the first copy iteration. For example, during the first copy iteration, the storage locations can be “map pages” containing “map data” to be copied from the source storage appliance 200 to the target storage appliance 300. The region of the source storage appliance 200 can receive a user data write, UDW, which can cause a change to data at one or more storage locations (e.g., LBAs or “pages”) that have already been copied from the source storage appliance 200 to the target storage appliance 300 for live migration. Such storage locations can be referred to as “dirty pages” containing “dirty data,” as described above. In a second copy iteration, the dirty data can be copied from the source storage appliance 200 to the target storage appliance 300 to keep the target storage appliance 300 up to date.

[0089] Still referring to Figure 1During the first replication iteration, source storage 200 can receive user data writes UDW to the following LBAs in the following order: LBA 0 at a first time (depicted as UDW 1); LBAs 8-10 at a second time (depicted as UDW 2); LBAs 14 and 15 at a third time (depicted as UDW 3); LBA 2 at a fourth time (depicted as UDW 4); and LBAs 7-9 at a fifth time (depicted as UDW 5, and depicted as UDW 2,5 for overlapping user data writes to LBAs 8 and 9 at the second and fifth times).

[0090] To track dirty pages (e.g., LBAs 0, 2, 7, 8-10, 14, and 15) to be replicated during the second replication iteration of the live migration process, live migration storage 250b can use a data structure DS in a first format to track dirty page locations (operation 1001). For example, data structure DS can be provided in the form of a bitmap (see FIG. 10A) or in the form of a scatter-gather list (see FIG. 10B). Information from data structure DS can be used to generate read submission queue entries RSQEs to enable system 1 to read data from source storage 200. Information from data structure DS can also be used to generate write submission queue entries WSQEs to enable system 1 to write data to target storage 300. Data structure DS can indicate locations of mapped pages during the first replication iteration of the live migration process. Data structure DS can indicate locations of dirty pages during the second or later replication iteration of the live migration process. Figure 3A ) or in the form of a scatter-gather list (see Figure 3B ). Information from data structure DS can be used to generate read submission queue entries RSQEs to enable system 1 to read data from source storage 200. Information from data structure DS can also be used to generate write submission queue entries WSQEs to enable system 1 to write data to target storage 300. Data structure DS can indicate locations of mapped pages during the first replication iteration of the live migration process. Data structure DS can indicate locations of dirty pages during the second or later replication iteration of the live migration process.

[0091] In some systems 1, source live migration storage 250b can send data structure DS to source live migration server 100b to resolve data structure DS. Source live migration server 100b can create corresponding read submission queue entries RSQEs (operation 1002). In some systems 1, source live migration server 100b can also create a write submission queue WSQ including write submission queue entries WSQEs to instruct target live migration server 100d and target live migration storage 250d how to write data from source storage 200 to target storage 300.

[0092] Read submission queue entries RSQEs can be sent from source live migration server 100b to source live migration storage 250b (operation 1003). Read submission queue entries RSQEs can include various fields with metadata (see FIG. 10C). Figure 3C)(operation 1004-1007), the metadata includes information to instruct the source live migrate storage device 250b how to read data from the source storage 200 to copy the data from the source storage 200 to the target storage 300. For example, depending on the replication iteration (operation 1004) from the source storage 200 to the source live migrate storage device 250b, the source live migrate storage device 250b can use a read commit queue entry RSQE to read dirty data or map data.

[0093] In some embodiments, the source live migrate storage device 250b can create a data queue DQ at the source live migrate storage device 250b (operation 1004). The data queue DQ can be stored (e.g., temporarily stored) in the source live migrate storage device cache 222b (operation 1004). The source live migrate server 100b can receive the data queue DQ from the source live migrate storage device 250b (operation 1005). The source live migrate server 100b can send the data queue DQ and a write commit queue WSQ to the target live migrate server 100d (operation 1006).

[0094] The target live migrate server 100d can send the write commit queue entry WSQE to the target live migrate storage device 250d for use in writing data to the target storage 300 (operation 1007). The write commit queue entry WSQE can include various fields (see Figure 3D ) with metadata that includes information to instruct the target live migrate storage device 250d how to write data from the source storage 200 to the target storage 300. The fields of the read commit queue entry RSQE can largely parallel (e.g., compare Figure 3C to Figure 3D ).

[0095] Figure 2 is a system diagram depicting a system for managing data migration using live migrate storage device creation of commit queue entries, in accordance with one or more embodiments of the present disclosure.

[0096] Referring to Figure 2 , in one or more embodiments, the source live migrate storage device 250b can create the SQEs (e.g., RSQE and / or WSQE) instead of the source live migrate server 100b creating the SQEs (operation 2001). Additionally, the live migrate storage device 250b can pre-populate one or more fields of the SQEs (e.g., can generate metadata therein) to remove the associated processing burden from the source live migrate server 100b and to speed up the live migration process.

[0097] In one or more embodiments, the source live migration storage device 250b can create a read commit queue entry RSQE, can pre-populate some or all fields of the read commit queue entry RSQE, and can send the read commit queue entry RSQE to the source live migration server 100b. The source live migration server 100b can: (i) complete populating the fields of the read commit queue entry RSQE, and (ii) add the read commit queue entry RSQE to the read commit queue RSQ (operation 2002). The source live migration server 100b can send a read command to the source live migration storage device 250b via the read commit queue entry RSQE (operation 2003). The source live migration storage device 250b can use the read commit queue entry RSQE to read corresponding data from the source storage 200 (operation 2004).

[0098] Alternatively, in one or more embodiments, the source live migration storage device 250b can read data from the source storage 200 based on a read commit queue entry RSQE created by the source live migration storage device 250b (operation 2003). For example, instead of sending a read commit queue entry RSQE to the source live migration server 100b and then waiting to receive a command from the source live migration server 100b (operations 2001, 2002, 2003, and 2004), the source live migration storage device 250b can have sufficient information from the read commit queue entry RSQE created by the source live migration storage device 250b to automatically retrieve data from the source storage 200 (operations 2001, 2003, and 2004).

[0099] In one or more embodiments, instead of the source live migration storage device 250b creating a data structure DS in bitmap form or in a scattered gather list form, the source live migration storage device 250b can use a write commit queue entry WSQE (and corresponding user data write UDW) corresponding to the source server commit queue SSSQ to generate a read commit queue entry RSQE. For example, the source live migration storage device 250b can be configured to convert a write commit queue entry WSQE corresponding to a dirty page into a read commit queue entry RSQE. In other words, the source live migration storage device 250b can extract metadata from one or more fields of a write commit queue entry WSQE from the source server 100a and create a read commit queue entry RSQE based on the extracted metadata.

[0100] In one or more embodiments, the source live migration storage device 250b can store the dirty data or the mapping data copied from the source storage 200 to a buffer on the source live migration server 100b. For example, the source live migration storage device 250b can send the data queue DQ to the buffer on the source live migration server 100b.

[0101] In one or more embodiments, the source live migration storage device 250b can store the dirty data or the mapping data copied from the source storage 200 to a buffer on the source live migration server 100b. For example, the source live migration storage device 250b can send the data queue DQ to the buffer on the source live migration server 100b.

[0102] In one or more embodiments, the source live migration storage device 250b can also create the write commit queue entries WSQEs for the target live migration server 100d or the target live migration storage device 250d to process when writing the data originated from the source storage 200 to the target storage 300 (operations 2004-2007).

[0103] Figure 3A is a diagram depicting a data structure as a bitmap, according to one or more embodiments of the present disclosure.

[0104] Referring to Figure 3A , Figure 1 and Figure 2 The data structure DS can be created in the form of a bitmap, where each LBA corresponds to a bit position in the bitmap. Depending on the iteration of the copy, the dirty pages or the mapping pages can be identified by a “1” bit. For example, for the dirty pages, the above-mentioned user data write UDW can be depicted by 1s corresponding to LBAs 0, 2, 7-10, 14, and 15. For the mapping pages, the mapping pages can be depicted by 1s corresponding to LBAs 0, 2, 7-10, 14, and 15. In one or more embodiments, based on this simple example, the source live migration storage device 250b can create four read commit queue entries RSQEs. For example: (i) a first read commit queue entry RSQE1 corresponding to LBA 0 can include a starting LBA (SLBA) field entry 0 (e.g., SLBA = 0) and a number of LBAs (NLBA) field entry 0 (e.g., NLBA = 0) (see Figure 3C and 3D(ii) a second read submission queue entry RSQE2 corresponding to LBAs 2-5 can include SLBA = 2 and NLBA = 3; (iii) a third read submission queue entry RSQE3 corresponding to LBAs 7-10 can include SLBA = 7 and NLBA = 3; and (iv) a fourth read submission queue entry RSQE4 corresponding to LBAs 14 and 15 can include SLBA = 14 and NLBA = 1.

[0105] Accordingly, the source live migration storage device 250b can be configured to convert the data structure DS as a bitmap into one or more pre-populated SQEs (e.g., one or more read submission queue entries (RSQEs) or one or more write submission queue entries (WSQEs)). In one or more embodiments, the SQEs can be provided to the source live migration server 100b in a data buffer in a manner similar to how read data or bitmap data can be returned to the source live migration server 100b. Optionally, the SQEs can be inserted into a submission queue selected by the source live migration server 100b.

[0106] Figure 3B is a diagram depicting a data structure as a scatter-gather list in accordance with one or more embodiments of the present disclosure.

[0107] Referring to Figure 3B , Figure 1 and Figure 2 The data structure DS can be created in the form of a scatter-gather list, where each user data write UDW or mapped page is indicated as an entry in the list (e.g., in a log) depending on the iteration of the copy. For example, the scatter-gather list can include five log entries corresponding to user data writes UDW or the above-mentioned mapped pages corresponding to LBAs 0, 2, 7-10, 14, and 15. As described above, for dirty pages, the user data writes UDW can occur in the following order: LBA 0 at a first time (depicted as UDW 1); LBAs 8-10 at a second time (depicted as UDW 2); LBAs 14 and 15 at a third time; LBA 2 at a fourth time; and LBAs 7-9 at a fifth time.

[0108] In one or more embodiments, based on this simple example, the source live migration storage device 250b can create five read submission queue entries RSQEs. For example: (i) a first read submission queue entry RSQE1 corresponding to LBAs 0 can include a start LBA (SLBA) field entry 0 (e.g., SLBA = 0) and a LBA number (NLBA) field entry 0 (e.g., NLBA = 0) (see Figure 3C and Figure 3D ); (ii) a second read submission queue entry RSQE2 corresponding to LBAs 8-10 can include SLBA = 8 and NLBA = 2; (iii) a third read submission queue entry RSQE3 corresponding to LBAs 14 and 15 can include SLBA = 14 and NLBA = 1; (iv) a fourth read submission queue entry RSQE4 corresponding to LBA 2 can include SLBA = 2 and NLBA = 0; and (iv) a fifth read submission queue entry RSQE5 corresponding to LBAs 7-9 can include SLBA = 7 and NLBA = 2.

[0109] Accordingly, the source live migration storage device 250b can be configured to convert the data structure DS as a scatter-gather list into one or more pre-populated SQEs (e.g., one or more read submission queue entries (RSQEs) or one or more write submission queue entries (WSQEs)).

[0110] In one or more embodiments, if overlap detection is enabled, the source live migration storage device 250b can be configured to combine the second read submission queue entry RSQE2 and the fifth read submission queue entry RSQE5. For example, if overlap detection is enabled, the source live migration storage device 250b can be configured to create a read submission queue entry including SLBA = 7 and NLBA = 3 corresponding to LBAs 7-10.

[0111] In one or more embodiments, a submit queue entry (SQE) can be provided to the source live migration server 100b in a data buffer, like command data. Alternatively, in one or more embodiments, the submit queue entry (SQE) can be placed in a log, instead of a log entry. Alternatively, in one or more embodiments, the submit queue entry (SQE) can be inserted into a submit queue selected by the source live migration server 100b. For example, the source live migration server 100b can select a submit queue designated specifically for such submit queue entries. For example, the source live migration server 100b can select a read submit queue for read SQEs and a write submit queue for write SQEs, such that read SQEs and write SQEs are organized in separate submit queues. In one or more embodiments, the source live migration server 100b can select to inspect and / or alter the submit queue entry. In one or more embodiments, the source live migration server 100b can select when to submit a doorbell update (e.g., a notification to the source live migration storage device 250b that indicates that a new SQE has been added to the submit queue for processing).

[0112] Figure 3C FIG. 2B is a diagram depicting fields of a read submit queue entry, in accordance with one or more embodiments of the present disclosure, Figure 3D FIG. 2C is a diagram depicting fields of a write submit queue entry, in accordance with one or more embodiments of the present disclosure.

[0113] Referring to Figure 3C and Figure 3D As described above, the source live migration storage device 250b can construct (e.g., create) the SQE for a read command (or a write command) in whole or in part for the source live migration server 100b. The SQE can be formatted according to the NVMe protocol. The opcode (OpCode) can be 02h for a read and 01h for a write. Thus, the OpCode field can be pre-populated based on the type of command. If the source live migration server 100b provides buffer space, the source live migration storage device 250b can fill in the metadata pointer (MD) field, the physical region page entry 1 (PRP1) field or the scatter gather list entry 1 (SGL1) field, and the physical region page entry 2 (PRP2) field or the scatter gather list entry 2 (SGL2) field data pointers. These pre-population operations performed by the source live migration storage device 250b can be performed for read, write, uncorrectable write, write zero, and copy commands. In other words, the pre-population can not be limited to just read and write.

[0114] In one or more embodiments, the following fields, among other unlisted fields, can be set to "off" or, alternatively, the source live migration server 100b can initialize the preferences for these fields: Force Unit Access (FUA); Limited Retry (LR); Fusion; and Dataset Management.

[0115] In one or more embodiments, the following fields, among other unlisted fields, can be copied from the write command or the host can initialize the preferences for these fields: Namespace Identifier (NSID); PRP / SGL; Expected Logical Block Storage Tag (ELBST) / Expected Initial Logical Block Reference Tag (EILBRT); Expected Logical Block Application Tag (ELBAT); Expected Logical Block Application Tag Mask (ELBATM); Storage Tag Check (STC); and Protection Information Field (PRINFO).

[0116] While examples of specific fields are discussed above, the present disclosure is not so limited. For example, different combinations of fields, including the above-mentioned fields and / or other fields, can be set to "off," initialized to a preference, or copied from the write command.

[0117] In one or more embodiments, the Command Identifier (Cmd ID) field can be left to the source live migration server 100b to fill. Alternatively, the source live migration server 100b can assign a pool of command identifiers for use by the source live migration storage device 250b.

[0118] Figure 4 is an example data queue in accordance with one or more embodiments of the present disclosure.

[0119] Referring to Figure 4 In one or more embodiments, the following fields, among other unlisted fields, can be set to "off" or, alternatively, the source live migration server 100b can initialize the preferences for these fields: Force Unit Access (FUA); Limited Retry (LR); Fusion; and Dataset Management. Figure 3C and Figure 3DThe SQE fields referred to as PRP1, PRP2, SGL1, SGL2, and MD can be left for the source live migration server 100b to fill. Alternatively, the source live migration server 100b can allocate a data queue DQ in the source live migration server memory 120b. In one or more embodiments, the source live migration storage device 250b can allocate memory space in the data queue DQ. Data for read operations of the live migration process can be placed in the data queue DQ, and metadata describing where to find that data can be provided in the PRP, SGL, and MD fields of the read SQEs for the read. If the data for the read is larger than can be described in the read SQEs, the source live migration storage device 250b (see Figure 2 ) can create extended PRP and SGL structures (e.g., additional PRP and SGL structures). The PRP and SGL regions of the read SQEs for the read can be filled with metadata pointing to the extended PRP and SGL regions on the data queue DQ. Thus, the extended PRP and SGL regions and the relevant data of all PRP and SGL regions can be allocated from the data queue DQ region allocated by the source live migration server 100b. Figure 4 One implementation method is depicted in which the source live migration storage device 250b manages the data queue DQ region in a circular first-in first-out (FIFO) manner. As more existing memory space is used, the source live migration storage device 250b can place (e.g., allocate) more memory space at the end of the data queue DQ. When a region of the data queue DQ is no longer used, the source live migration server 100b can be responsible for communicating with the source live migration storage device 250b. For example, the source live migration server 100b can copy the data queue DQ from the source live migration server 100b to the target live migration server 100d before releasing the data queue DQ back to the source live migration storage device 250b. For example, a first region 406 of the data queue DQ can include PRP and MD that are filled and used by the next SQE to be sent to the target live migration server 100d; a second region 408 of the data queue DQ can be used by the source live migration server 100b for copying to the target live migration server 100d; and a third region 410 corresponding to the advanced portion 404 of the data queue DQ and the updated buffer release portion 402 of the data queue 402 can be identified as free space to be allocated by the source live migration storage device 250b to the end of the data queue DQ. In one or more embodiments, the source live migration server 100b can return memory space on the data queue DQ to the source live migration storage device 250b through a PCIe register write or management command.

[0120] Similarly, a command identifier consumed by the source live migration storage device 250b can be reused by the source live migration storage device 250b for a new SQE formation. For example, the source live migration storage device 250b can receive a command identifier corresponding to a read SQE from the source live migration server 100b. The live migration storage device 250b can read data from the source storage 200 identified by the read SQE. The live migration storage device 250b can complete the read SQE by writing the data corresponding to the read SQE into a data queue DQ, and form a write SQE using the command identifier corresponding to the read SQE. Accordingly, the completion time of fulfilling the read command and forming the corresponding write SQE can be reduced (e.g., the completion time can be zero) by the live migration storage device 250b reusing the command identifier to create the write SQE.

[0121] In one or more embodiments, one or more memory locations of the source live migration server memory 120b (see Figure 2 ) corresponding to the various queues described above can be contiguous. For example, the read commit queue RSQ can include RSQEs arranged in contiguous memory locations; the write commit queue WSQ can include WSQEs arranged in contiguous memory locations; and / or the data queue DQ can include data queue entries (DQEs) arranged in contiguous memory locations. Alternatively, in one or more embodiments, one or more memory locations of the source live migration server memory 120b (see Figure 2 ) corresponding to the various queues (e.g., RSQ, WSQ, and DQ) described above can be non-contiguous (e.g., can be separated by intervening memory locations). For example, instead of allocating 20 memory locations in a contiguous range of 33-52, 20 memory locations can be allocated in non-contiguous ranges of 20-29 and 34-44. The 20 memory locations can also be arranged out of order. For example, 10 memory locations 35-44 can be arranged before the remaining 10 memory locations 20-29. In such embodiments, the source live migration server 100b can communicate which data queue DQ regions have completed by sending completion communications to the source live migration storage device 250b. The live migration storage device 250b can track gaps between data queue DQ regions based on the completion communications.

[0122] Figure 5 is a flowchart that describes example operations for a method for managing data migration using live migration storage devices to create commit queue entries, in accordance with one or more embodiments of the present disclosure.

[0123] Referring to Figure 5 , the method 5000 can include one or more of the following operations. The source live migration storage device 250b (seeFigure 2 ) can create a read commit queue entry RSQE based on the data structure DS, which is a bitmap, a scatter-gather list, or a write commit queue entry, and indicates a location (e.g., LBA) of data at the source storage 200 to be copied to the target storage 300 from the source storage 200 (operation 5001). The source live migration storage 250b can create (e.g., generate) metadata in a field of the read commit queue entry, which includes information for reading the data from the source storage (operation 5002). The source live migration storage 250b can copy the data from the source storage 200 to the cache 222b of the source live migration storage 250b, or can cause the data to be stored on the source live migration server 100b (e.g., host) (operation 5003). The source live migration storage 250b can create a write commit queue entry WSQE for processing by the target live migration server 100d or the target live migration storage 250d (operation 5004). It should be understood that depending on the iteration of the copy, the data to be copied from the source storage 200 to the target storage 300 can be mapped data or dirty data. For example, mapped data would be copied in the first (or initial) copy iteration during the live migration process, while dirty data would be copied in the second (or each subsequent) copy iteration during the live migration process.

[0124] As used herein, when a “data structure” is described as “indicating” a location of data, it can mean that an element of the data structure (e.g., a bit of a bitmap in the case that the data structure is or includes a bitmap) points to or indicates the location of the data.

[0125] Accordingly, embodiments of the present disclosure provide improvements and advantages for communication of metadata for managing (e.g., performing) data migration (e.g., live migration). By pre-populating commands, a live migration process can be accelerated by performing tasks at a storage device (e.g., live migration storage device) that would otherwise be performed by a host (e.g., live migration server).

[0126] Example embodiments of the present disclosure can extend to the following statements, without being limited thereto:

[0127] Statement 1: An example method includes creating, by a storage device, a read commit queue entry that indicates a location of data at a source storage of the storage device to be copied to a target storage from the source storage, the read commit queue entry including a field that contains metadata that includes information for reading the data from the source storage.

[0128] Statement 2: An example method includes the method of statement 1, further comprising sending, by the storage device, a read commit queue entry to the host, receiving, by the storage device, a command based on the read commit queue entry, and reading, by the storage device, data from the source storage based on the command.

[0129] Statement 3: An example method includes the method of any of statements 1 and 2, further comprising reading data from the source storage based on the read commit queue entry being stored on the storage device.

[0130] Statement 4: An example method includes the method of any of statements 1-3, wherein the read commit queue entry is one of a plurality of entries of a read commit queue entry, and the entry includes information for reading data from the source storage.

[0131] Statement 5: An example method includes the method of any of statements 1-4, wherein the metadata includes a data pointer or a command identifier.

[0132] Statement 6: An example method includes the method of any of statements 1-5, wherein the read commit queue entry is created using information from a write commit queue entry corresponding to a user data write.

[0133] Statement 7: An example method includes the method of any of statements 1-5, further comprising generating a scatter gather list or bitmap based on user data writes received in the source storage, wherein the read commit queue entry is created based on the scatter gather list or bitmap.

[0134] Statement 8: An example method includes the method of any of statements 1-7, further comprising copying, by the storage device, data from the source storage to a cache of the storage device, or causing the data to be stored on the host.

[0135] Statement 9: An example method includes the method of any of statements 1-8, further comprising creating, by the storage device, a write commit queue entry for processing by a target live migration server or a target live migration storage device.

[0136] Statement 10: An example device for performing the method of any of statements 1-9 includes a controller and a memory.

[0137] Statement 11: An example system for performing the method of any of statements 1-9 includes a host and a storage device.

[0138] While embodiments of the disclosure have been particularly shown and described with reference to the embodiments herein, it will be understood by those of ordinary skill in the art that various changes in form and details can be made therein without departing from the spirit and scope of the disclosure as set forth in the following claims and their equivalents.

Claims

1. A method for managing data migration operations, the method comprising: based on determining that a location of data at a source storage device is updated due to a data write, creating, by a storage appliance, a read commit queue entry indicating a location of the data at the source storage device to be copied from a source storage device of the storage appliance to a target storage device, the read commit queue entry comprising: a first field comprising first information for reading the data from the source storage device, the first information originating from the storage appliance; and a second field comprising second information for reading the data from the source storage device.

2. The method of claim 1, further comprising: sending, by the storage appliance, the read commit queue entry to a host; receiving, by the storage appliance, a command based on the read commit queue entry; and reading, by the storage appliance, the data from the source storage device based on the command.

3. The method of claim 1, further comprising reading the data from the source storage device based on the read commit queue entry being stored on the storage appliance. the first information comprises a data pointer or a command identifier. the read commit queue entry is created using information from a write commit queue entry corresponding to the data write.

4. The method of claim 1, wherein, 6. The method of claim 1, further comprising generating a scatter gather list or a bitmap based on data writes, 5. The method of claim 1, wherein, the read commit queue entry is created based on the scatter gather list or the bitmap.

7. The method of claim 1, further comprising: wherein copying, by the storage appliance, the data from the source storage device into a cache of the storage appliance; or causing the data to be stored on a host.

8. The method of claim 1, further comprising creating, by the storage appliance, a write commit queue entry for processing by a target live migration server or a target live migration storage appliance.

9. A storage appliance for managing data migration operations, the storage appliance configured to: based on determining that a location of data at a source storage device is updated due to a data write, create a read commit queue entry indicating a location of the data at the source storage device to be copied from a source storage device of the storage appliance to a target storage device, the read commit queue entry comprising: a first field comprising first information for reading the data from the source storage device, the first information originating from the storage appliance; and a second field comprising second information for reading the data from the source storage device. the first information comprises a data pointer or a command identifier. the read commit queue entry is created using information from a write commit queue entry corresponding to the data write. the storage appliance is configured to:

10. The storage device of claim 9, wherein, generate a scatter gather list or a bitmap based on data writes received in the source storage device; and 11. The storage device of claim 9, wherein, the read commit queue entry is created based on the scatter gather list or the bitmap.

12. The storage device of claim 9, wherein, ​ ​ ​ ​ 13. The storage appliance of claim 9, configured to create a read commit queue entry for processing by a target live migration server or a target live migration storage appliance.

14. A system for managing data migration operations, the system comprising: a host; and a storage appliance, wherein the storage appliance is configured to: based on a determination that a location of data at a source storage of the storage appliance is updated due to a data write, create a read commit queue entry indicating a location of the data at the source storage to be copied from the source storage of the storage appliance to a target storage, the read commit queue entry comprising: a first field comprising first information for reading the data from the source storage, the first information originating from the storage appliance; and a second field comprising second information for reading the data from the source storage.

15. The system of claim 14, wherein, the storage appliance is configured to: send the read commit queue entry to the host; receive a command based on the read commit queue entry; and read the data from the source storage based on the command.

16. The system of claim 14, wherein, the storage appliance is configured to read the data from the source storage based on the read commit queue entry being stored on the storage appliance.

17. The system of claim 14, wherein, the read commit queue entry is created using information from a write commit queue entry corresponding to the data write.

18. The system of claim 14, wherein, the storage appliance is configured to create a write commit queue entry for processing by a target live migration server or a target live migration storage appliance.

Images