System and method for managing backing volume connections

Abstract

Systems and methods are disclosed for managing connections between at least one initiator server and backing volumes within at least one target server. In one embodiment, a target server comprises at least two backing volumes and at least one mid-layer module configured to be exposed over a data plane network. An orchestrator is configured to control a control plane of the network. The mid-layer module is configured to expose at least one backing volume of the target server to at least one initiator server and is further configured to terminate and establish backing volume connections while preserving an existing initiator-target connection. The disclosed architecture permits switching between backing volumes without requiring termination of initiator-target connections.

Description

[0001] SYSTEM AND METHOD FOR MANAGING BACKING VOLUME

[0002] CONNECTIONS

[0003] FIELD OF INVENTION

[0004] This invention relates to data storage systems in general, and more particularly to systems and methods for managing backing volume connections.

[0005] BACKGROUND OF INVENTION:

[0006] Modern computing network systems utilize backing volumes to store and expose data to servers distributed throughout a network. For the purposes of this disclosure, the term backing volume is used to broadly encompass any block-based storage device, including but not limited to hard disk drives (HDDs), solid state drives (SSDs), random access memory (RAM) drives, Linux devicemapper (DM) devices, logical volume manager (LVM) volumes, or LVM snapshots.

[0007] When an initiator server switches its connection from one version of a backing volume to an updated version, a time penalty is incurred during the disconnection / reconnection cycle. Conventionally, this time penalty has simply been absorbed as a cost of operation for managing computing networks, but as the number of said initiator servers increases - as is developing in a number of contemporary use cases - this time penalty becomes multiplied, introducing a critical barrier to operation.

[0008] LVM snapshots, a type of backing volume, store a version of a dataset captured at a particular time, typically hosted on a remote storage system within a target server and accessible by many initiator servers distributed throughout a network. Each time a new LVM snapshot of the same dataset is produced, it is published on storage media in its updated form, wherein said form contains the differences over previous LVM snapshots, known as “delta updates Depending on the type of network, the type of data in the dataset, and the application for which initiator servers connect to the network, a storage system may produce new LVM snapshots several times an hour, and said LVM snapshot and delta update files may occupy upwards of 80% of storage capacity in the available storage media.

[0009] Conventionally, the process by which an initiator server connects to a new backing volume within a storage system contained within a target server essentially comprises the following steps: 1. exposing the new backing volume through a storage target; 2. establishing a new connection from each initiator server to the exposed backing volume; 3. stopping an application running on each initiator server; 4. reconfiguring each initiator server to establish a new initiator-target connection; 5. restarting an application on each initiator server; 6. disconnecting the existing initiator sessions from the old backing volume; and 7. un-mapping or removing the old backing volume from the storage target so that it is no longer exposed to the initiator servers.

[0010] For example, in the training of machine learning (ML) algorithms distributed throughout a computing network, many hundreds, thousands, or even tens of thousands of initiator servers connect to a remote storage system within a target server on which backing volumes are stored and periodically updated. In conventional operation, the time penalty incurred when each initiator server disconnects from each version of a backing volume and connects to a new one introduces a critical restriction in operation, manifest either: as a limitation in the number of initiator servers on which ML algorithms connect to a training dataset; or as a limitation in the frequency in which those ML algorithms can be trained on newer updates of the backing volume of the training dataset, both of which significantly constrain the abilities of multiple ML algorithms from effectively training. In a separate example illustrating the same limitation in conventional network operation with respect to backing volume switching, asynchronous mirroring - a technique commonly employed in the art - may require backing volumes to be updated in periods as short as several seconds. Networks that operate asynchronous mirroring replicate backing volumes between different nodes, and thus, when the frequency of backing volume updates increase, are forced to undertake the backing volume switching procedure described above, and in doing so incur a significant time penalty that limits operation and significantly increases cost.

[0011] The conventional knowledge taught in the art pertaining to backing volume switching renders significant limitations in the two industrial applications described above, among other industrial applications well known in the art. Thus, there exists a need in the art for a system and method of configuring a network capable of switching backing volumes without incurring the time penalty of operating the aforementioned conventional backing volume switching procedure.

[0012] SUMMARY OF INVENTION:

[0013] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, devices and methods which are meant to be exemplary and illustrative and not limiting in scope. In various embodiments, one or more of the above-described problems have been reduced or eliminated, while other embodiments are directed to other advantages or improvements.

[0014] According to a first aspect of the invention, a computing system for exposing storage over a network comprises: (a) at least one target server that comprises at least two backing volumes and at least one mid-layer module, configured to be exposed over a data plane (DP) network; (b) at least one initiator server configured to access data hosted on the at least two backing volumes of the target server through dedicated initiator-target connection(s); (c) at least one orchestrator configured to interact with each of said servers and designated to control a control plane of the DP network, wherein the at least one mid-layer module is configured to expose at least one backing volume of the at least one target server to the at least one initiator server, wherein a backing volume mid-layer module connection can be terminated while preserving an initiator-target connection, and wherein a new backing volume mid-layer module connection can be established while preserving an initiator-target connection.

[0015] According to another aspect of the invention, the at least two backing volumes are volume snapshots.

[0016] According to another aspect of the invention, the at least one mid-layer module is at least one virtual snapshot module.

[0017] According to another aspect of the invention, the at least one mid-layer module comprises at least one mid-layer sub-module to which backing volumes from different datasets can be exposed.

[0018] According to another aspect of the invention, the at least one mid-layer sub-module is at least one virtual snapshot sub-module to which snapshots from different datasets can be exposed.

[0019] According to another aspect of the invention, the at least one mid-layer sub-module is at least one Linux device to which backing volumes from different datasets can be exposed.

[0020] According to another aspect of the invention, the at least one mid-layer sub-module communicates with a new backing volume to which a connection between the backing volume and the at least one mid-layer module is to be established. According to another aspect, such said communication aspect enables switching between backing volumes while preserving an existing initiator-target connection. According to another aspect of the invention, the initiator-target connections to which the midlayer module exposes at least one backing volume to the plurality of initiator servers is mediated over a network protocol.

[0021] According to another aspect of the invention, the network protocol is an iSCSI network protocol or fibre channel network protocol.

[0022] According to another aspect of the invention, the network protocol is a Non-Volatile Memory express (NVMe) over TCP network protocol or a NVMe over fabric network protocol.

[0023] According to another aspect of the invention, the at least one initiator server is in communication with the at least one target server through multiple initiator-target connections.

[0024] According to another aspect of the invention, a method of exposing storage over a network comprises the steps: (a) providing at least one target server that comprises at least two backing volumes and at least one mid-layer module; (b) exposing the at least one mid-layer module to the at least one initiator server over a data plane (DP) network; (c) configuring at least one orchestrator to interact with each of the servers and designated to control a control plane of the DP network; (d) exposing, by the at least one mid-layer module, at least one backing volume of the at least one target server to the at least one initiator server; (e) terminating a backing volume connection at the mid-layer module while preserving an initiator-target connection; and (f) establishing a backing volume connection at the mid-layer module while preserving an initiatortarget connection.

[0025] According to another aspect of the invention, wherein step (f) is operated before step (e).

[0026] According to another aspect of the invention, steps (f) and (e) are operated simultaneously. BRIEF DESCRIPTION OF THE FIGURES:

[0027] Some embodiments of the invention are described herein with reference to the accompanying figures. The description, together with the figures, makes apparent to a person having ordinary skill in the art how some embodiments may be practiced. The figures are for the purpose of illustrative description and no attempt is made to show structural details of an embodiment in more detail than is necessary for a fundamental understanding of the invention.

[0028] In the Figures:

[0029] FIG. 1 constitutes a block diagram of backing volume access in a conventional computing network.

[0030] FIG. 2 constitutes a block diagram of backing volume access using a mid-layer module, according to some embodiments of the present invention.

[0031] FIG. 3 constitutes a block diagram of a mid-layer module implemented using a device mapper (DM) linear device, according to some embodiments of the present invention.

[0032] DETAILED DESCRIPTION OF SOME EMBODIMENTS:

[0033] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well- known methods, procedures, and components, modules, units and / or circuits have not been described in detail so as not to obscure the invention. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For the sake of clarity, discussion of same or similar features or elements may not be repeated.

[0034] The terms “module” and “component” as used herein generally represent software, firmware, hardware, or combinations thereof, unless otherwise specified or understood by those of ordinary skill in the art. In the case of a software implementation, the module or component represents computer readable instructions (or program code) that implement specified tasks when executed on a processor to implement the functionality associated with the module or component. The program code may be stored in one or more non-transitory computer-readable memory devices.

[0035] Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Additionally, some of the described method embodiments or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.

[0036] The term “initiator-target” connection, as used herein, refers to the regulated connection between an initiator server and a target server, through which a backing volume hosted on the latter can be exposed to the former. Initiator-target connections that use conventional network computation protocols in conventional storage systems must be formed and reformed each time a new backing volume is produced at the target server, incurring a significant time penalty on the network.

[0037] The term “mid-layer module ”, as used herein, refers to a distinct module that operates within a network storage system and is hosted on storage media connected through remote connections to at least one initiator server, and that is capable of maintaining a connection with the at least one initiator server through a corresponding at least one initiator-target connection while being exposed to different backing volumes which it can in turn expose to the at least one initiator server through the corresponding plurality of initiator-target connections. When such backing volumes are LVM snapshots, then the mid-layer module will be considered as a virtualisation of the snapshots and thereby considered as a “virtual snapshot module

[0038] The term “mid-layer sub-module ”, as used herein, refers to a distinct module novel to the present invention that operates within a mid-layer module, and is responsible for the backing volumes generated and updated from a single database. Multiple mid-layer sub-modules may be contained on a single mid-layer module, each exposed to backing volumes from a particular dataset and periodic updates thereof, and each exposing said backing layers and periodic updates thereof to at least one initiator server through a corresponding at least one initiator-target connection, wherein newly updated backing-layers can be exposed to the at least one initiator server by the backinglayer sub-module using at least one initiator-target connection.

[0039] The term “linear device ” as used herein, refers to a layer within a target server running the Linux operating system. The term “DM linear device ” as used herein, refers to a device manager linear device, which is a simple form of a linear device layer within a Linux target server.

[0040] The term “APP module ” as used herein, refers to an application component on an initiator server. The APP module generates VO requests and communicates such requests to a corresponding initiator module of an initiator server.

[0041] The term “initiator module ”, as used herein, refers to a storage protocol component operating on an initiator server. The initiator module receives I / O requests for an APP module and establishes initiator-target connections with a target server over a data plane (DP).

[0042] Reference is made to FIG. 1, which constitutes a block diagram of backing volume access on conventional computing networks. Initiator servers 101a -lOln, contain APP modules 102a - 102n that connect to initiator modules 103a-103n which can establish initiator-target connections 104a and 104b with target server 112 which comprises storage system 105, which itself comprises NVMe / SCSI target module 106. In the storage system, NVMe / SCSI target module 106 is exposed to the initiator servers lOla-lOln through initiator-target connections 104a and 104b, and NVMe / SCSI target module 106 provides exposed device 107a access to first backing volume 108a via first initiator-target connection 104a. In conventional systems, each initiator-target connection is locked to a particular exposed device. When initiator servers lOla-lOln need to access second backing volume 108b, a separate initiator-target connection 104b must be established. This necessitates termination of first initiator-target connection 104a and creation of a new connection to second backing volume 108b, thereby incurring a time penalty on the system for the termination and re-establishing of initiator-target connections.

[0043] Reference is made to FIG. 2, which constitutes a block diagram of backing volume access using a mid-layer module, according to some embodiments of the present invention. Multiple initiator servers, servers 201a-201n, contain APP modules 202a-202n that connect to initiator modules 203a-203n which establish initiator-target connection 204 with storage server 212 which comprises target server 205, which itself comprises NVMe / SCSI target module 206. NVMe / SCSI target module 206 exposes exposed device 207 to mid- layer module 209 to which backing volumes 208a-208n can be switched without terminating initiator-target connection 204, thereby avoiding the time penalty incurred by terminating and re-establishing initiator-target connections. For example, once backing volume 208a has been exposed to servers 201a-201n, second backing volume 208b can be exposed without terminating initiator-target connection 204. Second backing volume 208b is exposed to Mid-layer module 209 (which may comprise mid-layer sub-modules), and in turn then exposed to exposed device 207 within NVMe / SCSI target module 206 within storage system 205, and is thus accessible by the initiator modules 203a-203n at initiator servers 201a-201n. Orchestrator 210 is in communication with initiator servers 201a-201n and target server 205 via control plane 211. Orchestrator 210 issues commands to initiator modules 203a- 203n and to target server 205 to configure connection parameters and manage permissions, thereby coordinating an initiator-target interaction.

[0044] According to other embodiments of the invention, multiple NVMe / SCSI target modules are accessible by the same initiator servers, and thus a plurality of initiator servers are able to access multiple targets to which multiple mid-layer modules are capable of exposing backing volumes of different datasets hosted on the same storage system remote from the initiator servers.

[0045] According to some embodiments, initiator servers 201a-201n are in communication with target server 205 through multiple simultaneous initiator-target connections 204, and thus through multiple paths. This method is known in the field as “multipathing” and can, when configured in the present invention, confer a number of benefits to a network system, including but not limited to: redundancy (maintenance of connection even if one path fails); load balancing (to manage the distribution of I / O traffic to optimize available bandwidth); scalability (adaptability to larger data traffic demands).

[0046] Reference is made to FIG. 3, which constitutes a block diagram of a mid-layer module implemented using a DM linear device, according to some embodiments.

[0047] In the case illustrated by FIG. 3, a storage system employs DM linear device 309 as a mid-layer module, wherein DM linear device 309 is configurable as a mid-layer module within the storage system and may be assigned a unique World-Wide Name (WWN) / Universally Unique Identifier (UUID) when exposed to initiator servers 301a-301n through an NVMe / SCSI target module 306.

[0048] A WWN / UUID is not assigned to backing volumes 308a-308n, because the backing volumes 308a-308n will change and thus require reconfiguration. In the case illustrated by FIG. 3, DM linear device 309 (which may contain DM linear device sub-modules) operates on backing volumes 308a-308n, over which groups of volumes are organized in volume groups 313a-313n, which themselves are contained within physical volumes 314a-314n. Orchestrator 310 is in communication with initiator servers 301a-301n and target server 305 via control plane 311. Orchestrator 310 issues commands to initiator modules 303a-303n and to target server 305 to configure connection parameters and manage permissions, thereby coordinating an initiator-target interaction.

[0049] According to some embodiments of the invention, initiator-target connection 304 exists as part of a network fabric such as RDMA (remote direct memory access), fibre channel (FC), or transmission control protocol (TCP). In this configuration, DM linear device 309 may establish initiator-target connection 304 with initiator servers 301a-301n, wherein control functions of initiator-target connection 304 are managed by DM linear device 309, according to some embodiments. According to other embodiments, the control functions of the initiator-target connection 304 are managed by the operator of the network.

[0050] Although the present invention has been described with reference to specific embodiments, this description is not meant to be construed in a limited sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention will become apparent to persons skilled in the art upon reference to the description of the invention. It is, therefore, contemplated that the appended claims will cover such modifications that fall within the scope of the invention.

Claims

CLAIMS1. A computing system for exposing storage over a network, comprising: a. at least one target server that comprises at least two backing volumes and at least one mid-layer module, configured to be exposed to at least one initiator server over a data plane (DP) network; b. at least one orchestrator configured to interact with each of the servers and designated to control a control plane (CP) of the DP network; wherein the at least one mid-layer module is configured to expose at least one backing volume of the at least one target server to the at least one initiator server; wherein a backing volume mid-layer module connection can be terminated while preserving an initiator-target connection; and wherein a backing volume mid-layer module connection can be established while preserving an initiator-target connection.

2. The system of claim 1, wherein the at least two backing volumes are volume snapshots.

3. The system of claim 1, wherein the at least one mid-layer module is at least one virtual snapshot module.

4. The system of claim 1, wherein the initiator-target connections to which the mid-layer module exposes at least one backing volume to the plurality of initiator servers is mediated over a network protocol.

5. The system of claim 4, wherein the network protocol is an iSCSI network protocol or fibre channel network protocol.

6. The system of claim 4, wherein the network protocol is a Non-Volatile Memory express (NVMe) over TCP network protocol or a NVMe over fabric network protocol.

7. The system of claim 1, wherein the at least one initiator server is in communication with the target server through multiple initiator-target connections.

8. The system of claim 1, wherein the at least one mid-layer module comprises at least one mid-layer sub-module to which backing volumes from different datasets can be exposed.

9. The system of claim 8, wherein the at least one mid-layer sub-module is at least one virtual snapshot sub-module to which snapshots from different datasets can be exposed.

10. The system of claim 8, wherein the at least one mid-layer sub-module is at least one Linux device to which backing volumes from different datasets can be exposed.

11. The system of claim 8, wherein at least one mid-layer sub-module communicates with a new backing volume to which a connection between the backing volume and at least one mid-layer module is to be established.

12. The system of claim 11, wherein said mid-layer sub-module communication enables switching between backing volumes while preserving an initiator-target connection.

13. Amethod of exposing storage over a network, comprising the steps: a. providing at least one target server that comprises at least two backing volumes and at least one mid-layer module;b. exposing the at least one mid-layer module to the at least one initiator server over a data plane (DP) network; c. configuring at least one orchestrator to interact with each of the servers and designated to control a control plane of the DP network; d. exposing, by the at least one mid-layer module, at least one backing volume of the at least one target server to the at least one initiator server; e. terminating a backing volume connection at the mid-layer module while preserving an initiator-target connection; and f. establishing a backing volume connection at the mid-layer module while preserving an initiator-target connection.

14. The method of claim 13, wherein step (f) is operated before step (e).

15. The method of claim 13, wherein steps (f) and (e) are operated simultaneously.

Images