Dynamic logical thin volume

Abstract

The invention relates to a dynamic logical thin volume that enables a physical or virtual block device shared over a TCP / IP network or storage space networks to be used and managed as shared storage space by multiple physical servers to support thin - provisioned virtual disk files.

Description

[0001] DYNAMIC LOGICAL THIN VOLUME

[0002] Technical Field of the Invention

[0003] The invention relates to a dynamic logical thin volume that enables a physical or virtual block device shared over a TCP / IP network or storage area networks to be used and managed as shared storage area by multiple physical servers to support thin-provisioned virtual disk files.

[0004] State of the Art

[0005] Shared storage area allows multiple users to access the data in the storage area simultaneously. This is important so that if one physical server fails, other physical servers can seamlessly take over the workloads of the failed server. Otherwise, other servers cannot instantly access the data in the local storage area of the physical server and the failover process cannot take place. This feature is necessary to prevent outages during maintenance, load balancing, and hardware failures. In clustered physical server environments used for virtualization, virtual disk files of virtual machines are hosted in shared storage areas for these reasons.

[0006] Virtual disk file formats of virtual machines support thin provisioning. When thin provisioning is enabled, only the blocks that the virtual system uses are written to the storage area, thus saving storage space. For example, if a virtual machine with a 16 GiB thin-provisioned virtual disk file sees a 16 GiB hard block device, but the guest operating system hosts 7 GiB of data on that persistent block device, that virtual disk file takes up only 7 GiB of space in storage area. In this way, thin provisioning allows larger disk images to be created from available storage blocks. By creating large disk images for virtual machines, when the virtual machine writes to a new block, the capacity of the storage area is used more efficiently by expanding the virtual disk file behind without resizing the virtual machine's file system.

[0007] Shared storage areas are divided into two classes based on whether a block device or clustered block devices are accessed at the file level or at the block level. File-level shared storage formats a block device with a shared file system and shares it at the file level over the TCP / IP network stack via specialized application layer protocols designed for that shared file system. In addition, in Hyper-Converged Infrastructures, the clustering and sharing of block devices on physical servers at the file level is performed through distributed file systems and again, related specialized application layer protocols.

[0008] Regardless of whether virtual disk files are stored as files on the physical server's file system, virtual machines interact with these files at the block level. File-level shared storage areas add a layer of abstraction to enable file-by-file manipulation. Block-level shared storage areas interact directly with block devices, resulting in higher performance than file-level shares.

[0009] Block-level shared storage areas are shared with the TCP / IP network stack or storagespecific network stacks. In these structures, the physical server and the storage are separate units. In the literature, in order to share this shared storage within the physical server cluster, the block device is formatted with a shared-disk file system. Since these shared-disk file systems are mostly closed-source, they cannot be read or written to by end-user systems and other operating systems. When a failure to access the management network occurs on a physical server in the cluster, the virtual machines running on the failed server must continue to run and the thin-provisioned virtual disk files in the shared storage area must continue to expand. Some shared-disk file systems require the management network connections of the physical servers in the cluster to be operational for cluster coordination and locking mechanisms necessary to ensure data consistency. They can cause a deadlock when the management network connection between servers is lost or when the locking server goes down.

[0010] Another shared-use solution for block-level shared storage areas is logical volume management. A logical volume manager can divide one or more block devices into larger logical volumes than a block device in the group by grouping them into the same volume group. These logical volumes are used by physical servers as virtual block devices. When a logical volume manager is used to host virtual disks of virtual machines, a logical volume is created for a virtual machine.

[0011] In the state of the art, an implementation uses the logical volume manager for shared storage purposes only for thick-provisioned virtual disks. In another implementation, thin- provisioned disks are supported for only one physical server. In the state of the art, there are no solutions that use the logical volume manager for thin-provisioned virtual disks shared between physical servers, because the implementations in the literature that use the logical volume manager are dependent on the management network for shared use.

[0012] In the state of the art, for shared storage shared at the block level over TCP / IP networks or storage area networks, there are closed-source shared-disk file systems without read and write operations and / or solutions that perform lock management over the management network. Instead of these solutions, a system is needed that handles the block device's management lock communication by writing to the block device's metadata over the same connection through which it accesses storage. In this way, when the thin-provisioned virtual disk files on the physical server whose access to the management network is cut off need to be enlarged, the corresponding logical volume can be enlarged by obtaining a time-stamped lock. Since the lock is written to the block device with a timestamp, the deadlock problem is also eliminated.

[0013] Brief Description and Aim of the Invention

[0014] The invention relates to a dynamic logical thin volume that enables a physical or virtual block device shared over a TCP / IP network or storage area networks to be used and managed as shared storage area by multiple physical servers to support thin-provisioned virtual disk files.

[0015] One object of the invention is to enable the block device to write the management lock communication to the block device's metadata over the connection through which it accesses the storage. In this way, when the thin-provisioned virtual disk files on the physical server whose access to the management network is cut off need to be enlarged, the corresponding logical volume can be enlarged by obtaining a time-stamped lock.

[0016] Another object of the invention is to eliminate the deadlock problem with the lock written to the block device with the timestamp.

[0017] Description of the Drawings

[0018] Figure 1. The drawing showing the flow diagrams of the method of the invention.

[0019] Figure 2. The drawing showing the schematic view of the system of the invention. Figure 3. The drawing showing the schematic view of the system of the invention.

[0020] Description of the References in Figures

[0021] 1 . TCP / IP network switching device

[0022] 2. Storage area network switching device

[0023] 3. Physical server

[0024] 4. Virtual machine

[0025] 5. User interface

[0026] 6. Application programming interface

[0027] 7. Distributed key-value store module

[0028] 8. Virtual disk

[0029] 9. Controller module

[0030] 10. Storage controller module

[0031] 1 1 . Volume group lock manager

[0032] 12. Block device

[0033] 13. Logical volume manager

[0034] 14. Volume group

[0035] 15. Logical volume

[0036] 16. Logical volume extent

[0037] 17. Metadata

[0038] E: Yes

[0039] H: No

[0040] 1001 . Each physical server (3) is clustered through the TCP / IP network switching device (1 ) with the distributed key-value store module (7) thereon and a management network is created,

[0041] 2001 . A block device (12) is shared with physical servers (3) through a TCP / IP network switching device (1 ) or storage area network switching device (2)

[0042] 2002. A request is made to manage this block device (12) as a shared storage area by the logical volume manager (13) through the user interface (5), the user interface (5) forwards the request to the application programming interface (6), the application programming interface (6) checks that the block device (12) in the request is accessible to the physical servers (3) and that it is empty, then records the corresponding operation in the distributed key-value store module (7) 2003. The controller module (9) on one of the physical servers (3) follows the distributed key-value store module (7) to see the request, and, through the logical volume manager (13), formats the block device (12) to create and attach a new volume group (14) to it

[0043] 3001. A request is made through the user interface (5) to create a new virtual machine (4) to host the virtual disk (8) on this shared storage, the user interface (5) forwards the request to the application interface (6), and the application programming interface (6) checks that the storage area is accessible, then writes to the distributed key-value store module (7)

[0044] 3002. The controller module (9) of the corresponding physical server (3) waits to acquire a lock via lock manager

[0045] 3003. The volume group lock manager (1 1 ) waits if the volume group (14) has a lock on the metadata (17) and the time-stamp has not exceeded the timeout period

[0046] 3004. If the lock does not exist, has been removed, or the lock has timed out, the volume group lock manager (1 1 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting physical server (3)

[0047] 3005. The controller module (9) creates a logical volume (15) of a specified initial size with the logical volume manager (13), formats the logical volume (15) and maps the disk as a virtual block device (12), the controller module (9) creates thin and / or thick-provisioned virtual disk (8) files in the mapped virtual block device (12)

[0048] 4001 . Storage controller module (10) tracks the virtual block device (12) sizes and usage of virtual machines on the physical server (3) hosting thin-provisioned virtual disk (8) files

[0049] 4002. When the storage controller module (10) detects that the space utilization of the logical volume (15) is approaching a certain percentage of the maximum, it tries to obtain a lock for the corresponding volume group (14) through the volume group lock manager (11 )

[0050] 4003. The volume group lock manager (1 1 ) waits if the volume group (14) has a lock on the metadata (17) and the time-stamp has not exceeded the timeout period (4003),

[0051] 4004. If the lock does not exist, has been removed, or the lock has timed out, the lock manager (11 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting server

[0052] 4005. The storage controller module (10) adds new logical volume extent (16) to the logical volume (15) through the logical volume manager (13) and expands the size of the file system within Detailed Description of the Invention

[0053] The invention relates to a dynamic logical thin volume that enables a physical or virtual block device shared over a TCP / IP network or storage area networks to be used and managed as shared storage area by multiple physical servers to support thin-provisioned virtual disk files.

[0054] The TCP / IP network switching device (1 ) performs transmission of packets on the TCP / IP network to the target devices.

[0055] The storage area network switching device (2) performs block data transmission by connecting storage systems to servers in storage area networks.

[0056] A physical server (3) is the physical hardware on which system components that provide functionality for other programs or devices run.

[0057] Virtual machines (4) are virtual servers on which the necessary functionalities are provided to run an operating system through virtualization of physical devices in order to run more than one operating system on the physical hardware.

[0058] The user interface (5) is the interface that gives the end user the function of managing the appearance and features in the system. In the preferred embodiment of the invention, there is at least one located on each physical server (3).

[0059] An application programming interface (6) is an interface that allows the user interface (5) to interact with system applications. There is at least one located on each physical server (3).

[0060] The distributed key-value store module (7) performs consistent data sharing when accessing or writing cluster data on the physical server (3). There is one located on each physical server (3).

[0061] A virtual disk (8) is a file that is virtualized or emulated and presented to the virtual machine (4) and contains data inside a block device that is abstracted from physical storage. Virtual disk formats can support encryption, snapshot, thin provisioning, copy- on-write, and compression features. More than one thin or thick-provisioned virtual disk (8) can be used in a virtual machine (4).

[0062] The controller module (9) is responsible for the management of the physical server (3) on which it is located and exchanges information with other servers through the distributed key-value store module (7). There is at least one located on each physical server (3).

[0063] The storage controller module (10) hosts all the data, checks the health of the storage areas and manages the storage areas. There is at least one located on each physical server (3).

[0064] The volume group lock manager (11 ) manages multiple simultaneous operations or access by multiple users to specific data sources, preventing conflicts and ensuring data consistency. There is at least one located on each physical server (3).

[0065] A block device (12) is a data storage device that stores and transmits data in blocks.

[0066] Logical volume manager (13) is a software system that allows to manage storage area more flexibly. By grouping physical or virtual block devices and creating logical volumes, it divides storage resources and presents them as virtual block devices. There is one located on each physical server (3).

[0067] A volume group (14) is a storage management component that is provided as part of logical volume (15) management. It brings together multiple physical or virtual block devices and manages them as a single logical volume. This consolidates storage area, creating a larger and more manageable structure. In order to perform block device (12) based locking, one is created for each block device (12).

[0068] A logical volume (15) is created within a volume group (14) and this logical volume (15) is formatted with a file system just like a block device (12) and used for data storage needs. In the preferred implementation of the invention there is at least one for each virtual machine (4). Logical volume extent (16) refers to a storage area of a specific size within a volume group (14). Logical volumes (15) consist of one or more logical volume extents (16). When the size of the Logical Volume (15) is desired to be expanded, unused logical volume extents (16) are added to it.

[0069] Metadata (17) is the data required for storage layout management of logical volumes. This metadata (17) hosted at the beginning of the corresponding block device (12) contains the size of the logical volumes (15), which logical volume extents they occupy (16), and access rights. Lock information is also written here.

[0070] Each physical server (3) is clustered through the TCP / IP network switching device (1 ) with the distributed key-value store module (7) thereon and a management network is created.

[0071] The end user shares a block device (12) with physical servers (3) through a TCP / IP network switching device (1 ) or storage area network switching device (2) to create a shared storage area where the virtual machine (4) can host virtual disk (8) files. Through the user interface (5), a request is made for this block device (12) to be managed as a shared storage area by the logical volume manager (13). The user interface (5) forwards the request to the application programming interface (6). The application programming interface (6) checks that the block device (12) in the request is accessible to the physical servers (3) and that it is empty, then records the corresponding operation in the distributed key-value store module (7) The controller module (9) on one of the physical servers (3) follows the distributed key-value store module (7) to see the request. Through the logical volume manager (13), it formats the block device (12) to create and attach a new volume group (14) to it. As a result, shared storage area is provided to the system. This process is repeated for each new creation of shared storage area.

[0072] A request is made to create a new virtual machine (4) whose virtual disk (8) will be hosted on this shared storage through the user interface (5). The user interface (5) forwards the request to the application programming interface (6). The application programming interface (6) checks that the storage area is accessible, then writes to the distributed keyvalue store module (7). The controller module (9) of the corresponding physical server (3) waits to acquire a lock via lock manager. The volume group lock manager (11 ) waits if the volume group (14) has a lock on the metadata (17) and the timestamp has not exceeded the timeout period. If the lock does not exist, has been removed, or the lock has timed out, the lock manager (1 1 ) attempts to lock the volume group. If the lock owner is the requesting physical server (3), it is determined successful. The controller module (9) creates a logical volume (15) of a specific initial size with the logical volume manager (13). It formats the logical volume (15) and maps the disk as a virtual block device (12). Finally, it creates thin and / or thick-provisioned virtual disk (8) files in the mapped virtual block device (12). This process is repeated for each request to create a new virtual machine (4).

[0073] Although it is not directly related to the invention, in Figure 3, within the framework of the concept of High Availability, if it is desired not to lose the absolute majority in the event of a server loss in a cluster, there must be at least three servers in the cluster. For this reason, since the most preferred minimum number of installation examples in the literature is three, three servers are shown in the figure. The solution of the invention can work with one or more servers, that is, regardless of the number of servers. In Figure 3, the inside of the three physical servers is not exactly the same, the number of virtual machines is determined in order to accommodate the drawing. Zero, one or more virtual machines can be set up on a server. The intent is to show that zero, one, or more virtual disks used in a virtual machine are held on a logical volume, and that this logical volume occupies one, and possibly more, logical volume extent on the shared disk.

[0074] Figure 3 shows five logical volume extents (16), but when creating a logical volume, it must be created to accommodate at least one logical volume extent. For example, to create a logical volume with a capacity of 4 GiB (gibibytes), a total of 1024 logical volume extents are used, each of which is for example 4 MiB (mebibytes) in size. When the logical volume is desired to be expanded (4005th process), a number of logical volume extents are added to the logical volume, the total of which is the amount to be expanded.

[0075] Figure 3 shows the virtual disk (8) located in the logical volume (15). Figure 3 shows that zero, one, or more virtual disks (8) are hosted in a logical volume (15). A logical volume (15) is created for each virtual machine (4). The virtual disks (8) of that virtual machine (4) are all hosted within this logical volume (15).

[0076] The dynamic logical thin volume operation method includes the following process steps; Each physical server (3) is clustered through the TCP / IP network switching device (1 ) with the distributed key-value store module (7) thereon and a management network is created (1001 ),

[0077] A block device (12) is shared with physical servers (3) through a TCP / IP network switching device (1 ) or storage area network switching device (2) (2001 ),

[0078] A request is made to manage this block device (12) as a shared storage area by the logical volume manager (13) through the user interface (5), the user interface (5) forwards the request to the application programming interface (6), the application programming interface (6) checks that the block device (12) in the request is accessible to the physical servers (3) and that it is empty, then records the corresponding operation in the distributed key-value store module (7) (2002),

[0079] The controller module (9) on one of the physical servers (3) follows the distributed keyvalue store module (7) to see the request, and, through the logical volume manager (13), formats the block device (12) to create and attach a new volume group (14) to it (2003),

[0080] A request is made through the user interface (5) to create a new virtual machine (4) to host the virtual disk (8) on this shared storage, the user interface (5) forwards the request to the application interface (6), and the application programming interface (6) writes to the distributed key-value store module (7) after checking that the storage area is accessible (3001 )

[0081] The controller module (9) of the corresponding physical server (3) waits to acquire a lock via lock manager (3002),

[0082] The volume group lock manager (11 ) waits if the volume group (14) has a lock on the metadata (17) and the timestamp has not exceeded the timeout period (3003),

[0083] If the lock does not exist, has been removed, or the lock has timed out, the volume group lock manager (11 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting physical server (3) (3004), The controller module (9) creates a logical volume (15) of a specified initial size with the logical volume manager (13), formats the logical volume (15) and maps the disk as a virtual block device (12), the controller module (9) creates thin and / or thick-provisioned virtual disk (8) files in the mapped virtual block device (12) (3005),

[0084] Storage controller module (10) tracks the virtual block device (12) sizes and usage of virtual machines on the physical server (3) hosting thin-provisioned virtual disk (8) files (4001 ),

[0085] When the storage controller module (10) detects that the space utilization of the logical volume (15) is approaching a certain percentage of the maximum, it tries to obtain a lock for the corresponding volume group (14) through the volume group lock manager (1 1 ) (4002),

[0086] The volume group lock manager (1 1 ) waits if the volume group (14) has a lock on the metadata (17) and the timestamp has not exceeded the timeout period (4003),

[0087] If the lock does not exist, has been removed, or the lock has timed out, the lock manager (11 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting server (4004),

[0088] The storage controller module (10) adds new logical volume extents (16) to the logical volume (15) through the logical volume manager (13) and expands the size of the file system within (4005).

Claims

CLAIMS1 . A dynamic logical thin volume that enables a physical or virtual block device shared over a TCP / IP network or storage area networks to be used and managed as shared storage area by multiple physical servers to support thin-provisioned virtual disk files, characterized in that it comprises;- at least one TCP / IP network switching device (1 ) that forwards packets on the TCP / IP network to the target devices,- at least one storage area network switching device (2) that transmits block data by connecting storage systems to servers in storage area networks,- at least one physical server (3), which is the physical hardware on which the system components run, containing at least one of each of the user interface (5), application programming interface (6), distributed key-value store module (7), controller module (9), storage controller module (10), volume group lock manager (11 ), and logical volume manager (13),- at least one virtual machine (4) on which the necessary functionalities are provided to run an operating system through virtualization of physical devices in order to run more than one operating system on the physical hardware,- at least one user interface (5) that gives the end user the function of managing appearance and features in the system, where the block device (12) can be requested to be managed as a shared storage area by the logical volume manager (13),- at least one application programming interface (6) that allows the user interface (5) to interact with system applications, receives the request from the user interface (5), checks that the block device (12) in the request is accessible to the physical servers (3) and that it is empty, then records the corresponding operation in the distributed key-value store module (7),- at least one distributed key-value store module (7) that performs consistent data sharing when accessing or writing cluster data on the physical server (3),- at least one virtual disk (8) that is virtualized or emulated and presented to the virtual machine (4) and contains data inside a block device that isabstracted from physical storage, and which can enable encryption, snapshot, thin provisioning, copy-on-write, and compression features.- at least one controller module (9) responsible for the management of the physical server (3) on which it is located and exchanges information with other servers through the distributed key-value store module (7),- at least one storage controller module (10) that records data, checks the health of the storage area, and manages storage areas,- at least one volume group lock manager (11 ) that manages multiple simultaneous operations or access by multiple users to data sources, preventing conflicts and ensuring data consistency,- at least one block device (12), which is a data storage device that stores and transmits data in blocks,- at least one logical volume manager (13) that groups physical or virtual block devices and creates logical volumes (15), divides storage resources and presents them as new virtual block devices,- at least one volume group (14) that brings together multiple physical or virtual block devices and manages them as a single logical volume,- at least one logical volume (15) consisting of one or more logical volume extents (16), which is created within the volume group (14) and formatted with the file system and used for data storage needs,- at least one logical volume extent (16), which is a storage area of a size to be specified within the volume group (14),- at least one metadata (17) that contains data about the storage layout management of logical volumes, where the lock information is written, and is hosted at the beginning of the corresponding block device.

2. The operation method of the dynamic logical thin volume, characterized in that it comprises the following process steps;- Each physical server (3) is clustered through the TCP / IP network switching device (1 ) with the distributed key-value store module (7) thereon and a management network is created (1001 ),- A block device (12) is shared with physical servers (3) through a TCP / IP network switching device (1 ) or storage area network switching device (2) (2001 ),- A request is made to manage this block device (12) as a shared storage area by the logical volume manager (13) through the user interface (5), the user interface (5) forwards the request to the application programming interface (6), the application programming interface (6) checks that the block device (12) in the request is accessible to the physical servers (3) and that it is empty, then records the corresponding operation in the distributed key-value store module (7) (2002),- The controller module (9) on one of the physical servers (3) follows the distributed key-value store module (7) to see the request, and, through the logical volume manager (13), formats the block device (12) to create and attach a new volume group (14) to it (2003),- A request is made through the user interface (5) to create a new virtual machine (4) to host the virtual disk (8) on this shared storage, the user interface (5) forwards the request to the application interface (6), and the application programming interface (6) writes to the distributed key-value store module (7) after checking that the storage area is accessible (3001 )- The controller module (9) of the corresponding physical server (3) waits to acquire a lock via lock manager (3002),- The volume group lock manager (1 1 ) waits if the volume group (14) has a lock on the metadata (17) and the timestamp has not exceeded the timeout period (3003),If the lock does not exist, has been removed, or the lock has timed out, the volume group lock manager (11 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting physical server (3) (3004),- The controller module (9) creates a logical volume (15) of a specified initial size with the logical volume manager (13), formats the logical volume (15) and maps the disk as a virtual block device (12), the controller module (9) creates thin and / or thick-provisioned virtual disk (8) files in the mapped virtual block device (12) (3005),- Storage controller module (10) tracks the virtual block device (12) sizes and usage of virtual machines on the physical server (3) hosting thin- provisioned virtual disk (8) files (4001 ),- When the storage controller module (10) detects that the space utilization of the logical volume (15) is approaching a certain percentage of themaximum, it tries to obtain a lock for the corresponding volume group (14) through the volume group lock manager (1 1 ) (4002),- The volume group lock manager (1 1 ) waits if the volume group (14) has a lock on the metadata (17) and the timestamp has not exceeded the timeout period (4003),If the lock does not exist, has been removed, or the lock has timed out, the lock manager (11 ) will attempt to lock it, and it is determined successful if the lock owner is the requesting server (4004),- The storage controller module (10) adds new logical volume extents (16) to the logical volume (15) through the logical volume manager (13) and expands the size of the file system within (4005).

3. A dynamic logical thin volume according to claim 1 , characterized in that it comprises a virtual machine (4) that contains at least one thin or thick-provisioned virtual disk (8).

4. A dynamic logical thin volume according to claim 1 , characterized in that it comprises metadata (17) containing the size of the logical volumes (15), which logical volume extents (16) they occupy, lock information, and access rights.

5. A dynamic logical thin volume according to claim 1 , characterized in that it comprises the volume group (14), created one for each block device (12) in order to perform block device (12) based locking.

6. A dynamic logical thin volume according to claim 1 , characterized in that it comprises the logical volume (15) to which unused logical volume extents (16) can be added when its size is desired to be expanded.

7. A dynamic logical thin volume according to claim 1 , characterized in that it comprises the user interface (5) that allows requests to be made to create a new virtual machine (4) whose virtual disk (8) will be hosted on this shared storage.

Images