Method for improving distributed storage performance in virtual machine environment
By introducing Longhorn into the virtual machine environment and adopting SSD cache and optimizing the network architecture, the problem of low storage performance and data synchronization efficiency of Longhorn is solved, and efficient IO performance and storage service capabilities are achieved.
Patent Information
- Application Number
- CN202510301391.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-14
- Publication Date
- 2025-06-20
AI Technical Summary
Longhorn has problems in virtual machine environments with low storage read and write performance and low data synchronization efficiency, which affects the read and write efficiency of business pods.
Longhorn was introduced into the virtual machine environment, high-speed storage device SSD was used as the cache layer, and Linux Bcache scheduling algorithm was optimized to improve IO read and write performance. At the same time, Longhorn's replica synchronization network architecture is optimized to improve data synchronization efficiency by setting up a synchronization private network.
It significantly improves Longhorn's IO performance in the virtual machine environment, improves overall performance and storage service capabilities, and can better support the efficient operation of business pods.
Smart Images
Figure CN120179174A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of cloud computing, and particularly relates to a method for improving the performance of distributed storage in a virtual machine environment. Background Art
[0002] In traditional distributed storage products, Kubernetes can support a large number of external storage systems, such as NFS, GlusterFS, Ceph, OpenStack Cinder, etc. Currently, the mainstream deployment method of these storage systems is still to deploy and maintain them separately outside the Kubernetes cluster, and these products often cannot meet the requirements of the virtual machine environment for storage performance.
[0003] Longhorn is a lightweight, reliable and powerful distributed block storage system based on Kubernetes designed for cloud native platforms. It runs directly on the Kubernetes platform, runs in the form of containers and microservices, creates a dedicated storage controller for each block device volume, and synchronously replicates the volume across multiple replicas stored on multiple nodes.
[0004] Based on this architecture analysis, it can be seen that the performance problems of Longhorn are likely to occur in two aspects:
[0005] On the one hand, the read and write performance of the storage used by Longhorn will directly affect the speed of data being written to disk, and thus directly affect the read and write performance of the storage.
[0006] On the other hand, since Longhorn needs to replicate data among each replica, the network transmission efficiency among each replica will also greatly affect the data synchronization speed, and thus will affect the read and write efficiency of the business pods using the Longhorn block storage.
[0007] To solve the above problems, the present invention proposes a method for improving the performance of distributed storage in a virtual machine environment. Summary of the Invention
[0008] In order to make up for the defects of the prior art, the present invention provides a simple and efficient method for improving the performance of distributed storage in a virtual machine environment.
[0009] The present invention is realized by the following technical solutions:
[0010] A method for improving the performance of distributed storage in a virtual machine environment, characterized by comprising the following steps:
[0011] Step S1: Introduce the cloud-native distributed block storage type Longhorn into the virtual machine environment. Use high-speed storage devices SSD as the cache layer, and at the same time optimize the Linux Bcache scheduling algorithm to maximize the optimization of the IO read and write performance of the distributed storage Longhorn.
[0012] In the step S1, adopt the Bcache caching scheme. Add a small amount of high-speed storage devices SSD to the large-capacity traditional mechanical disk HDD. Use the high-speed storage devices SSD to provide cache support for the traditional mechanical disk HDD. Cache the data whose access frequency exceeds the custom threshold into the high-speed storage devices SSD. When the application needs to read this data, directly obtain it from the high-speed storage devices SSD, thereby greatly improving the data access speed.
[0013] In the step S1, enable the writeback write policy of Bcache. First write the data into the high-speed storage devices SSD for caching, then write it into the traditional mechanical disk HDD, and then perform data synchronization between replicas by Longhorn to ensure the high availability of data.
[0014] Step S2: Optimize the replica synchronization network architecture of Longhorn. Configure several network cards for the instance container pod, and set up a synchronization private network between distributed nodes. Use the K8S network for normal access to the service, and use the synchronization private network for data writing and synchronization to improve the synchronization efficiency between replicas, further improve the performance of Longhorn, and build a high-performance version of the cloud-native distributed storage Longhorn.
[0015] In the step S2, set the network of the instance container pod to the host network to replace the original K8S network. Change the normal access to the service to use the host network to get rid of the dependence and limitation on the K8S network.
[0016] A system for improving the performance of distributed storage in a virtual machine environment, including:
[0017] A cache configuration module, responsible for using high-speed storage devices SSD as the cache layer, and at the same time optimizing the Linux Bcache scheduling algorithm to maximize the optimization of the IO read and write performance of the distributed storage Longhorn.
[0018] A network configuration module, responsible for optimizing the replica synchronization network architecture of Longhorn. Configure several network cards for the instance container pod, and set up a synchronization private network between distributed nodes. Use the K8S network for normal access to the service, and use the synchronization private network for data writing and synchronization to improve the synchronization efficiency between replicas, further improve the performance of Longhorn, and build a high-performance version of the cloud-native distributed storage Longhorn.
[0019] The cache configuration module adopts the Bcache caching scheme, adding a small amount of high-speed storage devices SSD to a large-capacity traditional mechanical disk HDD, and using the high-speed storage device SSD to provide cache support for the traditional mechanical disk HDD; caching data with access frequencies exceeding a custom threshold into the high-speed storage device SSD. When the application needs to read this data, it directly obtains it from the high-speed storage device SSD, thereby greatly improving the data access speed.
[0020] The cache configuration module enables the writeback policy of Bcache, first writing data into the high-speed storage device SSD for caching, then writing it into the traditional mechanical disk HDD, and then Longhorn performs data synchronization between replicas to ensure the high availability of data.
[0021] The network configuration module sets the network of the instance container pod to the host network, replacing the original K8S network. For the normal access of the service, it uses the host network instead to get rid of the dependence and restrictions on the K8S network.
[0022] A device for improving the performance of distributed storage in a virtual machine environment, characterized in that it includes a memory and a processor; the memory is used to store computer programs, and the processor is used to implement the above method steps when executing the computer programs.
[0023] A readable storage medium, characterized in that a computer program is stored on the readable storage medium, and the computer program implements the above method steps when executed by a processor.
[0024] The beneficial effect of the present invention is that the method for improving the performance of distributed storage in the virtual machine environment greatly improves the IO performance of the cloud-native distributed storage Longhorn in the virtual machine environment. While improving the overall performance of the virtual machine environment, it can also improve the storage service ability of the entire cloud platform, better support the user's business pod, and further promote the development of the information security level and the information technology industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0026] Appendix Figure 1 It is a schematic diagram of the Bcache caching architecture used by Longhorn of the present invention.
[0027] Appendix Figure 2 This is a schematic diagram of the Longhorn network optimization architecture of the present invention. Detailed implementation manners
[0028] In order to enable those skilled in the art of the present technology to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.
[0029] As a cloud-native distributed block storage service system, Longhorn provides block storage services for application pods. If the disks used by Longhorn itself are high-performance Nvme or SSDs, there will be no performance problems. However, considering the actual business itself, since the cost of high-performance storage is relatively high, it is not widely adopted in actual business. Instead, traditional HDD hard disks are more commonly used to provide large-capacity storage services. Moreover, the IO performance of HDDs themselves is not high. How to improve storage performance at the lowest cost in this scenario is the key point of optimization.
[0030] Longhorn provides storage services externally in the form of microservices. Therefore, its services are essentially a bunch of network communication-based interfaces. There are two types of instances in Longhorn: the instance_manager_e instance engine and the instance_manager_r instance replica. Both instances are provided with services externally through the http server by pods running in the cluster. The instance replicas synchronize data before replicas through the K8S network and share the same network with all pods, which is limited by the capabilities of the K8S network plugin itself and is also affected by other pods.
[0031] The method for improving the performance of distributed storage in the virtual machine environment includes the following steps:
[0032] Step S1: Introduce the cloud-native distributed block storage type Longhorn into the virtual machine environment, use high-speed storage devices SSDs as the cache layer, and at the same time optimize the Linux Bcache scheduling algorithm to maximize the optimization of the IO read and write performance of the distributed storage Longhorn;
[0033] In step S1, the Bcache caching scheme is adopted. A small amount of high-speed storage devices SSDs are added to a large-capacity traditional mechanical hard disk HDD, and the high-speed storage devices SSDs are used to provide cache support for the traditional mechanical hard disk HDD. The data whose access frequency exceeds a custom threshold is cached in the high-speed storage device SSD. When the application needs to read this data, it can directly obtain it from the high-speed storage device SSD, thereby greatly improving the data access speed.
[0034] In step S1, the writeback write policy of Bcache is enabled. The data is first written to the high-speed storage device SSD for caching, and then written to the traditional mechanical hard disk HDD, and then the data synchronization between replicas is performed by Longhorn to ensure the high availability of the data.
[0035] The specific environment is as follows:
[0036] Longhorn uses 3 replicas, which are located on three different physical nodes respectively. Each node is configured with an 8T HDD ( / dev / vdb) to store the user data of Longhorn and a 200G SSD ( / dev / sdb) for caching.
[0037] The implementation solution is as follows:
[0038] a) Create a Bcache cache: make-bcache -C / dev / sdb -B / dev / vdb. After creation, a new disk / dev / bcache0 can be seen.
[0039] b) Modify the cache policy: echo writeback > / sys / block / bcache0 / bcache / cache_mode;
[0040] c) Format and mount: mkfs.ext4 / dev / bcache0; mount / dev / bcache0 / longhorndata;
[0041] d) Modify the Longhorn configuration, add a tag named bcache, and at the same time add a Longhorn disk, specifying the directory as / longhorndata of the node;
[0042] e) Configure the storage class SC (StorageClass) of Longhorn to use the disk with the tag bcache;
[0043] f) After the business container pod creates a Persistent Volume Claim (PVC) using the storage class SC configured as bcache, it can use the high-speed storage device SSD to optimize the storage performance.
[0044] Step S2: Optimize the replica synchronization network architecture of Longhorn. Configure several network cards for the instance container pod and set up a synchronization private network between distributed nodes. Use the K8S network for normal business access, and use the synchronization private network for data writing and synchronization to improve the synchronization efficiency between replicas, further improve the performance of Longhorn, and build a high-performance version of the cloud-native distributed storage Longhorn.
[0045] In the step S2, set the network of the instance container pod to the host network to replace the original K8S network, and use the host network for normal business access to get rid of the dependence and restrictions on the K8S network.
[0046] The system for improving the performance of distributed storage in the virtual machine environment includes:
[0047] The cache configuration module is responsible for using the high-speed storage device SSD as the cache layer, and at the same time optimizing the Linux Bcache scheduling algorithm to maximize the optimization of the IO read and write performance of the distributed storage Longhorn.
[0048] The network configuration module is responsible for optimizing the replica synchronization network architecture of Longhorn. Configure several network cards for the instance container pod and set up a synchronization private network between distributed nodes. Use the K8S network for normal business access, and use the synchronization private network for data writing and synchronization to improve the synchronization efficiency between replicas, further improve the performance of Longhorn, and build a high-performance version of the cloud-native distributed storage Longhorn.
[0049] The cache configuration module adopts the Bcache cache scheme, adding a small amount of high-speed storage device SSD to the large-capacity traditional mechanical disk HDD, and using the high-speed storage device SSD to provide cache support for the traditional mechanical disk HDD. Cache the data whose access frequency exceeds the custom threshold into the high-speed storage device SSD. When the application needs to read this data, directly obtain it from the high-speed storage device SSD, thus greatly improving the data access speed.
[0050] The cache configuration module enables the writeback write policy of Bcache, first writes the data into the high-speed storage device SSD for caching, then writes it into the traditional mechanical disk HDD, and then Longhorn performs data synchronization between replicas to ensure the high availability of data.
[0051] The network configuration module sets the network of the instance container pod to the host network, replacing the original K8S network. For normal access to the service, the host network is used instead to get rid of the dependence on and limitations of the K8S network.
[0052] The device for improving the performance of distributed storage in the virtual machine environment includes a memory and a processor; the memory is used to store computer programs, and the processor is used to implement the above method steps when executing the computer programs.
[0053] A computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the above method steps are implemented.
[0054] The above embodiments are only one of the specific implementation manners of the present invention, and the general changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for improving distributed storage performance in a virtual machine environment, characterized in that: The following steps are involved: Step S1, introducing the cloud native distributed block storage type Longhorn into the virtual machine environment, using the high-speed storage device SSD as the cache layer, and optimizing the Linux Bcache scheduling algorithm, thereby optimizing the IO read and write performance of the distributed storage Longhorn; Step S2: Optimize the replica synchronization network architecture of Longhorn, configure several network cards for the instance container pod, and set up a synchronous private network between distributed nodes; use the K8S network for normal business access, and use the synchronous private network for data writing and synchronization to improve the synchronization efficiency between replicas and further improve the performance of Longhorn.
2. The method for improving distributed storage performance in a virtual machine environment according to claim 1, characterized in that In step S1, a Bcache caching solution is adopted, and a high-speed storage device SSD is added to a large-capacity traditional mechanical disk HDD, and the high-speed storage device SSD is used to provide cache support for the traditional mechanical disk HDD; data with an access frequency exceeding a custom threshold is cached in the high-speed storage device SSD, and when the application needs to read this data, it is directly obtained from the high-speed storage device SSD, thereby greatly improving the data access speed.
3. The method for improving distributed storage performance in a virtual machine environment according to claim 2, characterized in that: In step S1, the writeback strategy of Bcache is enabled, and data is preferentially written to the high-speed storage device SSD for caching, and then written to the traditional mechanical disk HDD. Then Longhorn synchronizes the data between the replicas to ensure high data availability.
4. The method for improving distributed storage performance in a virtual machine environment according to claim 1, characterized in that: In step S2, the network of the instance container pod is set to the host network to replace the original K8S network, and the host network is used for normal access to the business to get rid of the dependence and restrictions on the K8S network.
5. A system for improving distributed storage performance in a virtual machine environment, characterized in that: include: The cache configuration module is responsible for using high-speed storage devices SSD as the cache layer, while optimizing the Linux Bcache scheduling algorithm, thereby optimizing the IO read and write performance of distributed storage Longhorn; The network configuration module is responsible for optimizing Longhorn's replica synchronization network architecture, configuring several network cards for the instance container pod, and setting up a synchronous private network between distributed nodes; the K8S network is used for normal business access, and the synchronous private network is used for data writing and synchronization to improve the synchronization efficiency between replicas and further improve Longhorn's performance.
6. The system for improving distributed storage performance in a virtual machine environment according to claim 5, characterized in that: The cache configuration module adopts the Bcache cache solution, adding a high-speed storage device SSD to a large-capacity traditional mechanical disk HDD, and using the high-speed storage device SSD to provide cache support for the traditional mechanical disk HDD; data with an access frequency exceeding a custom threshold is cached in the high-speed storage device SSD, and when the application needs to read this data, it is directly obtained from the high-speed storage device SSD, thereby greatly improving the data access speed.
7. The system for improving distributed storage performance in a virtual machine environment according to claim 6, characterized in that: The cache configuration module enables the writeback strategy of Bcache, and preferentially writes data to the high-speed storage device SSD for caching, and then writes it to the traditional mechanical disk HDD. Longhorn then synchronizes the data between the copies to ensure high data availability.
8. The system for improving distributed storage performance in a virtual machine environment according to claim 5, characterized in that: The network configuration module sets the network of the instance container pod to the host network, replacing the original K8S network, and uses the host network for normal access to the business to get rid of the dependence and restrictions on the K8S network.
9. A device for improving distributed storage performance in a virtual machine environment, characterized in that: The method comprises a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the method steps as claimed in any one of claims 1 to 4 when executing the computer program.
10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 4 are implemented.