A Method, Device and Storage Medium for Dynamically Configuring VirtIO Virtual Queues

Abstract

The present invention relates to a method, device, and storage medium for dynamically configuring VirtIO virtual queues. When the VirtIO performs IO operations with a high frequency, the present invention monitors whether each traversed virtual queue is fully utilized. For any fully utilized virtual queue, on the premise that there is sufficient virtual machine memory, the size of the virtual queue is expanded. And if the expansion of the virtual queue significantly improves the IO performance, the expansion is retained; if it does not significantly improve the IO performance, the original size before the expansion is restored. When the VirtIO performs IO operations with a low frequency and the idle ratio of the virtual queue continuously exceeds the preset idle ratio threshold for a first set time, the size of the virtual queue is reduced to reclaim the idle memory resources on the virtual queue. And if the reduction of the virtual queue does not significantly affect the IO performance, the reduction is retained; if it significantly affects the IO performance, the original size before the reduction is restored. The present invention realizes the dynamic configuration of virtual queues, taking into account both IO performance and virtual machine resource utilization rate.

Description

Technical Field

[0001] The present invention relates to the field of virtual queue configuration, and in particular, to a method, device, and storage medium for dynamically configuring a VirtIO virtual queue. Background Art

[0002] KVM, whose full name is Kernel-based Virtual Machine, is a hardware-assisted virtualization technology based on the linux kernel. KVM is a full virtualization technology with hardware virtualization support, so it can support running almost all operating systems on the corresponding hardware. KVM is a basic module of the kernel, and the user space provides the virtual machine with simulated hardware through QEMU.

[0003] VirtIO is a semi-virtualized IO simulation technology adopted by KVM. The VirtIO technology includes a front end, a back end, and a virtual queue for communication between the front end and the back end. The front end is mainly located inside the virtual machine and is the driver corresponding to the VirtIO device simulated inside the virtual machine. Currently, common front-end drivers mainly include VirtIO_net, VirtIO_balloon, VirtIO_scsi, etc. The back-end driver is mainly simulated by QEMU and can also be simulated by the kernel, such as vhost-net. The virtual queue is a virtual queue for data sharing between the front end and the back end, mainly created by the virtual machine front end. The front-end driver is mainly used to receive user requests. After receiving a user request, it will save the request information to the virtual queue and then notify the back-end driver. After receiving the notification, the back-end driver will obtain the request information from the virtual queue for processing. During the operation of the system, the size of the VirtIO virtual queue is fixed. During the startup process, when QEMU simulates the VirtIO device, it will write the size of the virtual queue into the configuration space of the VirtIO device. During the operation of the virtual machine, the size of the virtual queue remains unchanged. This will cause an increase in the number of vmexit times during frequent IO operations, affecting the performance of the virtual machine. When the IO operation frequency is low, it will cause waste of some resources of the virtual queue, resulting in insufficient utilization of virtual machine resources. Summary of the Invention

[0004] To solve the above technical problems or at least partially solve the above technical problems, the present invention provides a method, device, and storage medium for dynamically configuring a VirtIO virtual queue.

[0005] In a first aspect, the present invention provides a method for dynamically configuring a VirtIO virtual queue, including:

[0006] During the VirtIO execution of I / O operations, all virtual queues created on the VirtIO driver are traversed; when the frequency of VirtIO executing I / O operations is high, it is monitored whether each traversed virtual queue is fully utilized. For any virtual queue that is fully utilized, on the premise that the virtual machine memory is sufficient, the size of the virtual queue is expanded, and it is analyzed whether the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard. If so, the expanded size of the virtual queue is retained; otherwise, the virtual queue is restored to its original size before the expansion.

[0007] When the frequency of VirtIO executing I / O operations is low, for any virtual queue whose idle ratio continuously exceeds the preset idle ratio threshold for the first set time, the size of the virtual queue is reduced to reclaim the idle memory resources on the virtual queue, and it is analyzed whether the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard. If so, the original size of the virtual queue before the reduction is restored; otherwise, the reduced size of the virtual queue is retained.

[0008] Furthermore, analyzing whether the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard includes:

[0009] Collect the first speed of I / O before the virtual queue is expanded;

[0010] Collect the second speed of I / O after the virtual queue is expanded;

[0011] Analyze whether the ratio of the part by which the second speed exceeds the first speed to the first speed exceeds the preset first ratio threshold;

[0012] If so, it is considered that the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard.

[0013] Furthermore, analyzing whether the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard includes:

[0014] Calculate the third speed of I / O before the virtual queue is reduced;

[0015] Calculate the fourth speed of I / O after the virtual queue is reduced;

[0016] Analyze whether the ratio of the part by which the third speed exceeds the fourth speed to the fourth speed is greater than the preset second ratio threshold;

[0017] If so, it is considered that the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard.

[0018] Furthermore, during the startup process of the virtual machine, the initial size of the virtual queue is automatically allocated according to the total memory size of the virtual machine and the proportion of each virtual queue of VirtIO in the total memory, and the VirtIO driver constructs the corresponding virtual queue based on the initial size.

[0019] Further, a preset IO operation frequency threshold is set. The real-time IO operation frequency of VirtIO is monitored to collect the real-time IO operation frequency, and if the real-time IO operation frequency is less than the preset IO operation frequency threshold, it is considered that the IO operation frequency of VirtIO is low; otherwise, it is considered that the IO operation frequency of VirtIO is high.

[0020] Further, a unit amount for increasing or decreasing the virtual queue size is preset. Each time the virtual queue size is increased or decreased, the size of one unit amount is increased or decreased.

[0021] Further, a second set time is predefined, and a number threshold is predefined;

[0022] Within the second set time, if the number of times that a virtual queue continuously increases first and then returns to the same virtual queue size reaches the preset number threshold; then the process of increasing the virtual queue size is stopped accordingly;

[0023] Within the second set time, if the number of times that a virtual queue continuously decreases first and then returns to the same virtual queue size reaches the preset number threshold; then the process of decreasing the virtual queue size is stopped accordingly.

[0024] Further, it is detected that the virtual queue continuously increases first and then returns, or continuously decreases first and then returns to the same virtual queue size twice, and the virtual queue size is used as the upper limit or lower limit of the virtual queue size;

[0025] If the virtual queue size reaches the upper limit, the process of increasing the virtual queue is stopped accordingly;

[0026] If the virtual queue size reaches the lower limit, the process of decreasing the virtual queue is stopped accordingly.

[0027] In a second aspect, the present invention provides a VirtIO virtual queue dynamic configuration device, including: a processing unit, a bus unit, and a storage unit. The bus unit is connected to the storage unit and the processing unit. The storage unit stores a computer program, and when the computer program is executed by the processing unit, the VirtIO virtual queue dynamic configuration method described above is implemented.

[0028] In a third aspect, the present invention provides a storage medium for implementing the VirtIO virtual queue dynamic configuration method. The storage medium stores a computer program, and when the computer program is executed by a processor, the VirtIO virtual queue dynamic configuration method described above is implemented.

[0029] The above technical solutions provided by the embodiments of the present invention have the following advantages compared with the prior art:

[0030] During the VirtIO execution of IO operations, all virtual queues created on the VirtIO driver are traversed. When VirtIO executes IO operations at a high frequency, it monitors whether each traversed virtual queue is fully utilized. For any virtual queue that is fully utilized, on the premise that the virtual machine memory is sufficient, the size of the virtual queue is expanded. And if expanding the virtual queue significantly improves the IO performance, the expansion is retained; if it does not significantly improve the IO performance, it is restored to the original size before expansion. The virtual queue that is fully utilized is expanded only when it can improve the IO performance, and when the IO performance cannot be improved, the virtual queue is restored to the original size before expansion. While taking into account improving the IO performance and reducing the number of vmexits, it avoids wasting virtual machine resources caused by excessive expansion of the virtual queue.

[0031] When VirtIO executes IO operations at a low frequency, for any virtual queue whose idle ratio continuously exceeds the preset idle ratio threshold for the first set time, the size of the virtual queue is reduced to reclaim the idle memory resources on the virtual queue. And if reducing the virtual queue does not significantly affect the IO performance, the reduction is retained; when it significantly affects the IO performance, the virtual queue is restored to the original size before reduction. While taking into account reducing the number of vmexits and maintaining the IO performance, it reclaims the idle memory resources of the virtual queue as much as possible to ensure the performance of the virtual machine. Brief Description of the Drawings

[0032] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments in accordance with the present invention, and are used together with the specification to explain the principles of the present invention.

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0034] Figure 1 It is a flowchart of a method for dynamically configuring VirtIO virtual queues provided by an embodiment of the present invention;

[0035] Figure 2 It is a flowchart of expanding the fully utilized virtual queue when VirtIO executes IO operations at a high frequency after the virtual machine starts up, provided by an embodiment of the present invention;

[0036] Figure 3 It is a flowchart of analyzing whether the optimization of IO performance before and after expanding the virtual queue exceeds the set standard, provided by an embodiment of the present invention;

[0037] Figure 4A flowchart provided by an embodiment of the present invention for reducing all idle virtual queues when performing IO operations at a low frequency of VirtIO after the virtual machine is started;

[0038] Figure 5 A flowchart provided by an embodiment of the present invention for analyzing whether the reduction in IO performance before and after the reduction of the virtual queue exceeds a set standard;

[0039] Figure 6 A schematic diagram of a VirtIO virtual queue dynamic configuration device provided by an embodiment of the present invention. Detailed implementation manners

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without further limitations, an element defined by the phrase "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0042] Embodiment 1

[0043] Refer to Figure 1 As shown, the present invention provides a VirtIO virtual queue dynamic configuration method, including:

[0044] During the startup process of the virtual machine, the size of the virtual queue is initialized, and the VirtIO driver creates an initial virtual queue according to the initialized size of the virtual queue. Specifically, during the startup process of the virtual machine, the initial size of the virtual queue is automatically allocated according to the total memory size of the virtual machine and the proportion of each virtual queue of VirtIO in the total memory, and the VirtIO driver constructs a corresponding virtual queue based on the initial size.

[0045] After the virtual machine is started, when performing IO operations at a high frequency of VirtIO, expand the virtual queues that are all in use. In the specific implementation process, refer to Figure 2 As shown, including:

[0046] Traverse all virtual queues created on the VirtIO driver.

[0047] Detect whether VirtIO performs IO operations at a high frequency. In the specific implementation process, a preset IO operation frequency threshold is set, the execution frequency of VirtIO for IO operations is monitored in real time to collect the real-time IO operation frequency, and if the real-time IO operation frequency is less than the preset IO operation frequency threshold, it is considered that VirtIO performs IO operations at a low frequency, otherwise it is considered that VirtIO performs IO operations at a high frequency.

[0048] When the execution frequency of IO operations is high, monitor whether each traversed virtual queue is fully used. If the traversed virtual queue is not fully used, continue to traverse without finishing traversing the virtual queue.

[0049] When it is detected that the traversed virtual queue is fully used, detect whether the current virtual machine memory is sufficient. For any fully used virtual queue, on the premise that the virtual machine memory is sufficient, expand the size of the virtual queue. On the premise that the virtual machine memory is insufficient, do not perform any processing on the virtual queue. In the specific implementation process, a preset unit amount for increasing the size of the virtual queue is set. Each time the size of the virtual queue is expanded, the virtual queue to be expanded is increased by one unit amount of size.

[0050] Analyze whether the IO performance optimization before and after the expansion of the virtual queue exceeds the set standard. If so, the virtual queue retains the expanded size, otherwise the virtual queue restores its original size. In the specific implementation process, refer to Figure 3 As shown, analyzing whether the IO performance optimization before and after the expansion of the virtual queue exceeds the set standard includes:

[0051] Collect the first speed of IO before the virtual queue is expanded;

[0052] Collect the second speed of IO after the virtual queue is expanded;

[0053] Analyze whether the ratio of the part by which the second speed exceeds the first speed to the first speed exceeds the preset first ratio threshold;

[0054] If so, it is considered that the IO performance optimization before and after the expansion of the virtual queue exceeds the set standard, and the virtual queue retains the expanded size;

[0055] If so, it is considered that the IO performance optimization before and after the expansion of the virtual queue does not exceed the set standard, and the virtual queue restores its original size.

[0056] After the virtual machine is started, when VirtIO performs IO operations at a low frequency, reduce the virtual queues that are all idle to reclaim the idle memory resources on the virtual queues. In the specific implementation process, refer to Figure 4 As shown, it includes:

[0057] Traverse all virtual queues created on the VirtIO driver.

[0058] Detect whether VirtIO performs IO operations at a high frequency.

[0059] When the frequency of performing IO operations is low, monitor whether each traversed virtual queue has free memory. Specifically, the method for monitoring whether a virtual queue has free memory is as follows: If the free ratio of the virtual queue continues to exceed the preset free ratio threshold for the first set time, it is considered that the virtual queue has free memory.

[0060] If the traversed virtual queue has free memory, reduce the size of the virtual queue to reclaim the free memory resources on the virtual queue. In the specific implementation process, a preset unit amount for reducing the size of the virtual queue is set. Each time the size of the virtual queue is reduced, the size of the virtual queue to be reduced is reduced by one unit amount.

[0061] If the traversed virtual queue has no free memory, do not perform any processing on this virtual queue.

[0062] Continue the traversal process without traversing all virtual queues.

[0063] Analyze whether the reduction in IO performance before and after reducing the virtual queue exceeds the set standard. If so, restore the virtual queue to its original size; otherwise, keep the reduced size of the virtual queue. In the specific implementation process, refer to Figure 5 As shown, the analysis of whether the reduction in IO performance before and after reducing the virtual queue exceeds the set standard includes:

[0064] Calculate the third speed of IO before the virtual queue is reduced;

[0065] Calculate the fourth speed of IO after the virtual queue is reduced;

[0066] Analyze whether the ratio of the part by which the third speed exceeds the fourth speed to the fourth speed is greater than the preset second ratio threshold;

[0067] If so, it is considered that the reduction in IO performance before and after reducing the virtual queue exceeds the set standard, and restore the virtual queue to its original size;

[0068] Otherwise, it is considered that the reduction in IO performance before and after reducing the virtual queue does not exceed the set standard, and keep the reduced size of the virtual queue.

[0069] During the process of VirtIO performing I / O operations, all virtual queues created on the VirtIO driver are traversed; when the frequency of VirtIO performing I / O operations is high, it is monitored whether each traversed virtual queue is fully utilized. For any fully utilized virtual queue, on the premise that the virtual machine memory is sufficient, the size of the virtual queue is expanded, and if the expansion of the virtual queue significantly improves the I / O performance, the expansion is retained, otherwise it is restored to the original size before the expansion. The virtual queue is expanded only when it can improve the I / O performance, and if the I / O performance cannot be improved, the virtual queue is restored to the original size before the expansion. While taking into account improving the I / O performance and reducing the number of vmexits, it is necessary to avoid wasting virtual machine resources caused by excessive expansion of the virtual queue.

[0070] When the frequency of VirtIO performing I / O operations is low and the idle ratio of the virtual queue continues to exceed the preset idle ratio threshold for the first set time, the size of the virtual queue is reduced to reclaim the idle memory resources on the virtual queue, and if the reduction of the virtual queue does not significantly affect the I / O performance, the reduction is retained, and if it significantly affects the I / O performance, the virtual queue is restored to the original size before the reduction. While taking into account reducing the number of vmexits and maintaining the I / O performance, it is necessary to reclaim the idle memory resources of the virtual queue as much as possible to ensure the performance of the virtual machine.

[0071] In the specific implementation process, when increasing the size of the virtual queue cannot significantly improve the I / O performance, or when reducing the size of the virtual queue will significantly reduce the I / O performance, each time the virtual queue is traversed, an invalid virtual queue change will occur, that is, first increase and then restore, or first decrease and then restore. To avoid this situation, a feasible way is:

[0072] Preset the second set time and the threshold of the number of times in advance;

[0073] Within the second set time, if the number of times a virtual queue continuously increases and then restores to the same virtual queue size reaches the preset threshold of the number of times; then the process of increasing the size of the virtual queue is stopped accordingly.

[0074] Within the second set time, if the number of times a virtual queue continuously decreases and then restores to the same virtual queue size reaches the preset threshold of the number of times; then the process of reducing the size of the virtual queue is stopped accordingly.

[0075] Another feasible way is:

[0076] Detect that the virtual queue continuously increases and then restores twice, or continuously decreases and then restores twice to the same virtual queue size, and take this virtual queue size as the upper limit or lower limit of the virtual queue size.

[0077] If the virtual queue size reaches the upper limit, the process of increasing the virtual queue is stopped accordingly.

[0078] If the size of the virtual queue reaches the lower limit, the reduction process of the virtual queue is stopped accordingly.

[0079] Embodiment 2

[0080] Refer to Figure 6 As shown, an embodiment of the present invention provides a VirtIO virtual queue dynamic configuration device, including: a processing unit, a bus unit, and a storage unit. The bus unit is connected to the storage unit and the processing unit. The storage unit stores a computer program, and when the computer program is executed by the processing unit, the VirtIO virtual queue dynamic configuration method is implemented.

[0081] Embodiment 3

[0082] An embodiment of the present invention provides a storage medium for implementing the VirtIO virtual queue dynamic configuration method. The storage medium stores a computer program, and when the computer program is executed by a processor, the VirtIO virtual queue dynamic configuration method is implemented.

[0083] In the embodiments provided by the present invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, indirect couplings or communication connections of structures or units, and can be in electrical, mechanical or other forms.

[0084] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0086] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for dynamically configuring a VirtIO virtual queue, characterized in that Including: During the process of VirtIO performing I / O operations, all virtual queues created on the VirtIO driver are traversed; when VirtIO performs I / O operations at a high frequency, it monitors whether each traversed virtual queue is fully utilized. For any virtual queue that is fully utilized, on the premise that the virtual machine memory is sufficient, the size of the virtual queue is expanded, and it is analyzed whether the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard. If so, the expanded size of the virtual queue is retained; otherwise, the virtual queue is restored to its original size before the expansion; among them, analyzing whether the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard includes: collecting the first speed of I / O before the virtual queue is expanded; collecting the second speed of I / O after the virtual queue is expanded; analyzing whether the ratio of the part by which the second speed exceeds the first speed to the first speed exceeds a preset first ratio threshold; if so, it is considered that the I / O performance optimization before and after the expansion of the virtual queue exceeds the set standard; When VirtIO performs I / O operations at a low frequency, for any virtual queue whose idle ratio continuously exceeds the preset idle ratio threshold for a first set time, the size of the virtual queue is reduced to reclaim the idle memory resources on the virtual queue, and it is analyzed whether the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard. If so, the virtual queue is restored to its original size before the reduction; otherwise, the reduced size of the virtual queue is retained; among them, analyzing whether the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard includes: calculating the third speed of I / O before the virtual queue is reduced; calculating the fourth speed of I / O after the virtual queue is reduced; analyzing whether the ratio of the part by which the third speed exceeds the fourth speed to the fourth speed is greater than a preset second ratio threshold; if so, it is considered that the reduction in I / O performance before and after the reduction of the virtual queue exceeds the set standard.

2. The VirtIO virtual queue dynamic configuration method according to claim 1, wherein During the startup process of the virtual machine, the initial size of the virtual queue is automatically allocated according to the total memory size of the virtual machine and the proportion of each virtual queue of VirtIO in the total memory, and the VirtIO driver constructs the corresponding virtual queue based on the initial size.

3. The VirtIO virtual queue dynamic configuration method according to claim 1, wherein A preset I / O operation frequency threshold is set, the real-time I / O operation frequency is collected by monitoring the I / O operation frequency of VirtIO in real time, and if the real-time I / O operation frequency is less than the preset I / O operation frequency threshold, it is considered that VirtIO performs I / O operations at a low frequency; otherwise, it is considered that VirtIO performs I / O operations at a high frequency.

4. The VirtIO virtual queue dynamic configuration method according to claim 1, characterized in that A preset unit amount for increasing or decreasing the size of the virtual queue is set, and each time the size of the virtual queue is increased or decreased, the size is increased or decreased by one unit amount.

5. The VirtIO virtual queue dynamic configuration method according to claim 1, characterized in that A second set time is specified in advance, and a number threshold is specified in advance; Within the second set time, if the number of times that a virtual queue continuously increases first and then restores to the same virtual queue size reaches the preset number threshold; then the process of increasing the size of the virtual queue is stopped accordingly; Within the second set time, if the number of times that a virtual queue continuously decreases first and then restores to the same virtual queue size reaches the preset number threshold; then the process of decreasing the size of the virtual queue is stopped accordingly.

6. The VirtIO virtual queue dynamic configuration method according to claim 1, wherein Detect that the virtual queue first increases and then recovers twice in a row, or first decreases and then recovers to the same virtual queue size twice in a row, and use this virtual queue size as the upper or lower limit of the virtual queue size; If the virtual queue size reaches the upper limit, the corresponding increase process of the virtual queue is stopped; If the virtual queue size reaches the lower limit, the corresponding decrease process of the virtual queue is stopped.

7. A VirtIO virtual queue dynamic configuration device, characterized in that It includes: A processing unit, a bus unit, and a storage unit. The bus unit is connected to the storage unit and the processing unit. The storage unit stores a computer program. When the computer program is executed by the processing unit, it implements the VirtIO virtual queue dynamic configuration method as described in any one of claims 1-6.

8. A storage medium for implementing a method for dynamically configuring a VirtIO virtual queue, the storage medium storing a computer program, characterized in that When the computer program is executed by the processor, it implements the VirtIO virtual queue dynamic configuration method as described in any one of claims 1-6.

Images