A heterogeneous virtualization multi-tenant resource isolation and scheduling method and device

Abstract

The present invention relates to a heterogeneous virtualization multi-tenant resource isolation and scheduling method, including: an administrator adding a filter; the administrator creating a tenant and creating a tenant identifier for the tenant; the tenant creating a corresponding tenant group and a tenant template, and formulating a filtering rule for the filter corresponding to the tenant group according to the isolation requirements; the tenant sending a virtual machine creation request, and the filter determining whether the tenant is creating a virtual machine for the first time. If so, according to the requirements of the tenant template, the virtual machine is deployed on the physical host with the largest available memory resource. If not, the filter preferentially selects the physical host where the tenant already exists and determines whether the physical host meets the filtering rule. If so, according to the requirements of the tenant template, the virtual machine is deployed on the physical host. The present invention relates to a heterogeneous virtualization multi-tenant resource isolation and scheduling device. The present invention isolates and aggregates and schedules the virtual machines deployed by the same tenant, without the need for the administrator to configure a host group, improving the tenant isolation efficiency and security.

Description

Technical Field

[0001] The present invention relates to the technical field of computer platform server virtualization design, and in particular to a heterogeneous virtualization multi-tenant resource isolation and scheduling method and device, which is applied to the adaptation technology of the Xinchuang cloud platform based on the domestic heterogeneous mainstream chip server resource pool. Background Art

[0002] Multi-tenant is a fundamental technology used in the cloud, which realizes the scale cost and security of the cloud by sharing hardware or software. The highly shared resource architecture brought by cloud service multi-tenancy can improve resource utilization and reduce the unit resource cost. However, due to the decreased isolation between tenants, multi-tenant technology will bring a series of problems to tenants, especially in the cloud platform environment based on the underlying hardware resources of domestic heterogeneous CPU chips such as LoongArch, Feiteng, Kunpeng, and Haiguang. Taking public cloud as an example, poor sharing isolation among multi-tenants will bring security problems. For example, AWS EC2 and Google GCE, etc., allow their customers to create and run virtual machines on the shared physical infrastructure, which may lead to the coexistence of a virtual machine and a malicious virtual machine, whether intentional or unintentional, on a server. Research shows that when two virtual machines coexist on a physical server, there is a possibility of breaking through the virtualization logical isolation to obtain the confidential information of other virtual machines / users or reducing their performance. If each tenant is assigned a virtual machine and the application is not multi-tenant, performance isolation is achieved through the virtual machine, which results in large-request tenants being unable to obtain more resources than those allocated to their virtual machines, and there may be an idle situation for a certain application instance. If a multi-tenant application cluster is deployed on a physical infrastructure cluster and an instance of the same application runs in each virtual machine and all tenants can access it, then the peak requests of one tenant will significantly affect other tenants. Therefore, how to isolate and schedule virtualized multi-tenant resources in a heterogeneous environment has become an urgent problem to be solved.

[0003] Isolation can be achieved at different levels using different technologies: Physical-level isolation is the most complete isolation, where different tenants use different physical hardware, including computing and storage servers and network devices, etc. Isolation at the operating system level of computing provides relatively high security isolation through virtual machines and, on the other hand, combines other technologies such as Linux cgroup to achieve performance guarantee; isolation at the middleware level of computing, such as sharing underlying resources such as the operating system through containers and using technologies such as Linux namespace to provide general isolation; isolation at the logical level of the network, using methods such as VLAN, VxLAN, GRE, etc. to achieve logical isolation of network traffic; isolation at the application level of storage, such as shared storage like Ceph and Swift, etc., to provide tenant isolation at the application level.

[0004] Traditional OpenStack virtualization tenant isolation is achieved through filters. Administrators need to create a separate host group for each tenant, and this part of the host resources serves these batches of tenants alone. This isolation method has the following defects: 1. The resource requirements of tenants are not fixed. Therefore, when an administrator creates a host group, since the resource requirements of tenants cannot be determined, the resources provided by the host group must be greater than the tenant resource requirements and have a surplus, resulting in waste of host group resources; 2. Host groups can only be created by administrators, and tenants themselves cannot complete it; 3. When the number of tenant hosts is large or increasing, tenant hosts need to expand resources, but tenants cannot complete it independently. Summary of the Invention

[0005] To solve the above problems, the purpose of the present invention is to provide a heterogeneous virtualization multi-tenant resource isolation and scheduling method. It is an isolation based on computing resources / virtual machines, which isolates and aggregates and schedules virtual machines deployed by the same tenant, without the need for administrators to configure host groups, improving tenant isolation efficiency and security.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] Technical Solution 1

[0008] A heterogeneous virtualization multi-tenant resource isolation and scheduling method includes the following steps: The administrator adds filters to the configuration item description file of the nova module; the administrator creates a tenant and creates a tenant identifier for the tenant; the tenant creates corresponding tenant groups for different heterogeneous resources, configures the tenant templates corresponding to the tenant groups, and compiles the filtering rules of the filters corresponding to the tenant groups according to the isolation requirements; the tenant sends a request to create a virtual machine, and the request contains the tenant identifier and the tenant template. The filter determines whether the tenant is creating a virtual machine for the first time. If so, according to the requirements of the tenant template, the virtual machine is deployed on the physical host that meets the requirements of the tenant template and has the largest free memory resource; if not, it is a request to create a virtual machine again. The filter preferentially selects the physical host where the tenant already exists and determines whether the physical host meets the filtering rules. If so, according to the requirements of the tenant template, the virtual machine is deployed on the physical host, so that virtual hosts with the same tenant and the same heterogeneous resources are deployed on the same physical host.

[0009] Preferably, when the virtual machine is re-created, if the physical host that meets the filter requirements has insufficient resources and cannot deploy the virtual machine according to the tenant template, a competition scheduling mechanism is started. Specifically: it is judged whether the current tenant is the preferred tenant of the physical host. If so, the virtual machines deployed by the low-priority tenants on the physical host or the virtual machines without the isolation requirement are transferred from the current physical host to free up sufficient resource space, and then the virtual machine is deployed on the physical host according to the tenant template, and finally the tenant exclusively uses the physical host at the logical level. If not, the virtual machine re-created by the tenant is deployed on another physical host.

[0010] Preferably, the determination criterion for the preferred tenant is: calculate the tenant priority according to the memory ratio, CPU ratio, and storage ratio of all virtual machines created by the tenant on the physical host. If the tenant priority is high, it is the preferred tenant.

[0011] Preferably, after the tenant deploys the virtual machine on the physical host, calculate and adjust the tenant priority level of the tenant on the current physical host.

[0012] Preferably, the judgment process for the physical host that meets the filter requirements to have insufficient resources is: compare the node attributes, memory space, and hard disk space of the physical host with the resources requested by the tenant template; if the resources are sufficient, deploy the virtual machine on the physical host according to the tenant template requirements. If the resources are insufficient, start the competition scheduling mechanism.

[0013] Preferably, when the tenant creates a tenant group, a virtual machine naming rule is also set, which includes a tenant identifier, a CPU architecture, and an incrementing number; the specific steps for deploying the virtual machine on the physical host are: the resource scheduler starts a creation instruction on the physical host, creates a virtual machine according to the tenant template requirements, and then, according to the virtual machine naming rule, supplements the virtual machine name and increments the tail number of the virtual machine name in sequence to avoid duplication.

[0014] Preferably, after the virtual machine is deployed, the relevant information of the virtual machine is synchronized to the openstack database; the filter queries through the database to check whether there is a virtual machine with the same tenant identifier. If not, it is judged that the tenant is creating a virtual machine for the first time. If so, it is a re-creation of the virtual machine.

[0015] In order to solve the above problems, the purpose of the present invention is to provide a heterogeneous virtualization multi-tenant resource isolation scheduling device.

[0016] Technical solution two

[0017] A heterogeneous virtualization multi-tenant resource isolation and scheduling device, including a processor and a memory. A computer program that can be run by the processor is stored in the memory. When the processor runs this computer program, the method steps described in Technical Solution 1 are implemented.

[0018] To solve the above problems, the purpose of the present invention is to provide a computer-readable storage medium.

[0019] Technical Solution 3

[0020] A computer-readable storage medium stores a computer program that can be run by a processor. When the processor runs this computer program, the method steps described in Technical Solution 1 are implemented.

[0021] The present invention has the following beneficial effects:

[0022] 1. For the heterogeneous virtualization multi-tenant resource isolation and scheduling method of the present invention, after the administrator creates a tenant, the tenant group is independently created, the tenant template is set, and the filter rule is compiled. Tenant isolation is completed through the tenant's compilation of the filter rule. Through the parameter matching between the tenant template configuration and the physical host, the same tenant virtual machines are automatically aggregated. There is no need for the administrator to configure the host group, and the virtual machines can be independently extended and automatically aggregated, ensuring that the virtual machines of the tenant are deployed on the same physical host, improving the tenant isolation efficiency, and realizing that the virtual machines of the same tenant in the multi-tenant environment of the cloud platform are all deployed on the same physical host, thereby constructing physical isolation and meeting the high security and high performance requirements of the tenant.

[0023] 2. For the heterogeneous virtualization multi-tenant resource isolation and scheduling method of the present invention, through the tenant group, tenant template, and filter, when the tenant creates a virtual machine, it can independently create a virtual machine on the corresponding physical host according to the differences in heterogeneous resources (pre-selected Kunpeng, Feiteng, or Loongson resources), realizing the automatic aggregation of the same tenant virtual machines in the heterogeneous resource environment and constructing physical isolation.

[0024] 3. For the heterogeneous virtualization multi-tenant resource isolation and scheduling method of the present invention, through the competition scheduling mechanism, the virtual machines deployed by the dominant tenant are highly aggregated, and gradually realize that the tenant exclusively uses the physical host at the logical level, further improving the high security and high performance of the virtual machines of the dominant tenant.

[0025] 4. The heterogeneous virtualization multi-tenant resource isolation and scheduling device and a readable computer storage medium of the present invention have the same beneficial effects as this method. Description of the Drawings

[0026] Figure 1 It is a flowchart of a heterogeneous virtualization multi-tenant resource isolation and scheduling method of the present invention;

[0027] Figure 2 This is a flowchart of the competition scheduling mechanism used in the isolation scheduling method of the present invention. Specific embodiments

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Embodiment 1

[0030] See Figure 1 , a heterogeneous virtualization multi-tenant resource isolation scheduling method, including the following steps:

[0031] Step 1: The administrator adds a filter to the configuration item description file of the nova module. This filter is a default filter (scheduler_default_filters) and is provided for tenants to compile and use. The creation example is as follows: scheduler_default_filters = AggregateInstanceExtraSpecsFilter, RetryFilter, RamFilter, DiskFilter, ComputeFilter, ComputeCapabilitiesFilter, ServerGroupAffinityFilter, AggregateMultiTenancyIsolation, AggregateImagePropertiesIsolation

[0032] Step 2: The administrator creates a tenant and creates a tenant identifier for the tenant. This tenant identifier is generally the tenant ID. The tenant creates corresponding tenant groups for different heterogeneous resources, configures the tenant templates corresponding to the tenant groups, and compiles the filtering rules of the filters corresponding to the tenant groups according to the isolation requirements. When creating the tenant template, the virtual machine naming rule can also be set, which includes the tenant identifier, CPU architecture, and sequentially incrementing number. It is used to rename the newly created virtual machine when the virtual machine deployment is completed, and sequentially increment the tail number of the virtual machine name to avoid duplicate virtual machine names.

[0033] Among heterogeneous resources, only CPUs and GPUs cannot be scheduled. The architectures that can be adopted by a CPU or GPU include the ARM architecture or the X86 architecture. Since the allocation rules of the ARM resource pool and the X86 resource pool are different, ARM virtual machines and X86 virtual machines cannot be aggregated together. Therefore, the present invention proposes to create corresponding tenant groups according to the CPU or GPU architecture and configure tenant templates corresponding to the tenant groups. For example, if a tenant needs to create an ARM virtual machine, a tenant group corresponding to the ARM virtual machine needs to be created, and the tenant template corresponding to the tenant group is configured. The configuration parameters of the tenant template mainly include tenant identification, virtual machine naming, CPU type, number of CPUs, memory requirements, hard disk requirements, etc. If the tenant also needs to create an X86 virtual machine, a corresponding tenant group needs to be re-created and the corresponding tenant template needs to be configured.

[0034] The filter added by the administrator is the default filter (scheduler_default_filters). After this filter is added, it is provided to tenants with isolation requirements. When creating a filtering rule, use the AggregateInstanceExtraSpecsFilter parameter to configure the filter for filtering. If there is no isolation requirement, this filter can be not configured. The filter configuration instructions are as follows:

[0035] Filter configuration AggregateInstanceExtraSpecsFilter: Configure custom filtering attributes and complete the filter configuration through the Flavor metadata filter.

[0036] Filter configuration RetryFilter, RamFilter, DiskFilter, ComputeFilter: Filter nodes where the attempt fails, nodes with memory or hard disk issues, and nodes where the service is not working properly;

[0037] Filter configuration ComputeCapabilitiesFilter: Filter heterogeneous resources through metadata, such as different CPU types like x86_64, ARM64, longarch, etc.;

[0038] Filter configuration ImagePropertiesFilter: Filter matching physical hosts according to the properties of the selected image. Select KVM or QEMU by configuring metadata.

[0039] Configure the tenant tenant template and set the virtual host naming rule. The example is as follows:

[0040] | Property | Value |

[0041] +----------------------------+-------------------+

[0042] | OS-FLV-DISABLED:disabled | False |

[0043] | OS-FLV-EXT-DATA:ephemeral | 0 |

[0044] | disk | 80 |

[0045] | extra_specs | {u'ssd': u'true'} |

[0046] | id | 6 |

[0047] | name |Tenant_ID.arrch64.xx|

[0048] | os-flavor-access:is_public | True |

[0049] | ram | 8192 |

[0050] | rxtx_factor | 1.0 |

[0051] | swap | |

[0052] | vcpus | 4 |

[0053] Step 3: The tenant issues a request to create a virtual machine, and this request contains the tenant identifier and the tenant template. The filter determines whether this tenant is creating a virtual machine for the first time. Preferably, the tenant template is provided with a template identifier, and this template identifier contains heterogeneous resource types. When receiving the request to create a virtual machine, the filter determines the resources required by the tenant according to the tenant identifier and the template identifier. After the virtual machine is deployed, the relevant information of the virtual machine is synchronized to the openstack database; the filter can query through the database to check whether there is a virtual machine with the same tenant identifier; if not, it determines that this tenant is creating a virtual machine for the first time, and if so, it is creating a virtual machine again.

[0054] For the tenant creating a virtual machine for the first time, the resource scheduler deploys the virtual machine on the physical host that meets the requirements of the tenant template and has the largest available memory, so as to provide sufficient space for the subsequent resource aggregation of this tenant;

[0055] For a tenant who recreates a virtual machine, this filter preferentially selects a physical host where the tenant already exists and determines whether the physical host meets the filtering rules. If so, according to the requirements of the tenant template, the virtual machine is deployed on this physical host, so that virtual hosts with the same tenant and heterogeneous resources are deployed on the same physical host.

[0056] The specific steps of deploying the virtual machine on this physical host are as follows: The resource scheduler starts a creation instruction on this physical host. According to the requirements of the tenant template, a virtual machine is created. Then, according to the naming rule of the virtual machine, the virtual machine name is supplemented, and the tail number of the virtual machine name is incremented sequentially to avoid duplication. Virtual machines of the same tenant form a virtual machine cluster in the database record, serving as the resource pool of this tenant. The relevant information of the deployed virtual machine is synchronized to the openstack database and added to the tenant resource pool.

[0057] In this embodiment, the tenant can create a tenant group by itself according to heterogeneous resources, set a tenant template and compile a filter rule to perform aggregated scheduling on the virtual machines created by this tenant. It realizes that the tenant can create or expand virtual machines independently and can perform aggregated scheduling on the virtual machines created by this tenant through the self-compiled filter rule, ensuring that the virtual machines of this tenant are deployed on the same physical host, and realizing that the virtual machines of the same tenant in a multi-tenant environment are all deployed on the same physical host, thereby building physical isolation to meet the high security and high performance requirements of the tenant. Further, in this embodiment, through the tenant group, tenant template and filter, when the tenant creates a virtual machine, it can independently create a virtual machine on the corresponding physical host according to the differences in heterogeneous resources (pre-selected Kunpeng, Feiteng or Loongson resources), realizing automatic aggregation of virtual machines of the same tenant in a heterogeneous resource environment and building physical isolation.

[0058] Embodiment 2

[0059] This embodiment solves the problem of how to enable the dominant tenant on a physical host to exclusively use the physical host at the logical level when the resources of the physical host are exhausted.

[0060] Please refer to Figure 2For a request from a tenant to create a virtual machine again, this filter preferentially selects a physical host where the tenant already exists and determines whether the physical host meets the filtering rules. If it does, further determine whether there is insufficient resources on the physical host. Specifically: compare the node attributes, memory space, and hard disk space of the physical host with the resources requested in the tenant template. If the resources are sufficient, deploy the virtual machine on the physical host according to the requirements of the tenant template. If the resources are insufficient, start the competition scheduling mechanism: determine whether the current tenant is the dominant tenant of the physical host. If so, move the virtual machines deployed by low-priority tenants or virtual machines without the isolation requirement on the physical host away from the current physical host to free up enough resource space, and then deploy the virtual machine on the physical host according to the tenant template, ultimately enabling the tenant to exclusively use the physical host at the logical level. If not, deploy the virtual machine created again by the tenant on another physical host.

[0061] The determination criterion for the dominant tenant is: calculate the tenant priority based on the memory ratio, CPU ratio, and storage ratio of all virtual machines created by the tenant on the physical host. If the tenant priority is high, it is the dominant tenant. Preferably, after the tenant completes the virtual machine deployment on the physical host, calculate and adjust the tenant priority level of the tenant on the current physical host.

[0062] In the second embodiment, through the competition scheduling mechanism, the virtual machines deployed by the dominant tenant are highly aggregated, gradually enabling the tenant to exclusively use the physical host at the logical level, and further improving the high security and high performance of the virtual machines of the dominant tenant.

[0063] Based on the same inventive concept, the present application provides a heterogeneous virtualization multi-tenant resource isolation scheduling device corresponding to the first and second embodiments.

[0064] Embodiment Three

[0065] A heterogeneous virtualization multi-tenant resource isolation scheduling device includes a processor and a memory. The memory stores a computer program that can be run by the processor. When the processor runs this computer program, the method steps described in the first and second embodiments are implemented, as Figure 1 and Figure 2 shown.

[0066] Based on the same inventive concept, the present application provides a computer-readable storage medium corresponding to the first and second embodiments.

[0067] Embodiment Four

[0068] A computer-readable storage medium stores a computer program that can be run by the processor. When the processor runs this computer program, the method steps described in the first and second embodiments are implemented, asFigure 1 and Figure 2 as shown

[0069] For the implementation steps not elaborated in Embodiment Three and Embodiment Four, refer to Embodiment One and Embodiment Two. Meanwhile, Embodiment Three and Embodiment Four have the same beneficial effects as Embodiment One and Embodiment Two.

[0070] The above are only specific implementation manners of the present invention, and do not thus limit the patent scope of the present invention. Any equivalent structural transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present invention.

Claims

1. A heterogeneous virtualization multi-tenant resource isolation and scheduling method, characterized in that, it includes the following steps: The administrator adds a filter in the configuration item description file of the nova module; The administrator creates a tenant and creates a tenant identifier for this tenant; The tenant creates corresponding tenant groups for different heterogeneous resources, configures the tenant templates corresponding to the tenant groups, and compiles the filtering rules of the filters corresponding to the tenant groups according to the isolation requirements; The tenant issues a virtual machine creation request, which contains the tenant identifier and the tenant template, and the filter determines whether this tenant is creating a virtual machine for the first time, if so, according to the requirements of the tenant template, deploy the virtual machine on the physical host that meets the requirements of the tenant template and has the largest free memory resource; if not, it is to create a virtual machine again. This filter preferentially selects the physical host where this tenant already exists and judges whether this physical host meets this filtering rule. If so, according to the requirements of the tenant template, deploy the virtual machine on this physical host, so that virtual hosts with the same tenant and the same heterogeneous resources are deployed on the same physical host.

2. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 1, characterized in that: When creating the virtual machine again, if the physical host that meets the requirements of the filter has insufficient resources and cannot deploy the virtual machine according to the tenant template, start the competition scheduling mechanism, specifically: judge whether the current tenant is the dominant tenant of this physical host. If so, then transfer the virtual machines deployed by low-priority tenants or virtual machines without the isolation requirements on this physical host, vacate enough resource space, and then deploy the virtual machine on this physical host according to the tenant template, and finally make this tenant exclusively use this physical host at the logical level. If not, then deploy the virtual machine created again by this tenant on another physical host.

3. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 2, characterized in that: The determination criterion of the dominant tenant is: calculate the tenant priority according to the memory ratio, cpu ratio, and storage ratio of all virtual machines created by the tenant on this physical host. If the tenant priority is high, it is the dominant tenant.

4. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 2, characterized in that: After the tenant completes the virtual machine deployment on the physical host, calculate and adjust the tenant priority level of this tenant on the current physical host.

5. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 2, characterized in that: The judgment process for the physical host that meets the requirements of the filter to have insufficient resources is: compare the node attributes, memory space, and hard disk space of this physical host with the resources applied for by the tenant template; If the resources are sufficient, deploy the virtual machine on this physical host according to the requirements of the tenant template. If the resources are insufficient, start the competition scheduling mechanism.

6. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 1, characterized in that: When the tenant creates a tenant group, a virtual machine naming rule is also set, which includes a tenant identifier, a CPU architecture, and a sequentially incrementing number; the specific steps for deploying the virtual machine on the physical host are as follows: The resource scheduler starts a creation instruction on the physical host, creates a virtual machine according to the requirements of the tenant template, and then supplements the virtual machine name according to the virtual machine naming rule, and sequentially increments the tail number of the virtual machine name to avoid duplication.

7. A heterogeneous virtualization multi-tenant resource isolation and scheduling method according to claim 1, characterized in that: After the virtual machine is deployed, the relevant information of the virtual machine is synchronized to the openstack database; the filter retrieves and queries the database to check whether there is a virtual machine with the same tenant identifier. If not, it is determined that the tenant is creating a virtual machine for the first time. If so, it is a re-creation of the virtual machine.

8. A heterogeneous virtualization multi-tenant resource isolation and scheduling device, characterized in that: It includes a processor and a memory. The memory stores a computer program that can be run by the processor. When the processor runs this computer program, the method steps described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that can be run by the processor. When the processor runs this computer program, the method steps described in any one of claims 1 to 7 are implemented.

Images