A multi-tenant resource isolation method and device in an ascension server cluster

Abstract

The application relates to the field of resource isolation of Ascend servers, in particular to a multi-tenant resource isolation method and device in an Ascend server cluster, which comprises the following steps: creating a virt-api; creating a virt-controller; creating a virt-handler; creating a plurality of virt-launchers; establishing communication between the virt-handler and all the virt-launchers; creating a qemu; and establishing connection between the qemu and an Ascend chip. The kubevirt is applied to the Ascend server, so that the kubevirt can support the Ascend npu, compatibility adaptation problems are solved, the Ascend platform can also allocate computing power in the mode of virtual machines, the virtual machines can be arranged by using Kubernetes, the virtual machines of the Ascend platform can have a global resource perspective, computing power resources can be allocated across hosts, and multi-tenant isolation is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of resource isolation of Ascend servers, in particular to a multi-tenant resource isolation method and device in an Ascend server cluster. BACKGROUND

[0002] On the Ascend platform, the allocation of NPU resources is mainly realized through direct allocation of bare metal servers, containerization and hardware virtualization, and the characteristics of these allocation methods are as follows.

[0003] 1. For a bare metal server, all NPU resources of a single server and the server are allocated to a single task. The advantage of this method is that the degree of resource isolation is high, there is no interference between tasks, and the performance is high. The disadvantage is that the resource utilization is low, which may cause resource waste, and the resources cannot be dynamically adjusted, which requires manual pre-configuration.

[0004] 2. For the containerization method, the use of NPU memory and bandwidth by the container is limited by Linux Cgroup. On the container orchestration platform Kubernetes, Kubernetes Device Plugin is used to abstract NPU resources as schedulable "devices" to realize container-level allocation. The advantage is that it supports dynamic expansion and contraction, and the resource utilization is high in a multi-task scenario. The disadvantage is that the degree of resource isolation is relatively low, and there is a certain degree of competition for some host resources such as bandwidth and disk.

[0005] 3. For virtual machines, the server host is divided into one or more virtual machines, and the NPU is transmitted to the virtual machine to achieve performance close to a physical machine. The advantage of this method is that the isolation level is high, and the disadvantage is that the overhead of creating, destroying, starting and stopping a virtual machine is relatively large. Compared with containers, it does not currently support cross-host migration.

[0006] For the above-mentioned several resource allocation methods that the Ascend server can currently support, they all have certain disadvantages, such as the bare metal allocation method relies on manual configuration, the containerization method can realize tenant resource isolation, but the isolation degree is relatively low, and there is a problem of resource competition between different tenants, and the virtual machine method only supports the qemu and kvm solution on a single node, and cannot allocate resources by tenant. Based on this, the inventors have developed a multi-tenant resource isolation method and device in an Ascend server cluster. SUMMARY

[0007] In order to better support the Ascend platform by Kubevirt, the application provides a multi-tenant resource isolation method and device in an Ascend server cluster, which applies kubevirt to the Ascend server, so that kubevirt can support the Ascend npu, not only solves the compatibility adaptation problem, but also enables the Ascend platform to allocate computing power in the form of a virtual machine, and the virtual machine can be arranged by using Kubernetes, so that the virtual machine of the Ascend platform can have a global resource perspective and can allocate computing power resources across hosts, and multi-tenant isolation.

[0008] In a first aspect, the application provides a multi-tenant resource isolation method in an Ascend server cluster, which adopts the following technical solution:

[0009] A multi-tenant resource isolation method in an Ascend server cluster includes the following steps:

[0010] Create virt-api, which is used to provide the API interface of kubevirt and serve as an entrance for users or components to interact with kubevirt;

[0011] Create virt-controller, which is the core control component of kubevirt and is used to manage the entire life cycle of virtual machines and realize automatic control of virtual machine resources;

[0012] Create virt-handler, which is used to manage VM instances on the Pod node;

[0013] Create multiple virt-launchers, which are used to manage virtual machines and are responsible for interacting with qemu virtual tools, mapping Kubernetes device resources to virtual machines, and managing the start and stop of virtual machines;

[0014] Establish communication between virt-handler and all virt-launchers to monitor and control the VM state;

[0015] Create qemu, which is used to simulate the hardware devices required by the virtual machine;

[0016] Establish the connection between qemu and the Ascend chip, add support for the Ascend chip in the vfio_bar_quirk_setup function of qemu, and judge whether it is an Ascend chip by vendor_id and device_id, if so, initialize the memory area required by the Ascend chip and register with BAR.

[0017] Preferably, the device resources include CPU, memory, storage, and network.

[0018] Preferably, the hardware device includes a CPU, memory, hard disk, and network card.

[0019] Secondly, this application provides a multi-tenant resource isolation device for an Ascend server cluster, which adopts the following technical solution:

[0020] A multi-tenant resource isolation device in an Ascend server cluster includes the following modules:

[0021] The virt-api module is used to create virt-api, which provides the API interface for kubevirt and serves as the entry point for users or components to interact with kubevirt.

[0022] The virt-controller module is used to create virt-controllers. Virt-controllers are the core control component of kubevirt, responsible for managing the entire lifecycle of virtual machines and enabling automated control of virtual machine resources.

[0023] The virt-handler module is used to create virt-handlers, which are used to manage the VM instances on the Pod node.

[0024] The virt-launcher module is used to create multiple virt-launchers. The virt-launcher is used to manage virtual machines and is responsible for interacting with the QEMU virtualization tool, mapping Kubernetes-allocated device resources to virtual machines, and managing the start and stop of virtual machines.

[0025] The communication establishment unit is used to establish communication between virt-handler and all virt-launchers, and to monitor and control the VM state.

[0026] The Qemu module is used to create Qemu instances, which are used to emulate the hardware devices required for virtual machines.

[0027] The Ascend chip support module is used to establish a connection between QEMU and the Ascend chip. It adds support for the Ascend chip in the vfio_bar_quirk_setup function of QEMU. It determines whether it is an Ascend chip by using vendor_id and device_id. If it is, it initializes the memory area required by the Ascend chip and registers it with BAR.

[0028] Preferably, the device resources include CPU, memory, storage, and network.

[0029] Preferably, the hardware device comprises a CPU, a memory, a hard disk, and a network card.

[0030] In a third aspect, the present application provides an electronic device, which adopts the technical scheme as follows:

[0031] An electronic device, comprising:

[0032] one or more processors;

[0033] a memory;

[0034] one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: perform the multi-tenant resource isolation method in an Ascend server cluster according to any possible implementation manner of the first aspect.

[0035] In a fourth aspect, the present application provides a computer readable storage medium, which adopts the technical scheme as follows:

[0036] A computer readable storage medium, comprising: a computer program stored therein and capable of being loaded and executed by a processor to implement the multi-tenant resource isolation method in an Ascend server cluster according to any possible implementation manner of the first aspect.

[0037] In summary, the present application includes the following beneficial technical effects:

[0038] 1. The present application applies kubevirt to the Ascend server, so that kubevirt can support the Ascend npu, not only solves the compatibility adaptation problem, but also makes the Ascend platform be able to allocate computing power in the form of virtual machine, and the virtual machine can be arranged by Kubernetes, so that the virtual machine of the Ascend platform can have a global resource perspective, can allocate computing power resources across hosts, and multi-tenant isolation.

[0039] 2. Compared with the current bare metal server distribution, containerization, and hardware virtualization on a single server available on the Ascend platform, the present application not only solves the compatibility adaptation of kubevirt and the Ascend server, so that the virtualization of the Ascend platform can realize unified management of cluster resources by means of Kubernetes, but also realizes that multiple virtual machines can isolate resources by tenants, and by means of the scheduling ability of Kubernetes, the virtual machines can be scheduled to the optimal node according to node resources, affinity and other rules, and the resources are more reasonably utilized.

[0040] 3. The application allocates resources in units of virtual machines by using kubevirt, and different virtual machines allocated by different tenants are divided by a network plug-in in Kubernetes, so that virtual machines of different tenants cannot access each other, while virtual machines under the same tenant can be interconnected. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a system diagram of the multi-tenant resource isolation method in the embodiment of the application.

[0042] Figure 2 is a structural schematic diagram of establishing connection between qemu and Ascend chips in step S7 of the embodiment of the application.

[0043] Figure 3 is an architecture diagram of kubevirt in the embodiment of the application. DETAILED DESCRIPTION

[0044] The application will be further described in detail below with reference to the accompanying drawings.

[0045] The embodiment of the application discloses a multi-tenant resource isolation method in an Ascend server cluster.

[0046] Referring to Figure 1 The multi-tenant resource isolation method in the Ascend server cluster includes the following steps:

[0047] S1, creating virt-api, virt-api is used to provide an API interface of kubevirt and serve as an entrance for users or components to interact with kubevirt;

[0048] S2, creating virt-controller, virt-controller is a core control component of kubevirt, used to be responsible for managing the entire life cycle of a virtual machine and realizing automatic control of virtual machine resources;

[0049] S3, creating virt-handler, virt-handler is used to manage VM instances of a Pod node where virt-handler is located;

[0050] S4, creating a plurality of virt-launcher, virt-launcher is used to manage virtual machines and be responsible for interacting with qemu virtual tools, mapping Kubernetes allocated device resources into virtual machines, and managing start and stop of virtual machines;

[0051] Among them, the device resources include cpu, memory, storage, network and other resources.

[0052] The virt-handler interacts with and manages the virt-launcher in each Pod instance, thereby managing the virtual machine, and can interact with the qemu virtual tool to map the device resources allocated by Kubernetes into the virtual machine and manage the start and stop of the virtual machine;

[0053] S5, the virt-handler communicates with all virt-launchers to monitor and control the state of the VM;

[0054] S6, create qemu, which is used to simulate the hardware devices required by the virtual machine;

[0055] The hardware devices include CPU, memory, hard disk, network card, etc.

[0056] S7, establish the connection between qemu and the Ascend chip, add support for the Ascend chip in the vfio_bar_quirk_setup function of qemu, judge whether it is an Ascend chip by vendor_id and device_id, if it is, initialize the memory area required by the Ascend chip and register with BAR, for details, please refer to Figure 2 .

[0057] Through the above steps, a kubevirt is constructed, and the architecture of kubevirt is as shown in Figure 3 , which can be applied to and support the Ascend server.

[0058] The kubectl command line tool inputs the command input by the user to the API-server server, wherein the virt-controller is used to manage the entire life cycle of the virtual machine and realize the automatic control of the virtual machine resources, and the virt-api is used to interact with the virt-handler, thereby mapping the corresponding cpu, memory, storage, network and other device resources allocated by Kubernetes for the virt-controller into the virtual machine based on the command input by the user and managing the start and stop of the virtual machine.

[0059] The virt-launcher can interact with the qemu tool through the libvirtd virtualization management software.

[0060] The virt-handler also interacts with the API-server server to present the allocated resources to the user.

[0061] In the kubeviet, based on the use of each tenant, each tenant can be allocated corresponding device resources, so that resource isolation is formed between multi-tenants, virtual machines under the same tenant can be interconnected, and different tenants are isolated from each other, so that virtual machines of different tenants cannot access each other, to form resource isolation between multi-tenants. And it also adds corresponding support for Ascend chips in the vfio_bar_quirk_setup function of qemu, such as judging whether it is an Ascend chip by vendor_id and device_id, if so, initializing the memory area required by the chip and registering it to BAR.

[0062] In addition, the embodiment of the application further discloses a multi-tenant resource isolation device in an Ascend server cluster, which is used to implement the multi-tenant resource isolation method in the Ascend server cluster.

[0063] The multi-tenant resource isolation device in the Ascend server cluster comprises the following modules or units:

[0064] The virt-api module is used to create a virt-api, and the virt-api is used to provide an API interface of kubevirt and serve as an entrance for users or components to interact with kubevirt.

[0065] The virt-controller module is used to create a virt-controller, and the virt-controller is a core control component of kubevirt and is used to manage the entire life cycle of a virtual machine and realize automatic control of virtual machine resources.

[0066] The virt-handler module is used to create a virt-handler, and the virt-handler is used to manage VM instances of a Pod node.

[0067] The virt-launcher module is used to create a plurality of virt-launchers, and the virt-launcher is used to manage virtual machines and is responsible for interacting with a qemu virtual tool, mapping device resources allocated by Kubernetes into virtual machines, and managing start and stop of the virtual machines.

[0068] The communication establishment unit is used to establish communication between the virt-handler and all virt-launchers, and monitor and control VM states.

[0069] The Qemu module is used to create a qemu, and the qemu is used to simulate hardware devices required by a virtual machine.

[0070] The Ascend chip support module is configured to establish a connection between the qemu and the Ascend chip, and support for the Ascend chip is added in a vfio_bar_quirk_setup function of the qemu, whether the Ascend chip is determined according to a vendor_id and a device_id, and if so, a memory area required by the Ascend chip is initialized and registered to a BAR.

[0071] The electronic device provided in the embodiments of the present application includes a processor and a memory. The processor and the memory are connected, for example, through a bus. Optionally, the electronic device can further include a transceiver. It should be noted that the transceiver in actual application is not limited to one, and the structure of the electronic device does not constitute a limitation on the embodiments of the present application.

[0072] The processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0073] The bus can include a path for transmitting information between the above-mentioned components. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0074] The memory can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto.

[0075] The memory is used to store application program codes for executing the scheme of the present application, and the processor is used to control the execution. The processor is used to execute the application program codes stored in the memory to realize the content shown in the embodiment of the method for isolating multi-tenant resources in an Ascend server cluster.

[0076] The electronic device includes, but is not limited to, a mobile terminal of a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (for example, a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. It can also be a server or the like.

[0077] The application further provides a computer readable storage medium, which stores a computer program. When the computer program is run on a computer, the computer can execute the corresponding content in the method for multi-tenant resource isolation in an Ascend server cluster.

[0078] It should be understood that, although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

Claims

1. A method for multi-tenant resource isolation in an Ascend server cluster, characterized in that, Includes the following steps: Create virt-api, which provides the API interface for kubevirt and serves as the entry point for users or components to interact with kubevirt; Create virt-controller, which is the core control component of kubevirt. It is responsible for managing the entire lifecycle of virtual machines and realizing automated control of virtual machine resources. Create a virt-handler, which is used to manage the VM instances on the Pod node. Multiple virt-launchers are created. Each virt-launcher manages virtual machines and is responsible for interacting with the QEMU virtualization tool, mapping Kubernetes-allocated device resources to virtual machines, and managing the start and stop of virtual machines. Establish communication between virt-handler and all virt-launchers to monitor and control VM status; Create a QEMU instance, which is used to emulate the hardware devices required for virtual machines. Establish a connection between QEMU and the Ascend chip. Add support for the Ascend chip in the vfio_bar_quirk_setup function of QEMU. Determine whether it is an Ascend chip by using vendor_id and device_id. If it is, initialize the memory area required by the Ascend chip and register it with BAR.

2. The method of claim 1, wherein, The device resources include CPU, memory, storage, and network.

3. The method of claim 1, wherein, The hardware devices include a CPU, memory, hard drive, and network card.

4. A multi-tenant resource isolation device in an Ascend server cluster, characterized in that, Includes the following modules: The virt-api module is used to create virt-api, which provides the API interface for kubevirt and serves as the entry point for users or components to interact with kubevirt. The virt-controller module is used to create virt-controllers. Virt-controllers are the core control component of kubevirt, responsible for managing the entire lifecycle of virtual machines and enabling automated control of virtual machine resources. The virt-handler module is used to create virt-handlers, which are used to manage the VM instances on the Pod node. The virt-launcher module is used to create multiple virt-launchers. The virt-launcher is used to manage virtual machines and is responsible for interacting with the QEMU virtualization tool, mapping Kubernetes-allocated device resources to virtual machines, and managing the start and stop of virtual machines. The communication establishment unit is used to establish communication between virt-handler and all virt-launchers, and to monitor and control the VM state. The Qemu module is used to create Qemu instances, which are used to emulate the hardware devices required for virtual machines. The Ascend chip support module is used for establishing the connection between qemu and the Ascend chip, and the support for the Ascend chip is added in a vfio_bar_quirk_setup function of qemu, whether the Ascend chip is judged through a vendor_id and a device_id, if yes, the memory area required by the Ascend chip is initialized and is registered to a BAR.

5. The multi-tenant resource isolation device in an Ascend server cluster according to claim 4, characterized in that, The device resources include a CPU, a memory, a storage, and a network.

6. The multi-tenant resource isolation device in an Ascend server cluster according to claim 4, characterized in that, The hardware device includes a CPU, a memory, a hard disk, and a network card.

7. An electronic device, comprising: It includes: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to execute the multi-tenant resource isolation method in an Ascend server cluster according to any one of claims 1 to 3.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the multi-tenant resource isolation method in an Ascend server cluster according to any one of claims 1 to 3.

Images