Data protection with static resource partitioning for data processing accelerators

Abstract

Embodiments of the present disclosure disclose methods and systems for a virtualized environment of a data processing (DP) accelerator. In one embodiment, the data processing (DP) accelerator includes: one or more static partition resources and one or more virtual functions (VFs), each virtual function being associated with one of the one or more static partition resources. A virtual machine (VM) of a host is assigned one of the one or more VFs to access the static partition resource associated with the assigned VF. The VM cannot access the rest of the one or more static partition resources of the DP accelerator.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to data protection. More specifically, embodiments of the present invention relate to systems and methods for data protection with static resource partitioning for a data processing (DP) accelerator. Background Art

[0002] Sensitive transactions are increasingly being performed by data processing (DP) accelerators, such as artificial intelligence (AI) accelerators or coprocessors. This increases the need for better partitioning / isolation between DP accelerators.

[0003] For example, when a communication channel between DPAs is possible, a malicious DPA installed in an existing cluster may obtain AI models / data from other DPAs without permission, so that the AI ​​models and data processed by the DPA may be unprotected. Summary of the invention

[0004] The present application provides a data processing (DP) accelerator, which may include: one or more static partition resources; and one or more virtual functions (VFs), each virtual function being associated with one of the one or more static partition resources, wherein a virtual machine (VM) of a host is assigned one of the one or more VFs to access the static partition resources associated with the assigned VF, and wherein the VM cannot access the rest of the one or more static partition resources of the DP accelerator.

[0005] The present application provides a data processing (DP) system, which may include: a host; and a DP accelerator connected to the host. The DP accelerator may include: one or more static partition resources; and one or more virtual functions (VFs), each virtual function being associated with one of the one or more static partition resources, wherein a virtual machine (VM) of the host is allocated with one of the one or more VFs to access the static partition resource associated with the allocated VF, and wherein the VM cannot access the rest of the one or more static partition resources of the DP accelerator.

[0006] The present application provides a computer-implemented method, which may include: receiving a request from an application through a virtual function (VF) to execute instructions using resources at a data processing (DP) accelerator through a DP accelerator, wherein the reception is direct communication from a virtual machine (VM), wherein the VF is assigned to only one VM, and the VF is one of multiple VFs of the DP accelerator; executing instructions and generating results using DP accelerator resources based on the request; determining one or more memory addresses of the VM to store the results; and transmitting the results to the VM using the VF based on the one or more memory addresses of the VM.

[0007] The present application provides a computer program product, which may include a computer program, which when executed by a processor can implement the following method: receiving a request from an application through a virtual function (VF) to execute instructions using resources at a data processing (DP) accelerator through a DP accelerator, wherein the reception is direct communication from a virtual machine (VM), wherein the VF is only assigned to one VM, and the VF is one of multiple VFs of the DP accelerator; executing instructions and generating results using DP accelerator resources based on the request; determining one or more memory addresses of the VM to store the results; and transmitting the results to the VM using the VF based on the one or more memory addresses of the VM. Among them, the instruction may be an instruction for training an AI model or performing reasoning based on an AI model; the hardware I / O information of the VF may be located at a driver of the VM at a host hosting an application running on the VM; the DP accelerator may include an SR-IOV pass-through device; the VM of the host may initially be assigned a VF by a virtual machine manager (VMM) of the host; data packets may be transmitted between the VM and a virtual function corresponding to the VM without passing through the VMM; and data packets may be transmitted between the VM and the VF corresponding to the VM via direct memory access (DMA) of a processing unit independent of the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the invention are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0009] Figure 1 is a block diagram illustrating an example of a system configuration of a host and a data processing (DP) accelerator according to some embodiments.

[0010] Figure 2 is a block diagram illustrating an example of a host communicating with a data processing (DP) accelerator according to one embodiment.

[0011] Figure 3 is a block diagram illustrating an example of static resource partitioning of a DP accelerator according to one embodiment.

[0012] Figure 4 is a flow chart illustrating an exemplary method of a DP accelerator according to one embodiment.

[0013] Figure 5 is a block diagram illustrating an example of dynamic resource isolation for a DP accelerator according to one embodiment.

[0014] Figure 6 is a flow chart illustrating an exemplary method of a DP accelerator according to one embodiment. DETAILED DESCRIPTION

[0015] Various embodiments and aspects of the present invention will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and accompanying drawings may illustrate the present invention, but are not intended to limit the present invention. Many specific details are described to provide a thorough understanding of various embodiments of the present invention. However, in some instances, in order to provide a concise discussion of embodiments of the present invention, known or conventional details are not described.

[0016] References to "one embodiment" or "an embodiment" in the specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0017] According to a first aspect of the present disclosure, a data processing (DP) accelerator includes: one or more static partition resources and one or more virtual functions (VFs), each of the one or more VFs being associated with one of the one or more static partition resources. A virtual machine (VM) of a host is assigned one of the one or more VFs to access the static partition resource associated with the assigned VF. The VM cannot access the rest of the one or more static partition resources of the DP accelerator.

[0018] According to a second aspect of the present disclosure, a virtual function (VF) receives a request from an application to execute an instruction by a data processing (DP) accelerator using resources of a DP accelerator from a virtual machine (VM), wherein the reception is a direct communication, wherein the VF is dedicated to the VM, and the VF is one of a plurality of VFs of the DP accelerator. The DP accelerator executes the instruction based on the request using the DP accelerator resources and generates a result. The DP accelerator determines one or more memory addresses of the VM. The DP accelerator transmits the result to the VM using the VF based on the one or more memory addresses of the VM.

[0019] According to a third aspect of the present disclosure, a data processing (DP) accelerator includes: a resource management unit and one or more resources managed by the resource management unit. The DP accelerator includes: one or more virtual functions (VFs), each virtual function is associated with one of the one or more resources, wherein a virtual machine (VM) of a host is assigned one of the one or more VFs to access the resources associated with the assigned VF, and wherein the VM cannot access the rest of the one or more resources.

[0020] According to a fourth aspect of the present disclosure, a virtual function (VF) receives a request for a data processing (DP) accelerator resource from an application, wherein the reception is a direct communication from a virtual machine (VM), and the VF is dedicated to the VM, wherein the VF is one of a plurality of VFs of the DP accelerator. The DP accelerator determines a first resource isolation of the DP accelerator assigned to the VF. The DP accelerator determines that the first resource isolation does not satisfy the size of the request. The DP accelerator dynamically updates the first resource isolation to a second resource isolation through a resource management unit of the DP accelerator to satisfy the size of the request.

[0021] Figure 1 is a block diagram illustrating an example of a system configuration of a host and a data processing (DP) accelerator according to some embodiments. Figure 1 , the system configuration 100 includes, but is not limited to: one or more client devices 101 and 102 communicatively connected to the DP server 104 via a network 103. The client devices 101 and 102 can be any type of client device, such as a personal computer (e.g., a desktop computer, a laptop computer, and a tablet computer), a "thin" client, a personal digital assistant (PDA), a web-enabled device, a Smartwatch or a mobile phone (e.g., a Smartphone), etc. Alternatively, the client devices 101 and 102 can be other servers. The network 103 can be any type of wired or wireless network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof.

[0022] The server (e.g., host) 104 may be any kind of server or server cluster, such as a web or cloud server, an application server, a backend server, or a combination thereof. The server 104 also includes an interface (not shown) to allow clients, such as client devices 101 and 102, to access resources or services provided by the server 104 (such as resources and services provided by the DP accelerator via the server 104). For example, the server 104 may be a cloud server or a server in a data center that provides various cloud services to clients, such as cloud storage, cloud computing services, machine learning training services, data mining services, etc. The server 104 may be configured as part of a software as a service (SaaS) or platform as a service (PaaS) system on a cloud, which may be a private cloud, a public cloud, or a hybrid cloud. The interface may include a web interface, an application programming interface (API), and / or a command line interface (CLI).

[0023] For example, a client (in this example, a user application (e.g., a web browser, application) of client device 101) may send or transmit instructions for execution (e.g., artificial intelligence (AI) training, reasoning instructions, etc.) to server 104, and the instructions are received by server 104 via an interface over network 103. In response to the instructions, server 104 communicates with DP accelerators 105 to 107 to complete the execution of the instructions. In some embodiments, the instructions are machine learning type instructions, where the DP accelerator, as a dedicated machine or processor, can execute the instructions many times faster than by server 104. Therefore, server 104 can control / manage the execution of one or more DP accelerators in a distributed manner. Server 104 then returns the execution results to client devices 101 and 102. The DP accelerator or AI accelerator may include: one or more dedicated processors, such as the Baidu artificial intelligence (AI) chipset available from Baidu, or the DP accelerator may be an AI chipset from NVIDIA, Intel, or some other AI chipset provider.

[0024] According to one embodiment, each application that accesses any one of the DP accelerators 105 to 107 connected to the data processing server 104 (also referred to as the host) can verify that the application is provided by a trusted source or vendor. Each application can be launched and executed within a virtual machine (VM), or an execution environment (EE) or container, or a VM within a container, that is specifically configured and executed by the central processing unit (CPU) of the host 104. When the application is configured to access any one of the DP accelerators 105 to 107, a connection can be established between the host 104 and the corresponding one of the DP accelerators 105 to 107.

[0025] Figure 2 is a block diagram illustrating an example of a host communicating with a data processing (DP) accelerator according to one embodiment. Figure 2, the system 200 provides a virtualized environment for a host / server and / or DP accelerator to share common and / or dedicated resources of a DP accelerator. In one embodiment, a host machine or server 104 may be a system that hosts one or more virtual machines (VMs) 201A to 201N, and each VM 201A to 201N may host an operating system that may respectively run user applications 203A to 203N and drivers 205A to 205N. VMs 201A to 201N may be managed by a virtual machine manager (VMM) 209. VMM 209 (sometimes referred to as a hypervisor) may run on hardware 213 (e.g., a central processing unit (CPU), and optionally, a security module (e.g., a trusted platform module (TPM))) of the host machine 104. The host machine 104 is typically a CPU system that may control and manage the execution of user applications 203 and / or DP accelerators 105 to 107 on the host machine 104.

[0026] In one embodiment, the host system 104 includes: hardware with one or more CPUs 213, wherein the CPU 213 is equipped with a security module (such as a trusted platform module (TPM)) within the host machine 104. The TPM is a dedicated chip on an endpoint device that stores cryptographic keys (e.g., RSA cryptographic keys) dedicated to the host system for hardware authentication. Each TPM chip may contain one or more RSA key pairs (e.g., public and private key pairs) called endorsement keys (EK) or endorsement credentials (EC), i.e., root keys. The key pairs are kept inside the TPM chip and cannot be accessed by software. The key parts of the firmware and software can then be hashed by the EK or EC before they are executed to protect the system from unauthorized firmware and software modifications. Therefore, the TPM chip on the host machine can be used as a root of trust for secure boot.

[0027] The TPM chip including the secure boot can protect the VMM 209 from unauthorized access. The VMM 209 can include a driver provided by the DP accelerator vendor, which specifies the hardware descriptors of the DP accelerators 105 to 107 for the host 104 to communicate with the DP accelerators 105 to 107. Since the communication channel 215 of the DP accelerators 105 to 107 can be exclusively occupied by the VMM 209, the communication channel 215 can be protected by the TPM chip.

[0028] Reference Figure 2, system 200 may include: host system 104 and DP accelerators 105 to 107. DP accelerators 105 to 107 may include: Baidu AI chipset or any other AI chipset that can perform AI intensive computing tasks, such as NVIDIA graphics processing unit (GPU). In one embodiment, DP accelerators 105 to 107 are peripheral component interconnect express (PCIE) components. In one embodiment, communication channel 215 includes: PCIE channel.

[0029] In one embodiment, each VM 201A to 201N may represent a secure execution environment or an independent container, such as a docking container. Here, the host machine 104 may instantiate one or more secure execution environments or independent containers, and each VM 201A to 201N may be executed in a secure execution environment or an independent container. The secure execution environment / container may be forced to be secure by TPM / CPU 213. An example of a secure execution environment may be Intel Software Guard Extensions (SGX), or AMD Secure Encrypted Virtualization (SEV), or any non-secure execution environment. Intel SGX and / or AMD SEV may include: a set of central processing unit (CPU) instruction codes, wherein the code allows user-level code to allocate a private area of ​​the CPU's memory, which is protected from the impact of processes running at a higher privilege level. Here, the secure execution environment may protect user applications 203 and drivers 205, wherein user applications 203 and drivers 205 may be provided by end users and DP accelerator vendors, respectively. Here, the driver 205 may convert the API call into a command for execution, configuration, and / or control of the DP accelerators 105 to 107 .

[0030] In one embodiment, the DP accelerators 105 to 107 implement the single root I / O virtualization (SR-IOV) implementation standard to allow hardware-based virtualization. The SR-IOV standard enables efficient sharing of devices between virtual machines, and implements the SR-IOV standard in hardware to obtain I / O performance comparable to native performance. The SR-IOV specification defines standards that enable virtual machines to connect directly to I / O devices. For example, each DP accelerator device (such as an SR-IOV-enabled AI accelerator that can be presented as a diverse, separate physical DP accelerator device with appropriate hardware and OS support) has its own PCIE configuration space.

[0031] In one embodiment, each DP accelerator 105 to 107 has at least one physical function (PF) and one or more virtual functions (VFs). SR-IOV can use PFs 231 to 233 to manage the SR-IOV and / or VF functions of each DP accelerator 105 to 107. The VF / PF of the DP accelerator 105 to 107 can be mapped to each VM. The VM can then directly access the resources of the DP accelerator via PFs 231 to 233 and VFs 235 to 237. In this way, each VM 201 can access the unique and / or shared resources of the DP accelerator 105 to 107. Note that a VF is a lightweight PCIE function that shares one or more (dedicated or public) resources with a PF and / or other VFs associated with the PF.

[0032] A physical function (PF) is a PCI function that supports SR-IOV capabilities as defined by the SR-IOV specification. A physical function contains an SR-IOV capability structure and manages SR-IOV functions. A PF is a fully characterized PCIE function that can be discovered, managed, and operated like a PCIE device. A PF can be used to configure and control PCIE devices. A VF can be created by a PF. After creation, a VF can be directly assigned to an I / O domain. This enables VFs to share physical devices and perform I / O without the overhead of a central processing unit (CPU) and hypervisor.

[0033] Reference Figure 2 In one embodiment, the driver 205 may provide hardware input / output descriptors (not shown) of physical functions (PFs) 231 to 233 and / or virtual functions (VFs) 235 to 237. The VMSs 201A to 201N of the host 104 may access the driver 205 and communicate with the PFs 231 to 233 and / or the VFs 235 to 237 via the hardware input / output descriptors within the driver 205.

[0034] The host machine 104 can be set up as follows: a memory-secure Linux distribution is installed on a system equipped with a TPM secure boot. The installation can be performed offline during the manufacturing or preparation phase. The installation can also ensure that the user application 203 is programmed using a memory-secure programming language.

[0035] After installation, the system can then be started through a TPM-based secure boot. TPM secure boot ensures that only signed / certified hardware and VMM are started in the kernel space that provides accelerator services. Note that a hypervisor or virtual machine manager (VMM) is computer software, firmware, or hardware that creates and runs virtual machines. Note that kernel space is a declarative area or scope in which a kernel (i.e., a predetermined set (e.g., predefined) of functions for execution) is identified to provide functions and services to user applications. In the event that the integrity of the system is compromised, TPM secure boot may fail to start and instead shut down the system.

[0036] After secure boot, VMM 209 may start an instance of VM 201. Next, user application 203 and driver 205 may be started within the operating system of VM 201. In one embodiment, user application 203 and driver 205 are statically linked and started together. In another embodiment, driver 205 is first started in VM 201, and then user application 203 is dynamically loaded in VM 201. In another embodiment, user application 203 is first started in VM 201, and then driver 205 is dynamically loaded in VM 201.

[0037] In summary, the system 200 provides a hardware-based virtualization environment using one or more VMs located at the host 104 and an SR-IOV-enabled DP accelerator with static partitioning and / or resources. The system 200 may include: a TPM-based secure boot protection layer and a secure execution environment layer. In addition, the system 200 may provide a memory-safe user space by ensuring that other applications on the host machine are implemented in a memory-safe programming language, and the system 200 may also ensure security within the application by eliminating potential memory corruption / vulnerabilities. In one embodiment, the system 200 may include: an application that uses a side-channel-free algorithm to defend against side-channel attacks (such as cache-based side-channel attacks).

[0038] Figure 3 is a block diagram showing an example of a host communicating with a DP accelerator according to one embodiment. Figure 3 , the DP accelerator 105 may be statically partitioned into one or more resource partitions, and each VF (#1 ​​to 4) 235 may be associated with a specific static partition of resources. Here, in this example, the resources 225 of the DP accelerator 105 may be statically partitioned into four resource partitions 225A to 225D by the manufacturer / supplier of the DP accelerator. Each static partition of resources may include: multiple I / O resources, data computing processing units, graphics processing units, AI processing units, memory / cache, hard disk drive space, registers, etc.

[0039] In one embodiment, each static partition of a resource is distinct from other partitions of the resource (or is dedicated to a particular VF). In one embodiment, one static partition of a resource cannot access another static partition of the resource, and a VM assigned to a VF corresponding to a static partition of the resource cannot access another static partition of the resource. In one embodiment, static partitions are unmodifiable except for deletion and repartitioning of the static partition.

[0040] In one embodiment, drivers 205A to 205N provide hardware input and output descriptors to VMs 201A to 201N, respectively, to access VF 235 / PF 231. In one embodiment, each VF 235 may or may not share one or more resources of DP accelerator 105 with PF 231 and / or other VFs 235. Thus, only VMs assigned to a particular VF may access resources associated with the VF. In other words, other VMs cannot access the assigned VF.

[0041] In one embodiment, the number of VFs is equal to the number of static partition resources. In one embodiment, when a VM is instantiated, the VMM (or hypervisor) may automatically assign VFs to the VM. In another embodiment, the hypervisor assigns the VFs to the containers containing the VMs. In one embodiment, when the number of VMs connected to the resources is greater than the number of VFs, the hypervisor can no longer assign VFs to the VMs. In this case, the hypervisor may queue the VM for the next available VF, or discard the assignment, or migrate the VM to another host server system (which may result in additional waiting time / processing cost for the user application) to use the available VFs of another host server system.

[0042] In one embodiment, the DP accelerator 105 includes: a queue 301 and (optionally) a data classifier 303. A VF 235 (e.g., VF#1) may receive a request from a VM 201A (e.g., VM#1) assigned to the VF 235 (e.g., VF#1). The request may be of I / O type, machine learning, data processing, AI training, requested AI inference type, etc. When the DP accelerator 105 completes the request, the DP accelerator 105 may determine an identifier of the VM 201A based on the request, such as a media access control (MAC) address. The data classifier 303 may then classify the result based on the identifier and send the result to the queue 301 (e.g., queue#1) associated with the MAC address. The queue 301 then queues the result to be transmitted from the VF 235 (e.g., VF#1) to the VM 201A (e.g., VM#1) using direct memory access.

[0043] In another embodiment, when the DP accelerator 105 completes the request, the DP accelerator 105 may determine one or more memory addresses at the VM to store the results of the request. The data classifier 303 may then classify the results based on the memory addresses and send the results to the queue 301 (e.g., queue #1) associated with the memory address of the VM. The queue 301 then queues the results to be transferred from the VF 235 (e.g., VF #1) to the VM 201A (e.g., VM #1) using direct memory access. Here, direct memory access (DMA) is a method that allows an input / output (I / O) device to bypass the CPU of the host 104 and directly send data to or receive data from the main memory of the host 104.

[0044] Figure 4 4 is a flow chart illustrating an exemplary method of a DP accelerator according to one embodiment. Process 400 may be performed by processing logic that may include software, hardware, or a combination thereof. For example, process 400 may be performed by a host system (such as, Figure 3 DP accelerator 105) is executed. Figure 4 At block 401, processing logic (e.g., a VF) receives a request from an application to execute an instruction by a data processing (DP) accelerator using resources at the DP accelerator, wherein the reception is a direct communication from a virtual machine (VM), wherein the VF is dedicated to the VM (e.g., the VF is assigned to only one VM), and the VF is one of multiple VFs of the DP accelerator. At block 402, the processing logic executes the instruction using the DP accelerator resources based on the request and generates a result. At block 403, the processing logic determines one or more memory addresses of the VM to store the result. At block 404, based on the one or more memory addresses of the VM, the processing logic transmits the result to the VM using the VF.

[0045] In one embodiment, the instructions are instructions to train an AI model or perform inference based on an AI model.In one embodiment, the hardware I / O information of the VF is located at a driver of the VM at a host hosting an application running on the VM.

[0046] In one embodiment, the DP accelerator includes: an SR-IOV pass-through device. In one embodiment, a VM of a host is initially allocated a VF by a virtual machine manager (VMM) of the host.

[0047] In one embodiment, data packets are transferred between a VM and a VF corresponding to the VM without passing through a VMM. In one embodiment, data packets are transferred between a VM and a VF corresponding to the VM via direct memory access (DMA) of a processing unit independent of the host.

[0048] Figure 5is a block diagram illustrating an example of dynamic resource isolation of a DP accelerator according to one embodiment. Figure 5 , for one example, the DP accelerator 107 includes: a resource pool 505 and a resource management unit 507. The resource management unit 507 can manage and implement resource isolation of the resources 227 of the resource pool 505. In one embodiment, the resource pool 505 can dynamically instantiate multiple resource isolations 227A to 227N in real time. For example, VFs 237#1 to 4 can be associated with PF 233. In this case, the resource pool 505 can instantiate n resource isolations 227A to 227N for VFs 237#1 to 4, respectively. VM 201B can be assigned to any one of the four VFs 237#1 to 4 (here, VF#2). VFS#1 to 4 can access any resource isolation 227A to 227N according to a request from the VM at the VF assigned to the VM. Each resource isolation may include: multiple I / O resources, data computing processing units, graphics processing units, AI processing units, memory / cache, hard disk drive space, registers, etc.

[0049] In one embodiment, the DP accelerator 107 includes: a queue 501 and (optionally) a data classifier 503. A VF 237 (e.g., VF#2) may receive a request from a VM 201B (e.g., VM#2) assigned to the VF 237 (e.g., VF#2) (assigned by the VMM 209 of the host 104). The request may be of I / O type, machine learning, data processing, AI training, requested AI inference type, etc. Next, a resource isolation from the resource pool 505 may be dynamically instantiated for the request based on the size of the request, such as resource 227B#2.

[0050] In one embodiment, in response to determining that the request (or subsequent request) requires a larger amount of resources, VM #2 sends a request to resource management unit 507 (via Figure 2 The VMM 209 of the resource management unit 507 transmits a control command to control the resource management unit 507 to dynamically update the resource 227B#2. Here, the resource 227B#2 can be updated based on the request size to have a larger amount of resources. The update can use resources available to the resource pool 505 or resources maintained by the resource pool 505 (for example, public resources available to the resource pool 505 (not shown)).

[0051] When the DP accelerator 107 completes the request, the DP accelerator 107 may determine an identifier, such as a media access control (MAC) address, for VM#2 based on the request. The data classifier 503 may then classify the result based on the identifier and send the result to a queue 501 (e.g., queue #2) associated with the identifier. The queue 501 then queues the result to be transmitted from the VF 237 (e.g., VF#2) to the VM 201B (e.g., VM#2) using direct memory access. In another embodiment, when the DP accelerator 107 completes the request, the DP accelerator 107 may determine one or more memory addresses at the VM to store the result of the request. The data classifier 503 may then classify the result based on the memory address and send the result to a queue 501 (e.g., queue #2) associated with the memory address of the VM. The queue 501 then queues the result to be transmitted from the VF 237 (e.g., VF#2) to the VM 201B (e.g., VM#2) using direct memory access.

[0052] Figure 6 600 is a flow chart illustrating an exemplary method of a DP accelerator according to one embodiment. Process 600 may be performed by processing logic that may include software, hardware, or a combination thereof. For example, process 600 may be performed by a host system (such as, Figure 5 DP accelerator 107) is executed. Figure 6 At box 601, processing logic (e.g., a VF) receives a request for a DP accelerator resource from an application, wherein the reception is a direct communication from a virtual machine (VM), wherein the VF is assigned to only one VM and not to other VMs, and the VF is one of multiple VFs of the DP accelerator. At box 602, the processing logic determines that a first isolation of the DP accelerator resource is assigned to the VF. At box 603, the processing logic determines that the first isolation of the resource does not satisfy the size of the request. At box 604, the processing logic dynamically updates the first resource isolation to a second resource isolation through a resource management unit of the DP accelerator to satisfy the request size.

[0053] In one embodiment, the request includes: a request to train an AI model. In one embodiment, the hardware I / O information of the VF is located at a driver of a VM at a host hosting the application.

[0054] In one embodiment, the DP accelerator comprises: a single root input output virtualization (SR-IOV) pass-through device. In one embodiment, a VM of a host is initially assigned a VF by a virtual machine manager (VMM) of the host.

[0055] In one embodiment, data packets are transferred directly between a VM and a VF assigned to the VM without passing through a VMM. In one embodiment, data packets are transferred between a VM and a VF corresponding to the VM via direct memory access (DMA) of a processing unit independent of the host.

[0056] Note that some or all of the components shown and described above may be implemented in software, hardware, or a combination thereof. For example, such components may be implemented as software installed and stored in a permanent storage device, which may be loaded and executed by a processor (not shown) in a memory to perform the processes or operations described throughout this application. Alternatively, such components may be implemented as executable code programmed or embedded in dedicated hardware (such as an integrated circuit (e.g., an application specific integrated circuit or ASIC), a digital signal processor (DSP), or a field programmable gate array (FPGA)), which may be accessed from an application via a corresponding driver and / or operating system. In addition, such components may be implemented as dedicated hardware logic in a processor or processor core as part of an instruction set accessible by a software component via one or more dedicated instructions.

[0057] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities.

[0058] It should be remembered, however, that all of these terms and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise clearly indicated from the above discussion, it should be understood that throughout the specification, discussions utilizing terms such as those set forth in the following claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission, or display devices.

[0059] Embodiments of the present disclosure also relate to devices for performing the operations described herein. Such a computer program is stored in a non-transitory computer-readable medium. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device).

[0060] The processes or methods depicted in the preceding figures may be performed by processing logic, wherein the processing logic includes hardware (e.g., circuitry, dedicated logic, etc.), software (e.g., embodied on a non-transitory computer-readable medium), or a combination of both. Although the processes or methods are described above in terms of some sequential operations, it should be understood that some of the operations described may be performed in a different order. In addition, some operations may be performed in parallel rather than sequentially.

[0061] The embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the embodiments of the present disclosure as described herein.

[0062] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. Obviously, various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and the drawings are to be considered illustrative rather than restrictive.

Claims

1. Data processing accelerator, include: One or more static partition resources; one or more virtual functions, each virtual function being associated with one of the one or more static partitioned resources, wherein a virtual machine of a host machine is assigned one of the one or more virtual functions to access the static partitioned resource associated with the assigned virtual function, and wherein the virtual machine is unable to access a remainder of the one or more static partitioned resources of the data processing accelerator; and A data classifier that is communicatively connected to one or more I / O queues is configured to: classify the results from the one or more static partitioned resources according to an identifier of a virtual machine that sends a request for the results or a memory address at the virtual machine for storing the results, and send the classified results to the I / O queue corresponding to the identifier or the memory address, so that the classified results are transmitted from the virtual function to the virtual machine via the I / O queue.

2. The data processing accelerator according to claim 1, further comprising: include: Single root I / O virtualization through the device.

3. The data processing accelerator according to claim 1, in, The virtual machine directly transmits a data packet to the virtual function via hardware access to the virtual function using a virtual function driver running on the virtual machine.

4. The data processing accelerator according to claim 1, in, The host's virtual machine manager allocates a virtual machine of the host to communicate with the virtual function.

5. The data processing accelerator according to claim 4, in, The virtual machine manager of the host allocates at most one virtual function to the virtual machine, and wherein, if the one or more virtual functions are not available for allocation, the virtual machine manager waits for a next one of the one or more virtual functions to become available for allocation, or the virtual machine manager rejects the allocation, or the virtual machine manager migrates the virtual machine to another host for allocation to the virtual function of the other host.

6. The data processing accelerator according to claim 4, in, Data packets are transmitted directly between the virtual machine and the virtual function assigned to the virtual machine without passing through the virtual machine manager.

7. Data processing system, include: Host; as well as A data processing accelerator connected to the host, the data processing accelerator comprising: one or more static partitioned resources; one or more virtual functions, each virtual function being associated with one of the one or more static partitioned resources, wherein a virtual machine of a host is assigned one of the one or more virtual functions to access the static partitioned resource associated with the assigned virtual function, and wherein the virtual machine is unable to access the rest of the one or more static partitioned resources of the data processing accelerator; and A data classifier that is communicatively connected to one or more I / O queues is configured to: classify the results from the one or more static partitioned resources according to an identifier of a virtual machine that sends a request for the results or a memory address at the virtual machine for storing the results, and send the classified results to the I / O queue corresponding to the identifier or the memory address, so that the classified results are transmitted from the virtual function to the virtual machine via the I / O queue.

8. The data processing system according to claim 7, in, The data processing accelerator includes a single input-output virtualization pass-through device.

9. The data processing system according to claim 8, in, The virtual machine directly transmits a data packet to the virtual function via hardware access to the virtual function using a virtual function driver running on the virtual machine.

10. The data processing system according to claim 7, in, The host's virtual machine manager allocates the host's virtual machine to communicate with the virtual function.

11. The data processing system according to claim 10, in, The virtual machine manager of the host allocates at most one virtual function to the virtual machine, and wherein, if the one or more virtual functions are not available for allocation, the virtual machine manager waits for a next one of the one or more virtual functions to become available for allocation, or the virtual machine manager rejects the allocation, or the virtual machine manager migrates the virtual machine to another host for allocation to the virtual function of the other host.

12. The data processing system according to claim 7, in, Data packets are transmitted directly between the virtual machine and the virtual function assigned to the virtual machine without passing through the virtual machine manager.

13. Computer-implemented methods, include: Receiving a request from an application in a virtual machine through a virtual function to execute instructions through a data processing accelerator using resources at the data processing accelerator, wherein the receiving is a direct communication from the virtual machine, wherein the virtual function is assigned to only one virtual machine, and wherein the virtual function is one of a plurality of virtual functions of the data processing accelerator; Execute the instructions and generate results using the data processing accelerator resources based on the request; and The results are classified according to the identifier of the virtual machine that sends the request or the memory address at the virtual machine for storing the results, and the classified results are sent to the I / O queue corresponding to the identifier or the memory address, so that the classified results are transmitted from the virtual function to the virtual machine via the I / O queue.

14. The computer-implemented method of claim 13, in, The instructions are instructions for training an AI model or performing reasoning based on an AI model.

15. The computer-implemented method of claim 13, in, The hardware I / O information of the virtual function is located at a driver of the virtual machine at a host hosting the application running on the virtual machine.

16. The computer-implemented method of claim 13, in, The data processing accelerator includes a single input-output virtualization pass-through device.

17. The computer-implemented method of claim 15, in, The virtual machine of the host is initially assigned the virtual function by a virtual machine manager of the host.

18. The computer-implemented method of claim 17, in, Data packets are transmitted between the virtual machine and the virtual function corresponding to the virtual machine without passing through the virtual machine manager.

19. The computer-implemented method of claim 15, in, Data packets are transferred between the virtual machine and the virtual function corresponding to the virtual machine via direct memory access independent of a processing unit of the host machine.

20. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 13 to 19.

Images