Method for virtual machine migration with checkpoint authentication in virtualized environments

Abstract

Systems and methods are disclosed for migrating a virtual machine (VM) with virtual functions that map resources of an artificial intelligence (AI) accelerator to the VM. A driver for the AI accelerator can generate a checkpoint of a VM process that makes calls to the AI accelerator, and the checkpoint can include a list and configuration of resources mapped to the AI accelerator through the virtual functions. The driver can also access code, data, and memory of the AI accelerator to generate a checkpoint of the AI accelerator state. Either or both of these checkpoint frames can then be used when the VM is migrated to a new host to ensure that the VM can be successfully restored on the new host with the appropriate AI accelerator resources. One or both checkpoint frames can be captured based on an event to anticipate a need to migrate the VM.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to one or more artificial intelligence accelerators connected to a host that is linked to a virtual machine. More specifically, embodiments of this disclosure relate to migrating virtual machines that use artificial intelligence accelerators. Background Technology

[0002] Artificial intelligence (AI) models (also known as “machine learning models”) have recently been widely utilized as AI technologies have been deployed in various fields, such as image classification, medical diagnosis, or autonomous driving. Similar to executable images or binary images in software applications, AI models can perform inference based on a set of attributes to be classified as features during training. Training AI models requires a significant investment in collecting, organizing, and filtering data to generate AI models that produce useful predictions. Furthermore, predictions generated using AI models may contain sensitive personal data that users would wish to protect.

[0003] Generating predictions from an AI model is a computationally intensive process. To provide sufficient computing power for one or more users, one or more AI accelerators can be connected to the host machine of one or more virtual machines. To provide sufficient computing power for computationally intensive tasks such as training AI models, AI accelerators can be organized into clusters, and then into multiple groups, with each group assigned to a single virtual machine. For less intensive tasks, a single virtual machine can have a single AI accelerator assigned to it.

[0004] For several well-known reasons, virtual machines may need to be migrated to different hosts. Existing virtual machine migration techniques do not protect the state of the one or more AI accelerators during migration. AI applications that generate one or more AI tasks (at least some of which are executed on AI accelerators) may fail or be interrupted after migration to another host. Failures may include failure to capture the configuration, memory contents, and compute state of the AI ​​accelerator, as well as failure to capture the compute state of the AI ​​tasks within the VM. Summary of the Invention

[0005] In a first aspect, a computer-implemented method is provided for migrating a source virtual machine (VM-S) of an application that is performing virtual functions to access an artificial intelligence (AI) accelerator, the method comprising:

[0006] The checkpoint of one state of the VM-S is stored in a storage device for multiple states of the VM-S, wherein each state of the VM-S includes the state of the VM-S resources, the state of the application, and the state of the AI ​​accelerator mapping AI resources to the virtual functions of the VM-S.

[0007] In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state to be used to perform the migration:

[0008] Record, and then stop one or more ongoing AI tasks;

[0009] Using the selected checkpoint, migrate the VM-S, the application, the one or more AI tasks, and the virtual functions to the target VM on the target host.

[0010] (VM-T); and

[0011] In response to receiving a notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the VM-S hypervisor commands the target host hypervisor to restart one or more recorded AI tasks and restart the application.

[0012] In a second aspect, a computer-readable medium programmed with executable instructions is provided, which, when executed by a processing system having at least one hardware processor communicatively connected to at least one artificial intelligence (AI) processor, perform operations to migrate a source virtual machine (VM-S) of an application executing virtual functions accessing said at least one AI processor, said operations including:

[0013] The checkpoint of one state of the VM-S is stored in a storage device for multiple states of the VM-S, wherein each state of the VM-S includes the state of the VM-S resources, the state of the application, and the state of the AI ​​accelerator mapping AI resources to the virtual functions of the VM-S.

[0014] In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state to be used to perform the migration:

[0015] Record, and then stop one or more ongoing AI tasks;

[0016] Using the selected checkpoint, migrate the VM-S, the application, the one or more AI tasks, and the virtual functions to the target VM on the target host.

[0017] (VM-T); and

[0018] In response to receiving a notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the VM-S hypervisor commands the target host hypervisor to restart one or more recorded AI tasks and restart the application.

[0019] In a third aspect, a system is provided comprising at least one hardware processor coupled to an instruction-programmable memory, the instructions, when executed by the at least one hardware processor communicatively coupled to an artificial intelligence (AI) accelerator, causing the system to perform operations for migrating a source virtual machine (VM-S) of an application performing virtual functions accessing the AI ​​accelerator, the operations including:

[0020] The checkpoint of one state of the VM-S is stored in a storage device for multiple states of the VM-S, wherein each state of the VM-S includes the state of the VM-S resources, the state of the application, and the state of the virtual functions of the AI ​​accelerator.

[0021] In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state to be used to perform the migration:

[0022] Record, and then stop one or more ongoing AI tasks;

[0023] Using the selected checkpoint, migrate the VM-S, the application, the one or more AI tasks, and the virtual functions to the target VM on the target host.

[0024] (VM-T); and

[0025] In response to receiving a notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the VM-S hypervisor commands the target host hypervisor to restart one or more recorded AI tasks and restart the application.

[0026] In a fourth aspect, a system is provided comprising at least one hardware processor coupled to an instruction-programmable memory, the instructions, when executed by the at least one hardware processor, causing the system to perform operations for migrating a source virtual machine VM-S that is performing virtual functions for accessing an artificial intelligence (AI) accelerator, the operations including:

[0027] The target host's hypervisor receives a checkpoint from the source virtual machine VM-S associated with the virtualization function VF, the VF mapping AI processor resources to the VM-S; and receives a request for the VM-S hosted as the target virtual machine VM-T.

[0028] The target host's management program allocates and configures resources for hosting the VM-S as the VM-T and the VM-S's VF based on the checkpoint;

[0029] The management program of the target host receives the data frame of the VM-S and stores the data frame to generate the VM-T;

[0030] The management program of the target host receives the recorded status of the unfinished AI task of the VM-S, and restarts the unfinished AI task on the VM-T.

[0031] In a fifth aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to the first aspect. Attached Figure Description

[0032] Embodiments of this disclosure are illustrated in the figures by way of example rather than limitation, wherein similar reference numerals indicate similar elements.

[0033] Figure 1 This is a block diagram illustrating a security processing system according to one embodiment, which can migrate virtual machines using checkpoint authentication and / or artificial intelligence (AI) accelerator state verification.

[0034] Figure 2A and Figure 2B This is a block diagram illustrating a secure computing environment between one or more hosts and one or more artificial intelligence accelerators according to one embodiment.

[0035] Figure 3 The diagram illustrates a block diagram of a host according to an embodiment, which controls a cluster of artificial intelligence accelerators, each cluster having a virtualization function that maps the resources of a group of AI accelerators within the cluster to virtual machines, and each AI accelerator having secure resources and non-secure resources.

[0036] Figure 4A This is a block diagram illustrating components of a data processing system according to an embodiment, the data processing system having an artificial intelligence (AI) accelerator to implement a method for virtual machine migration using checkpoint authentication in a virtualized environment.

[0037] Figure 4B This is a block diagram illustrating components of a data processing system according to an embodiment, the data processing system having an artificial intelligence (AI) accelerator to implement a method for virtual machine migration using state verification with an AI accelerator in a virtualized environment.

[0038] Figure 5A The illustration depicts a method for migrating a data processing system with an AI accelerator in a virtualized environment using checkpoint authentication, from the perspective of the host hypervisor of the source virtual machine to be migrated, according to an embodiment.

[0039] Figure 5B The illustration depicts a method for migrating a data processing system with an AI accelerator in a virtualized environment using AI accelerator state verification, from the perspective of the host hypervisor of the source virtual machine to be migrated, according to an embodiment.

[0040] Figure 6 The illustration depicts a method, according to an embodiment, for generating checkpoints from the perspective of a source manager on the host of the virtual machine to be migrated, for use in a method for migrating virtual machines using checkpoint authentication in a virtualization environment.

[0041] Figure 7 The illustration illustrates a method, according to an embodiment, for determining, from the perspective of the source manager hosting the virtual machine to be migrated, whether to utilize checkpoint authentication to migrate a virtual machine with an AI accelerator in a virtualized environment.

[0042] Figure 8 The illustration depicts a method for migrating a virtual machine with an AI accelerator data processing system in a virtualization environment using checkpoint authentication, from the perspective of the source manager hosting the virtual machine to be migrated, according to an embodiment.

[0043] Figure 9 The illustration depicts a method for performing post-migration cleanup on a source host computing device after migrating a virtual machine of a data processing system with an AI accelerator in a virtualized environment using checkpoint authentication.

[0044] Figure 10 The illustration depicts a method, according to some embodiments, for migrating a virtual machine of a data processing system with an AI accelerator in a virtualized environment, utilizing checkpoint authentication and optionally AI accelerator status verification, from the perspective of a target hypervisor on a host for the migrated virtual machine. Detailed Implementation

[0045] Various embodiments and aspects of this disclosure will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and accompanying drawings illustrate this disclosure and are not to be construed as limiting the disclosure. Numerous specific details are described to provide a thorough understanding of various embodiments of this disclosure. However, in some instances, well-known or conventional details have not been described in order to provide a concise discussion of embodiments of this disclosure.

[0046] The use of "an embodiment" or "an embodiment" in the specification means that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this disclosure. The phrase "in an embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0047] The following examples relate to using artificial intelligence (AI) accelerators to increase the processing throughput of certain types of operations that can be offloaded to (or delegated to) the AI ​​accelerator from a host device. The host device hosts one or more virtual machines (VMs). At least one VM on the host may be associated with a virtual function that maps the resources of the AI ​​accelerator to the VM. The virtual function enumerates the resources within the AI ​​accelerator mapped to the VM and the configuration of those resources within the accelerator. Drivers within the VM can track the scheduling and computational state of tasks to be processed by the AI ​​accelerator. Drivers can also obtain the code, data, and memory of the AI ​​accelerator mapped to the VM.

[0048] As used herein, a “virtual function” is a mapping of a set of resources within a group of AI accelerators in an artificial intelligence (AI) accelerator or AI accelerator cluster to a virtual machine. This set of resources is referred to individually and collectively as an “AI resource” throughout this document. An AI accelerator or AI accelerator cluster is referred to as an “AI accelerator” throughout this document, unless a distinction between an AI accelerator and an AI accelerator cluster is described.

[0049] AI accelerators can be general-purpose processing units (GPUs), artificial intelligence (AI) accelerators, math coprocessors, digital signal processors (DSPs), or other types of processors. AI accelerators can be proprietary designs, such as... AI accelerators or other GPUs, etc. Although embodiments are illustrated and described with the host device securely connected to one or more AI accelerators, the concepts described herein can be implemented more generally as distributed processing systems.

[0050] Multiple AI accelerators can be linked in a cluster managed by a host device that has drivers for one or more AI accelerators that interpret application processing requests into processing tasks. The host device can support one or more virtual machines (VMs), each with a user associated with its corresponding VM. The drivers can implement virtualization functions that map the resources of the AI ​​accelerators to the VMs. The drivers can include a scheduler that schedules application processing requests from the multiple VMs for processing by one or more AI accelerators. In one embodiment, the drivers can analyze the processing requests in the scheduler to determine how to group the one or more AI accelerators in the cluster, and whether to command one or more AI accelerators to disconnect from a group and enter a low-power state to reduce heat and conserve energy.

[0051] The host device and the AI ​​accelerator can be interconnected via a high-speed bus, such as Peripheral Component Interconnect High Speed ​​(PCIe) or other high-speed buses. The host device and the AI ​​accelerator can exchange keys and establish a secure channel via the PCIe bus before performing operations according to the aspects of the invention described below. Some of these operations include: the AI ​​accelerator using an artificial intelligence (AI) model to perform inference using data provided by the host device. Before the AI ​​model inference is trusted by the host device, the host device can use the AI ​​accelerator to perform one or more verification tests described below, including determining a watermark for the AI ​​model. In some embodiments and operations, the AI ​​accelerator is unaware that the host device is testing the validity of the results produced by the AI ​​accelerator.

[0052] The host device may include a central processing unit (CPU) coupled to one or more AI accelerators. Each AI accelerator may be coupled to the CPU via a bus or interconnect. AI accelerators may be implemented as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other forms of integrated circuits (ICs). Alternatively, the host processor may be part of a main data processing system, while the AI ​​accelerator may be one of many distributed systems (e.g., cloud computing systems, such as Software as a Service (SaaS) or Platform as a Service (PaaS) systems) that are secondary systems that can remotely offload their data processing tasks to the main system via a network. The link between the host processor and the AI ​​accelerator may be a peripheral component interconnect high-speed (PCIe) link or a network connection (such as an Ethernet connection). Each AI accelerator may include one or more link registers that enable (link connected) or disable (link disconnected) the communication link with another AI accelerator.

[0053] In a first aspect, a computer-implemented method for migrating a source virtual machine (VM-S) of an application executing virtual functions accessing an artificial intelligence (AI) accelerator may include: storing a checkpoint of one state of the VM-S to a storage device containing multiple states of the VM-S. Each state of the VM-S may include the state of the VM-S's resources, the state of the application, and the state of the AI ​​accelerator's virtual functions mapping AI resources to the VM-S. In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state of the VM-S to be used for the migration, the method may further include: recording, then stopping one or more executing AI tasks, and migrating the VM-S, the application, the one or more AI tasks, and the virtual functions to a target VM (VM-T) on a target host using the selected checkpoint. In response to receiving notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the recorded one or more AI tasks and the application may be restarted on the VM-T. The virtual functions map the resources of the AI ​​accelerator to the VM-S, and the user of the VM-S is the only user who can access the resources of the AI ​​accelerator that are mapped to the VM-S through the virtual functions. In one embodiment, the virtual function maps resources of multiple AI accelerators to the VM-S, and the checkpoint includes the communication configuration between the multiple AI accelerators. The user of the VM-S is the only user who can access the resources of the multiple AI accelerators mapped to the VM-S through the virtual function. In another embodiment, the method further includes receiving a notification from the target manager that the migration of the VM-S is complete and that the one or more recorded AI tasks have been successfully restarted. In response to this notification, post-migration cleanup can be performed on the VM-S. Post-migration cleanup can include: erasing at least the secure memory of the AI ​​accelerators, including any AI inference, AI models, secure computation, or portions thereof; and erasing the memory of the VM-S associated with the AI ​​virtual function and any calls to the virtual function by applications. Verifying the checkpoint's signature and freshness date can include: decrypting the checkpoint's signature using the VM-S's public key, determining that the checkpoint's date and timestamp are within a threshold date and time range, and verifying the checkpoint's hash. In an embodiment, a checkpoint may include: a record of one or more AI tasks in progress; configuration information of resources within the one or more AI accelerators communicatively connected to the VM-S; the date and timestamp of the checkpoint; and a snapshot of the VM-S's memory, including virtual function scheduling information and communication buffers within the one or more AI accelerators.

[0054] In a second aspect, a method for migrating a source virtual machine (VM-S) executing an application that accesses a virtual function (VF) of an artificial intelligence (AI) accelerator includes: a hypervisor of a target host receiving a checkpoint from the source VM-S associated with the VF, the VF mapping AI processor resources to the VM-S; and receiving a request for the VM-S hosted as a target VM-T. The hypervisor of the target host allocates and configures resources for the VM-S hosted as VM-T and the VF of the VM-S based on the checkpoint. The hypervisor of the target host receives data frames from the VM-S and stores these data frames to generate the VM-T. The hypervisor of the target host receives a recorded state of unfinished AI tasks on the VM-S and restarts the unfinished AI tasks on the VM-T. In an embodiment, verifying the checkpoints of the VM-S and VF includes: decrypting the signature of the checkpoint using the public key of the VM-S; determining that the date and timestamp of the checkpoint fall within a predetermined range; and recalculating the hash of the checkpoint and determining whether the recalculated hash matches the hash stored in the checkpoint. In response to successful verification of the checkpoint, the migration of the VM-S to the target host's hypervisor continues, thereby generating a VM-T on the target host.

[0055] In a third aspect, a method implemented on a computer for migrating a source virtual machine (VM-S) of an application executing a virtual function that accesses an artificial intelligence (AI) accelerator includes: in response to receiving a command to migrate the VM-S and the virtual function, and in response to receiving a selection of a checkpoint for the VM-S and the virtual function to be used to perform the migration, recording and then stopping one or more AI tasks being executed by the application. The method further includes: generating or selecting a state of the AI ​​accelerator associated with the virtual function, and then transmitting the checkpoint and the AI ​​accelerator state to a hypervisor on the target host to generate a target virtual machine (VM-T) to be migrated.

[0056] In response to receiving notification that the target host has verified the checkpoint and AI accelerator state and that the target host has generated and configured resources for generating VM-T, the target host migrates VM-S and virtual functions to VM-T. The migration includes the target host loading the AI ​​accelerator with data from the AI ​​accelerator state frame. In an embodiment, the method further includes performing post-migration cleanup on VM-S and virtual functions in response to receiving notification that the application and AI task have restarted on VM-T. Post-migration cleanup at VM-S may include: (1) erasing at least the secure memory of the AI ​​accelerator, including any AI inference, AI models, intermediate results of secure computation, or portions thereof; and (2) erasing the memory of VM-S associated with the virtual functions and any calls to the virtual functions by the application. In an embodiment, the checkpoint storing the state of VM-S and virtual functions may include a storage device storing checkpoints of the state of VM-S and VF into a plurality of checkpoints of VM-S. Each checkpoint of VM-S may include the state of VM-S resources, the state of the application, and the state of the virtual functions associated with the resources of the AI ​​accelerator. In an embodiment, the checkpoint may further include a record of one or more ongoing AI tasks, configuration information of resources within the AI ​​accelerator communicatively connected to the VM-S, and a snapshot of the VM-S's memory. The checkpoint may further include virtual function scheduling information and communication buffers within the one or more AI accelerators, as well as the date and timestamp of the checkpoint. In an embodiment, generating the state of the AI ​​accelerator may include: (1) storing the date and timestamp of the state in the AI ​​accelerator state; (2) storing the contents of the memory within the AI ​​accelerator in the AI ​​accelerator state, including one or more registers associated with the AI ​​accelerator's processor, and caches, queues, or pipelines of pending instructions to be processed by the AI ​​accelerator; and (3) generating a hash of the AI ​​accelerator's state and digitally signing the state, hash, date, and timestamp. In an embodiment, the AI ​​accelerator state may further include one or more register settings indicating that the AI ​​accelerator is configured to communicate with one or more other AI accelerators in the AI ​​accelerator cluster. In an embodiment, verifying the signature and freshness of the AI ​​accelerator state may include: using the VM-S public key to decrypt the signature of the AI ​​state, determining that the date and timestamp of the AI ​​accelerator state are within a threshold date and time range, and verifying the hash of the AI ​​accelerator state.

[0057] Any of the above functions can be programmed as executable instructions onto one or more non-transitory computer-readable media. When the executable instructions are executed by a processing system having at least one hardware processor, the processing system causes the function to be implemented. Any of the above functions can be implemented by a processing system having at least one hardware processor coupled to memory programmed with executable instructions, which, upon execution, cause the processing system to implement the function.

[0058] Figure 1 This is a block diagram illustrating a security processing system 100 according to one embodiment, which can migrate virtual machines using checkpoint authentication and / or artificial intelligence (AI) accelerator state verification. Reference Figure 1 System configuration 100 includes, but is not limited to, one or more client devices 101 to 102, which are communicatively connected via network 103 to a source data processing (DP) server 104-S (e.g., a host) and a target data DP server 104-T. DP server 104-S can host one or more clients. The one or more clients can be virtual machines. As described herein, any virtual machine on DP server 104-S can be migrated to the target DP server 104-T.

[0059] Client devices 101 to 102 can be any type of client device, such as a personal computer (e.g., desktop computer, laptop computer, and tablet computer), a "thin" client, a personal digital assistant (PDA), a web-enabled device, a smartwatch, or a mobile phone (e.g., a smartphone). Alternatively, client devices 101 to 102 can be virtual machines on DP servers 104-S or 104-T. Network 103 can be any type of wired or wireless network, such as a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a high-speed bus, or a combination thereof.

[0060] Multiple servers (e.g., hosts) 104-S and 104-T (collectively referred to as DP Server 104 unless otherwise stated) can be any type of server or server cluster, such as a web or cloud server, application server, backend server, or a combination thereof. Server 104 further includes interfaces (not shown) to allow clients (e.g., client devices 101-102) to access resources or services provided by server 104 (e.g., resources and services provided by an AI accelerator via server 104). For example, server 104 can be a cloud server or a server in a data center providing various cloud services to clients, such as cloud storage, cloud computing services, artificial intelligence training services, data mining services, etc. Server 104 can be configured as part of a Software as a Service (SaaS) or Platform as a Service (PaaS) system via a cloud, which can be a private cloud, public cloud, or hybrid cloud. The interfaces may include a web interface, an application programming interface (API), and / or a command-line interface (CLI).

[0061] For example, the client can be a user application (e.g., a web browser, application) of client device 101. The client can send or transmit instructions for execution (e.g., AI training, AI inference instructions, etc.) to server 104, and the server 104 receives the instructions via an interface through network 103. In response to the instructions, server 104 communicates with AI accelerators 105 to 107 to execute the instructions. Source DP server 104-S can communicatively connect to one or more AI accelerators. Client virtual machines hosted by DP server 104-T (which runs applications using one or more of AI accelerators 105-T to 107-T) can be migrated to target DP server 104-T to run on the corresponding AI accelerators 105-T to 107-T. In some embodiments, the instructions are machine learning type instructions, wherein the AI ​​accelerator, as a dedicated machine or processor, can execute the instructions many times faster than the instructions executed by a general-purpose processor. Therefore, server 104 can control / manage the execution jobs for the one or more AI accelerators in a distributed manner. Then, server 104 returns the execution result to client devices 101 to 102 or a virtual machine on server 104. The AI ​​accelerator may include one or more dedicated processors, such as those available from Baidu, Obtained Artificial intelligence (AI) chipsets, or alternatively, AI accelerators, can be AI chipsets from another AI chipset provider.

[0062] According to one embodiment, each application accessing any of the AI ​​accelerators 105-S..107-S or 105-T..107-T (collectively referred to as 105..107, unless otherwise indicated) hosted by data processing server 104 (also referred to as host) can verify that the application is provided by a trusted source or vendor. Each application can be initiated and executed within user memory space and executed by the central processing unit (CPU) of host 104. When an application is configured to access any of the AI ​​accelerators 105 to 107, an fuzzy connection can be established between host 104 and the corresponding AI accelerator 105 to 107, thereby protecting the data exchanged between host 104 and AI accelerators 105 to 107 from attacks / intrusions from malware.

[0063] Figure 2A This is a block diagram illustrating a secure computing environment 200 between one or more hosts and one or more artificial intelligence (AI) accelerators according to some embodiments. In one embodiment, with or without hardware modifications to the AI ​​accelerators 105 to 107, system 200 provides protection for obfuscated communication between host 104 and AI accelerators 105 to 107. Host machine or server 104 can be depicted as a system having one or more layers to be protected against intrusion, such as user applications(s) 205, runtime libraries(s) 206, drivers(s) 209, operating system(s) 211, hypervisor(s) 212, and hardware(s) 213 (e.g., central processing unit (CPU) 201 and storage(s) 204). Below applications 205 and runtime libraries 206, one or more drivers(s) 209 can be installed to interface with hardware 213 and / or AI accelerators 105 to 107.

[0064] Driver 209 may include a scheduler 209A that schedules processing tasks requested by one or more user applications 205. Driver 209 may further include an analyzer 209B with logic that analyzes the processing tasks scheduled to execute on AI accelerators 105 through 107 to determine how to optimally configure AI accelerators 105 through 107 based on scheduling criteria such as processing throughput, energy consumption, and heat generated by the AI ​​accelerators. Driver 209 may further include one or more policies involving configuring AI accelerators to implement scheduling criteria. Configuring AI accelerators may include grouping AI accelerators into one or more groups and removing one or more AI accelerators from one or more groups. Driver 209 may further include a checkpointer 209C. Checkpointer 209C can take snapshots of the state of user applications 205, memory within VM 201, the state of scheduler 209A, the state of analyzer 209B, and the configuration of virtual functions within VM 201. As used in this article, a virtual function is a mapping of a set of resources within an artificial intelligence (AI) accelerator (e.g., 105) or AI accelerator cluster 105..107 to virtual machines. See below for reference. Figure 3 , Figure 4A and Figure 4B Describe the virtual functionality.

[0065] AI accelerators not assigned to a group within the AI ​​accelerator cluster can be set to a low-power state to save energy and reduce heat. A low-power state may include reducing the AI ​​accelerator's clock speed or entering a standby state, in which the AI ​​accelerator remains communicatively connected to the host device and can enter a running state, in which the AI ​​accelerator is ready to receive processing tasks from the host device. Alternatively, AI accelerators not assigned to a group within the cluster can remain powered on, allowing driver 209 to assign work to individual AI accelerators that are not members of a group of AI accelerators.

[0066] Configuring an AI accelerator may further include: commanding one or more AI accelerators to establish communication links (link connections) with one or more other AI accelerators to form a group of AI accelerators within the AI ​​accelerator cluster. Configuring an AI accelerator may further include: commanding one or more DP accelerators to disconnect communication links (link disconnections) between an AI accelerator and one or more other AI accelerators. Link connections and disconnections of AI accelerators can be managed by one or more link registers in each AI accelerator.

[0067] In a policy-based partitioning implementation, the AI ​​accelerator configuration policy is a single policy describing the communication links (link connected or disconnected) of each AI accelerator. While the configuration of each AI accelerator may (and typically will) differ from other AI accelerators, each AI accelerator's configuration is included within a single policy, and each AI accelerator in the cluster receives the same policy. Each AI accelerator then configures itself by configuring a portion of the AI ​​accelerator according to the policy description. Policy-based partitioning can be based on analysis of processing tasks in scheduler 209A. This analysis can determine the optimal allocation of AI accelerators to groups. In one embodiment, time-sharing processing tasks within a group of processors or across multiple groups of processors minimizes energy consumption and generated heat to optimize throughput. The advantages of partitioning AI accelerators into multiple groups in a policy-based manner include: rapid partitioning of AI accelerators, flexible scheduling of processing tasks within or across groups, time-sharing operation of AI accelerators, and time-sharing operation of groups.

[0068] In a dynamic partitioning embodiment, an AI accelerator policy is generated for each AI accelerator. Driver 209 can dynamically change the configuration of each AI accelerator, including reorganizing groups of AI accelerators, removing one or more AI accelerators from all groups, and setting those AI accelerators to a low-power state. In a dynamic partitioning embodiment, each group of AI accelerators is assigned to a single user, rather than time-sharing AI accelerators among users. Driver 209 may include an analyzer 209B that analyzes processing tasks within scheduler 209A to determine the optimal grouping of AI accelerators. The analysis can generate configurations for one or more AI accelerators, and these configurations can be deployed to each such AI accelerator to be reconfigured. The advantages of dynamic partitioning include energy savings by setting one or more processors to a low-power state, and user-dedicated processing for one AI accelerator or a group of AI accelerators instead of time-sharing among users.

[0069] Hardware 213 may include a processing system 201 having one or more processors 201. Hardware 213 may further include storage devices(s) 204. Storage devices(s) 204 may include one or more artificial intelligence (AI) models 202 and one or more kernels 203. Kernel 203 may include a signing kernel, a watermarking kernel, an encryption and / or decryption kernel, etc. The signing kernel, when executed, can digitally sign any input according to the kernel's programming. The watermarking kernel can extract watermarks from data objects (e.g., AI models or other data objects). The watermarking kernel can also embed watermarks into AI models, inference outputs, or other data objects.

[0070] A watermark kernel (e.g., a kernel that inherits a watermark) can inherit a watermark from another data object and embed that watermark into different objects, such as inference output or an AI model. As used in this paper, a watermark is an identifier associated with or embedded in an AI model or inference generated by an AI model. For example, a watermark can be embedded in one or more weight variables or bias variables. Alternatively, one or more nodes (e.g., dummy nodes that the AI ​​model does not use or is unlikely to use) can be created to embed or store the watermark.

[0071] The host machine 104 may be a CPU system that controls and manages the execution of jobs on the host machine 104 and / or AI accelerators 105 to 107. To protect / obscure the communication channel 215 between the AI ​​accelerators 105 to 107 and the host machine 104, different components may be needed to protect different layers of the host system that are vulnerable to data intrusion or attacks.

[0072] According to some embodiments, system 200 includes host system 104 and AI accelerators 105 to 107. Any number of AI accelerators can be present. The AI ​​accelerators may include... An AI chipset or another AI chipset (such as a graphics processing unit (GPU) capable of performing artificial intelligence (AI) intensive computational tasks). In one embodiment, host system 104 includes hardware 213 within host machine 104 having one or more CPUs, which are optionally equipped with a security module (such as an optional Trusted Platform Module (TPM)). The optional TPM is a dedicated chip on the endpoint device that stores host system-specific cryptographic keys (e.g., RSA cryptographic keys) for hardware authentication. Each TPM chip may contain one or more RSA key pairs (e.g., public and private key pairs) (referred to as endorsement keys (EKs) or endorsement credentials (ECs)), i.e., root keys. The key pairs are kept internal to the optional TPM chip and are inaccessible by software. Critical parts of the firmware and software can then be hashed by the EK or EC before execution to protect the system from unauthorized firmware and software modifications. Thus, the optional TPM chip on the host machine can be used as a root of trust for secure boot.

[0073] The optional TPM chip also protects the (multiple) drivers 209 and operating system (OS) 211 in the operating kernel space for communication with AI accelerators 105 to 107. Here, driver 209 is provided by the AI ​​accelerator vendor and can act as the driver 209 for user application 205 to control the (multiple) communication channels 215 between the host and the AI ​​accelerator. Because the optional TPM chip and secure boot processor protect the OS 211 and driver 209 in their kernel space, the TPM also effectively protects driver 209 and OS 211.

[0074] Since the communication channel 215 used for AI accelerators 105 to 107 can be exclusively used by the OS 211 and driver 209, the communication channel 215 can be protected by an optional TPM chip. In one embodiment, the communication channel 215 includes a Peripheral Component Interconnect (PCI) or Peripheral Component Interconnect High Speed ​​(PCIE) channel. In one embodiment, the communication channel 215 is an ambiguous communication channel.

[0075] Runtime library 206 can translate application programming interface (API) calls into commands for executing, configuring, and / or controlling the AI ​​accelerator. In one embodiment, runtime library 206 provides a set of predetermined (e.g., predefined) kernels for execution by a user application. In one embodiment, these kernels may be stored as kernel 203 in storage device(s) 204.

[0076] (Multiple) operating systems 211 can be any Release version OS or OS or other operating systems.

[0077] The system can be booted via optional TPM-based Secure Boot. Optional TPM Secure Boot ensures that only the signed / certified operating system 211 and accelerator driver 209 are initiated within the kernel space providing accelerator services. In one embodiment, the operating system 211 can be loaded via a hypervisor (212). The hypervisor or virtual machine manager 212 is computer software, firmware, or hardware that creates and runs virtual machines. Kernel space is a declared region or scope where the kernel (i.e., a predefined set of functions to be executed) is identified to provide functionality and services to user applications. If the integrity of the system is compromised, optional TPM Secure Boot may fail to boot and instead shut down the system.

[0078] Upon startup, runtime library 206 runs user application 205. In one embodiment, user application 205 and runtime library 206 are statically linked and launched together. In another embodiment, runtime library 206 is launched first, and then user application 205 is dynamically loaded. Statically linked libraries are those linked to the application at compile time. Dynamic loading can be performed by a dynamic linker. The dynamic linker loads and links shared libraries so that the user application can run at runtime. Here, user application 205 and runtime library 206 are visible to each other at runtime; for example, all process data is visible to each other.

[0079] In one embodiment, user application 205 may invoke only one kernel from a set of kernels predefined by runtime library 206. In another aspect, user application 205 and runtime library 206 are hardened with a side-channel-free algorithm to defend against side-channel attacks, such as cache-based side-channel attacks. Side-channel attacks are any attacks based on information obtained from the implementation of a computer system, rather than weaknesses in the implemented algorithm itself (e.g., cryptanalysis and software errors). Examples of side-channel attacks include cache attacks, which are attacks based on an attacker's ability to monitor shared physical system caches in a virtualized or cloud environment. Hardening may include masking the cache, or output generated by the algorithm to be placed on the cache. Finally, when the user application finishes execution, it terminates its execution and exits.

[0080] In one embodiment, the kernel group 203 includes an obfuscation kernel algorithm. In one embodiment, the obfuscation kernel algorithm can be a symmetric or asymmetric algorithm. Symmetric obfuscation algorithms can use the same algorithm to obfuscate and deobfuscate data communication. Asymmetric obfuscation algorithms require a pair of algorithms, where the first in the pair is used for obfuscation and the second in the pair is used for deobfuscation, and vice versa. In another embodiment, the asymmetric obfuscation algorithm includes a single obfuscation algorithm for obfuscating a dataset, but the dataset is not intended to be deobfuscated; for example, a peer-to-peer deobfuscation algorithm does not exist.

[0081] Obfuscation refers to blurring the intended meaning of communication by making the message difficult to understand (often using confusing and ambiguous language). Obfuscated data is more difficult and complex to reverse engineer. Obfuscation algorithms can be applied before data is transmitted to obscure (encrypt / decrypt) data communication, thereby reducing the chance of eavesdropping. In one embodiment, the obfuscation algorithm may further include an encryption scheme to further encrypt the obfuscated data to achieve an additional layer of protection. Unlike encryption, which can be computationally intensive, obfuscation algorithms can simplify computation.

[0082] Obfuscation techniques can include, but are not limited to, letter obfuscation, name obfuscation, data obfuscation, and control flow obfuscation. Letter obfuscation is the process of replacing one or more letters in data with specific substitute letters, thereby rendering the data meaningless. Examples of letter obfuscation include letter rotation functions, where each letter is shifted or rotated a predetermined number of positions along the alphabet. Another example is reordering or shuffling letters based on a specific pattern. Name obfuscation is the process of replacing a specific target string with a meaningless string. Control flow obfuscation can use additive code (inserting dead code, inserting uncontrolled jumps, inserting alternative structures) to change the order of control flow in a program, thereby hiding the true control flow of the algorithm / AI model.

[0083] In summary, System 200 provides multi-layered protection for AI accelerators (for data transmission, including machine learning models, training data, and inference output) to prevent loss of data confidentiality and integrity. System 200 may include an optional TPM-based secure boot protection layer and a kernel verification / verification layer. System 200 may include applications that use side-channel-free algorithms to defend against side-channel attacks (such as cache-based side-channel attacks).

[0084] Runtime 206 can provide an obfuscation kernel algorithm to obfuscate data communication between host 104 and AI accelerators 105-107. In one embodiment, obfuscation can be paired with a cryptographic scheme. In another embodiment, obfuscation is the sole protection method, and cryptographic hardware is unnecessary for the AI ​​accelerators.

[0085] Figure 2B This is a block diagram illustrating a secure computing environment between one or more hosts and one or more artificial intelligence (AI) accelerators according to one embodiment. In one embodiment, the Host Channel Manager (HCM) 250 includes an optional authentication module 251, an optional termination module 252, an optional key manager 253, an optional (multiple) key store 254, and an optional cryptographic engine 255. The optional authentication module 251 can authenticate user applications running on the host server 104 to authorize access to or use of the resources of the AI ​​accelerator 105. The HCM 250 can communicate with the Accelerator Channel Manager (ACM) 280 of the AI ​​accelerator 105 via a communication channel 215.

[0086] Optional termination module 252 can terminate the connection (e.g., the channel associated with the connection will be terminated). Optional key manager 253 can manage (e.g., create or destroy) asymmetric key pairs or symmetric keys to encrypt / decrypt one or more data packets for different secure data exchange channels. Here, each user application (as...) Figure 2AThe user application 205 (as part of the application) can correspond to or map to different secure data exchange channels in a one-to-many relationship, and each data exchange channel can correspond to one AI accelerator 105. Each application can utilize multiple session keys, where each session key is used to correspond to a secure channel of one AI accelerator (e.g., accelerators 105 to 107). An optional (multi-)key store 254 can store encrypted asymmetric key pairs or symmetric keys. An optional cryptographic engine 255 can encrypt or decrypt data packets for data exchanged through any of the secure channels. Note that some of these modules can be integrated into a smaller set of modules.

[0087] In one embodiment, AI accelerator 105 includes an ACM 280, non-sensitive resources 290, and sensitive resources 270. ACM 280 is a peer module corresponding to HCM 250, responsible for managing communication between host 104 and AI accelerator 105, such as resource access control. ACM 280 includes a link configuration module 281 that cooperates with HCM 250 of host server 104 to establish a communication channel 215 between host server 104 and AI accelerator 105. ACM 280 further includes a resource manager 282. Resource manager 282 enforces restricted access to sensitive resources 270 and non-sensitive resources 290. In one embodiment, sensitive resource 270 occupies a first address space range within AI accelerator 105. Non-sensitive resource 290 occupies a second address space range within AI accelerator 105. In one embodiment, the first and second address spaces are mutually exclusive and do not overlap. In one embodiment, resource manager 282 further includes logic (e.g., access control logic) that allows host server 104 to access sensitive resource 270 and non-sensitive resource 280. In one embodiment, resource manager 282 enforces access and configuration policies received from host server 104, as further described below.

[0088] Sensitive resource 270 may include an optional key manager 271, an optional multiple key store 272, a true random number generator 273, an optional cryptographic engine 274, and a memory / storage device 277. The optional key manager 271 may manage (e.g., generate, store, and / or destroy) asymmetric key pairs or symmetric keys. The optional multiple key store 272 may store cryptographic asymmetric key pairs or symmetric keys in a secure storage device within sensitive resource 270. The true random number generator 273 may generate seeds for key generation and use by cryptographic engine 274, such as for AI accelerators used for authentication links. The optional cryptographic engine 274 may encrypt or decrypt key information or data packets used for data exchange. Memory / storage device 277 may include storage for AI model 275 and kernel 276. Kernel 276 may include multiple watermark kernels (including inherited watermark kernels, watermark-enabled kernels, watermark signature kernels, etc.), encryption and decryption kernels, and associated data.

[0089] AI accelerator 105 may further include non-sensitive resources 290. Non-sensitive resources 290 may include one or more processors or processing logic 291 and memory / storage devices 292. The processors or processing logic 192 are capable of executing instructions or programs to perform various processing tasks, such as AI tasks (e.g., machine learning processes).

[0090] Link configuration module 281 is responsible for establishing or connecting links or paths from one AI accelerator to another, or terminating or disconnecting links or paths from one AI accelerator to another. In one embodiment, in response to a request to join a group of AI accelerators (e.g., from a host), link configuration module 281 establishes links or paths from the corresponding AI accelerator to at least some AI accelerators in that group or cluster, enabling the AI ​​accelerator to communicate with other AI accelerators, such as accessing their resources for AI processing. Similarly, in response to a request to switch from a first group of AI accelerators to a second group, link configuration module 281 terminates existing links from the corresponding AI accelerator in the first group and establishes new links to the second group of AI accelerators.

[0091] In one embodiment, the AI ​​accelerator 105 further includes an AI processing unit (not shown), which may include an AI training unit and an AI inference unit. The AI ​​training unit and the AI ​​inference unit may be integrated into a single unit in sensitive resource 270. The AI ​​training module is configured to train an AI model using a set of training data. The AI ​​model to be trained and the training data may be received from host system 104 via communication link 215. In one embodiment, the training data may be stored in non-sensitive resource 290. The AI ​​model inference unit may be configured to execute the trained AI model on a set of input data (e.g., a set of input features) to infer and classify the input data. For example, an image may be input into the AI ​​model to classify whether the image contains people, scenery, etc. The trained AI model and input data may also be received from host system 104 via interface 140 via communication link 215.

[0092] In one embodiment, a watermarking unit (not shown) in sensitive resource 270 may include a watermark generator and a watermark engraver (also referred to as a "watermark implanter"). The watermarking unit (not shown) may include a watermark kernel executor or kernel processor (not shown) of sensitive resource 270 to execute kernel 276. In an embodiment, the kernel may be received from host 104 or retrieved from persistent or non-persistent storage, and executed in kernel memory 276 in sensitive resource 270 of AI accelerator 105. The watermark generator is configured to generate a watermark using a predetermined watermarking algorithm. Alternatively, the watermark generator may inherit a watermark from an existing watermark or extract a watermark from another data structure or data object (such as an artificial intelligence model or a set of input data), which may be received from host system 104. The watermark implanter is configured to engrave or implant a watermark into a data structure (such as an artificial intelligence model or output data generated by an artificial intelligence model). The watermarked artificial intelligence model or output data may be returned from AI accelerator 105 to host system 104 via communication link 215. Note that AI accelerators 105 to 107 have the same or similar structure or components, and the description of AI accelerators throughout this application will apply to all AI accelerators.

[0093] Figure 3 This is a block diagram of a host 104 according to an embodiment, which controls a cluster 310 of artificial intelligence accelerators, each cluster having a virtualization function that maps the resources of a group 311 AI accelerators within the cluster to virtual machines on the host, each AI accelerator having secure and non-secure resources.

[0094] Host 104 may include (multiple) applications 205 (e.g., artificial intelligence (AI) applications), runtime libraries 206, one or more drivers 209, operating system 211, and hardware 213, as referenced above. Figure 2A and Figure 2B Each of the above has been described and will not be repeated here. In the virtual computing embodiment, host 104 may further include a hypervisor 212, such as vSphere / or Hypervisor 212 can be a Type 1 "bare-metal" or "native" hypervisor that runs directly on the physical server. In one embodiment, hypervisor 212 can be a Type 2 hypervisor that is loaded into and managed by operating system 211 like any other application. In either case, hypervisor 212 can support one or more virtual machines (not shown) on host 104. In this aspect, virtual machines (not shown) can be considered as... Figure 1 Client devices 101 and 102.

[0095] Artificial Intelligence (AI) Accelerator Cluster 310 may include the above references Figure 2A and Figure 2B The AI ​​accelerators described are 105 to 107. (Reference) Figure 3 AI accelerator cluster 310 may include, for example, eight (8) AI accelerators labeled A to H. Each AI accelerator in accelerator cluster 310 may have one or more communication links 215 to one or more other AI accelerators in accelerator cluster 310. (See above reference) Figure 2A and Figure 2B The AI ​​accelerator communication link 215 is described. Each AI accelerator in cluster 310 is configured according to a policy received from host 104 driver 209. Each AI accelerator in cluster 310 may have sensitive resources 270 and non-sensitive resources 290.

[0096] exist Figure 3In the example shown, AI accelerator AD is configured as four (4) AI accelerators in a first group 311. The resources of the AI ​​accelerators in the first group 311 are configured and managed by virtual function VF1 and associated with a first virtual machine. AI accelerator EH is configured as four (4) AI accelerators in a second group 312. The resources of the AI ​​accelerators in the second group 312 are configured and managed by virtual function VF2 and associated with a second virtual machine. The resources of the two groups 311 and 312 are mutually exclusive, and a user in either group cannot access the resources of the other group. In the AI ​​accelerators of the first group 311, each AI accelerator has a direct communication link to another accelerator, such as AB, AC, BD, and CD, or has a communication path to another accelerator via one or more intervening accelerators, such as ABD, ACD, etc. The second group 312 is shown as having a direct communication link between each AI accelerator in the second group 312 and each other AI accelerator in the second group 312. Driver 209 can generate a policy in which each AI accelerator in a group has a direct communication link with each or some of the other AI accelerators in the group. In the case of the first group 311, driver 209 can generate a policy that further includes, for example, instructions for enabling AI accelerators A and D to establish communication links with each other and AI accelerators B and C to establish communication links with each other. Cluster 310 can contain any number of AI accelerators configured into any number of groups.

[0097] In a static policy-based embodiment, a single policy defines the configuration for each AI accelerator and is transmitted from driver 209 to all AI accelerators in cluster 310. In another embodiment, driver 209 may transmit the policy to all AI accelerators in the cluster in a single broadcast message. Each AI accelerator reads the policy and either establishes a communication link (link connection) or disconnects a communication link (link disconnection) with one or more AI accelerators in cluster 310, thereby configuring the AI ​​accelerator into one or more groups. Figure 3In the cluster 310, there are eight (8) AI accelerators configured in two groups of four (4) AI accelerators each. Each AI accelerator in a group has a direct communication link to each other in the group, or an indirect communication path to each other via one or more AI accelerators (which have direct communication links to the one or more AI accelerators). In a static policy-based environment, the scheduler 209A of the driver 209 can schedule processing tasks on one or more groups of the cluster 310 using time slicing among users of the application 205 and / or the virtual machine. In an embodiment, each group of accelerators in the accelerator cluster 310 may have a different and separate scheduler 209A. The static policy can be changed by the driver 209 to generate a new policy describing the configuration of each AI accelerator in the cluster 310.

[0098] Each AI accelerator in cluster 310 (e.g., link configuration module 281 and / or resource manager 282) reconfigures itself according to a policy, thereby establishing (link connection) or disconnecting a communication link (link disconnection) between that AI accelerator and one or more other AI accelerators in cluster 310. Configuration based on static policies is fast because the configuration is transmitted in a single (e.g., broadcast) message, and each AI accelerator configures itself substantially in parallel with the other AI accelerators in cluster 310. Since the policy for all AI accelerators is transmitted to all AI accelerators simultaneously, configuration can occur very quickly. For example, if the policy includes instructions for AI accelerator "A" to generate a link to AI accelerator "B", then the policy also includes instructions for AI accelerator B to generate a link to AI accelerator A. Each AI accelerator can disconnect its own end of the link substantially simultaneously, thereby disconnecting the link between AI accelerator A and AI accelerator B very quickly. In one embodiment, a single policy can be expressed as an adjacency list of AI accelerators.

[0099] Static policy-based configuration is also effective because it supports time-slice scheduling among different users and allows the allocation of user processing tasks to more than one AI accelerator group in cluster 310. Static policies can be generated by determining the characteristics of the processing tasks in scheduler 209A from analyzer 209B. For example, scheduler 209A may include a large number of tasks that use the same AI model to perform inference or further train the AI ​​model. The analyzer can generate policies that configure several AI accelerators to prepare for performing inference or training on the AI ​​model. Configuration may include identifying groupings of AI accelerators and loading one or more AI models into the sensitive memory of one or more AI accelerators to prepare for processing tasks in scheduler 209A.

[0100] In a dynamic policy-based embodiment, driver 209 can individually configure each AI accelerator in cluster 310 to implement an AI accelerator configuration. Policies are transmitted individually to each AI accelerator. In practice, in a dynamic policy-based embodiment, the policies transmitted to each AI accelerator are typically different from each other. An AI accelerator receives a policy and configures itself accordingly. Configuration includes an AI accelerator configuring itself into or out of a group within cluster 310. Depending on the policy, an AI accelerator configures itself into a group by forming a communication link (link connection) with at least one AI accelerator in that group. An AI accelerator leaves a group by disconnecting the communication link between itself and all AI accelerators in the group (link disconnect). After configuration, if an AI accelerator is not a member of any group of AI accelerators, it can be set to a low-power model to reduce heat and save energy. In one embodiment, scheduler 209A assigns an AI accelerator or group of AI accelerators to each user or application that the scheduler 209A is scheduling processing tasks to be targeted by cluster 310.

[0101] Figure 4A This is a block diagram illustrating components of a data processing system according to an embodiment, the data processing system having an artificial intelligence (AI) accelerator to implement a method for virtual machine migration using checkpoint authentication in a virtualized environment.

[0102] The source host (HOST-S) 401 can support multiple virtual machines (VMs), such as the first (source) VM (VM1-S) to be migrated to the target host (HOST-T) 451 via network 103. Network 103 can be any network, as referenced above. Figure 1 As described, HOST-S 401 can also support additional VMs, such as VM2 and VM3. VM1-S, VM2, and VM3 (each labeled "402") can each include at least one application 403 and at least one driver 404. Driver 404 can include one or more function libraries and application programming interfaces (APIs) that enable VM402 containing driver 404 to communicate with one or more artificial intelligence (AI) accelerators 410, which are communicatively connected to VM 402 via hypervisor 405, CPU 406, and bus 407.

[0103] Hypervisor X 405 can be any type of hypervisor, including a "bare-metal" hypervisor running on the hardware of HOST-S 401, or a hypervisor that can run an operating system (not shown) on the host hardware (such as CPU 406 and memory (not shown)). CPU 406 can be any type of CPU, such as a general-purpose processor, multi-core processor, pipelined processor, parallel processor, etc. Bus 407 can be any type of high-speed bus, such as Peripheral Component Interconnect High Speed ​​(PCIe), Fibre Channel bus, or other types of high-speed bus. (See above reference) Figure 2A , Figure 2B and Figure 3 As described in communication channel 215, communication conducted via bus 407 can be encrypted. Bus 407 communicatively connects CPU 406 to the one or more artificial intelligence (AI) accelerators 410. Each VM may have a separate encrypted communication channel 215, which uses one or more keys that are different from the keys used by the encrypted communication channels 215 of each other VM.

[0104] Each AI accelerator 410 can host one or more virtual functions, such as VF1, VF2, ... VFn, each labeled with reference numeral 411 in Figure 4. Virtual function 411 maps resources 412 (e.g., RES1, RES2, ... RESn of accelerator ACC1 410) to specific host virtual machines 402. Each virtual machine 402 has a user. The virtual function 411 associated with a specific VM 402 (e.g., VM1-S) can only be accessed by the user of that specific VM 402 (e.g., VM1-S). Virtual machine resources are each labeled with reference numeral 412 in Figure 4. (See above for reference.) Figure 2B Virtual machine resources 412 are described, and include resources such as: non-sensitive resources 290 (including processing logic 291 and memory / storage device 292), accelerator channel management 280 (including link configuration 281 and resource manager 282), and sensitive resources 270 (including (multiple) AI models 275, (multiple) kernels 276, memory / storage device 277, and key manager 271, (multiple) key stores 272, true random number generator 273, and cryptographic engine 274). As described more fully below, after migrating a virtual machine (e.g., VM1-S) to a target host (e.g., HOST-T 451), at least sensitive resources should be erased such that after migrating the migrated virtual function of VM1-S to the target host HOST-T 451 and assigning the currently unused resources of the migrated virtual function of VM1-S to the new VM, the new VM will not have access to the sensitive data of the migrated VM1-S and the sensitive data associated with the virtual function (associated with VM1-S).

[0105] The target host (e.g., HOST-T 451) may have the same or similar hardware and software configuration as HOST-S 401. Accelerators 410 and 460 should be of the same or similar type, such as having compatible instruction sets for their respective processors. HOST-T 451 should have sufficient available resources, as VM-S might require, to allow VM1-S to migrate to VM1-T. Qualitatively, HOST-S 401 and HOST-T 451 should have compatible operating hardware and software. For example, HOST-S 401 accelerator 410 may be from the same manufacturer and model compatible with accelerator ACC2 460 on HOST-T 451; otherwise, the migration may not be successful.

[0106] Checkpoint 420 is a snapshot of the state of VM1-S up to and including virtual functions 411 (e.g., VF1) being migrated as part of the migration of VM1-S from HOST-S 401 to HOST-T 451. The checkpoint for VM1-S and the associated virtual functions may include the following information. In this embodiment, the checkpoint does not include information contained within resources 412 inside accelerator 410. The following list of information included in the checkpoint is for illustration and not limitation. Those skilled in the art can add or remove information from the checkpoint 420 for virtual machines and virtual functions to be migrated, as shown in the table below.

[0107]

[0108]

[0109] Checkpoint device 209C can be based on Figure 6The event described herein generates checkpoint frames 420. Checkpoint frames 420 can be generated, for example, at a specified time increment, upon detection of a system anomaly or failure, or upon receiving an instruction to retrieve checkpoint frames 420. Such instructions can originate from a user, such as an administrator or end user. The size of each checkpoint frame 420 can be approximately, for example, 1 gigabyte (GB). In an embodiment, checkpointer 209 may include a circular buffer that stores up to a specified number k of checkpoint frames 420. When the buffer is full, the next added frame overwrites the oldest checkpoint frame 420. When it is time to migrate virtual machines and virtual functions, the user can select a specific checkpoint frame 420 to use for performing the migration, the checkpoint frame representing the known state of the running application 403 preferred for migration by the user. In an embodiment, by default, the migration uses the most recent checkpoint frame 420. In an embodiment, during the migration of the source VM1-S, the checkpoint frame 420, its hash, and its date and timestamp can be digitally signed before the checkpoint frame 420 is transferred from the source VM1-S to the target host HOST-T 451.

[0110] When the hypervisor 455 of the target host HOST-T 451 receives the checkpoint frame 420, the hypervisor 455 can use the public key of VM1-S to decrypt the checkpoint frame 420, verify that the date and timestamp fall within a predetermined time window, and verify the hash of the checkpoint frame. Verifying the date and timestamp confirms the freshness of the checkpoint frame 420. If the hypervisor 455 of the target HOST-T 451 verifies the checkpoint frame 420, then the hypervisor 455 of HOST-T 451 can allocate resources at HOST-T 451 for the source VM1-S to generate VM1-T 452.

[0111] Now for reference Figure 4BCheckpointer 209 can further acquire AI accelerator status frame 421. AI accelerator status frame 421 differs from checkpoint frame 420 in that it captures information internal to AI accelerator(s) 410. The captured content of the AI ​​accelerator status frame may include the contents of one or more registers within the AI ​​accelerator, secure memory, and insecure memory (including, for example, AI models, kernels, intermediate inference computations, etc.). AI accelerator status frame 421 can be acquired synchronously with checkpoint frame 420, such that the information acquired at AI accelerator status frame 421 is "fresh" (current) relative to the most recent checkpoint frame 420 for the VM1-S to be migrated and its associated virtual function (which maps the allocation of AI accelerator 410 resources to virtual machines, such as VM1-S). In an embodiment, AI accelerator status frame 421 can be acquired after checkpoint frame 420 and after pending AI tasks of the currently executing application 403 have been stopped. This embodiment avoids storing the state of the AI ​​accelerator in the AI ​​accelerator state frame 421, which corresponds to the part of the process or thread that may be difficult to reliably restart after migration.

[0112] AI accelerator status frame 421 may contain the following information. This information is illustrative and not limiting. Those skilled in the art can add or remove information from the table for a specific system installation. During the migration of VM1-S, the frame, its hash, and its date and timestamp can be digitally signed with the private key of AI accelerator 410 or the private key of virtual machine VM1-S before the hypervisor 455 transmits the AI ​​accelerator status frame 421 to the target host HOST-T 451. When it is time to migrate virtual machine VM1-S and virtual functions, the user can select a specific AI accelerator status frame 421, or the frame 421 can be generated in response to the selection of checkpoint frame 420 and in response to receiving an instruction to migrate source VM1-S to target HOST-T 451. In embodiments, by default, the migration uses the AI ​​accelerator status frame 421 associated with the most recent checkpoint frame 420. In this embodiment, during the migration of the source VM1-S, the AI ​​accelerator status frame 421, its hash, and its date and timestamp can be digitally signed before the hypervisor 455 transmits the AI ​​accelerator status frame 421 from the source VM1-S to the target host HOST-T 451.

[0113] When the target host's management program 455 receives the AI ​​accelerator status frame 421, the management program can use the public key of VM1-S or, in this embodiment, the public key of the AI ​​accelerator 410 of VM1-S to decrypt the AI ​​accelerator status frame 421 to verify that the date and timestamp fall within a predetermined time window and to verify the hash of the AI ​​accelerator status frame 421. The check of the date and timestamp verifies the freshness of the AI ​​accelerator status frame 421. If the target host-T 451's management program 455 verifies the AI ​​accelerator status frame 421, then the host-T 451's management program 455 can copy the contents of the AI ​​accelerator status frame to the AI ​​accelerator ACC2 460 on VM1-T 452.

[0114]

[0115]

[0116] Figure 5A The illustration depicts a method 500 for migrating a data processing system with an AI accelerator in a virtualization environment using checkpoint authentication, from the perspective of a source hypervisor hosting the virtual machine to be migrated, according to an embodiment. Method 500 can be practiced on a source virtual machine (e.g., VM1-S) to be migrated to a target host (e.g., HOST-T 451 of the migrated virtual machine VM1-T).

[0117] In operation 600, the logic of VM1-S can determine whether to store a checkpoint frame 420 of VM1-S, which is running an application 403 utilizing one or more artificial intelligence (AI) accelerators (e.g., ACC1 410). The checkpoint frame 420 contains a snapshot of VM1-S, including application 403, the application's execution threads, a scheduler 209A containing the execution threads, application-related memory allocated by VM1-S, and virtual functions that map the resources of one or more AI accelerators to VM1-S, as referenced above. Figure 4A As described. In an embodiment, optionally, generating the checkpoint frame 420 may also trigger the acquisition of the AI ​​accelerator status frame 421. In an embodiment, the AI ​​accelerator status frame 421 may be generated and stored after one or more AI tasks associated with application 403 have been paused or stopped in operation 800. Reference below Figure 6 Please describe operation 600 in detail.

[0118] In operation 700, VM1-S can determine whether to migrate VM1-S. This decision can be based on receiving user commands, such as from an administrator or end user. In an embodiment, the decision to migrate VM1-S can be based on an anomaly or failure threshold that is higher than a certain threshold. See below for further details. Figure 7 Please describe operation 700 in detail.

[0119] In operation 800, in response to receiving a command to migrate VM1-S, the application, and the virtual functions for the associated AI accelerator to the target host 451, and in response to receiving a selection of a checkpoint frame 420 to be used for performing the migration, checkpointer 209C records the status of one or more ongoing AI tasks associated with the running application, and then stops or pauses said one or more ongoing AI tasks. VM1-S then begins process 800 for migrating VM1-S and the virtual functions to the target host. See below for reference. Figure 8 To describe operation 800.

[0120] In operation 900, in response to VM1-S receiving a notification from the hypervisor 455 of the target host 451 that the hypervisor 455 has successfully verified checkpoint 420 and the migration is complete, the source host's hypervisor commands the hypervisor 455 on the target host 451 to restart the migrated applications and recorded tasks in VM1-T. Optionally, VM1-S performs post-migration cleanup on VM1-S and the one or more AI accelerators associated with VM1-S via virtualization functions. See below for reference. Figure 9 This describes operation 900. Method 500 ends.

[0121] Figure 5B The illustration depicts a method 550 for migrating a virtual machine on a data processing system with an AI accelerator in a virtualized environment, from the perspective of a source hypervisor hosting the source virtual machine to be migrated, according to an embodiment. Method 550 can be practiced on the source virtual machine (e.g., VM1-S) to be migrated to the target host (e.g., HOST 2451 as the migrated virtual machine VM1-T).

[0122] In operation 800, in response to receiving a command to migrate VM1-S, the application running on VM1-S, and the virtual functions for the associated AI accelerator to target host 451, and in response to receiving a selection of a checkpoint frame 420 to be used for performing the migration, checkpointer 209C records the status of one or more ongoing AI tasks associated with the running application, and then stops or pauses said one or more ongoing AI tasks. VM1-S then begins process 800 for migrating VM1-S and the virtual functions to the target host. See below for reference. Figure 8 To describe operation 800.

[0123] In operation 551, after selecting checkpoint frame 420, VM1-S then generates or selects AI accelerator status frame 421 associated with the virtual functions of VM1-S. (See above for reference.) Figure 4B AI accelerator status frame 421 is described. A hash of AI accelerator status frame 421 is generated, along with its date and timestamp. The AI ​​accelerator status frame 421, the hash, and the date and timestamp are digitally signed using the private key of VM1-S or, in an embodiment, the private key of AI accelerator 410 (which is associated with a virtual function that maps AI resources to VM1-S). The digitally signed AI accelerator status frame 421 is then transmitted to the hypervisor 455 of the target host 451.

[0124] In operation 900, in response to receiving notification from the hypervisor 455 on the target host 451 that the checkpoint frame 420 and AI accelerator status frame 421 have been successfully verified and the migration is complete, the hypervisor 455 on the target host 541 restarts the application and recorded AI tasks within the migrated virtual machine VM1-T. Optionally, VM1-S can perform post-migration cleanup. See below for reference. Figure 9 The following describes operation 900 in detail, which includes post-migration cleanup of VM1-S and the one or more AI accelerators associated with VM1-S via virtualization. Method 550 ends.

[0125] Figure 6 The illustration depicts a method 600, according to an embodiment, for generating checkpoint frames from the perspective of a source manager hosting a virtual machine to be migrated, for use in a method for migrating a virtual machine using checkpoint authentication in a virtualized environment.

[0126] In operation 601, the hypervisor 405 in host 401 monitors the status of the source virtual machine (e.g., VM1-S), network status, AI accelerator status, and job completion progress.

[0127] In operation 602, it is determined whether the time increment used to generate checkpoint frame 420 has expired. This time increment can be set by a user or administrator and can be dynamically adjusted based on conditions. In an embodiment, the user adjusts the time increment, such as anticipating the need to migrate VM1-S, such as if the application running on VM1-S is not making sufficient progress, or for other reasons. In an embodiment, the time increment is fixed. In an embodiment, the time increment dynamically increases or decreases relative to the frequency or absence of faults, such that checkpoint frame 420 is generated more frequently if faults increase, or less frequently if faults decrease. If it is time to generate checkpoint frame 420, method 600 continues at operation 605; otherwise, method 600 continues at operation 603.

[0128] In operation 603, it is determined whether an anomaly or fault has occurred. A fault counter can be configured to handle one or more different types of faults with varying degrees of importance. Processor anomalies are far more significant than, for example, network faults in a network that supports retrying failed transmissions or receptions. Therefore, a processor fault can trigger the generation of checkpoint frame 420 with a count lower than the network fault count. If the occurrence of an anomaly or fault exceeds the fault count configured for the anomaly or fault type, method 600 continues at operation 605; otherwise, method 600 continues at operation 604.

[0129] In operation 604, it is determined whether the job progress is less than a threshold percentage of completion. In embodiments, the job progress can have multiple types of job progress counters. Each type of job progress counter can be triggered, for example, by calling a specific piece of source code or by invoking a specific AI function within an AI accelerator (such as a job progress counter for training an AI model or a counter for AI inference). The counter can be based on the expected execution time versus the actual execution time or other metrics. If the job progress counter indicates that the progress is less than a threshold percentage of the progress counter type, method 600 continues at operation 605; otherwise, method 600 terminates.

[0130] In operation 605, VM1-S generates VM1-S checkpoint frame 420, the running application, and the virtual function that maps AI accelerator resources to VM1-S.

[0131] In operation 606, optionally, the AI ​​accelerator status frame 421 may be generated after the checkpoint frame 420 is generated. Method 600 ends.

[0132] Figure 7The illustration illustrates a method 700 for determining, from the perspective of the source manager hosting the virtual machine to be migrated, whether to migrate a virtual machine of a data processing system with an AI accelerator in a virtualized environment using checkpoint authentication and / or AI accelerator status verification.

[0133] In Operation 701, the flag indicating whether to migrate the virtual machine (VM) is set to false.

[0134] In operation 702, it is determined whether the VM logic has received a user command to migrate the VM. In an embodiment, the migration command may originate from the user of the VM, who may be monitoring the progress of an AI application being executed. The reasons a user may choose to migrate the VM can be as known in the art: for example, the process is not progressing as adequately as might be expected, the particular host machine is heavily loaded or has limited resources, and there is a lack of factors contributing to the progress. If a user command to migrate the VM is received, method 700 continues at operation 705; otherwise, method 700 continues at operation 703.

[0135] In operation 703, it can be determined whether a command to migrate the VM has been received from the administrator. The administrator can periodically monitor the load on the server, the progress of one or more applications, and the availability of resources. The administrator can choose to send a migration command in response to a user request or based on the administrator's own judgment. If the administrator issues a command to migrate the VM, method 700 continues at operation 705; otherwise, method 700 continues at operation 704.

[0136] In operation 704, it can be determined whether the count of an anomaly or fault type has exceeded a threshold. Different thresholds may exist for different types of faults. For example, before triggering automatic migration based on fault counts, the count of processor anomalies may be very low, while the count of network faults may be much higher compared to processor faults. In an embodiment, instead of automatically initiating VM migration based on automatically detected conditions, a notification suggesting VM migration based on detected faults can be sent to the administrator. If the number of occurrences of any type of fault or anomaly exceeds the threshold associated with that fault or anomaly type, method 700 continues at operation 705; otherwise, method 700 terminates.

[0137] In operation 705, the migration flag is set to true. A selection of checkpoints to be used for the migration is also received. In the case of a user command or administrator command used to initiate the migration, the command may also include a checkpoint frame 420 to be used for the migration. In the case of an automatically initiated migration command, the checkpoint frame 420 may be generated automatically, or the most recent checkpoint frame 420 may be selected. In an embodiment, if the most recently stored checkpoint frame 420 is older than a threshold time, a new checkpoint frame 420 is generated.

[0138] In operation 706, optionally, an AI accelerator state frame 421 can be generated. In the case of an automatically generated migration command, the AI ​​accelerator state frame 421 can be automatically generated based on fault conditions and can be used with the migration. If either option is selected or an AI accelerator state frame is generated, method 550 can be executed. Figure 5B Otherwise, execute method 500. Figure 5A Method 700 ends.

[0139] Figure 8 The illustration depicts a method 800 for migrating a virtual machine with an AI accelerator data processing system in a virtualization environment using checkpoint authentication from the perspective of a source manager hosting the virtual machine to be migrated, according to an embodiment.

[0140] In operation 801, a selection is received for a target (destination) server (e.g., host 451) that will host the virtual machine being migrated (e.g., VM1-S).

[0141] In operation 802, one or more running AI tasks of a running application on VM1-S are stopped or paused. In an embodiment, one or more running AI tasks are allowed to complete, while others are paused or stopped.

[0142] In operation 803, the selected checkpoint frame 420 is transferred to the target host 451. The hypervisor 405 of VM1-S waits for a response from the target host confirming that the signature, date, timestamp, and hash of the checkpoint frame 420 have been verified.

[0143] In operation 804, the hypervisor 405 or driver 209 logs the AI ​​applications running on VM1-S and any associated unfinished AI tasks, and stops all unfinished AI tasks.

[0144] In operation 805, the VM1-S hypervisor 405 sends the recorded status of incomplete AI jobs to the hypervisor 455 of the target host 451. Method 800 ends.

[0145] Figure 9 The illustration depicts a method 900 for performing post-migration cleanup on a source virtual machine after migrating a virtual machine of a data processing system with an AI accelerator in a virtualized environment using checkpoint authentication.

[0146] In operation 901, the hypervisor 405 of the source virtual machine (VM1-S) receives a notification from the hypervisor 455 of the target host 451 that the signature, date, timestamp, and hash of the checkpoint frame 420 have all been verified. In an embodiment, the notification may also include an indication that the signature, date, timestamp, and hash of the AI ​​accelerator status frame 421 have been verified. The notification may further indicate that the migration of VM1-S to the target host 451 is complete, and that the application and incomplete AI tasks have been restarted at the migrated virtual machine (VM1-T) on the target host 451.

[0147] In operation 902, the hypervisor 405 and / or driver 404 of the source host 401 may at least erase the security memory of the AI ​​accelerator used by the source VM1-S. The hypervisor 405 and / or driver 404 may also erase memory used by applications on the VM1-S that make calls to APIs or drivers associated with the AI ​​accelerator via virtual functions associated with the VM1-S.

[0148] In operation 903, the hypervisor 405 of the source host 401 can deallocate resources of VM1-S, including AI accelerator resources used by VM1-S and associated with virtual functions that map AI accelerator resources to VM1-S. Method 900 ends.

[0149] Figure 10 The illustration depicts a method 1000 for migrating a virtual machine with an AI accelerator data processing system in a virtualization environment using checkpoint authentication, from the perspective of a target hypervisor of a host hosting the virtual machine to be migrated, according to some embodiments.

[0150] In operation 1001, the hypervisor 455 of the target host machine 451 receives a checkpoint frame 420 from a source virtual machine (e.g., VM1-S) associated with a virtualization function that maps AI processor resources to VM1-S. The hypervisor 455 also receives a request for VM1-S hosted as a migrated virtual machine (VM1-T) on the target host 451.

[0151] In operation 1002, the hypervisor 455 on host 451 calculates and reserves resources for generating VM1-S as VM1-T on host 451. The hypervisor 455 allocates and configures resources for hosting VM1-S and its associated virtual functions based on the received checkpoint frame 420.

[0152] In operation 1003, the hypervisor 455 at the target host 451 receives data frames as part of migrating VM1-S to VM1-T, and acknowledges these data frames to the hypervisor 405 at the source host 401. The hypervisor 455 stores the received frames on host 451, enabling the hypervisor 455 to generate VM1-T.

[0153] In operation 1004, optionally, the hypervisor 455 at the target host 451 receives a signed AI accelerator status frame 421 from the hypervisor 505 at the source host 401. The hypervisor 455 uses the public key of VM1-S or the public key of the VM1-S accelerator to decrypt the signed AI accelerator frame 421. The hypervisor 455 verifies the date and timestamp in frame 421 and verifies the digest of frame 421. If the signed AI accelerator status frame 421 is successfully verified, the hypervisor 455 loads the data from the AI ​​accelerator status frame 421 into the AI ​​accelerator and configures the AI ​​accelerator according to the data in the AI ​​accelerator status frame 421.

[0154] In operation 1005, the hypervisor 455 of the target host 451 receives the recorded status of unfinished AI tasks of the application running on VM1-S. VM1-T restarts the application and the unfinished AI tasks on VM1-T.

[0155] In operation 1006, the hypervisor 455 on the target host 451 sends a notification to the source hypervisor 405 on the source host 401, indicating that the restart of the application and the unfinished AI tasks were successful and that the migration from VM1-S to VM1-T was successful.

[0156] Some of the parts described in detail above have been presented based on algorithms and symbolic representations of operations on data bits within computer memory. These algorithmic descriptions and representations are the most efficient way for those skilled in the art of data processing to communicate the essence of their work to others skilled in the art. Here, an algorithm is generally conceived as a self-consistent sequence of operations that leads to a desired result. These operations are those that require physical manipulation of physical quantities.

[0157] However, it should be remembered that all these terms and similar terms will be associated with appropriate physical quantities and are merely convenient labels suitable for those quantities. Unless otherwise expressly stated (as is apparent from the above discussion), it should be understood that throughout the specification, discussions using terms (such as those set forth in the following claims) refer to the operation and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the registers and memories of the computer system and converts said data into other data similarly represented as physical quantities within the computer system's memory or registers or other such information storage, transmission, or display devices.

[0158] Embodiments of this disclosure also relate to an apparatus for performing the operations described herein. Such a computer program is stored in a non-transitory computer-readable medium. Machine-readable media include any mechanism for storing information in a machine-readable (e.g., computer-readable) form. For example, machine-readable (e.g., computer-readable) media include machine-readable (e.g., computer-readable) storage media (e.g., read-only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices).

[0159] The processes or methods depicted in the preceding figures can be executed by processing logic, which includes hardware (e.g., circuitry, special-purpose logic, etc.), software (e.g., embodied in a non-transitory computer-readable medium), or a combination of both. Although the processes or methods have been described above in terms of some sequential operations, it should be understood that some of the described operations can be performed in a different order. Furthermore, some operations can be performed in parallel rather than sequentially.

[0160] Embodiments of this disclosure are described without reference to any particular programming language. It will be understood that the teachings of the embodiments of this disclosure as described herein can be implemented using various programming languages.

[0161] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and drawings are to be considered illustrative rather than restrictive.

Claims

1. A method for migrating a virtual machine (VM-S) in a virtualization environment using checkpoint authentication, the method comprising: The checkpoint of one state of the VM-S is stored in a storage device for multiple states of the VM-S, wherein each state of the VM-S includes the state of the VM-S resources, the state of the application, and the state of the AI ​​accelerator mapping AI resources to the virtual functions of the VM-S. In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state to be used to perform the migration: Record, and then stop one or more ongoing AI tasks; Using the selected checkpoint, migrate the VM-S, the application, the one or more AI tasks, and the virtual functionality to the target virtual machine VM-T on the target host; and In response to receiving a notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the VM-S hypervisor commands the target host hypervisor to restart one or more recorded AI tasks and restart the application.

2. The method according to claim 1, wherein, The virtual function maps the resources of the AI ​​accelerator to the VM-S, and the user of the VM-S is the only user who can access the resources of the AI ​​accelerator that are mapped to the VM-S through the virtual function.

3. The method according to claim 2, wherein, The virtual function maps the resources of multiple AI accelerators to the VM-S, and the checkpoint includes the communication configuration between the multiple AI accelerators, and the user of the VM-S is the only user who can access the resources of the multiple AI accelerators mapped to the VM-S through the virtual function.

4. The method of claim 1, further comprising: In response to receiving a notification from the target management program that the migration of the VM-S is complete and the one or more recorded AI tasks have been successfully restarted: At least erase the secure memory of the AI ​​accelerator, including any AI inference, AI models, intermediate computations for secure computing, or portions thereof; as well as Erase the memory of the VM-S associated with the virtual function, as well as any calls to the virtual function made by the application.

5. The method according to claim 1, wherein, Verification includes verifying the signature and freshness date of the checkpoint, including: Use the public key of the VM-S to decrypt the signature of the checkpoint; Determine that the date and timestamp of the checkpoint are within the threshold date and time range; and Verify the hash of the checkpoint of the VM-S.

6. The method according to claim 1, wherein, The checkpoints include: A record of one or more AI tasks in progress; Configuration information of resources within one or more AI accelerators that are communicatively connected to the VM-S; The date and timestamp of the checkpoint; and A snapshot of the VM-S's memory, including virtual function scheduling information and communication buffers within the one or more AI accelerators.

7. A computer-readable medium programmed with executable instructions, which, when executed by a processing system having at least one hardware processor communicatively connected to at least one artificial intelligence (AI) processor, perform the method of any one of claims 1-6.

8. A system for migrating virtual machines using checkpoint authentication in a virtualized environment, comprising at least one hardware processor coupled to instruction-programmable memory, the instructions, when executed by the at least one hardware processor communicatively coupled to an artificial intelligence (AI) accelerator, causing the system to perform operations for migrating a source virtual machine VM-S that is executing an application accessing virtual functions of the AI ​​accelerator, the operations including: The checkpoint of one state of the VM-S is stored in a storage device for multiple states of the VM-S, wherein each state of the VM-S includes the state of the VM-S resources, the state of the application, and the state of the virtual functions of the AI ​​accelerator. In response to receiving a command to migrate the VM-S and the virtual functions, and receiving a selection of a checkpoint for the state to be used to perform the migration: Record, and then stop one or more ongoing AI tasks; Using the selected checkpoint, migrate the VM-S, the application, the one or more AI tasks, and the virtual functionality to the target virtual machine VM-T on the target host; and In response to receiving a notification from the target hypervisor that the checkpoint has been successfully verified by the target hypervisor and that the migration has been successfully completed, the VM-S hypervisor commands the target host hypervisor to restart one or more recorded AI tasks and restart the application.

9. The system according to claim 8, wherein, The virtual function maps the resources of the AI ​​accelerator to the VM-S, and the user of the VM-S is the only user who can access the resources of the AI ​​accelerator that are mapped to the VM-S through the virtual function.

10. The system according to claim 9, wherein, The virtual function maps the resources of multiple AI accelerators to the VM-S, and the checkpoint includes the communication configuration between the multiple AI accelerators, and the user of the VM-S is the only user who can access the resources of the multiple AI accelerators mapped to the VM-S through the virtual function.

11. The system according to claim 8, wherein, The operation further includes: In response to receiving a notification from the target management program that the migration of the VM-S is complete and the one or more recorded AI tasks have been successfully restarted: At least erase the secure memory of the AI ​​accelerator, including any AI inference, AI models, intermediate computations for secure computing, or portions thereof; and Erase the memory of the VM-S associated with the virtual function, as well as any calls to the virtual function made by the application.

12. The system according to claim 8, wherein, Verification includes verifying the signature and freshness date of the checkpoint, including: Use the public key of the VM-S to decrypt the signature of the checkpoint; Determine that the date and timestamp of the checkpoint are within the threshold date and time range; and Verify the hash of the checkpoint of the VM-S.

13. The system according to claim 8, wherein, The checkpoints include: A record of one or more AI tasks in progress; Configuration information of resources within one or more AI accelerators that are communicatively connected to the VM-S; The date and timestamp of the checkpoint; and A snapshot of the VM-S's memory, including virtual function scheduling information and communication buffers within the one or more AI accelerators.

14. A system for migrating virtual machines using checkpoint authentication in a virtualized environment, comprising at least one hardware processor coupled to instruction-programmable memory, the instructions, when executed by the at least one hardware processor, causing the system to perform operations for migrating a source virtual machine VM-S that is executing virtual functions for accessing an artificial intelligence (AI) accelerator, the operations including: The target host's hypervisor receives a checkpoint from the source virtual machine VM-S associated with the virtualization function VF, the VF mapping AI processor resources to the VM-S; and receives a request for the VM-S hosted as the target virtual machine VM-T. The target host's management program allocates and configures resources for hosting the VM-S as the VM-T and the VM-S's VF based on the checkpoint; The management program of the target host receives the data frame of the VM-S and stores the data frame to generate the VM-T; The management program of the target host receives the recorded status of the unfinished AI task of the VM-S, and restarts the unfinished AI task on the VM-T.

15. The system according to claim 14, wherein, The operation further includes: Verify the checkpoint received from the VM-S, wherein the verification includes: Use the public key of the VM-S to decrypt the signature of the checkpoint; The date and timestamp of the checkpoint are determined to fall within a predetermined range; Recalculate the hash of the checkpoint and determine whether the recalculated hash matches the hash stored in the checkpoint; and In response to the successful verification of the checkpoint, the management procedure for migrating the VM-S to the target host as VM-T continues.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images