A GPU remote hot migration method, system, device and storage medium

Abstract

The present invention provides a GPU remote hot migration method, system, device and storage medium. The GPU remote hot migration method includes: setting a computing task in a first physical machine; a second physical machine equipped with a GPU calculates the computing task of the first physical machine; the second physical machine responds to the hot migration request of the control end, and the control end sends an indication of suspending the computing task process to the first physical machine, and the second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine equipped with a GPU; after the memory data is transmitted, the first physical machine and the second physical machine continue to execute the computing task. The present invention suspends the computing task of the first physical machine during the hot migration process, then performs data migration, and continues to execute the task after the transmission, thereby solving the problem of discontinuity of service caused by migration, and the problem of long-term interruption of tasks affecting the overall performance of the system and user experience.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a GPU remote hot migration method, system, device and storage medium. Background Art

[0002] In recent years, GPUs (graphics processing units) have been widely used in high-performance computing, deep learning, and scientific computing. With the increase in the complexity of computing tasks and the amount of data, how to efficiently manage and schedule GPU resources has become an important research topic. In cloud computing and distributed computing environments, the application of GPU hot migration technology can significantly improve the flexibility and resource utilization of the system. GPU hot migration technology includes steps such as generating snapshots, migrating snapshots, and restoring snapshots. Through these steps, computing tasks can be hot migrated between different GPU nodes, thereby improving the reliability and resource utilization of the system.

[0003] Traditional GPU computing task migration usually requires stopping the execution of the task, waiting for the task status to be saved, and then migrating the task to the new GPU node. In systems with high availability requirements, this interruption-based migration method will lead to service discontinuity, fail to meet the needs of real-time computing, and cause long-term interruption of tasks, affecting the overall performance of the system and user experience. Summary of the invention

[0004] In view of this, the embodiments of the present application are committed to providing a GPU remote hot migration method, system, device and storage medium, which are used to solve the problems in the prior art that interruptive migration leads to service discontinuity and long-term task interruption affects the overall system performance and user experience.

[0005] According to a first aspect of the present invention, the present invention provides a GPU remote hot migration method, comprising:

[0006] Setting the computing task in the first physical machine;

[0007] A second physical machine equipped with a GPU computes the computing task of the first physical machine;

[0008] The second physical machine responds to the hot migration request of the control end, and the control end sends an instruction to suspend the computing task process to the first physical machine, and the second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine on which a GPU is installed;

[0009] After the memory data is transmitted, the first physical machine and the second physical machine continue to execute the computing task.

[0010] Preferably, the first physical machine deploys a client, and the second physical machine and the third physical machine deploy a server.

[0011] Preferably, after the second physical machine responds to the hot migration request of the control terminal, the following steps are included:

[0012] The control end confirms whether the local memory of the second physical machine meets the snapshot condition;

[0013] When the local memory of the second physical machine meets the snapshot condition, the control end confirms whether the third physical machine meets the conditions for receiving the snapshot, loading the snapshot and resuming the operation, or the second physical machine confirms whether the third physical machine meets the conditions for receiving the snapshot, loading the snapshot and resuming the operation, and returns the confirmation information to the control end;

[0014] When the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, the hot migration is performed.

[0015] Preferably, the control end sends an instruction to suspend the computing task process to the first physical machine, including:

[0016] When the first physical machine executes the computing task process suspension indication, the address of the first physical machine connected to the first physical machine in the second physical machine is converted into the address of the third physical machine, and a connection between the second physical machine and the third physical machine is established.

[0017] Preferably, the second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine on which a GPU is installed, including:

[0018] The second physical machine dumps the memory data related to the computing task process of the first physical machine to the local disk of the second physical machine, and unloads the data related to the computing task process in the GPU of the second physical machine;

[0019] The second physical machine transfers the memory data in the local disk to the third physical machine.

[0020] Preferably, after the second physical machine transfers the memory data in the local disk to the third physical machine, the method further includes:

[0021] The third physical machine loads the received memory data into the local memory of the third physical machine, and restores the computing task related data in the GPU of the third physical machine.

[0022] Preferably, after the memory data is transmitted, the first physical machine and the third physical machine continue to execute the computing task, including:

[0023] After the second physical machine transfers the memory data to the third physical machine, the control end notifies the first physical machine to resume the computing task process. At the same time, the first physical machine establishes a memory mapping relationship with the third physical machine, and the third physical machine continues to execute the computing task.

[0024] According to a second aspect of the present invention, a GPU remote hot migration system is also provided, comprising:

[0025] The first physical machine is used to place computing tasks;

[0026] A second physical machine, having a GPU installed therein, for computing the computing task of the first physical machine and responding to a hot migration request from the control terminal;

[0027] A third physical machine, having a GPU installed therein, for receiving memory data related to the computing task process transmitted by the second physical machine;

[0028] The control end is used to send the hot migration request to the second physical machine and send an instruction to suspend the computing task process to the first physical machine.

[0029] According to a third aspect of the present invention, an electronic device is also provided, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the GPU remote hot migration method.

[0030] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the GPU remote hot migration method.

[0031] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0032] According to the first aspect of the present invention, during migration, the control end sends a migration command to the second physical machine, and at the same time notifies the first physical machine to suspend the task being executed, and then dumps the data of the computing task executed by the first physical machine to the third physical machine through the second physical machine. After the dump, the first physical machine and the second physical machine continue to execute the suspended computing tasks. This method ensures the consistency of data and the continuity of tasks during the hot migration process, thereby improving the overall performance of the system and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic flow chart of a GPU remote hot migration method according to an embodiment of the present invention is shown;

[0034] Figure 2 The present invention shows Figure 1 A schematic flow chart after responding to the hot migration request in step S300;

[0035] Figure 3 A schematic flow chart of a multi-terminal interaction scenario according to an embodiment of the present invention is shown;

[0036] Figure 4 A schematic flow chart of a GPU remote hot migration system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0038] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0039] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0040] Figure 1 FIG. 4 shows a schematic flow chart of a GPU remote hot migration method according to an embodiment of the present invention. Figure 1 As shown, the present invention provides a GPU remote hot migration method, comprising:

[0041] Step S100, setting a computing task in a first physical machine;

[0042] Step S200, a second physical machine equipped with a GPU computes a computing task of the first physical machine;

[0043] Step S300, the second physical machine responds to the hot migration request of the control end, and the control end sends an instruction to suspend the computing task process to the first physical machine, and the second physical machine transmits the memory data related to the computing task process of the first physical machine to the third physical machine on which the GPU is installed;

[0044] Step S400: After the memory data is transmitted, the first physical machine and the second physical machine continue to execute the computing task.

[0045] During migration, the control end sends a migration command to the second physical machine, and at the same time notifies the first physical machine to suspend the task being executed. Then, the second physical machine dumps the data of the computing task executed by the first physical machine to the third physical machine. After the dump, the first physical machine and the second physical machine continue to execute the suspended computing task. This method ensures the consistency of data and the continuity of tasks during the hot migration process, thereby improving the overall performance of the system and user experience.

[0046] The first physical machine deploys the client, and the second and third physical machines deploy the server.

[0047] Figure 2 The present invention shows Figure 1 A schematic flow chart after responding to the hot migration request in step S300;

[0048] like Figure 2 As shown, in step S300, after the second physical machine responds to the hot migration request of the control end, it includes:

[0049] Step S310, the control end confirms whether the local memory of the second physical machine meets the snapshot condition;

[0050] Step S320, when the local memory of the second physical machine meets the snapshot condition, the control end confirms whether the third physical machine meets the conditions for receiving the snapshot, loading the snapshot and resuming the operation, or the second physical machine confirms whether the third physical machine meets the conditions for receiving the snapshot, loading the snapshot and resuming the operation, and returns the confirmation information to the control end;

[0051] Step S330 : When the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming operation, hot migration is performed.

[0052] In this step, the control end first checks whether the local memory of the second physical machine meets the snapshot conditions, and then checks whether the third physical machine meets the conditions of receiving snapshots, loading snapshots and resuming operations. When all the above conditions are met, hot migration can be performed. In this process, the control end confirms the conditions of the second physical machine and the third physical machine respectively, and this confirmation mechanism can ensure the reliability of migration.

[0053] In step S300, the control end sends a suspend computing task process instruction to the first physical machine, including:

[0054] Step S340, when the first physical machine executes the computing task process suspension indication, the address of the first physical machine connected to the first physical machine in the second physical machine is converted into the address of the third physical machine, and the connection between the second physical machine and the third physical machine is established.

[0055] In this step, when the first physical machine executes a computing task, the second physical machine needs to establish a connection with the first physical machine and perform computing on the second physical machine. When migrating, the address of the first physical machine on which the second physical machine establishes a connection will be changed to the address of the third physical machine. By changing the address, the second physical machine establishes a connection with the third physical machine. This dynamic address conversion mechanism simplifies network configuration and improves system flexibility.

[0056] In step S300, the second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine on which a GPU is installed, including:

[0057] Step S350, the second physical machine dumps the memory data related to the computing task process of the first physical machine to the local disk of the second physical machine, and unloads the computing task process related data in the GPU of the second physical machine;

[0058] Step S360: The second physical machine transfers the memory data in the local disk to the third physical machine.

[0059] The second physical machine dumps the memory data lines related to the computing task process of the first physical machine to the local disk of the second physical machine, and then dumps the content of the local disk to the third physical machine. Using the local disk as an intermediary enhances the stability and security of data transmission. If the data is not transferred to the disk first, when an unexpected situation occurs, the data that needs to be hot migrated may be lost. After the second physical machine dumps the content to the local disk, the data in the GPU is unloaded to prevent data loss caused by dump failure.

[0060] In step S360, after the second physical machine transfers the memory data in the local disk to the third physical machine, the process further includes:

[0061] The third physical machine loads the received memory data into the local memory of the third physical machine, and resumes the computing task in the GPU of the third physical machine to perform related data.

[0062] In this step, after receiving the memory data, the third physical machine can restore the computing task in the local memory and GPU. This mechanism ensures that the computing task can continue seamlessly after migration.

[0063] In step S400, after the memory data is transmitted, the first physical machine and the third physical machine continue to perform the computing task, including:

[0064] After the second physical machine transfers the memory data to the third physical machine, the control end notifies the first physical machine to resume the computing task process, and at the same time, the first physical machine establishes a memory mapping relationship with the third physical machine, and the third physical machine continues to execute the computing task.

[0065] In this step, a mapping relationship is established between the first physical machine and the third physical machine to ensure the consistency of data and the correctness of calculation results.

[0066] In one embodiment of the present invention, Figure 3 FIG. 4 shows a schematic flow chart of a multi-terminal interaction scenario according to an embodiment of the present invention; Figure 3 As shown, we are going to deploy remote computing client A (first physical machine), remote computing server B (second physical machine) equipped with GPU, and remote computing server C (third physical machine) equipped with GPU. First, we set computing tasks for client A, then start the client computing tasks, and server B runs the GPU computing tasks of client A. Data will be generated in this process.

[0067] When server B is running the GPU task of client A, and server B receives a request for hot migration of the task from server B to server C, the control end first checks whether server B meets the snapshot conditions. If the snapshot conditions are met, it then checks whether server C meets the migration requirements.

[0068] When server B meets the snapshot conditions and server C meets the migration requirements, server B saves the data about client A process locally, and the control end suspends client A process. Server B and server C establish a connection through the address of server C, and server C receives the data sent by server B.

[0069] After server B saves the data about client A's process locally, it releases the video memory and computing power occupied by client A's computing task in the GPU. After server C receives the data sent by server B, it loads the data sent by server B and restores the video memory. At this time, client A resumes the suspended client process, and at the same time, client A establishes a connection with server C, restores the memory mapping, and continues to execute the task. Server C establishes a connection with client A and runs client A's computing task.

[0070] In one embodiment of the present invention, as shown in the figure, there are three servers, wherein the first server runs the client for client A, the second and third servers run the server for server B and server C, and a remote computing task is run on client A using the GPU on server B for calculation.

[0071] Now initiate a hot migration request to migrate the computing task from server B to server C. At this time, server B receives the hot migration task and first checks whether the conditions for server B to establish a snapshot are met, establishes a connection between server B and server C, and confirms whether server C meets the conditions for restoring the snapshot.

[0072] Server B notifies client A to execute a command to suspend the client process. The code is "pkill -STOP -x my_process". The server BDumps (Dump Memory Data) and the memory related to the client A process to the local disk, and unloads the video memory and code blocks related to the server A process on the GPU in server B, releases the video memory, the code is "cudaFree", cancels the module registration, the code is "cuModuleUnload", and destroys the CUDA (Compute Unified Device Architecture, parallel computing platform and programming model) context, the code is "cuCtxDestroy".

[0073] Server B sends the dumped memory data to server C. After receiving the data, server C loads it locally and restores the data in the GPU through the following steps: code "cuCtxCreate" loads the CUDA module, code "cuModuleLoad" gets the CUDA function address, code "cuModuleGetFunction", and then allocates memory, code "cudaMalloc", and transfers the memory to the video memory, code "cudaMemcpy".

[0074] After completion, client A is notified, and the client executes the code "pkill -CONT -x my_process" command to resume the process. The process is connected to server C through configuration, and then the memory mapping relationship between client A and server C is established, and then the computing task is continued.

[0075] Figure 4 FIG. 4 shows a schematic flow chart of a GPU remote hot migration system according to an embodiment of the present invention. Figure 4 As shown, a GPU remote hot migration system is also provided, including:

[0076] A first physical machine 510 is used to place computing tasks;

[0077] The second physical machine 520 has a GPU installed therein, and is used to calculate the computing task of the first physical machine and respond to the hot migration request of the control terminal;

[0078] A third physical machine 530, having a GPU installed therein, for receiving memory data related to the computing task process transmitted by the second physical machine;

[0079] The control terminal 540 is used to send the hot migration request to the second physical machine, and send an instruction to suspend the computing task process to the first physical machine.

[0080] In particular, an electronic device is also provided, the electronic device includes a processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the GPU remote hot migration method, for example, to execute the above-described Figure 1 Steps S100 to S400 of the method described above are also performed. Figure 2 The method comprises steps S310 to S330.

[0081] In particular, a computer-readable storage medium is also provided, the computer-readable storage medium storing computer-executable instructions, when the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the GPU remote hot migration method, for example, to execute the above-described Figure 1 Steps S100 to S400 of the method described above are also performed. Figure 2 The method comprises steps S310 to S330.

[0082] The processor is used to complete various control logics of the device, which can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor can also be any traditional processor, microprocessor or state machine. The processor can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0083] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0084] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0085] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0086] The above are only some specific implementation methods of the present application, and any other improvements made based on the concept of the present application are deemed to be within the protection scope of the present application.

Claims

1. A GPU remote hot migration method, characterized in that: include: Setting the computing task in the first physical machine; A second physical machine equipped with a GPU computes the computing task of the first physical machine; The second physical machine responds to the hot migration request of the control end, and the control end sends an instruction to suspend the computing task process to the first physical machine, and the second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine on which a GPU is installed; After the second physical machine responds to the hot migration request of the control end, the control end includes: the control end confirms whether the local memory of the second physical machine meets the snapshot condition; when the local memory of the second physical machine meets the snapshot condition, the control end confirms whether the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, or the second physical machine confirms whether the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, and returns the confirmation information to the control end; when the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, the hot migration is performed; The control end sends an instruction to suspend the computing task process to the first physical machine, including: when the first physical machine executes the instruction to suspend the computing task process, the address of the first physical machine connected to the first physical machine in the second physical machine is converted into the address of the third physical machine, and a connection is established between the second physical machine and the third physical machine; After the memory data is transmitted, the first physical machine and the third physical machine continue to execute the computing task.

2. The method according to claim 1, characterized in that The first physical machine deploys a client, and the second physical machine and the third physical machine deploy a server.

3. The method according to claim 1, characterized in that The second physical machine transmits memory data related to the computing task process of the first physical machine to a third physical machine on which a GPU is installed, including: The second physical machine dumps the memory data related to the computing task process of the first physical machine to the local disk of the second physical machine, and unloads the data related to the computing task process in the GPU of the second physical machine; The second physical machine transfers the memory data in the local disk to the third physical machine.

4. The method according to claim 3, characterized in that After the second physical machine transfers the memory data in the local disk to the third physical machine, the method further includes: The third physical machine loads the received memory data into the local memory of the third physical machine, and restores the computing task related data in the GPU of the third physical machine.

5. The method according to claim 1, characterized in that After the memory data is transmitted, the first physical machine and the third physical machine continue to execute the computing task, including: After the second physical machine transfers the memory data to the third physical machine, the control end notifies the first physical machine to resume the computing task process. At the same time, the first physical machine establishes a memory mapping relationship with the third physical machine, and the third physical machine continues to execute the computing task.

6. A GPU remote hot migration system, characterized in that: include: The first physical machine is used to place computing tasks; A second physical machine, having a GPU installed therein, for computing the computing task of the first physical machine and responding to a hot migration request from the control terminal; A third physical machine, having a GPU installed therein, for receiving memory data related to the computing task process transmitted by the second physical machine; A control end, used for sending the hot migration request to the second physical machine, and sending an instruction to suspend the computing task process to the first physical machine; After the second physical machine responds to the hot migration request of the control end, the control end includes: the control end confirms whether the local memory of the second physical machine meets the snapshot condition; when the local memory of the second physical machine meets the snapshot condition, the control end confirms whether the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, or the second physical machine confirms whether the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, and returns the confirmation information to the control end; when the third physical machine meets the conditions of receiving the snapshot, loading the snapshot and resuming the operation, the hot migration is performed; The control end sends an instruction to suspend the computing task process to the first physical machine, including: when the first physical machine executes the instruction to suspend the computing task process, the address of the first physical machine connected to the first physical machine in the second physical machine is converted into the address of the third physical machine, and a connection is established between the second physical machine and the third physical machine; After the memory data is transmitted, the first physical machine and the third physical machine continue to execute the computing task.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the GPU remote hot migration method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the GPU remote hot migration method according to any one of claims 1 to 5.

Images