Low-delay GPU server cold start method and system

Abstract

The invention provides a low-delay GPU server cold start method and system. The method comprises the steps that S1, a checkpoint tool is called to obtain an application mirror image; s2, obtaining a GPU data access sequence through offline sampling; s3, transferring the right of ownership of the GPU data to a Restore Server; s4, recovering according to a GPU data access sequence obtained by offline sampling; s5, after the to-be-recovered process is recovered, the to-be-recovered process is converted into the target process to start running, and the GPU data continues to be recovered in the background Restore Server. According to the method, all the Restore processes transfer the ownership of a GPU data area to the Server, and GPU data recovery is carried out in a unified mode in the background; the recovery of the CPU data and the GPU data is decoupled to different processes, parallel recovery can be realized, and the recovery process is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of serverless technology, and in particular to a low-latency GPU serverless cold start method and system. Background Art

[0002] As technology advances, serverless technology is also evolving. Users upload their application code to the serverless platform, which then dynamically scales capacity based on load. The key to scaling capacity is to quickly start new application instances, a process called cold start.

[0003] Since the cold start process is time-consuming and expensive, the industry generally adopts some caching strategies to reduce the number of cold starts, such as reserving a certain number of additional instances, extending the instance scaling-down time, lowering the scaling-down standard, etc.; however, although these methods are simple to implement, they have high overhead, and the corresponding effects are heavily dependent on specific caching strategies. They can only bypass some cold start times and cannot directly accelerate the cold start process.

[0004] In the Chinese patent document with publication number CN111559278A, a system and method for ultra-low temperature cold start of a series hybrid electric vehicle is disclosed, the system includes: a high-voltage subsystem and a control subsystem; wherein the high-voltage subsystem mainly includes a power battery, an anti-reverse diode, an intelligent power unit, a drive motor, and a battery heating film, and the intelligent power unit includes a generator and an engine; the control subsystem includes a vehicle controller, a battery manager, and a distribution box controller, and the battery manager and the distribution box controller are both connected to the vehicle controller. However, this patent document is mainly intended to solve the short circuit problem caused by charging of the power battery after the series hybrid electric vehicle is started at low temperature, which is different from the technical problem to be solved by this application.

[0005] Reference Figure 1 The figure shows the system architecture of the prior art 1. It includes two parts: GPU application and GPU server. Different from the traditional way of directly accessing and using GPU by application, this technology adopts a proxy-based design. It separates the part interacting with GPU from the application and uses a pre-started GPU server as a proxy. The application sends GPU commands to the server, and the server agent interacts with GPU.

[0006] Reference Figure 2 As shown, it is a flow chart of the method of prior art 1. The specific steps are:

[0007] Step 1: Prepare the GPU server offline; Step 2: Enter the online part and start the application; Step 3: Establish a connection with the pre-established GPU server; Step 4: Load the model.

[0008] Reference Figure 3 As shown, this is the disadvantage of the prior art 1. This solution can only optimize the GPU-related initialization part, and cannot reduce the overhead of loading the framework and loading the model. Moreover, this solution requires additional communication with the GPU service process when the program is running, which introduces additional communication overhead.

[0009] Reference Figure 4 The figure shows the system architecture of the second prior art, which includes the reorganized model files and the block memory management system. The model files are reorganized in the order of access so that they can be read sequentially when loaded. The block memory management system loads the model files into the memory in block granularity through multithreading and Direct IO, and copies the data from the memory to the video memory in parallel, thereby speeding up the cold start process.

[0010] Reference Figure 5 As shown, it is a flow chart of the method of the prior art 2. The specific steps are as follows:

[0011] Step 1: The application loads the framework. Step 2: The application simultaneously loads the model from the hard disk to the memory and performs partial initialization of the GPU (such as creating a GPU context), and then copies the model from the memory to the video memory. Step 3: Complete the remaining GPU initialization, such as loading the operator library.

[0012] Reference Figure 6 As shown, this is the disadvantage of the prior art 2. Although this method can effectively reduce the overhead of loading the model, it cannot avoid the overhead of loading the framework and GPU-related initialization.

[0013] Reference Figure 7 The system architecture diagram of the prior art 3 is shown in FIG. It includes an application image file and a Checkpoint / Restore tool. In the offline stage, the Checkpoint / Restore tool will checkpoint the application that has been started into an application image file. In the online stage, the application is directly started by restoring from the application image file, thereby bypassing the initialization process.

[0014] Reference Figure 8 As shown, it is a flow chart of the method of prior art 3. The specific steps are as follows:

[0015] Step 1: Create a recovery process; Step 2: Restore most of the state of the process, among which the recovery of GPU data is the most time-consuming; Step 3: Restore the memory state, among which the recovery of CPU data is the most time-consuming; Step 4: Restore the remaining states that depend on the memory state, and then wake up the process.

[0016] Reference Fig. 9As shown, this is the disadvantage of the prior art 3. Although this method can bypass the initialization phase, it introduces the overhead of new recovery process state, recovery GPU data and recovery CPU data.

[0017] Existing technologies can only optimize part of the cold start phase or introduce a lot of additional overhead. Summary of the invention

[0018] In view of the defects in the prior art, the object of the present invention is to provide a low-latency GPU serverless cold start method and system.

[0019] A low-latency GPU serverless cold start method provided by the present invention includes:

[0020] Step S1: Call the checkpoint tool to obtain the application image;

[0021] Step S2: Obtain GPU data access sequence through offline sampling;

[0022] Step S3: After restoring the state of the GPU kernel driver of the process to be restored, the ownership of the GPU data is transferred to the Restore Server;

[0023] Step S4: Based on the application image, the Restore Server is instructed to perform recovery according to the GPU data access sequence obtained by offline sampling, and at the same time, the process to be recovered is restored to the state of other processes;

[0024] Step S5: When the process to be restored is completed, it is transformed into the target process and starts running, so that the GPU data continues to be restored in the background Restore Server.

[0025] Preferably, step S2 includes restoring the application once in advance, without performing page table mapping of the GPU data area; exporting the order in which the application triggers GPU page faults through a modified Linux kernel, and using this as the GPU data access order.

[0026] Preferably, step S3 comprises:

[0027] When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored;

[0028] AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

[0029] Preferably, the recovery process in step S4 includes reading data from the Checkpoint image and writing the data to the corresponding video memory area through DMA.

[0030] Preferably, step S5 comprises:

[0031] If the target process accesses the unrecovered GPU data area, a page fault will be triggered; the modified kernel page fault handling function checks the set flag bit, if it has been recovered, it restores the mapping, if not, it waits for the recovery to be completed; after the restore server completes the recovery, it notifies the kernel through the ioctl interface and sets the corresponding flag bit.

[0032] A low-latency GPU serverless cold start system provided by the present invention includes:

[0033] Image Storage system, used to store application images;

[0034] Restore Server, used for parallel recovery of GPU data and on-demand recovery;

[0035] The kernel page table manager is used to support on-demand recovery through the page fault mechanism;

[0036] Checkpoint / Restore system, used to support application state preservation and recovery;

[0037] The Checkpoint / Restore system calls the checkpoint tool to obtain the application image and stores it in the Image Storage system, and obtains the GPU data access sequence through offline sampling;

[0038] After the recovery process recovers the state of the GPU kernel driver, the ownership of the GPU data is transferred to the RestoreServer;

[0039] Based on the application image, the Restore Server performs recovery according to the GPU data access sequence obtained by offline sampling, and the process to be recovered recovers the status of other processes;

[0040] When the process to be restored is restored, it is transformed into the running target process; the restoration of GPU data is continued by the Restore Server in the background.

[0041] Preferably, the process of obtaining the GPU data access sequence by offline sampling by the Checkpoint / Restore system includes:

[0042] Restore the application once in advance, and do not perform page table mapping of the GPU data area at this time;

[0043] The modified Linux kernel exports the order in which applications trigger GPU page faults, and uses this as the GPU data access order.

[0044] Preferably, after the process to be restored restores the state of the GPU kernel driver, the process of transferring the GPU data ownership to the Restore Server includes:

[0045] When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored;

[0046] AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

[0047] Preferably, the process of the Restore Server restoring according to the GPU data access sequence obtained by offline sampling includes:

[0048] Read data from the Checkpoint image and write it to the corresponding video memory area via DMA.

[0049] Preferably, when the process to be restored is restored, it is transformed into a running target process; the restoration of GPU data is continued by the Restore Server in the background, and if the target process accesses the unrestored GPU data area, a page fault will be triggered; the modified kernel page fault processing function checks the set flag bit, if it has been restored, the mapping is restored, if not restored, wait for the restoration to be completed; after the restoration of the Restore Server is completed, the kernel is notified through the ioctl interface and the corresponding flag bit is set.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention introduces a parallel Restore mechanism based on ownership transfer and a system service RestoreServer. Each Restore process transfers the ownership of the GPU data area to the Server, and the Server performs GPU data recovery in a unified manner in the background. The recovery of CPU data and GPU data is decoupled to different processes, so that they can be restored in parallel, which speeds up the recovery process.

[0052] 2. The present invention introduces an on-demand recovery mechanism based on page faults. For CPU data, the mmap system call of the Linux system is used for implementation. For GPU data, the corresponding GPU driver is modified to support on-demand recovery, and a sampling-based background recovery mechanism is designed. The Restore Server performs continuous recovery in the background according to the pre-sampled order. The application can start execution without restoring all data, which speeds up the recovery process.

[0053] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0055] Figure 1 This is a system architecture diagram of prior art 1.

[0056] Figure 2 The present invention is a flow chart of a method of prior art 1.

[0057] Figure 3 Schematic diagram of the shortcomings of the prior art 1.

[0058] Figure 4This is a system architecture diagram of the second prior art.

[0059] Figure 5 It is a flow chart of the method of prior art 2.

[0060] Figure 6 It is a schematic diagram of the shortcomings of the second prior art.

[0061] Figure 7 This is a system architecture diagram of prior art three.

[0062] Figure 8 It is a flow chart of the method of prior art three.

[0063] Fig. 9 It is a schematic diagram of the shortcomings of the prior art three.

[0064] Fig.10 The figure is a flow chart of the method of the present invention.

[0065] Fig.11 It is a system architecture diagram of the present invention.

[0066] Fig.12 This is a software and hardware based implementation of the present invention.

[0067] Fig.13 2 is a component structure diagram in an embodiment of the present invention.

[0068] Fig.14 2 is a comparison diagram of the effects in the embodiments of the present invention. DETAILED DESCRIPTION

[0069] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0070] Reference Fig.10 As shown, a low-latency GPU serverless cold start method includes:

[0071] Step S1: Call the checkpoint tool to obtain the application image;

[0072] Step S2: Obtain the GPU data access sequence through offline sampling: restore the application once in advance, and do not perform page table mapping of the GPU data area at this time; export the order in which the application triggers GPU page faults through the modified Linux kernel, and use this as the GPU data access sequence.

[0073] Step S3: After the process to be restored restores the state of the GPU kernel driver, the ownership of the GPU data is transferred to the Restore Server; when the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag bit will be set to indicate whether the corresponding area has been restored;

[0074] AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

[0075] Step S4: Based on the application image, the Restore Server is ordered to perform recovery according to the GPU data access sequence obtained by offline sampling, and at the same time, the process to be recovered is restored to other process states; the recovery process includes reading data from the Checkpoint image and writing it to the corresponding video memory area through DMA.

[0076] Step S5: When the process to be restored is restored, it is transformed into the target process and starts running, so that the GPU data continues to be restored in the background Restore Server. If the target process accesses the unrestored GPU data area, a pagefault will be triggered; the modified kernel page fault processing function checks the set flag bit, if it has been restored, it restores the mapping, if not, it waits for the restoration to be completed; after the Restore server completes the restoration, it notifies the kernel through the ioctl interface and sets the corresponding flag bit.

[0077] The present invention introduces a parallel Restore mechanism based on ownership transfer and a system service RestoreServer. Each Restore process transfers the ownership of the GPU data area to the Server, and the Server performs GPU data recovery in a unified manner in the background. The recovery of CPU data and GPU data is decoupled to different processes, so that they can be restored in parallel, which speeds up the recovery process.

[0078] The above are basic embodiments of the present invention. The technical solution of the present invention is further described below through two preferred embodiments.

[0079] Example 1

[0080] Reference Fig.10 As shown, a low-latency GPU serverless cold start method includes:

[0081] Hardware: Need to support memory access retry mechanism, such as AMD GPU XNACK feature;

[0082] Specifically, it includes the Checkpoint phase and the Restore phase;

[0083] Checkpoint phase:

[0084] 1. Call the checkpoint tool to obtain the application image.

[0085] Restore phase:

[0086] 1. Offline sampling to obtain the GPU data access order. The sampling method is to restore the application in advance, without performing page table mapping of the GPU data area, and then export the order in which the application triggers GPU pagefault through the modified Linux kernel, which is used as the GPU data access order.

[0087] 2. When starting recovery, the recovery process first recovers the state of the GPU kernel driver. The recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries in the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored.

[0088] 3. When the recovery starts, the process to be recovered transfers the ownership of GPU data to the Restore Server. Currently, AMDGPU manages the video memory or memory used by the driver through a data structure called buffer object. This structure can be exported as Linux dma-buf through the AMD GPU interface. The dma-buf will provide a file descriptor for user-mode programs to use. During the recovery process, after the process to be recovered restores the state of the GPU kernel driver, it will obtain a series of dma-buf file descriptors corresponding to the area to be recovered. Subsequently, the process will pass these file descriptors to the Restore Server through the Unix socket domain. The Restore server will then import these dma-buf as buffer objects to gain control of these video memory areas.

[0089] 4. Restore Server restores according to the previously sampled GPU data access sequence (the GPU data access sequence obtained by offline sampling in the first step). At this time, the process to be restored also restores the status of other processes. The recovery process reads data from the Checkpoint image and writes it to the corresponding video memory area through DMA.

[0090] 5. After the recovery process is completed, it will be transformed into the target process and start running. The GPU data will continue to be restored in the background RestoreServer. At this time, if the target process accesses the unrecovered GPU data area, a pagefault will be triggered. Then the modified kernel page fault processing function will check the set flag bit. If it has been restored, the mapping will be restored. If it has not been restored, it will wait for the recovery to be completed. After the restore server is restored, it will notify the kernel through the ioctl interface and set the corresponding flag bit.

[0091] Example 2

[0092] Reference Fig.12 and Fig.13 As shown, the present invention utilizes the Checkpoint / Restore mechanism to skip all initialization stages in the cold start, and introduces two mechanisms, namely, parallel recovery of CPU and GPU data and recovery on demand, to address the additional overhead brought by Checkpoint / Restore, thereby greatly reducing the additional overhead and speeding up the cold start process.

[0093] The CPU and GPU parallel recovery mechanisms include:

[0094] 1. Introduce a global Restore Server to assist in the recovery of GPU data. For the on-demand recovery part of the GPU data to be recovered, the operating system can only record the information of this part of the area and temporarily postpone mapping this part of the area to the GPU page table and CPU page table.

[0095] 2. The process to be restored transfers the ownership of the area to be restored of the GPU data to the Restore Server, which then performs the restoration. The background Restore Server will continue to restore this part of the data and update the metadata in the kernel to indicate whether the corresponding area has been restored. When the application accesses the corresponding area and triggers a page fault, the operating system will check the corresponding metadata and update the page table if it has been restored, otherwise it will wait.

[0096] The on-demand CPU and GPU data recovery method based on Checkpoint / Restore can start executing applications without restoring all data, further speeding up the recovery process. The operating system can record and export the address sequence of the application triggering pagefault, and the Restore process recovers according to this address sequence.

[0097] Reference Fig.14 As shown, From scratch means starting from scratch, cold cache means starting based on caching related frameworks, warm cache means cold cache performs additional preheating operations, CRIU is the Checkpoint / Restore cold start mechanism based on CRIU, and gCROP is the mechanism of the present invention.

[0098] It can be seen that no matter it is the smaller model MobileNet or the larger model GPT-2-Large, the cold start performance of the present invention is better than the existing starting from scratch and Checkpoint / Restore mechanism, and the performance of the method of the cache framework is comparable on the smaller model, and is better on the larger model.

[0099] The present invention also provides a low-latency GPU serverless cold start system, which can be implemented by executing the process steps of the low-latency GPU serverless cold start method, that is, those skilled in the art can understand the low-latency GPU serverless cold start method as a preferred implementation of the low-latency GPU serverless cold start system.

[0100] Reference Fig.11 Specifically, a low-latency GPU serverless cold start system includes:

[0101] Image Storage system, used to store application images;

[0102] Restore Server, used for parallel recovery of GPU data and on-demand recovery;

[0103] The kernel page table manager is used to support on-demand recovery through the page fault mechanism;

[0104] Checkpoint / Restore system, used to support application state preservation and recovery;

[0105] The Checkpoint / Restore system calls the checkpoint tool to obtain the application image and stores it in the Image Storage system, and obtains the GPU data access sequence through offline sampling;

[0106] After the recovery process recovers the state of the GPU kernel driver, the ownership of the GPU data is transferred to the RestoreServer;

[0107] Based on the application image, the Restore Server performs recovery according to the GPU data access sequence obtained by offline sampling, and the process to be recovered recovers the status of other processes;

[0108] When the process to be restored is restored, it is transformed into the running target process; the restoration of GPU data is continued by the Restore Server in the background.

[0109] The process of obtaining the GPU data access sequence by offline sampling in the Checkpoint / Restore system includes:

[0110] Restore the application once in advance, and do not perform page table mapping of the GPU data area at this time;

[0111] The modified Linux kernel exports the order in which applications trigger GPU page faults, and uses this as the GPU data access order.

[0112] After the process to be restored restores the state of the GPU kernel driver, the process of transferring the GPU data ownership to the Restore Server includes:

[0113] When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored;

[0114] AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

[0115] The process of the Restore Server restoring the GPU data access sequence obtained by offline sampling includes:

[0116] Read data from the Checkpoint image and write it to the corresponding video memory area via DMA.

[0117] When the process to be restored is restored, it is transformed into the running target process; the recovery of GPU data is continued by the Restore Server in the background. If the target process accesses the unrestored GPU data area, a page fault will be triggered; the modified kernel page fault processing function checks the set flag bit, if it has been restored, the mapping is restored, if not, it waits for the recovery to be completed; after the Restore Server completes the recovery, it notifies the kernel through the ioctl interface and sets the corresponding flag bit.

[0118] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0119] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A low-latency GPU serverless cold start method, characterized in that: include: Step S1: Call the checkpoint tool to obtain the application image; Step S2: Obtain GPU data access sequence through offline sampling; Step S3: After restoring the state of the GPU kernel driver of the process to be restored, the ownership of the GPU data is transferred to the RestoreServer; Step S4: Based on the application image, the Restore Server is instructed to perform recovery according to the GPU data access sequence obtained by offline sampling, and at the same time, the process to be recovered is restored to the state of other processes; Step S5: When the process to be restored is completed, it is transformed into the target process and starts running, so that the GPU data continues to be restored in the background Restore Server.

2. The low-latency GPU serverless cold start method according to claim 1, characterized in that: The step S2 includes restoring the application once in advance, without performing page table mapping of the GPU data area at this time; exporting the order in which the application triggers GPU page faults through a modified Linux kernel, and using this as the GPU data access order.

3. The low-latency GPU serverless cold start method according to claim 2, characterized in that: The step S3 comprises: When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored; AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

4. The low-latency GPU serverless cold start method according to claim 1, characterized in that: The recovery process in step S4 includes reading data from the Checkpoint image and writing it to the corresponding video memory area through DMA.

5. The low-latency GPU serverless cold start method according to claim 1, characterized in that: The step S5 comprises: If the target process accesses the unrecovered GPU data area, a page fault will be triggered; the modified kernel page fault handling function checks the set flag bit, if it has been recovered, it restores the mapping, if not, it waits for the recovery to be completed; after the restore server completes the recovery, it notifies the kernel through the ioctl interface and sets the corresponding flag bit.

6. A low-latency GPU serverless cold start system, characterized in that: include: Image Storage system, used to store application images; Restore Server, used for parallel recovery of GPU data and on-demand recovery; The kernel page table manager is used to support on-demand recovery through the page fault mechanism; Checkpoint / Restore system, used to support application state preservation and recovery; The Checkpoint / Restore system calls the checkpoint tool to obtain the application image and stores it in the Image Storage system, and obtains the GPU data access sequence through offline sampling; After the recovery process recovers the state of the GPU kernel driver, the ownership of the GPU data is transferred to the RestoreServer; Based on the application image, the Restore Server performs recovery according to the GPU data access sequence obtained by offline sampling, and the process to be recovered recovers the status of other processes; When the process to be restored is restored, it is transformed into the running target process; the restoration of GPU data is continued by the RestoreServer in the background.

7. The low-latency GPU serverless cold start system according to claim 6, characterized in that: The process of obtaining the GPU data access sequence by offline sampling in the Checkpoint / Restore system includes: Restore the application once in advance, and do not perform page table mapping of the GPU data area at this time; The modified Linux kernel exports the order in which applications trigger GPU page faults, and uses this as the GPU data access order.

8. The low-latency GPU serverless cold start system according to claim 7, characterized in that: After the process to be restored restores the state of the GPU kernel driver, the process of transferring the GPU data ownership to the Restore Server includes: When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of the GPU page table entries of the area that needs to be restored on demand will be skipped, and a flag will be set to indicate whether the corresponding area has been restored; AMD GPU manages the video memory or memory used by the driver through the buffer object data structure. During the recovery process, the process to be recovered will obtain some dma-buf file descriptors corresponding to the area to be recovered after restoring the state of the GPU kernel driver. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains control of the corresponding video memory area.

9. The low-latency GPU serverless cold start system according to claim 6, characterized in that: The process of the Restore Server restoring the GPU data access sequence obtained by offline sampling includes: Read data from the Checkpoint image and write it to the corresponding video memory area via DMA.

10. The low-latency GPU serverless cold start system according to claim 6, characterized in that: When the process to be restored is restored, it is transformed into the running target process; the recovery of GPU data is continued by the Restore Server in the background. If the target process accesses the unrestored GPU data area, a page fault will be triggered; the modified kernel page fault processing function checks the set flag bit, if it has been restored, the mapping is restored, if not, it waits for the recovery to be completed; after the Restore Server completes the recovery, it notifies the kernel through the ioctl interface and sets the corresponding flag bit.

Images