Low-latency gpu serverless cold start method and system

Abstract

The application provides a low-latency GPU serverless cold start method and system, comprising the following steps: S1, calling a checkpoint tool to obtain an application image; S2, obtaining a GPU data access order through offline sampling; S3, transferring the ownership of GPU data to a Restore Server; S4, restoring according to the GPU data access order obtained through offline sampling; and S5, when the process to be restored is restored, the process is changed into a target process to start running, so that the GPU data continues to be restored in the background Restore Server. In the application, the ownership of the GPU data region of each Restore process is transferred to the Server, and the restoration of the GPU data is uniformly performed in the background; the restoration of CPU data and GPU data is decoupled to different processes, and the CPU data and the GPU data can be restored in parallel, so that the restoration process is accelerated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of serverless, in particular, to a low-latency GPU serverless cold start method and system. BACKGROUND

[0002] With the progress of science and technology, the development of serverless technology has also changed. Users upload their application code to the serverless platform, and then the serverless platform dynamically scales according to the load. The key to scaling is to quickly start a new application instance, which is called cold start.

[0003] Because the cold start process takes a long time and has high overhead, the industry generally uses some caching strategies to reduce the number of cold starts, such as reserving a certain number of additional instances, extending the instance scaling time, and reducing the scaling standard. However, these methods are simple to implement, but have high overhead and their corresponding effects are heavily dependent on the specific caching strategy, and can only bypass part of the cold start times and cannot directly speed up the cold start process.

[0004] In the Chinese patent document with publication number CN111559278A, a series hybrid electric vehicle ultra-low temperature cold start system and method are disclosed. The system includes a high-voltage subsystem and a control subsystem. The high-voltage subsystem mainly includes a power battery, a reverse prevention diode, an intelligent power unit, a drive motor, and a battery heating film. The intelligent power unit includes a generator and an engine. The control subsystem includes a vehicle controller, a battery manager, and a distribution box controller. The battery manager and the distribution box controller are both connected to the vehicle controller. However, this patent document is mainly to solve the short circuit problem caused by charging the power battery after the series hybrid electric vehicle starts at low temperature, which is different from the technical problem to be solved in this application.

[0005] Referring to Figure 1 Figure 1 is a system architecture diagram of prior art one. It contains two parts: GPU application and GPU server. Unlike traditional applications that directly access and use GPUs, this technology uses a proxy-based design. It separates the part that interacts with the GPU from the application and uses a pre-started GPU server as a proxy. The application sends GPU commands to the server, which interacts with the GPU on behalf of the application.

[0006] Referring to Figure 2 Figure 2 is a method flowchart of prior art one. The specific steps are as follows:

[0007] Step 1: Offline preparation of GPU server; Step 2: Enter the online part and start the application; Step 3: Establish a connection with the pre-established GPU server; Step 4: Load the model.

[0008] Referring to Figure 3 Figure 1 shows the system architecture diagram of Prior Art 1. It contains the application and the loading framework. The loading framework is responsible for loading the model file into the memory and copying the data from the memory to the GPU memory. The application is responsible for the initialization of the GPU.

[0009] Referring to Figure 4 Figure 2 shows the system architecture diagram of Prior Art 2. It contains the reorganized model file and the chunked memory management system. The model file is reorganized according to the order of access, so that it can be read sequentially during loading. The chunked memory management system loads the model file into the memory in block granularity through multi-threading and Direct IO, and simultaneously copies the data from the memory to the GPU memory in parallel, thereby speeding up the cold start process.

[0010] Referring to Figure 5 Figure 3 shows the method flowchart of Prior Art 2. The specific steps are as follows:

[0011] Step 1: Load the framework; Step 2: Simultaneously load the model from the hard disk to the memory and perform partial initialization of the GPU (such as creating a GPU context) to copy the model from the memory to the GPU memory; Step 3: Complete the remaining GPU initialization, such as loading the operator library, etc.

[0012] Referring to Figure 6 Figure 4 shows the shortcomings of Prior Art 2. Although this method can effectively reduce the overhead of loading the model, it cannot avoid the overhead of loading the framework and the GPU-related initialization.

[0013] Referring to Figure 7 Figure 5 shows the system architecture diagram of Prior Art 3. It contains the application image file and the Checkpoint / Restore tool. In the offline stage, the Checkpoint / Restore tool checkpoints the application that has completed startup into an application image file, and in the online stage, it directly starts the application by restoring from the application image file, thereby bypassing the initialization process.

[0014] Referring to Figure 8 Figure 6 shows the method flowchart of Prior Art 3. The specific steps are as follows:

[0015] Step 1: Create a recovery process; Step 2: Restore most of the state of the process, with the recovery of GPU data being the most time-consuming; Step 3: Restore the memory state, with the recovery of CPU data being the most time-consuming; Step 4: Restore the remaining state that depends on the memory state, and then wake up the process.

[0016] Referring to Figure 9As shown, this is the drawback of the prior art three. Although this method can bypass the initialization phase, it introduces new overhead of restoring the process state, restoring GPU data and restoring CPU data.

[0017] The prior art can only optimize part of the cold start phase, or introduce a lot of additional overhead. SUMMARY

[0018] In view of the defects in the prior art, the purpose of the present application is to provide a low-latency GPU serverless cold start method and system.

[0019] According to the low-latency GPU serverless cold start method provided by the present application, the method comprises the following steps:

[0020] Step S1: calling a checkpoint tool to obtain an application image;

[0021] Step S2: obtaining a GPU data access order through offline sampling;

[0022] Step S3: after the to-be-restored process restores the state of the GPU kernel driver, transferring the ownership of the GPU data to the Restore Server;

[0023] Step S4: based on the application image, causing the Restore Server to restore according to the GPU data access order obtained through offline sampling, and causing the to-be-restored process to restore other process states;

[0024] Step S5: after the to-be-restored process is restored, the target process starts running, and the GPU data continues to be restored in the background Restore Server.

[0025] Preferably, the step S2 comprises pre-restoring the application once, at which time the page table mapping of the GPU data region is not performed; the sequence of triggering the GPU page fault of the application by the modified Linux kernel is exported, and the sequence is used as the GPU data access order.

[0026] Preferably, the step S3 comprises:

[0027] When the driver is an AMD driver, the restoration process is completed by calling the interface provided by the AMD driver in the kernel, at which time the restoration of the GPU page table item of the region that needs to be restored on demand is skipped, and a flag bit is set to indicate whether the corresponding region is restored or not;

[0028] The AMD GPU manages the memory used by the driver through the data structure of the buffer object. In the recovery process, after the process to be recovered restores the state of the GPU kernel driver, it will obtain some file descriptors of dma-buf corresponding to the to-be-recovered area. The process transmits these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf as buffer objects and obtains the control right of the corresponding memory area.

[0029] Preferably, the recovery process in the step S4 includes reading data from the mirror of the Checkpoint and writing to the corresponding memory area through DMA.

[0030] Preferably, the step S5 includes:

[0031] If the target process accesses the GPU data area that has not been recovered, a page fault will be triggered. The modified kernel page fault processing function checks the set flag. If the recovery is completed, the mapping is restored. If the recovery is not completed, the process waits until the recovery is completed. After the Restore Server completes the recovery, it notifies the kernel through the ioctl interface and sets the corresponding flag.

[0032] According to the low-latency GPU serverless cold start system provided by the application, the system includes:

[0033] An Image Storage system is configured to store application images.

[0034] A Restore Server is configured to recover GPU data in parallel and on demand.

[0035] A kernel page table manager is configured to support on-demand recovery through a page fault mechanism.

[0036] A Checkpoint / Restore system is configured to support the saving and recovery of application states.

[0037] The Checkpoint / Restore system calls the checkpoint tool to obtain application images and store them in the Image Storage system. GPU data access sequences are obtained through offline sampling.

[0038] After the process to be recovered restores the state of the GPU kernel driver, the GPU data ownership is transferred to the Restore Server.

[0039] Based on the application image, the Restore Server restores according to the GPU data access order obtained by offline sampling, and the process to be restored restores the state of other processes;

[0040] When the process to be restored is restored, it becomes the target process running in the system; the restoration of GPU data is continued by the Restore Server in the background.

[0041] Preferably, the process of obtaining the GPU data access order by offline sampling of the Checkpoint / Restore system comprises:

[0042] The application is restored once in advance, and the page table mapping of the GPU data region is not performed at this time;

[0043] The order of triggering the GPU page fault of the application by the modified Linux kernel is exported, and is used as the GPU data access order.

[0044] Preferably, the process of transferring the ownership of the GPU data to the Restore Server after the process to be restored restores the state of the GPU kernel driver comprises:

[0045] When the driver is an AMD driver, the restoration process is completed by calling the interface provided by the AMD driver in the kernel, at this time, the restoration of the GPU page table entry of the region that needs to be restored on demand is skipped, and a flag bit is set to indicate whether the corresponding region is restored;

[0046] The AMD GPU manages the video memory or the memory used by the driver through the data structure of the buffer object, in the restoration process, the process to be restored restores the state of the GPU kernel driver, and obtains some file descriptors of the dma-buf corresponding to the region to be restored, the process transfers the file descriptors to the Restore Server through the Unix socket domain, and the Restore Server imports the dma-buf as a buffer object to obtain the control right of the corresponding video memory region.

[0047] Preferably, the process of restoring by the Restore Server according to the GPU data access order obtained by offline sampling comprises:

[0048] Data is read from the image of the Checkpoint and is written to the corresponding video memory region through DMA.

[0049] Preferably, when the process to be recovered is completed, the target process is changed to running; the recovery of GPU data is continued by the Restore Server in the background, if the target process accesses the un-recovered GPU data area, a page fault is triggered; the modified kernel page fault processing function checks the set flag, if it has been recovered, the mapping is recovered, if it has not been recovered, it waits for the recovery to be completed; the Restore Server informs the kernel and sets the corresponding flag after the recovery is completed.

[0050] Compared with the prior art, the application has the beneficial effects as follows:

[0051] 1. The application introduces a parallel Restore mechanism based on ownership transfer, introduces a system service Restore Server, and each Restore process transfers the ownership of the GPU data area to the Server, so that the Server uniformly performs the recovery of the GPU data in the background; the recovery of the CPU data and the GPU data is decoupled to different processes, thereby the recovery can be performed in parallel, and the recovery process is accelerated.

[0052] 2. The application introduces an on-demand recovery mechanism based on page fault, for the CPU data, the mmap system call of the Linux system is used for implementation, for the GPU data, the corresponding GPU driver is modified to support on-demand recovery, and a background recovery mechanism based on sampling is designed, so that the Restore Server continuously recovers according to the pre-sampled sequence in the background, and an application does not need to be executed after all data is recovered, and the recovery process is accelerated.

[0053] Other beneficial effects of the application will be described in the specific embodiments through the introduction of specific technical features and technical solutions, and the person skilled in the art should understand the beneficial technical effects brought by the technical features and technical solutions through the introduction of the technical features and technical solutions. BRIEF DESCRIPTION OF DRAWINGS

[0054] Other features, objects and advantages of the application will become more apparent through the following detailed description of the non-limiting embodiments with reference to the accompanying drawings:

[0055] Figure 1 It is a system architecture diagram of prior art one.

[0056] Figure 2 It is a method flowchart of prior art one.

[0057] Figure 3 It is a defect diagram of prior art one.

[0058] Figure 4System architecture diagram for prior art two.

[0059] Figure 5 Method flow chart for prior art two.

[0060] Figure 6 Defect schematic diagram for prior art two.

[0061] Figure 7 System architecture diagram for prior art three.

[0062] Figure 8 Method flow chart for prior art three.

[0063] Figure 9 Defect schematic diagram for prior art three.

[0064] Figure 10 Method flow chart for the present application.

[0065] Figure 11 System architecture diagram for the present application.

[0066] Figure 12 Hardware and software-based implementation form for the present application.

[0067] Figure 13 Component structure diagram in an embodiment of the present application.

[0068] Figure 14 Effect comparison diagram in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These all belong to the protection scope of the present application.

[0070] Referring to Figure 10 A low-latency GPU serverless cold start method, comprising:

[0071] Step S1: call the checkpoint tool to obtain the application image;

[0072] Step S2: obtain the GPU data access order by offline sampling: restore the application once in advance, at this time do not perform page table mapping of the GPU data region; export the order of triggering the GPU page fault of the application by the modified Linux kernel, and use it as the GPU data access order.

[0073] Step S3: After the process to be restored restores the state of the GPU kernel driver, the GPU data ownership is transferred to the Restore Server; when the driver is an AMD driver, the restoration process is completed by calling the interface provided by the AMD driver in the kernel, at this time the restoration of the GPU page table entry of the area that needs to be restored on demand is skipped, and a flag bit is set to indicate whether the corresponding area is restored or not;

[0074] The AMD GPU manages the video memory or memory used by the driver through the data structure of the buffer object. In the restoration process, after the process to be restored restores the state of the GPU kernel driver, it will obtain some file descriptors of dma-buf corresponding to the to-be-restored area. The process transfers these file descriptors to the Restore Server through the Unix socket domain. The Restore server imports these dma-buf as buffer objects and obtains the control right of the corresponding video memory area.

[0075] Step S4: Based on the application image, the Restore Server restores according to the GPU data access order obtained by offline sampling, and the process to be restored restores the state of other processes; the restoration process includes reading data from the image of the Checkpoint and writing to the corresponding video memory area through DMA.

[0076] Step S5: After the process to be restored is restored, it is changed into the target process to start running, and the GPU data continues to be restored in the background Restore Server. If the target process accesses the GPU data area that has not been restored, a page fault will be triggered; the modified kernel page fault processing function checks the set flag bit, if it has been restored, the mapping is restored, if it has not been restored, it waits for the restoration to be completed; the Restore server notifies the kernel through the ioctl interface after the restoration is completed and sets the corresponding flag bit.

[0077] The application introduces a parallel Restore mechanism based on ownership transfer, introduces a system service RestoreServer, and the ownership of the GPU data area of each Restore process is transferred to the Server, which uniformly restores the GPU data in the background; the restoration of CPU data and GPU data is decoupled to different processes, and then parallel restoration is realized, which speeds up the restoration process.

[0078] The above is the basic embodiment of the application, and the technical solutions of the application will be further described through two preferred embodiments.

[0079] Embodiment 1

[0080] Figure 10 ReferenceFigure 10 A low-latency GPU serverless cold start method is shown, comprising:

[0081] Hardware: need to support memory retry mechanism, for example: AMD GPU XNACK feature;

[0082] Specifically includes Checkpoint phase and Restore phase;

[0083] Checkpoint phase:

[0084] 1. Call the checkpoint tool to get the application image.

[0085] Restore phase:

[0086] 1. Offline sampling to get GPU data access order. The sampling method is to restore the application once in advance, at this time do not perform page table mapping of GPU data area, and then export the order of triggering GPU page fault by modifying the Linux kernel to trigger GPU page fault, which is used as the GPU data access order.

[0087] 2. When starting recovery, the process to be restored first restores the state of the GPU kernel driver. The recovery process is completed by calling the interface provided by the AMD driver in the kernel, at this time the recovery of the GPU page table entry of the area that needs to be restored on demand is skipped, and a flag bit is set to indicate whether the corresponding area is recovered.

[0088] 3. When starting recovery, the process to be restored transfers the ownership of GPU data to the Restore Server. Currently AMD GPU manages the video memory or memory used by the driver through a data structure called buffer object. This structure can be exported to Linux dma-buf through the interface of AMD GPU, and the dma-buf will provide a file descriptor for user mode program. In the recovery process, after the process to be restored restores the state of the GPU kernel driver, it will get a series of file descriptors of dma-buf corresponding to the to-be-restored area, and then the process transfers these file descriptors to the Restore Server through the Unix socket domain. Subsequently, the Restore server imports these dma-bufs as buffer objects, thereby obtaining control of these video memory regions.

[0089] 4. The Restore Server restores according to the pre-sampled GPU data access order (the GPU data access order sampled offline in the first step), and at this time the process to be restored restores the states of other processes simultaneously. The restoration process is to read data from the mirror of the Checkpoint and write to the corresponding GPU memory area through DMA.

[0090] 5. After the process to be restored is restored, it is converted into the target process to start running, and the GPU data is continuously restored in the background Restore Server. At this time, if the target process accesses the GPU data area that has not been restored, a page fault is triggered, and then the modified kernel page fault processing function checks the set flag. If it has been restored, the mapping is restored, and if it has not been restored, it waits for the restoration to be completed. After the restoration of the Restore server is completed, the kernel is notified through the ioctl interface and the corresponding flag is set.

[0091] Embodiment 2

[0092] Referring to Figure 12 and Figure 13 As shown in the figures, the present application uses the Checkpoint / Restore mechanism to skip the entire initialization stage in the cold start, and in view of the additional overhead brought by Checkpoint / Restore, introduces two mechanisms of CPU and GPU data parallel restoration and on-demand restoration, thereby greatly reducing the additional overhead and speeding up the cold start process.

[0093] The CPU and GPU parallel restoration mechanism includes:

[0094] 1. A global Restore Server is introduced to assist in the restoration of GPU data; the operating system can only record the information of the on-demand restoration part of the GPU data to be restored, and temporarily suspend the mapping of this part of the region to the GPU page table and the CPU page table.

[0095] 2. The process to be restored transfers the ownership of the GPU data to be restored to the Restore Server for restoration. The background Restore Server continuously restores this part of data and updates the meta-information of the corresponding region in the kernel whether it has been restored; when the application accesses the corresponding region and triggers a page fault, the operating system checks the corresponding meta-information. If it has been restored, the page table is updated, otherwise it waits.

[0096] Checkpoint / Restore-based CPU and GPU data on-demand restore method can start the execution of the application without restoring all data, further accelerating the restore process. The operating system can record and export the address sequence of the page fault triggered by the application, and the restore process is restored according to the address sequence.

[0097] Referring to Figure 14 As shown in the figure, From scratch is a start from scratch, cold cache is a start based on caching related frameworks, warm cache is a preheating operation in addition to cold cache, CRIU is a checkpoint / restore cold start mechanism, and gCROP is the mechanism of the application.

[0098] It can be seen that, whether it is a smaller model MobileNet or a larger model GPT-2-Large, the cold start performance of the application is better than the existing zero start and checkpoint / restore mechanism, and the performance of the method of caching the framework is equivalent on a smaller model, and the performance is better on a larger model.

[0099] The application also provides a low-latency GPU serverless cold start system, which can be realized by executing the process steps of the low-latency GPU serverless cold start method, that is, the low-latency GPU serverless cold start method can be understood by those skilled in the art as the preferred embodiment of the low-latency GPU serverless cold start system.

[0100] Referring to Figure 11 As shown in the figure, specifically, a low-latency GPU serverless cold start system includes:

[0101] An image storage system for storing application images;

[0102] A restore server for parallel restoration of GPU data and on-demand restoration;

[0103] A kernel page table manager for supporting on-demand restoration through a page fault mechanism;

[0104] Checkpoint / Restore system for supporting application state saving and restoration;

[0105] Checkpoint / Restore system call checkpoint tool gets application image and stores it in Image Storage system, and GPU data access order is obtained through offline sampling;

[0106] After the process to be restored restores the state of GPU kernel driver, the GPU data ownership is transferred to the Restore Server;

[0107] Based on the application image, the Restore Server restores according to the GPU data access order obtained through offline sampling, and the process to be restored restores the state of other processes;

[0108] When the process to be restored is restored, it becomes the target process running in the system, and the restoration of GPU data is continued by the Restore Server in the background.

[0109] The process of obtaining GPU data access order through offline sampling by the Checkpoint / Restore system includes:

[0110] The application is pre-restored once, and the page table mapping of the GPU data region is not performed at this time;

[0111] The sequence of triggering GPU page fault by the application is exported through the modified Linux kernel, and is used as the GPU data access order.

[0112] The process of transferring the GPU data ownership to the Restore Server after the process to be restored restores the state of GPU kernel driver includes:

[0113] When the driver is AMD driver, the restoration process is completed by calling the interface provided by the AMD driver in the kernel, and at this time the restoration of the GPU page table entry of the region that needs to be restored on demand is skipped, and a flag bit is set to indicate whether the corresponding region is restored or not;

[0114] The AMD GPU manages the video memory or memory used by the driver through the data structure of buffer object, and in the restoration process, the process to be restored restores the state of GPU kernel driver and obtains some file descriptors of dma-buf corresponding to the to-be-restored region, and the process transfers these file descriptors to the Restore Server through Unix socket domain, and the Restore Server imports these dma-buf as buffer object and obtains the control right of the corresponding video memory region.

[0115] The process of recovery by the Restore Server according to the GPU data access order obtained by offline sampling includes:

[0116] Reading data from the mirror of the Checkpoint and writing to the corresponding GPU memory area through DMA.

[0117] When the process to be recovered is recovered, it is changed to the target process running; the recovery of the GPU data is continued by the Restore Server in the background, if the target process accesses the GPU data area that has not been recovered, a page fault is triggered; the modified kernel page fault processing function checks the set flag, if it has been recovered, the mapping is recovered, if it has not been recovered, it waits for the recovery to be completed; the Restore Server informs the kernel through the ioctl interface after the recovery is completed and sets the corresponding flag.

[0118] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module, unit thereof in a pure computer readable program code manner, the system provided by the present application and each device, module, unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to achieve the same functions by logically programming the method steps. Therefore, the system provided by the present application and each device, module, unit thereof can be considered as a hardware component, and the devices, modules, units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, units for implementing various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0119] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other at will.

Claims

1. A low-latency GPU serverless cold start method, characterized in that, include: Step S1: Use the checkpoint tool to obtain the application image; Step S2: Obtain the GPU data access order through offline sampling; Step S3: After restoring the GPU kernel driver state of the process to be restored, transfer ownership of the GPU data to RestoreServer; Step S4: Based on the application image, instruct the Restore Server to restore the GPU data access order obtained from offline sampling, and at the same time restore the state of other processes to the process to be restored; Step S5: Once the process to be restored is complete, it will switch to the target process and start running, allowing the GPU data to continue to be restored in the background Restore Server.

2. The low-latency GPU serverless cold start method according to claim 1, characterized in that, Step S2 includes pre-restoring the application once, without performing page table mapping for the GPU data region; exporting the order in which the application triggers GPU page faults through a modified Linux kernel, and using this as the GPU data access order.

3. The low-latency GPU serverless cold start method according to claim 2, characterized in that, Step S3 includes: When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of GPU page table entries in the regions that need to be recovered on demand will be skipped, and a flag will be set to indicate whether the corresponding regions have been recovered. AMD GPUs manage the video memory or system memory used by the driver through the buffer object data structure. During the recovery process, after the process to be recovered restores the state of the GPU kernel driver, it will obtain file descriptors of dma-buf corresponding to some regions to be recovered. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf into buffer objects and gains control of the corresponding video memory regions.

4. The low-latency GPU serverless cold start method according to claim 1, characterized in that, The recovery process in step S4 includes reading data from the Checkpoint image and writing it to the corresponding video memory area via DMA.

5. The low-latency GPU serverless cold start method according to claim 1, characterized in that, Step S5 includes: If the target process accesses an unrestored GPU data region, a page fault will be triggered. The modified kernel page fault handler checks the set flags. If the data has been restored, the mapping is restored. If not, it waits for the restoration to complete. After the Restore server completes the restoration, it notifies the kernel via the ioctl interface and sets the corresponding flags.

6. A low-latency GPU serverless cold start system, characterized in that, include: Image Storage system for storing application images; Restore Server is used for parallel recovery of GPU data and on-demand recovery. The kernel page table manager is used to support on-demand recovery via the page fault mechanism; The Checkpoint / Restore system is used to support the saving and restoration of application state; The Checkpoint / Restore system calls the checkpoint tool to obtain the application image and stores it in the Image Storage system. The GPU data access order is obtained through offline sampling. After the recovery process restores the state of the GPU kernel driver, ownership of the GPU data is transferred to the RestoreServer; Based on the application image, the Restore Server restores the GPU data access order obtained from offline sampling, while the process to be restored restores the state of other processes. Once the process to be restored is completed, it becomes the running target process; the restoration of GPU data is continued in the background by the RestoreServer.

7. The low-latency GPU serverless cold start system according to claim 6, characterized in that, The Checkpoint / Restore system obtains the GPU data access order through offline sampling, including the following process: The application is restored once beforehand, during which page table mapping of the GPU data region is not performed; The order in which applications trigger GPU page faults is exported by modifying the Linux kernel, and this order is used as the GPU data access order.

8. The low-latency GPU serverless cold start system according to claim 7, characterized in that, After the process to be restored restores the state of the GPU kernel driver, the process of transferring ownership of the GPU data to the Restore Server includes: When the driver is an AMD driver, the recovery process is completed by calling the interface provided by the AMD driver in the kernel. At this time, the recovery of GPU page table entries in the regions that need to be recovered on demand will be skipped, and a flag will be set to indicate whether the corresponding regions have been recovered. AMD GPUs manage the video memory or system memory used by the driver through the buffer object data structure. During the recovery process, after the process to be recovered restores the state of the GPU kernel driver, it will obtain file descriptors of dma-buf corresponding to some regions to be recovered. The process passes these file descriptors to the Restore Server through the Unix socket domain. The Restore Server imports these dma-buf into buffer objects and gains control of the corresponding video memory regions.

9. The low-latency GPU serverless cold start system according to claim 6, characterized in that, The process by which the Restore Server restores data based on the access order of GPU data obtained from offline sampling includes: Data is read from the Checkpoint image and written to the corresponding video memory area via DMA.

10. The low-latency GPU serverless cold start system according to claim 6, characterized in that, Once the process to be restored is complete, it becomes the running target process. The restoration of GPU data continues in the background by the Restore Server. If the target process accesses an unrestored GPU data area, a page fault will be triggered. The modified kernel page fault handler checks the set flags. If the data has been restored, the mapping is restored. If not, it waits for the restoration to complete. After the Restore Server completes the restoration, it notifies the kernel via the ioctl interface and sets the corresponding flags.

Images