State storage and recovery method and system suitable for universal graphics processing unit
By using pre-copy and differential copying technologies on a general graphics processor, the problems of program pause and redundant data during transparent state saving and recovery are solved, efficient state saving and recovery are achieved, and binary image size and recovery are reduced. time.
Patent Information
- Application Number
- CN202410769668.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-14
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-06-14
AI Technical Summary
In the transparent state saving and recovery process of general graphics processors, the Checkpoint stage introduces a long program pause time and generates a large-scale binary image, and there is a problem of redundancy in memory data on the CPU and GPU side.
Without stopping the kernel function, the GPGPU memory data is copied to the CPU side through the pre-copy and differential copy stages, and the memory state is restored as needed during the recovery stage, reducing the overhead of redundant data copying and program re-execution.
It realizes keeping kernel functions running for most of the time, reducing state saving overhead, shortening memory copy time, reducing binary image scale, and improving state recovery performance.
Smart Images

Figure CN118885331A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of general-purpose graphics processors, and in particular to a state saving and restoring method and system suitable for general-purpose graphics processors. Background Art
[0002] In the fields of machine learning, scientific computing, and 3D rendering, general-purpose graphics processing units (GPGPUs) are important hardware for performing calculations. Programs can complete data processing by sending computing tasks with kernel functions as the smallest granularity to GPGPUs. Kernels usually use several buffers stored in the GPGPU memory as input data, and write the results to several buffers after completing the corresponding calculations. In cloud computing and other scenarios, it is an important function to save and restore the state of the operating system (OS) process using GPGPU (Checkpoint / Restore, C / R), which can be used to support important system capabilities such as disaster recovery backup, hot migration, and fast startup: Checkpoint records the state of a running process on the CPU side and the GPGPU side as a binary image; Restore reloads the process state recorded in the binary image on the CPU and GPGPU to resume the execution of the process.
[0003] Based on whether the original program needs to be modified to implement the C / R function, the existing C / R solutions can be divided into non-transparent and transparent solutions. Non-transparent C / R requires users to manually add C / R functions to the program; transparent C / R directly completes the C / R of the process status on the OS without modifying the original program. In cloud and other scenarios, since it is difficult for operators to invade tenant programs, transparent C / R is a more important capability and is also the focus of this invention.
[0004] At present, the main method for transparent C / R of GPGPU is the memory copy scheme: when checking, the OS first stops the program to launch the kernel to the GPGPU, and waits for all the kernels launched to the GPGPU to complete execution. The OS then copies all the data in the GPGPU memory to the CPU side, and finally saves the process state as a binary image on the CPU side; when restoring, the OS first loads the memory data on the GPGPU side, and then resumes the operation of the program on the CPU side. This type of scheme avoids the long kernel re-execution overhead during restore, but the disadvantage is that the checkpoint introduces a long program pause time, and because of the redundancy of the memory data on the CPU and GPU sides, the checkpoint will generate a large binary image. Summary of the invention
[0005] In view of the defects in the prior art, an object of the present invention is to provide a state saving and restoring method and system applicable to a general-purpose graphics processor.
[0006] The state saving and restoring method for a general-purpose graphics processor provided by the present invention includes: copying the memory data of the GPGPU to the CPU side in the state saving phase without stopping the operation of the kernel function, and restoring the memory data of the GPGPU side from the CPU side in the state restoring phase;
[0007] The state preservation phase includes the pre-copy and differential copy phases;
[0008] During the pre-copy phase, the operating system performs the following steps:
[0009] Step 1: The operating system copies the buffer in the GPGPU memory and concurrently executes steps 2 to 4 in sequence;
[0010] Step 2: The operating system analyzes the memory access semantics of each non-stop kernel function launched by the program, and obtains the output buffer and input buffer of the non-stop kernel function for each buffer;
[0011] Step 3: The operating system determines the copy status of the output buffer of the non-stop kernel function: if the output buffer has not been copied by step 1, a copy operation is performed on it, that is, copy-on-write is performed, and a dirty bit is marked for it; if it has been copied by step 1, the operating system marks it dirty; if it is being copied by step 1, the operating system blocks and waits for the copy to be completed, and marks it dirty;
[0012] Step 4: The operating system sends the non-stop kernel function to the GPGPU for execution;
[0013] After the pre-copy phase is completed, in the differential copy phase, the operating system intercepts and caches all non-stop kernel functions emitted by the program and performs the following steps:
[0014] Step 5: The operating system determines the size of the dirty-bit marked buffers: if the size is lower than a preset threshold, the operating system copies all dirty-bit marked buffers to the CPU side; otherwise, execute step 6;
[0015] Step 6: The operating system saves the state of the CPU-side program and packages the CPU-side program state and all non-stop kernel functions intercepted in the pre-copy and differential copy stages into a binary image;
[0016] In the state recovery phase, the operating system restores the GPGPU memory state by performing the following steps:
[0017] Step 7: The operating system recovers the dirty-bit marked buffers that were not processed in the differential copy phase by reissuing the part of the pre-copy phase without stopping the kernel function;
[0018] Step 8: The operating system launches the non-stop kernel function that has not been executed and is cached in the differential copy phase.
[0019] Preferably, in step 1, the operating system verifies the redundant data existing on the CPU side and the GPGPU side based on a checksum calculation method, and omits the copy operation of this part of the GPGPU buffer, thereby reducing the size of the final binary image.
[0020] Preferably, the process of verifying the redundant data existing on the CPU side and the GPGPU side is: calculating the hash value of the memory segment on the GPGPU side and the memory segment corresponding to the CPU. If the hash values of the two are consistent, they are considered to be the same and are redundant data; wherein only the data copied from the CPU side to the GPGPU side is verified.
[0021] Preferably, in step 1 and step 2, if the operating system detects that the original program will perform a memory copy operation on the CPU and GPGPU sides, the operating system will save the introduced copy operation in a paused state.
[0022] Preferably, in step 2, memory semantic analysis is performed on the non-stop kernel function by analyzing function signatures and dynamic program analysis.
[0023] Preferably, if the function signature is documented, the memory accessed by the function is analyzed based on the document; if the function signature is an anonymous function, all its parameters are considered to be memory addresses to be accessed, and are matched with the address segment allocated by the current program. If they match, it is considered that the function has accessed the memory.
[0024] Preferably, the step 3 comprises:
[0025] Added step: The operating system copies the buffer data to the temporary memory area of GPGPU;
[0026] Commit step: The operating system copies the buffer data stored in the temporary memory area to the CPU side.
[0027] Preferably, in step 5, if the operating system determines that copying the dirty bit buffer introduces an overhead exceeding a preset time length, the copy operation is abandoned.
[0028] Preferably, in step 7, in step 7 and in step 8, the operating system restores the buffer to the GPGPU as needed: the operating system loads the input buffer it depends on before launching the non-stop kernel function to gradually restore the GPGPU memory state.
[0029] According to the state saving and restoring system applicable to a general-purpose graphics processor provided by the present invention, executing the state saving and restoring method applicable to a general-purpose graphics processor also includes:
[0030] The interception and analysis module of the non-stop kernel function: analyzes the memory access of the non-stop kernel function before the non-stop kernel function is launched;
[0031] State saving module: copy GPGPU memory;
[0032] State recovery module: restores GPGPU execution from a checkpoint;
[0033] The interception of the kernel function that does not stop is located in the same address space as the program whose state is saved and restored, or in a different address space;
[0034] In the scenario of the agent program, the interception and analysis module of the non-stop kernel function and the application are in the same address space, and the state saving module and the state recovery module are located in the agent program; they interact through inter-process communication and are all executed on the CPU.
[0035] Compared with the prior art, the present invention has the following beneficial effects:
[0036] (1) When saving the state, the kernel function can be kept running for most of the time, so there is no long-term program interruption, which reduces the overhead introduced by state saving to the original program operation;
[0037] (2) At runtime, the state preservation / state recovery is performed by using a differential copy or recalculation scheme according to the dirty memory size in the pre-copy phase, thereby maintaining good state preservation / state recovery performance under workloads with different memory access patterns;
[0038] (3) Redundant data on the CPU and GPGPU sides is eliminated when saving the state, which shortens the memory copy time when saving the state and reduces the size of the binary image.
[0039] (4) The on-demand recovery method is adopted during state recovery, which shortens the time it takes for the program to resume running. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0041] Figure 1a and Figure 1b They are respectively flow charts of a state saving and restoring method applicable to a general-purpose graphics processor;
[0042] Figure 2 Schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION
[0043] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0044] Example
[0045] As shown in FIG1 , the present invention shows a specific process of performing C / R (state save / state restore) on GPGPU memory based on Kernel (non-stop kernel function) memory access semantics. The following takes the C / R of three buffers (buffers, BufferA, BufferB, BufferC) in GPGPU as an example, and describes the following C / R steps in detail in conjunction with FIG1 , wherein steps 401 to 406 belong to the Checkpoint (state save) stage, and steps 407 to 408 belong to the Restore (state restore) stage:
[0046] In step 401, the OS (operating system) copies the buffers on the GPGPU to the Checkpoint area in the order of A, B, and C. Assume that the OS is copying BufferA at this time, and at the same time, steps 402 to 404 are executed synchronously;
[0047] In step 402, the OS intercepts the kernels sent by the program to the GPGPU and analyzes their memory access semantics. Assuming that the analysis result shows that the currently intercepted kernel uses BufferB as the input buffer and BufferA as the output buffer, then execute step 403;
[0048] In step 403, the OS detects that BufferA is being copied, and blocks until the copy of BufferA is completed, raises the dirty bit of BufferA, and then executes step 404;
[0049] In step 404, the OS sends the Kernel to the GPGPU for execution, and then re-executes 402;
[0050] If it is assumed that BufferB has not been copied in step 401, and the result of OS's analysis of Kernel in step 402 is to use BufferB as the output Buffer, then in step 403, OS detects that BufferB as the output Buffer has not been copied, and first performs COW (copy on write) on BufferB, raises the dirty bit of BufferB, and then executes step 404;
[0051] After the copy operation of all buffers is completed in step 401, step 405 is executed;
[0052] In step 405, the OS determines that most of the copied memory areas are marked as dirty, and does not perform differential copying, and directly executes step 406;
[0053] In step 406, the OS packages the process into a binary image on the CPU side and stores it in the form of a persistent file or memory data;
[0054] In step 407, the OS determines that the dirty memory is not supplemented with a copy in step 405, and then the OS re-executes part of the kernels during steps 402 to 404 to recover the dirty memory data;
[0055] In step 408, the OS executes the kernels that have not been executed and are cached by the OS during steps 405-406.
[0056] like Figure 2As shown, the present invention forwards the kernels emitted by the program to the semantic analysis module in the application process during Checkpoint, and uses function signature analysis, dynamic program analysis and other means to analyze the memory access semantics of the kernel, and combines the semantics to coordinate the normal kernel execution and the copy process of the GPGPU memory state; when restoring, the data stored in the binary image is used to complete the recovery of the GPGPU state.
[0057] The present invention also provides a state saving and restoring system applicable to a general-purpose graphics processor, comprising:
[0058] The interception and analysis module of the non-stop kernel function: analyzes the memory access of the non-stop kernel function before the non-stop kernel function is launched;
[0059] State saving module: copy GPGPU memory;
[0060] State recovery module: restores GPGPU execution from a checkpoint;
[0061] The interception of the kernel function that does not stop is located in the same address space as the program whose state is saved and restored, or in a different address space;
[0062] In the scenario of the agent program, the interception and analysis module of the non-stop kernel function and the application are in the same address space, and the state saving module and the state recovery module are located in the agent program; they interact through inter-process communication and are all executed on the CPU.
[0063] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.
[0064] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. A state saving and recovery method applicable to a general-purpose graphics processor, characterized in that: include: Without stopping the kernel function, the memory data of GPGPU is copied to the CPU side in the state saving phase, and the memory data of GPGPU side is restored from the CPU side in the state recovery phase; The state preservation phase includes the pre-copy and differential copy phases; During the pre-copy phase, the operating system performs the following steps: Step 1: The operating system copies the buffer in the GPGPU memory and concurrently executes steps 2 to 4 in sequence; Step 2: The operating system analyzes the memory access semantics of each non-stop kernel function launched by the program, and obtains the output buffer and input buffer of the non-stop kernel function for each buffer; Step 3: The operating system determines the copy status of the output buffer of the non-stop kernel function: if the output buffer has not been copied by step 1, a copy operation is performed on it, that is, a copy is performed on it during writing, and a dirty bit is marked for it; if it has been copied by step 1, the operating system marks it dirty; if it is being copied by step 1, the operating system blocks and waits for the copy to be completed, and marks it dirty; Step 4: The operating system sends the non-stop kernel function to the GPGPU for execution; After the pre-copy phase is completed, in the differential copy phase, the operating system intercepts and caches all non-stop kernel functions emitted by the program and performs the following steps: Step 5: The operating system determines the size of the dirty-bit marked buffers: if the size is lower than a preset threshold, the operating system copies all dirty-bit marked buffers to the CPU side; otherwise, execute step 6; Step 6: The operating system saves the state of the CPU-side program and packages the CPU-side program state and all non-stop kernel functions intercepted in the pre-copy and differential copy stages into a binary image; In the state recovery phase, the operating system restores the GPGPU memory state by performing the following steps: Step 7: The operating system recovers the dirty-bit marked buffers that were not processed in the differential copy phase by reissuing the part of the pre-copy phase without stopping the kernel function; Step 8: The operating system launches the non-stop kernel function that has not been executed and is cached in the differential copy phase.
2. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: In step 1, the operating system verifies the redundant data existing on the CPU side and the GPGPU side based on the method of calculating the checksum, and omits the copy operation of this part of the GPGPU buffer, thereby reducing the size of the final binary image.
3. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 2, characterized in that: The process of verifying the redundant data on the CPU side and the GPGPU side is as follows: the hash values of the memory segment on the GPGPU side and the memory segment corresponding to the CPU are calculated. If the hash values of the two are consistent, they are considered to be the same and are redundant data. Among them, only the data copied from the CPU side to the GPGPU side is verified.
4. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: In step 1 and step 2, if the operating system detects that the original program will perform a memory copy operation on the CPU and GPGPU sides, the operating system will suspend the state to save the introduced copy operation.
5. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: In the step 2, memory semantic analysis is performed on the non-stop kernel function by analyzing function signatures and dynamic program analysis.
6. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 5, characterized in that: If the function signature has a document, the memory accessed by the function is analyzed based on the document; if the function signature is an anonymous function, all its parameters are considered to be memory addresses to be accessed, and are matched with the address segment allocated by the current program. If they match, the function is considered to have accessed the memory.
7. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: The step 3 comprises: Added step: The operating system copies the buffer data to the temporary memory area of GPGPU; Commit step: The operating system copies the buffer data stored in the temporary memory area to the CPU side.
8. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: In step 5, if the operating system determines that copying the dirty bit buffer introduces an overhead exceeding a preset time length, the copy operation is abandoned.
9. The state saving and restoring method applicable to a general-purpose graphics processor according to claim 1, characterized in that: In step 7, step 7 and step 8, the operating system restores the buffer to the GPGPU as needed: the operating system loads the input buffer it depends on before launching the non-stop kernel function to gradually restore the GPGPU memory state.
10. A state saving and recovery system suitable for a general-purpose graphics processor, characterized in that: The state saving and restoring method applicable to a general-purpose graphics processor according to any one of claims 1 to 9 further includes: The interception and analysis module of the non-stop kernel function: analyzes the memory access of the non-stop kernel function before the non-stop kernel function is launched; State saving module: copy GPGPU memory; State recovery module: restores GPGPU execution from a checkpoint; The interception of the kernel function that does not stop is located in the same address space as the program whose state is saved and restored, or in a different address space; In the scenario of the agent program, the interception and analysis module of the non-stop kernel function and the application are in the same address space, and the state saving module and the state recovery module are located in the agent program; they interact through inter-process communication and are all executed on the CPU.
Citation Information
Patent Citations
Method for performing recovery and playback when running software
CN101154185A
Memory pre-copying and virtual machine migration method and system based on dirty rate prediction
CN112181601A
GPU task real-time preemption method and system
CN115408135A
Chip optimization method and device, electronic equipment and storage medium
CN117556756A
GPGPU (General Purpose Graphics Processing Unit) branch instruction scheduling method based on multistage redistribution mechanism
CN118012632A
