State saving and restoring method and system suitable for general-purpose graphics processors

Abstract

The application provides a state saving and restoring method and system suitable for general-purpose graphic processors, in the process of state saving, the method allows GPU application to continue to execute, meanwhile, before emitting a GPGPU kernel function, the method predicts memory accessed by the kernel function, and through methods such as write-time copying and setting a dirty bit, the accuracy of state saving data is ensured; in the process of state restoring, the method restores dirty memory through re-executing the kernel function, and accelerates the restoring process of the program through an on-demand loading mode. Compared with the prior art, the application has the characteristics of low overhead, wide hardware and workload adaptation range, and can be used to support important system requirements such as disaster recovery backup, hot migration and fast start.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of general-purpose graphic processor, in particular to a state saving and restoring method and system suitable for general-purpose graphic processor. BACKGROUND

[0002] In the fields of machine learning, scientific computing and 3D rendering, general-purpose graphic processor (GPGPU) is an important hardware for performing operations. A program can complete data processing by transmitting a computing task with a kernel function (Kernel) as the minimum granularity to the GPGPU. The Kernel usually takes a plurality of buffers stored in the GPGPU memory as input data, completes corresponding operations, and writes the results into a plurality of buffers. In cloud computing scenarios, checkpointing and restoring (C / R) of the state of an operating system (OS) process using the GPGPU is an important function, which can be used to support important system capabilities such as disaster recovery backup, live migration and fast start. Checkpointing records the state of a running process on the CPU side and the GPGPU side as a binary image, and restoring reloads the process state recorded in the binary image on the CPU and the GPGPU to resume the execution of the process.

[0003] Based on whether the original program needs to be modified to implement the C / R function, the existing C / R schemes can be divided into non-transparent and transparent schemes. Non-transparent C / R requires users to manually add C / R functions to the program; transparent C / R directly completes the C / R of the process state on the OS without modifying the original program. In cloud scenarios, since it is difficult for operators to intrude into the tenant program, transparent C / R is a more important capability, and is the field focused on by the present application.

[0004] Currently, the main method of transparent C / R for GPGPU is memory copy scheme: at Checkpoint, OS first suspends the program from launching Kernel to GPGPU, and waits for all the Kernels launched to GPGPU to be executed completely, then OS copies all the data in GPGPU memory to CPU side, and finally saves the process state as binary image on CPU side; at Restore, OS first loads the memory data on GPGPU side, and then restores the running of the program on CPU side. This kind of scheme avoids the long Kernel re-execution overhead at Restore, but the disadvantage is that it introduces a long program pause time at Checkpoint, and because of the redundancy of memory data on CPU and GPU sides, Checkpoint will generate a large binary image. SUMMARY

[0005] In view of the defects in the prior art, the purpose of the present application is to provide a state saving and restoring method and system suitable for general-purpose graphic processors.

[0006] The state saving and restoring method suitable for general-purpose graphic processors provided by the present application comprises: copying memory data of a GPGPU to a CPU side in a state saving stage without stopping the running of a kernel function, and restoring the memory data of the GPGPU side from the CPU side in a state restoring stage.

[0007] The state saving stage comprises a pre-copying stage and a differential copy stage.

[0008] In the pre-copying stage, the operating system performs the following steps:

[0009] Step 1: the operating system copies the buffer in the GPGPU memory, while concurrently performing steps 2 to 4 in sequence;

[0010] Step 2: the operating system analyzes the memory access semantics of each non-stop kernel function launched by the program, and obtains the output buffer and input buffer of each non-stop kernel function to each buffer;

[0011] Step 3: the operating system judges the copy status of the output buffer of the non-stop kernel function: if the output buffer has not been copied by step 1, it is copied, i.e. write-copy, and is marked with a dirty bit; if it has been copied by step 1, the operating system marks it with a dirty bit; if it is being copied by step 1, it is blocked and waits for the end of the copy, and is marked with a dirty bit;

[0012] Step 4: the operating system launches the non-stop kernel function to GPGPU for execution;

[0013] After the pre-copy phase is completed, in the differential copy phase, the operating system intercepts and caches all non-stop kernel functions launched by the program, and performs the following steps:

[0014] Step 5: The operating system judges the size of the cache marked with the dirty bit: if the size is lower than the preset threshold, the operating system re-copies all the caches marked with the dirty bit to the CPU side; otherwise, step 6 is performed.

[0015] Step 6: The operating system saves the state of the CPU-side program, and packs the CPU-side program state and all non-stop kernel functions intercepted in the pre-copy and differential copy phases into a binary image.

[0016] In the state recovery phase, the operating system recovers the memory state of the GPGPU by performing the following steps:

[0017] Step 7: The operating system recovers the caches marked with the dirty bit that are not processed in the differential copy phase by re-launching part of the non-stop kernel functions in the pre-copy phase.

[0018] Step 8: The operating system launches the non-stop kernel functions that have not been executed and are cached in the differential copy phase.

[0019] Preferably, in step 1, the operating system verifies the redundant data existing on the CPU side and the GPGPU side based on the checksum calculation method, and omits the copy operation of the part of the GPGPU cache, thereby reducing the size of the final binary image.

[0020] Preferably, the process of verifying the redundant data existing on the CPU side and the GPGPU side is: calculating the hash values of the memory segment on the GPGPU side and the corresponding memory segment on the CPU, and if the hash values of the two are consistent, it is considered that the two are the same, which is redundant data; wherein, only the data copied from the CPU side to the GPGPU side is verified.

[0021] Preferably, in steps 1 and 2, if the operating system detects that the original program will perform the memory copy operation of the CPU and the GPGPU side, the operating system will suspend the copy operation introduced by the state saving.

[0022] Preferably, in step 2, the memory semantic analysis of the non-stop kernel function is performed by analyzing the function signature and dynamic program analysis.

[0023] Preferably, if the function signature exists in the document, the memory accessed by the function is analyzed according to the document; if the function signature is an anonymous function, all its parameters are considered as the memory address accessed, and are matched with the address segment allocated by the current program, if matched, it is considered that the function has accessed the memory.

[0024] Preferably, step 3 comprises:

[0025] add step: the operating system copies the buffer data into a temporary memory area of the GPGPU;

[0026] commit step: the operating system copies the buffer data stored in the temporary memory area to the CPU side.

[0027] Preferably, in the step 5, the operating system gives up the copy operation if it is determined that the copy dirty bit buffer introduces an overhead exceeding a preset time length.

[0028] Preferably, in the step 7, the operating system re-recoveries the buffer to the GPGPU on demand: the operating system loads the input buffer on which the non-stop kernel function depends before the non-stop kernel function is launched, so as to gradually recover the memory state of the GPGPU.

[0029] According to the application, the state saving and recovery system for the general-purpose graphic processor is provided, and the state saving and recovery method for the general-purpose graphic processor is executed, and the system further comprises:

[0030] the interception and analysis module of the non-stop kernel function: the memory accessed by the non-stop kernel function is analyzed before the non-stop kernel function is launched;

[0031] the state saving module: the memory of the GPGPU is copied;

[0032] the state recovery module: the execution of the GPGPU is recovered from the checkpoint;

[0033] The interception of the non-stop kernel function is located in the same address space as the program saved and recovered by the state, or is located in different address spaces.

[0034] In the scenario of the proxy program, the interception and analysis module of the non-stop kernel function and the application are located in the same address space, the state saving module and the state recovery module are located in the proxy program, the interaction is performed through inter-process communication, and the state saving module and the state recovery module are executed on the CPU.

[0035] Compared with the prior art, the application has the following beneficial effects:

[0036] (1) The running of the non-stop kernel function can be maintained in most of the time during the state saving, so there is no long program interruption, and the overhead introduced by the state saving to the original program running is reduced;

[0037] (2) The state saving / state recovery is performed according to the size of the dirty memory in the pre-copy stage, so the state saving / state recovery performance can be maintained under different memory access patterns of the workload;

[0038] (3) The redundant data on the CPU side and the GPGPU side is eliminated during the state saving, the memory copy time during the state saving is shortened, and the size of the binary image is reduced;

[0039] (4) The on-demand restoring method is adopted during the state restoring, and the program running time is shortened. BRIEF DESCRIPTION OF DRAWINGS

[0040] Other features, objects, and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, when read in conjunction with the accompanying drawings:

[0041] Figure 1a and Figure 1b are flowcharts of the state saving and restoring methods suitable for general-purpose graphic processors, respectively.

[0042] Figure 2 is a schematic diagram of the system architecture of the application. DETAILED DESCRIPTION

[0043] The application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the application. These are within the scope of protection of the application.

[0044] EMBODIMENT

[0045] As shown in FIG. 1, it is the specific process of C / R (state saving / state restoring) of the GPGPU memory based on the Kernel (non-stop kernel function) memory access semantics of the application. The following C / R steps will be described in detail below by taking the C / R of three Buffers (Buffer A, Buffer B, and Buffer C) in the GPGPU as an example in combination with FIG. 1, wherein steps 401-406 belong to the Checkpoint (state saving) phase, and steps 407-408 belong to the Restore (state restoring) phase:

[0046] In step 401, the OS (operating system) copies the Buffers on the GPGPU to the Checkpoint area in the order of A, B, and C. It is assumed that the OS is performing the copy operation on Buffer A at this time, and steps 402-404 are synchronously executed at the same time;

[0047] In step 402, the OS intercepts the Kernel launched by the program to the GPGPU, and analyzes the memory access semantics. Assuming that the analysis result is that the currently intercepted Kernel takes BufferB as the input Buffer and takes BufferA as the output Buffer, then step 403 is executed;

[0048] In step 403, the OS detects that BufferA is being copied, and blocks until the BufferA copying is completed, then pulls the dirty bit of BufferA, and then step 404 is executed;

[0049] In step 404, the OS launches the Kernel to the GPGPU for execution, and then re-executes step 402;

[0050] If it is assumed that step 401 has not performed the copying operation on BufferB, and the analysis result of the Kernel by the OS in step 402 is that BufferB is taken as the output Buffer, then in step 403, the OS detects that the BufferB taken as the output Buffer has not been copied, and first performs COW (copy-on-write) on BufferB, then pulls the dirty bit of BufferB, and then step 404 is executed;

[0051] After step 401 completes the copying operation on all Buffers, step 405 is executed;

[0052] In step 405, the OS judges that most of the copied memory regions are marked with the dirty bit, and does not perform the differential copying, and directly executes step 406;

[0053] In step 406, the OS packages the process into a binary image on the CPU side, and stores in the form of a persistent file or memory data;

[0054] In step 407, the OS judges that the dirty memory is not supplemented in step 405, and then the OS re-executes the part of the Kernels during steps 402-404 to restore the dirty memory data;

[0055] In step 408, the OS executes the Kernels that have not been executed and are cached by the OS during steps 405-406.

[0056] As Figure 2As shown, the application forwards the Kernels launched by the program in the application program process at Checkpoint to the semantic analysis module, and analyzes the memory access semantics of the Kernels by means of function signature analysis, dynamic program analysis, etc., and combines the semantic coordination to execute the normal Kernels and the copying process of the GPGPU memory state; at Restore, the GPGPU state is restored by using the data stored in the binary image.

[0057] The application further provides a state saving and restoring system suitable for a general-purpose graphics processor, comprising:

[0058] The intercepting and analyzing module of the non-stop kernel function analyzes the access memory of the non-stop kernel function without stopping the launch of the non-stop kernel function;

[0059] The state saving module copies the memory of the GPGPU;

[0060] The state restoring module restores the execution of the GPGPU from the checkpoint;

[0061] The intercepting of the non-stop kernel function is located in the same address space as the program saved and restored by the state, or is located in different address spaces;

[0062] In the scenario of the proxy program, the intercepting and analyzing module of the non-stop kernel function and the application are located in the same address space, and the state saving module and the state restoring module are located in the proxy program; the interaction is performed through inter-process communication, and the state saving module and the state restoring module are executed on the CPU.

[0063] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the application in the form of pure computer readable program code, the same program can also be implemented in the form of logic gates, switches, special integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and each module thereof provided by the application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures in the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures in the hardware component.

[0064] The specific embodiments of the application are described above. It should be understood that the application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the application. The embodiments of the present application and the features in the embodiments can be arbitrarily combined with each other without conflict.

Claims

1. A state saving and restoring method suitable for a general-purpose graphic processor, characterized by, The application comprises the following steps: In the state saving stage, the memory data of the GPGPU is copied to the CPU side without stopping the kernel function, and in the state recovery stage, the memory data of the GPGPU side is recovered from the CPU side; The state saving stage comprises a pre-copying stage and a differential copying stage; In the pre-copying stage, the operating system performs the following steps: Step 1: the operating system copies the buffer in the GPGPU memory while concurrently performing steps 2 to 4 in sequence; Step 2: the operating system analyzes the memory access semantics of each non-stop kernel function launched by the program to obtain the output buffer and input buffer of the non-stop kernel function for each buffer; Step 3: the operating system judges the copying status of the output buffer of the non-stop kernel function: if the output buffer has not been copied by step 1, it is copied, that is, a copy-on-write is performed, and a dirty bit is marked; if it has been copied by step 1, the operating system marks a dirty bit for it; if it is being copied by step 1, it is blocked and waits for the copying to end, and a dirty bit is marked; Step 4: the operating system launches the non-stop kernel function to the GPGPU for execution; After the pre-copying stage is completed, in the differential copying stage, the operating system intercepts and buffers all non-stop kernel functions launched by the program and performs the following steps: Step 5: the operating system judges the size of the buffer marked with a dirty bit: if the size is below a preset threshold, the operating system re-copies all the buffers marked with a dirty bit to the CPU side; otherwise, step 6 is performed; Step 6: the operating system saves the state of the CPU side program and packs the CPU side program state and all non-stop kernel functions intercepted in the pre-copying and differential copying stages into a binary image; In the state recovery stage, the operating system recovers the memory state of the GPGPU by performing the following steps: Step 7: the operating system recovers the buffer marked with a dirty bit that has not been processed in the differential copying stage by re-launching part of the non-stop kernel functions in the pre-copying stage; Step 8: the operating system launches the non-stop kernel functions that have not been executed and are buffered in the differential copying stage.

2. The state saving and restoring method suitable for a general-purpose graphic processor according to claim 1, wherein, In step 1, the operating system verifies the redundant data existing on the CPU side and the GPGPU side based on a checksum calculation method, and omits the copying operation of the GPGPU buffer, thereby reducing the size of the final binary image.

3. The state saving and restoring method suitable for a general-purpose graphic processor according to claim 2, wherein, The process of verifying the redundant data existing on the CPU side and the GPGPU side is as follows: the hash values of the memory segment on the GPGPU side and the corresponding memory segment on the CPU are calculated, and if the hash values are consistent, it is considered that the two are the same, which is redundant data; wherein, only the data copied from the CPU side to the GPGPU side is verified.

4. The state saving and restoring method suitable for a general-purpose graphic processor according to Claim 1, wherein, In steps 1 and 2, if the operating system detects that the original program will perform a memory copying operation on the CPU and the GPGPU side, the operating system will suspend the copying operation introduced by the state saving.

5. The state saving and restoring method suitable for a general-purpose graphic processor according to Claim 1, wherein, In step 2, the memory semantics of the non-stop kernel function is analyzed by analyzing the function signature and dynamic program analysis.

6. The state saving and restoring method suitable for a general-purpose graphic processor according to claim 5, wherein, If the function signature is documented, the memory accessed by the function is analyzed according to the documentation; if the function signature is an anonymous function, all its parameters are considered as memory addresses accessed by the function, and are matched with the address segment allocated by the current program, if matched, the function is considered to have accessed the memory.

7. The state saving and restoring method suitable for a general-purpose graphic processor according to Claim 1, wherein, The step 3 comprises: An adding step: the operating system copies the buffer data to a temporary memory area of the GPGPU; A submitting step: the operating system copies the buffer data stored in the temporary memory area to the CPU side.

8. The state saving and restoring method suitable for a general-purpose graphic processor according to Claim 1, wherein, In the step 5, if the operating system judges that the overhead introduced by the copy dirty bit buffer exceeds the preset time length, the operating system gives up the copy operation.

9. The state saving and restoring method suitable for a general-purpose graphic processor according to Claim 1, wherein, In the step 7, the operating system restores the buffer to the GPGPU as needed: the operating system loads the input buffer relied on by the non-stop kernel function before the non-stop kernel function is launched, so as to gradually restore the memory state of the GPGPU.

10. A state saving and restoring system suitable for a general purpose graphics processor, the system comprising: a state saving module configured to save a state of the general purpose graphics processor; and a state restoring module configured to restore the state of the general purpose graphics processor. The state saving and restoring method suitable for a general-purpose graphics processor according to any one of claims 1 to 9 further comprises: An intercepting and analyzing module of the non-stop kernel function: analyzing the memory accessed by the non-stop kernel function before the non-stop kernel function is launched; A state saving module: copying the memory of the GPGPU; A state restoring module: restoring the execution of the GPGPU from the checkpoint; The intercepting of the non-stop kernel function is located in the same address space as the program saved and restored by the state, or in a different address space; In the scenario of the proxy program, the intercepting and analyzing module of the non-stop kernel function and the application are located in the same address space, and the state saving module and the state restoring module are located in the proxy program; the interaction is performed through inter-process communication, and both are executed on the CPU.

Images