GPU access data tracking method and system based on kernel parameter analysis and instrumentation verification
Through methods based on kernel parameter analysis and instrumentation verification, the video memory access range of GPU kernel is predicted and verified, which solves the problems of insufficient hardware support and large performance overhead in the existing technology, and realizes efficient and low-overhead GPU access data tracking.
Patent Information
- Application Number
- CN202411200303.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
Existing GPU access data tracking methods lack hardware support and have high performance overhead, especially methods that rely entirely on instrumentation tracking are severe in performance losses when executed at high frequency.
Using kernel parameter analysis and instrumentation verification methods, by intercepting GPU video memory calls and kernel startup calls, predicting and verifying the memory range accessed by kernel, generating kernels after instrumentation to verify the prediction results, reducing the performance overhead brought by instrumentation .
It reduces the impact of instrumentation on GPU performance, reduces the runtime performance overhead, and realizes efficient access data tracking, without relying on hardware support provided by GPU manufacturers. It has low hardware dependence and wide application prospects.
Smart Images
Figure CN119088699A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of GPU access data tracing, and in particular to a GPU access data tracing method and system based on kernel parameter analysis and plug-in verification. Background Art
[0002] GPU is a hardware accelerator specially designed for handling graphics computing and parallel processing tasks. It has a large number of parallel processing units and can process large amounts of data at the same time, making it particularly suitable for tasks such as graphics rendering, scientific computing, and machine learning. GPU kernel is a function of parallel computing tasks executed on the GPU. It defines the operations that each thread on the GPU will perform, and is usually executed in parallel on thousands of threads. GPU stub refers to adding additional code to the GPU program (kernel code) for performance analysis, debugging, and monitoring. These stub codes can collect various information about the program runtime, such as execution time, memory usage, etc.
[0003] Access data tracking refers to the process of monitoring and managing data that has been modified but not yet persisted (such as written back to disk or main memory) or migrated to other machines. One of the design goals of access data tracking is to be transparent to upper-layer applications, that is, applications can benefit from the access data tracking mechanism without special processing. This can simplify development work and improve the usability and stability of the system. GPU access data tracking can help realize the swapping of data between video memory and main memory, and can also help realize the hot migration of GPU tasks between different machines, which has high application value.
[0004] Currently, there are two main methods for tracking GPU access data: one is hardware-software co-design: the hardware provides access bits or other similar flags to track which data has been modified. This method currently lacks support from GPU manufacturers, and mainstream GPUs (such as NVIDIA GPUs) do not provide corresponding hardware interfaces for application software. The other is to track access data completely through instrumentation: insert an instrumentation code at each memory write instruction of the kernel, and the instrumentation code is responsible for recording the address of each write. This method requires high-frequency execution of relatively complex instrumentation code, which has a large performance overhead. Summary of the invention
[0005] In view of the defects in the prior art, the object of the present invention is to provide a GPU access data tracking method and system based on kernel parameter analysis and instrumentation verification.
[0006] A GPU access data tracking method based on kernel parameter analysis and instrumentation verification is provided according to the present invention, comprising:
[0007] Kernel instrumentation step: instrument the target kernel one by one to generate the instrumented kernel;
[0008] Access data prediction step: Analyze the parameters of the kernel to be started, predict the memory range modified by the kernel, and call the kernel after the stub to verify the prediction result;
[0009] Access data update step: update the access data tracking result according to the verification result; if the verification passes, add the current prediction range to the access data range; if the verification fails, update the access data to the entire allocated video memory range.
[0010] Preferably, the instrumented kernel can verify the video memory write address range, and the instrumented object is an intermediate representation of the binary kernel code.
[0011] Preferably, the access data prediction step includes intercepting a GPU call, and if it is a video memory allocation / release call, updating a video memory allocation record; the video memory allocation information includes the address and size of the video memory allocated to the GPU application.
[0012] Preferably, the access data prediction step further includes:
[0013] Intercept GPU calls. If it is a kernel startup call,
[0014] Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the parameters of the kernel include the untyped parameter bytes passed to the kernel at runtime;
[0015] Get the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as the prediction of the kernel write video memory range;
[0016] Execute the instrumented version of the kernel to be started, pass the predicted write memory range to the kernel as an additional parameter, and retrieve the predicted verification result after the kernel execution is completed.
[0017] Preferably, the scanning of the parameters of the kernel to be started includes enumerating a byte sequence of a pointer size in the parameter, and if the byte sequence falls within a certain allocated video memory range as interpreted by the pointer, the byte sequence is identified as a potential video memory pointer, and the allocated video memory range is considered to be video memory allocation information corresponding to the potential pointer.
[0018] Preferably, the predicted verification result is a Boolean value, which is the AND of each write address verification result during kernel execution; if the kernel verification result is true, it means that the verification is passed.
[0019] A GPU access data tracking system based on kernel parameter analysis and instrumentation verification is provided according to the present invention, comprising:
[0020] Kernel instrumentation module: instruments the target kernel one by one to generate the instrumented kernel;
[0021] Access data prediction module: analyze the parameters of the kernel to be started, predict the memory range modified by the kernel, and call the kernel after the stub to verify the prediction results;
[0022] Access data update module: updates access data tracking results according to verification results; if verification passes, adds the current prediction range to the access data range; if verification fails, updates the access data to the entire allocated video memory range.
[0023] Preferably, the instrumented kernel can verify the video memory write address range, and the instrumented object is an intermediate representation of the binary kernel code.
[0024] Preferably, the access data prediction module includes intercepting GPU calls, and if it is a video memory allocation / release call, updating the video memory allocation record; the video memory allocation information includes the address and size of the video memory allocated to the GPU application.
[0025] Preferably, the access data prediction module further includes:
[0026] Intercept GPU calls. If it is a kernel startup call,
[0027] Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the parameters of the kernel include the untyped parameter bytes passed to the kernel at runtime;
[0028] Get the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as the prediction of the kernel write video memory range;
[0029] Execute the instrumented version of the kernel to be started, pass the predicted write memory range to the kernel as an additional parameter, and retrieve the predicted verification result after the kernel execution is completed.
[0030] Preferably, the scanning of the parameters of the kernel to be started includes enumerating a byte sequence of a pointer size in the parameter, and if the byte sequence falls within a certain allocated video memory range as interpreted by the pointer, the byte sequence is identified as a potential video memory pointer, and the allocated video memory range is considered to be video memory allocation information corresponding to the potential pointer.
[0031] Preferably, the predicted verification result is a Boolean value, which is the AND of each write address verification result during kernel execution; if the kernel verification result is true, it means that the verification is passed.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] 1. The present invention first intercepts GPU video memory calls and kernel startup calls, and predicts the video memory accessed by the kernel through the parameters of kenrel, and then inserts the running kernel, so that the kernel can efficiently verify whether the predicted access is correct during operation; compared with the prior art, the present invention has the advantages of low hardware dependence and low operating overhead.
[0034] 2. The present invention is independent of the hardware support provided by GPU manufacturers, such as hardware access bits and supporting drivers; it adopts a prediction and verification method, which reduces the impact on kernel performance by ten times compared with the existing method of directly tracking GPU plug-ins, reduces the runtime performance overhead caused by plug-ins, and has broad application prospects.
[0035] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0037] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0038] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0039] Example 1
[0040] Reference Figure 1As shown, a GPU access data tracking method based on kernel parameter analysis and instrumentation verification includes a kernel instrumentation tool and a runtime access data tracking tool. The kernel instrumentation tool can insert code into the original kernel to verify whether the video memory modification range is within the predicted range. The runtime access data tracking tool can analyze the parameters of the kernel to be started, thereby predicting the video memory range that the kernel may modify, and calling the kernel instrumented by the kernel instrumentation tool to verify the correctness of the prediction result. The specific steps are as follows:
[0041] Step 1: All GPU kernels included in the program are instrumented one by one. The instrumented object is the intermediate representation of the binary kernel code (such as NVIDIA PTX), and an instrumented kernel that can verify the memory write address range is generated; the function of the instrumented code is to receive additional kernel parameters representing the predicted write address range, and each time the original write instruction in the kernel is executed, verify whether the write address is within the predicted write address range, if so, the verification result is true, otherwise it is false.
[0042] Step 2: Intercept GPU calls and execute steps 3 and 4 according to the call situation;
[0043] Step 3: For a video memory allocation / release call, update the video memory allocation record; the video memory allocation information refers to the address and size of the video memory that has been allocated to the GPU application.
[0044] Step 4: For kernel startup call, execute steps 5, 6, 7, and 8;
[0045] Step 5: Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the kernel parameters refer to the untyped parameter bytes passed to the kernel at runtime, and the parameter scanning refers to the byte sequence of the pointer size in the enumeration parameters. If the byte sequence falls within a certain allocated video memory range according to the pointer interpretation, the byte sequence is identified as a potential video memory pointer, and the allocated video memory range is considered to be the video memory allocation information corresponding to the potential pointer.
[0046] Step 6: Obtain the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as a prediction of the kernel write video memory range; the process of summarizing the allocation information is to obtain the union of the video memory allocation information corresponding to all potential video memory pointers.
[0047] Step 7: Execute the instrumented version of the kernel to be started, pass the predicted write memory range as an additional parameter to the kernel, and retrieve the predicted verification result after the kernel execution ends; the kernel verification result is a Boolean value, which is the AND of each write address verification result during the kernel execution process. A kernel verification result of true means that the verification is successful.
[0048] Step 8: Update the access data tracking result according to the verification result. If the verification is successful, add the current speculation range to the access data range. Otherwise, update it to the entire allocated video memory range.
[0049] The present invention is independent of the hardware support provided by the GPU manufacturer, such as hardware access bits and supporting drivers; it adopts a prediction and verification method, which reduces the impact on kernel performance by ten times compared with the existing method of directly tracing GPU plugging, reduces the runtime performance overhead caused by plugging, and has broad application prospects.
[0050] Example 2
[0051] Reference Figure 1 The following is a specific flow of the GPU-oriented access data tracking method based on kernel parameter analysis and plugging verification of the present invention. This embodiment takes two video memory allocations (A, F) and one GPU kernel call (B) as an example. Figure 1 The following steps for access data tracking are described in detail:
[0052] In step 401 , B obtains C through plugging. Compared with B, C adds the function of receiving the predicted video memory modification range and performing verification, and then executes step 402 .
[0053] In step 402, the allocation calls of A and F and the start call of B are intercepted in sequence, and then steps 403 and 404 are executed respectively.
[0054] In step 403, the starting addresses and sizes of the A and F allocations are recorded.
[0055] In step 404, B's startup call is intercepted and step 405 is executed.
[0056] In step 405 , assuming that the length of the byte sequence D in the parameter of B is exactly the pointer size, and the address pointed to by D as a pointer is included in the video memory allocation A, D is recorded as a potential video memory pointer, and step 406 is executed.
[0057] In step 406 , because A is the video memory allocation corresponding to the potential pointer D, the range of A is added to the kernel's prediction of modifying the video memory range, and step 407 is executed.
[0058] In step 407, C is executed instead of B, and the memory modification range prediction A in step 406 is passed to C as a parameter. The verification result obtained after C is executed is recorded as E.
[0059] In step 408, if E is verified successfully, add the current speculation range A to the access data range, otherwise update the access data range to all allocated video memory ranges A and F.
[0060] Example 3
[0061] A GPU access data tracking system based on kernel parameter analysis and instrumentation verification, including:
[0062] Kernel instrumentation module: instruments the target kernel one by one to generate the instrumented kernel;
[0063] Access data prediction module: analyze the parameters of the kernel to be started, predict the memory range modified by the kernel, and call the kernel after the stub to verify the prediction results;
[0064] Access data update module: updates access data tracking results according to verification results; if verification passes, adds the current prediction range to the access data range; if verification fails, updates the access data to the entire allocated video memory range.
[0065] The instrumented kernel can verify the video memory write address range, and the instrumented object is an intermediate representation of the binary kernel code.
[0066] The access data prediction module includes intercepting GPU calls, and if it is a video memory allocation / release call, updating the video memory allocation record; the video memory allocation information includes the address and size of the video memory allocated to the GPU application.
[0067] The access data prediction module also includes:
[0068] Intercept GPU calls. If it is a kernel startup call,
[0069] Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the parameters of the kernel include the untyped parameter bytes passed to the kernel at runtime;
[0070] Get the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as the prediction of the kernel write video memory range;
[0071] Execute the instrumented version of the kernel to be started, pass the predicted write memory range to the kernel as an additional parameter, and retrieve the predicted verification result after the kernel execution is completed.
[0072] The scanning of the parameters of the kernel to be started includes enumerating a byte sequence of the pointer size in the parameter. If the byte sequence falls within a certain allocated video memory range according to the pointer interpretation, the byte sequence is identified as a potential video memory pointer, and the allocated video memory range is considered as video memory allocation information corresponding to the potential pointer.
[0073] The predicted verification result is a Boolean value, which is the AND of each write address verification result during the kernel execution process; if the kernel verification result is true, it means that the verification is passed.
[0074] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.
[0075] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0076] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. A GPU access data tracking method based on kernel parameter analysis and instrumentation verification, characterized in that: include: Kernel instrumentation step: instrument the target kernel one by one to generate the instrumented kernel; Access data prediction step: Analyze the parameters of the kernel to be started, predict the memory range modified by the kernel, and call the kernel after the stub to verify the prediction result; Access data update step: update the access data tracking result according to the verification result; if the verification passes, add the current prediction range to the access data range; if the verification fails, update the access data to the entire allocated video memory range.
2. The GPU access data tracking method based on kernel parameter analysis and plug-in verification according to claim 1, characterized in that: The instrumented kernel can verify the video memory write address range, and the instrumented object is an intermediate representation of the binary kernel code.
3. The GPU access data tracking method based on kernel parameter analysis and plug-in verification according to claim 2, characterized in that: The access data prediction step includes intercepting GPU calls, and if it is a video memory allocation / release call, updating the video memory allocation record; the video memory allocation information includes the address and size of the video memory allocated to the GPU application.
4. The GPU access data tracking method based on kernel parameter analysis and plug-in verification according to claim 3 is characterized in that: The access data prediction step further includes: Intercept GPU calls. If it is a kernel startup call, Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the parameters of the kernel include the untyped parameter bytes passed to the kernel at runtime; Get the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as the prediction of the kernel write video memory range; Execute the instrumented version of the kernel to be started, pass the predicted write memory range to the kernel as an additional parameter, and retrieve the predicted verification result after the kernel execution is completed.
5. The GPU access data tracking method based on kernel parameter analysis and plug-in verification according to claim 4 is characterized in that: The scanning of the parameters of the kernel to be started includes enumerating a byte sequence of the pointer size in the parameter. If the byte sequence falls within a certain allocated video memory range according to the pointer interpretation, the byte sequence is identified as a potential video memory pointer, and the allocated video memory range is considered as video memory allocation information corresponding to the potential pointer.
6. The GPU access data tracking method based on kernel parameter analysis and plug-in verification according to claim 4 is characterized in that: The predicted verification result is a Boolean value, which is the AND of each write address verification result during the kernel execution process; if the kernel verification result is true, it means that the verification is passed.
7. A GPU access data tracking system based on kernel parameter analysis and instrumentation verification, characterized in that: include: Kernel instrumentation module: instruments the target kernel one by one to generate the instrumented kernel; Access data prediction module: analyze the parameters of the kernel to be started, predict the memory range modified by the kernel, and call the kernel after the instrumentation to verify the prediction results; Access data update module: updates access data tracking results according to verification results; if verification passes, adds the current prediction range to the access data range; if verification fails, updates the access data to the entire allocated video memory range.
8. The GPU access data tracking system based on kernel parameter analysis and instrumentation verification according to claim 7, characterized in that: The instrumented kernel can verify the video memory write address range, and the instrumented object is an intermediate representation of the binary kernel code.
9. The GPU access data tracking system based on kernel parameter analysis and instrumentation verification according to claim 8, characterized in that: The access data prediction module includes intercepting GPU calls, and if it is a video memory allocation / release call, updating the video memory allocation record; the video memory allocation information includes the address and size of the video memory allocated to the GPU application.
10. The GPU access data tracking system based on kernel parameter analysis and instrumentation verification according to claim 9, characterized in that: The access data prediction module also includes: Intercept GPU calls. If it is a kernel startup call, Scan the parameters of the kernel to be started, and extract the potential video memory pointer in the parameters according to the current video memory allocation information; the parameters of the kernel include the untyped parameter bytes passed to the kernel at runtime; Get the video memory allocation information corresponding to the potential video memory pointer, and use the summarized allocation information as the prediction of the kernel write video memory range; Execute the instrumented version of the kernel to be started, pass the predicted write memory range to the kernel as an additional parameter, and retrieve the predicted verification result after the kernel execution is completed.