Anti-fraud evaluation method and storage medium for automatic generation of GPU kernel function
By employing a dual-process architecture and an opaque evaluation runner in GPU kernel function evaluation, the problem of existing frameworks being easily tampered with is solved, resulting in more accurate and efficient evaluation results. It is applicable to the evaluation of various GPU kernel function generation systems and deep learning frameworks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN UNIV
- Filing Date
- 2026-03-30
- Publication Date
- 2026-07-21
AI Technical Summary
Existing GPU kernel function evaluation frameworks are easily tampered with, leading to inaccurate evaluation results. They suffer from problems such as runtime tampering, stream injection, reference implementation pollution, cross-call state exploitation, and GPU memory data theft, which affect the reliability of automatic GPU kernel function generation.
A dual-process architecture is adopted, placing the reference implementation in the evaluation process and the kernel function to be evaluated in an independent execution subprocess. Through the process isolation mechanism of the operating system and an opaque evaluation runner, critical function references are saved, and post-load checks and double-layer timing verification are performed to prevent tampering and state accumulation.
It effectively prevents the evaluation results from being manipulated, improves the accuracy and efficiency of the evaluation, supports multiple GPU kernel function writing methods, and is suitable for the evaluation of GPU kernel function generation systems driven by large language models and deep learning frameworks.
Smart Images
Figure CN122431744A_ABST
Abstract
Description
Technical Field
[0001] This invention mainly relates to the intersection of GPU kernel function automatic generation and high-performance computing evaluation, specifically referring to an anti-spoofing evaluation method and storage medium for GPU kernel function automatic generation. Background Technology
[0002] With the rapid development of deep learning and high-performance computing, the demand for writing GPU kernel functions is increasing. GPU kernel functions are computational programs that run on graphics processing units (GPUs) and are used to perform high-performance computing tasks such as matrix multiplication, attention mechanisms, and convolution. The performance of GPU kernel functions directly affects the training and inference efficiency of deep learning models.
[0003] Due to the frequent updates in GPU hardware architecture (e.g., NVIDIA's Ampere, Hopper, and Blackwell architectures iterate every 1-2 years), the diversity of hardware platforms (covering multiple generations of products from various manufacturers such as NVIDIA, AMD, and Intel), and the fact that different hardware architectures often require different algorithm designs and optimization strategies (rather than simple code recompilation), manually writing high-performance GPU kernel functions faces the challenge of combinatorial explosion. Therefore, automatically generating GPU kernel functions using Large Language Models (LLM) and Reinforcement Learning (RL) techniques has become an important research direction in this field.
[0004] Systems that automatically generate GPU kernel functions (hereinafter referred to as agents) require reliable evaluation frameworks to assess the correctness and performance of the generated kernel functions. Evaluation frameworks typically include two core functions: correctness verification and performance measurement. Correctness verification involves comparing the output of the kernel function under test with the output of a known correct reference kernel to determine if they are consistent within the allowable numerical error range. Performance measurement involves measuring the execution time of the kernel function under test using GPU timing events, then comparing it with the execution time of the reference implementation to calculate the speedup, which is the ratio of the reference implementation's execution time to the kernel function's execution time.
[0005] GPU timing events are a timing mechanism provided by the GPU programming interface (such as CUDA events in the NVIDIA CUDA platform and HIP events in the AMD ROCm platform). By recording start and end events on the GPU command stream (such as CUDAstream in the CUDA platform), the execution time of all GPU operations between two events can be precisely measured. The timing accuracy of GPU timing events is in the microsecond range and is not affected by CPU-side scheduling latency.
[0006] Currently, most existing GPU kernel function evaluation frameworks adopt a single-process architecture, meaning that the code of the kernel function to be evaluated and the evaluation code execute in the same Python process. Under this architecture, the code to be evaluated and the evaluation code share the following resources: (1) Python runtime environment: including global namespace, module system, built-in functions, etc. Due to the dynamic nature of the Python language, any code running in the same process can modify any object at runtime, including replacing functions, modifying class definitions, and tampering with module references.
[0007] (2) GPU device context: The GPU device context (e.g., the CUDA context in the CUDA platform) is a set of states maintained by the GPU driver, including allocated GPU memory, loaded kernel functions, GPU command streams, etc. In the same process, all code shares the same GPU device context, so they can access data in the GPU memory space from each other.
[0008] (3) Global configuration status: This includes floating-point precision configuration (such as the enabling status of TF32 mode), compiler optimization level, etc. Once these configurations are modified, they will affect all subsequent calculation operations in the same process.
[0009] In scenarios where agents are trained using reinforcement learning, the performance metrics provided by the evaluation framework serve as reward signals. Agents have an incentive to exploit vulnerabilities in the evaluation framework to obtain false high-performance metrics; this problem is known as reward hacking. Similarly, in scenarios where agents participate in kernel function writing competitions as autonomous agents, there is an incentive to exploit evaluation vulnerabilities to obtain high rankings.
[0010] The single-process architecture of existing evaluation frameworks allows agents to manipulate the evaluation process through various means. Among these, the most threatening attack method is runtime tampering that exploits the dynamic nature of Python.
[0011] In a single-process architecture, the code to be evaluated and the evaluation code run in the same Python interpreter. Python is a highly dynamic language that allows runtime modification of the attributes and behavior of arbitrary objects, including replacing function definitions, modifying class methods, and redirecting module references. The code to be evaluated can exploit this feature to tamper with critical functions on which the evaluation framework depends at load time, thereby bypassing both correctness verification and performance measurement.
[0012] Specifically, runtime tampering attacks can target three core aspects of the evaluation framework: (1) Tamper with the correctness comparison function. The code to be evaluated replaces the numerical comparison function used by the evaluation framework to compare the output with a function that always returns "pass", so that any output can pass the correctness verification, regardless of whether the actual calculation result is correct.
[0013] (2) Tampering with GPU timing events. The code to be evaluated replaces the GPU event class used for timing by the evaluation framework with a fake subclass. The timing method of this subclass always returns an extremely small time value (e.g., 0.001 milliseconds), which seriously distorts the performance measurement results.
[0014] (3) Tampering with GPU synchronization functions. The code to be evaluated replaces the device-wide synchronization function used by the evaluation framework to wait for the GPU to complete the computation with a no-operation function, so that the evaluation framework records the end event before the GPU computation is completed, resulting in the timing result being much shorter than the actual execution time.
[0015] Runtime tampering is particularly dangerous because Python's module import mechanism executes all top-level code in the imported module upon import. This means that the code under test can undergo such tampering before the evaluation framework even starts working—its critical functions have already been replaced before the evaluation framework begins functioning. More importantly, runtime tampering forms the basis for many other attack methods: for example, the code under test can use tampering with synchronized functions to facilitate stream injection attacks, or it can use tampering with module references to achieve reference implementation pollution.
[0016] Besides runtime tampering, single-process architectures also have the following attack surfaces: (1) Stream injection: The kernel function to be evaluated starts computation on a non-default GPU command stream that it creates and returns immediately, so that the timing events recorded on the default command stream cannot capture the actual execution time.
[0017] (2) Reference implementation pollution: The code to be evaluated modifies the global calculation configuration (such as enabling low-precision calculation mode) or directly replaces the function reference of the reference implementation, so as to reduce the output precision of the reference implementation or slow down the execution speed, thereby making the speedup ratio artificially high.
[0018] (3) Cross-call state utilization: The kernel function under evaluation maintains its state between multiple calls, caches output, or merges multiple independent computation tasks into a single kernel function launch to reduce the reported single execution time. In the NVFP4 competition organized by GPU Mode, an agent merged 15 independent tasks into a single kernel function launch, reducing the reported single execution time to about 1 / 15 of the actual time.
[0019] (4) GPU memory data theft: In a single-process architecture, the reference implementation and the kernel function to be evaluated share the same GPU device context. The kernel function to be evaluated can directly reuse the output data written by the reference implementation and pass the correctness verification without performing any calculations.
[0020] The aforementioned attack methods have been observed in real-world GPU kernel function automatic generation projects and competitions, and new attack methods are constantly being discovered as agent capabilities improve (especially those trained through reinforcement learning). Therefore, existing GPU kernel function evaluation frameworks based on single-process architectures suffer from the serious problem of easily manipulated evaluation results, hindering the reliable evaluation and healthy development of GPU kernel function automatic generation technology.
[0021] In summary, the existing technical solutions still have the following shortcomings: 1. Runtime tampering: Utilizing Python's dynamic features to replace comparison functions, timing functions, and synchronization functions, bypassing verification and timing; 2. Stream Injection: Computation is performed on a non-default GPU stream, and the default stream timer cannot capture the actual execution time. 3. Reference implementation pollution: Modify global configuration or replace reference functions to artificially increase speed ratio; 4. Cross-call state utilization: caching output and merging tasks to reduce report execution time; 5. GPU memory data theft: Directly reuse the output of the reference implementation; verification is passed without calculation. Summary of the Invention
[0022] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a simple, easy-to-implement, widely applicable, and efficient anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions.
[0023] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A method for anti-spoofing evaluation based on automatic generation of GPU kernel functions, comprising: Step S1: Initialize the evaluation process; load the trusted reference kernel function and store it in the evaluation process memory; read the test parameter set T; Step S2: Traverse the test parameters and start independent subprocesses; that is, take the unevaluated parameter t and create a new independent execution subprocess; the subprocess runs the opaque evaluation runner; Step S3: Subprocess initialization; pre-save key function references, synchronization, timing, numerical comparison, thread counting; load the kernel function code to be evaluated, and check it after loading; compare the current function with the pre-saved reference, and if they are inconsistent, it is determined to be tampered with, and the evaluation is terminated; Step S4: Correctness verification; The evaluation process generates K sets of new random inputs and sends them through inter-process communication; The evaluation process calculates the expected results using the reference implementation, performs strict numerical comparison, and terminates if it fails; Step S5: Performance measurement, used to prevent caching and fraud; Step S6: Cleanup and Looping; Kill the current child process to completely destroy all states and avoid cross-parameter pollution; Return to step S2 to process the next test parameter until all are completed; Step S7: Output the results; output the correctness of all parameters, execution time, and speedup ratio.
[0024] As a further improvement of the present invention: step S1 includes: The evaluation process loads the kernel function of the reference implementation and stores its reference in the evaluation process's memory space; The evaluation process reads the test parameter set T. Each element t in the test parameter set T represents a set of test parameters, including the shape and data type of the input tensor.
[0025] As a further improvement of the present invention: in step S2, it is determined whether there is an unevaluated test parameter in the test parameter set T; if there is, let the current test parameter be the first unevaluated test parameter t in T, and go to step S3; if there is no unevaluated, go to step S7.
[0026] As a further improvement of the present invention: In step S2, the evaluation process starts a new execution subprocess for the current test parameter t; the evaluation process creates the execution subprocess through the process creation mechanism of the operating system to ensure that the subprocess has an independent Python runtime environment and an independent GPU device context.
[0027] As a further improvement of the present invention: in step S3, the opaque evaluation runner performs initialization operations in the execution subprocess, including: Step S301: Save references to key evaluation functions; Before loading the code to be evaluated, save the current references of the following functions to local variables respectively: GPU full device synchronization function, GPU timing event constructor, numerical comparison function, and current active thread count function; Step S302: Load the code to be evaluated; The kernel function to be evaluated is loaded from the code to be evaluated using Python's module import mechanism; Step S303: Perform post-load checks; After step S302, the current GPU full-device synchronization function, GPU timing event constructor, and numerical comparison function are compared with the references saved in step S301. If the current reference of any function is inconsistent with the saved reference, it indicates that the code under test has tampered with the key evaluation function during the import process. The evaluation is terminated and a runtime tampering exception is reported. At the same time, the current active thread count is compared with the thread count saved in step S301. If the thread count increases, it indicates that the code under test has started a background thread during the import process. This exception information is recorded.
[0028] As a further improvement of the present invention: step S5 includes: Step S501: Dynamically determine the number of iterations M and the number of warm-up iterations W, and generate M sets of new random inputs; Step S502: Subprocess execution; Receive new input, clear GPU L2 cache; Create GPU timing event using pre-saved function, call kernel function → full device synchronization → record end event → calculate time consumption; Random probability correctness check; Step S503: Return the time consumption and output, and perform double-layer cross-validation in the evaluation process; Step S504: Take the median as the representative execution time and calculate the speedup ratio.
[0029] As a further improvement of the present invention: In step S4, for each set of correctness verification test inputs, the opaque evaluation runner in the subprocess performs the following operations: Step S4001: Receive a set of test inputs from the evaluation process; Step S4002: Call the kernel function to be evaluated, pass in the test input, and obtain the output tensor of the kernel function to be evaluated; Step S4003: Perform a full device synchronization operation using the saved GPU full device synchronization function reference; Step S4004: Verify that the output of the kernel function to be evaluated meets the following conditions: the output type is a standard tensor type and not a subclass of it; the size of the output storage space is greater than zero; the output data pointer is non-zero; the above verification is used to defend against lazy evaluation attacks, that is, the kernel function to be evaluated returns a fake tensor object that has not actually allocated storage space or has not actually been calculated. Step S4005: Send the output tensor of the kernel function to be evaluated to the evaluation process through an inter-process communication mechanism; Step S4006: The evaluation process calls the kernel function of the reference implementation within the evaluation process, passes in the same test input as in step S4001, and calculates the expected output; if the correctness verification of any of the K sets of test inputs fails, the evaluation is terminated and a correctness verification failure is reported.
[0030] As a further improvement of the present invention: In step S5, the evaluation process generates test inputs for M iterations for performance measurement; the test input for each iteration is a data tensor newly allocated on the GPU and generated using a random number generator according to the shape and data type specified by the current test parameter t; the value of M is dynamically determined based on the estimated single execution time.
[0031] As a further improvement of the present invention: In step S5, for each iteration of performance measurement, the opaque evaluation runner in the subprocess performs the following operations: Receive a set of test inputs from the evaluation process; clear the GPU's on-chip cache; The start and end events are created using the saved GPU timing event constructor reference, both of which enable timing functionality; the start event is recorded on the default GPU command stream; Call the kernel function to be evaluated, pass in the received test input, and obtain the output tensor; use the saved GPU full-device synchronization function reference to perform a full-device synchronization operation; record the end event on the default GPU command stream; perform the full-device synchronization operation again to ensure that the end event has been processed; The time difference between the start and end events is calculated using the timing functions of the start and end events, and is used as the execution time for this iteration; the correctness of the output of this iteration is checked with a random probability p. The execution time and output tensor of this iteration are sent to the evaluation process via inter-process communication. The evaluation process uses the same method as the above steps to measure the execution time of the reference implementation under the current test parameter t and calculates the speedup. The evaluation process kills the currently executing child process.
[0032] The present invention also provides a storage medium that can be read by a computer or processor, wherein the storage medium stores a computer program for executing any of the above methods.
[0033] Compared with the prior art, the advantages of the present invention are as follows: 1. The present invention provides an anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions. It is simple in principle, easy to implement, widely applicable, and can improve evaluation efficiency and accuracy. The present invention addresses the problem that the evaluation results are easily manipulated by the code under test due to the single-process architecture of the existing GPU kernel function evaluation framework. The method of the present invention can effectively prevent the code under test from obtaining false performance indicators through stream injection, runtime tampering, reference implementation pollution, output caching, cross-call batch processing, data theft, etc., without restricting the programming language and libraries used by the code under test.
[0034] 2. The anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions of the present invention, through the dual-process isolation architecture of the evaluation process and the execution subprocess, utilizes the process isolation mechanism of the operating system and the non-transferable cross-process characteristics of the GPU device context, fundamentally blocking attack methods that require shared process space, such as runtime tampering, reference implementation pollution, input memory theft and module import side effects.
[0035] 3. The anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions of the present invention provides an additional defense layer against stream injection and delayed evaluation attacks within the execution subprocess through the pre-saved reference mechanism and post-load check mechanism of the opaque evaluation runner.
[0036] 4. The anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions of the present invention eliminates the ability of the code to be evaluated to accumulate states between different test questions by starting a new execution subprocess for each set of test parameters, thus blocking cross-call batch attacks.
[0037] 5. This invention provides an anti-spoofing evaluation method and storage medium for automatically generated GPU kernel functions. By using regenerated random input data for each call, this invention prevents the code under evaluation from buffering outputs based on input pointers, input content, or input shapes. Through a two-layer cross-validation mechanism using GPU timing events and wall-clock timing, anomalies can still be detected even if the main timing method is tampered with. This invention does not limit the programming language or libraries used in the code under evaluation, supports various GPU kernel function writing methods and frameworks, and maintains submission flexibility while providing secure evaluation.
[0038] 6. The anti-spoofing evaluation method and storage medium for automatic generation of GPU kernel functions of the present invention can be applied to the training and evaluation of GPU kernel function automatic generation systems driven by large language models / reinforcement learning, automatic optimization and verification of operators of deep learning frameworks (such as PyTorch and TensorFlow), industrial-grade acceptance evaluation of automatically generated kernel functions in high-performance computing scenarios, and even kernel function performance competitions of multi-architecture GPUs such as NVIDIA, AMD, and Intel. Attached Figure Description
[0039] Figure 1 This is a flowchart illustrating the method of the present invention.
[0040] Figure 2 This is a detailed flowchart of the present invention in a specific embodiment. Detailed Implementation
[0041] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0042] This invention discloses an anti-spoofing evaluation method for automatic generation of GPU kernel functions. It employs a dual-process architecture that separates the evaluation process from the execution subprocess. The core idea is to run the reference implementation within the evaluation process and the kernel function to be evaluated within a separate execution subprocess, using the operating system's process isolation mechanism to block runtime tampering and reference implementation pollution attacks. An opaque evaluation runner is set up in the execution subprocess to save references to key evaluation functions before loading the code to be evaluated, using these saved references to perform synchronization and timing operations to defend against stream injection and delayed evaluation attacks. A new execution subprocess is started for each set of test parameters to eliminate state accumulation across test problems, and regenerated random input data is used in each call to the same test parameter to prevent output caching and cross-call batching. A two-layer cross-validation method using GPU timing events and external wall-clock timing is employed to improve the reliability of timing results.
[0043] like Figure 1 As shown, the present invention provides an anti-spoofing evaluation method for automatic generation of GPU kernel functions, comprising: Step S1: Initialize the evaluation process; load the trusted reference kernel function and save it in the evaluation process memory; read the test parameter set T (such as input shape and data type); Step S2: Traverse the test parameters and start independent subprocesses; that is, take the unevaluated parameter t and create a new independent execution subprocess (independent Python environment + independent GPU context); the subprocess runs the opaque evaluation runner; Step S3: Subprocess initialization; pre-save key function references, synchronization, timing, numerical comparison, thread counting; load the kernel function code to be evaluated, and check it after loading; compare the current function with the pre-saved reference, and if they are inconsistent, it is determined to be tampered with, and the evaluation is terminated; Step S4: Correctness Verification; The evaluation process generates K sets of new random inputs (K≥3) and sends them via inter-process communication. The child process calls the kernel function → synchronizes all devices → verifies the validity of the output → returns the result; The evaluation process calculates the expected result using the reference implementation, performs strict numerical comparison, and terminates if it fails. Step S5: Performance measurement, used to prevent caching and fraud; Step S6: Cleanup and Looping; Kill the current child process to completely destroy all states and avoid cross-parameter pollution; Return to step S2 to process the next test parameter until all are completed; Step S7: Output the results; output the correctness of all parameters, execution time, and speedup ratio.
[0044] The evaluation process is a trusted process responsible for loading the reference implementation, generating test input, initiating the execution of subprocesses, verifying correctness, and collecting performance measurement results. In a specific application example, step S1 includes: The evaluation process loads the kernel function of the reference implementation and stores its reference in the evaluation process's memory space; The evaluation process reads the test parameter set T. Each element t in the test parameter set T represents a set of test parameters, including the shape and data type of the input tensor.
[0045] In a specific application example, step S2 determines whether there is an unevaluated test parameter in the test parameter set T; if it exists, the current test parameter is set to the first unevaluated test parameter t in T, and the process proceeds to step S3; if it does not exist, the process proceeds to step S7.
[0046] In a specific application example, in step S2, the evaluation process starts a new execution subprocess for the current test parameter t. The evaluation process creates this execution subprocess using the operating system's process creation mechanism (such as fork-exec or spawn), ensuring that the subprocess has an independent Python runtime environment and an independent GPU device context. The independent Python runtime environment means that the Python global namespace, module system, and built-in functions in the execution subprocess are completely isolated from the evaluation process; any modifications to the Python runtime in the execution subprocess will not affect the evaluation process. The independent GPU device context means that the GPU memory allocation of the execution subprocess and the GPU memory allocation of the evaluation process reside in different virtual address spaces; the code in the execution subprocess cannot access the GPU memory data of the evaluation process. The execution subprocess runs an opaque evaluation runner. This opaque evaluation runner is a pre-written trusted script whose source code is not visible to the code to be evaluated.
[0047] In a specific application example, in step S3, the opaque evaluation runner performs the following initialization operations in the execution subprocess, and the specific process may include: Step S301: Save references to key evaluation functions; Before loading the code to be evaluated, the current references to the following functions are saved to local variables: the GPU full-device synchronization function, the GPU timing event constructor, the numerical comparison function, and the current active thread count function. The names of these local variables are determined by the implementation of the opaque evaluation runner and are not visible to the code to be evaluated.
[0048] Step S302: Load the code to be evaluated; The GPU kernel function to be evaluated is loaded from the code to be evaluated using Python's module import mechanism. The code to be evaluated can use any programming language and library to write GPU kernel functions and can contain any Python code. During the import process, the code to be evaluated may execute arbitrary module-level code, including modifying Python global state and replacing standard library functions.
[0049] Step S303: Perform post-load checks; After step S302, the current GPU full-device synchronization function, GPU timing event constructor, and numerical comparison function are compared with the references saved in step S301. If the current reference of any function is inconsistent with the saved reference, it indicates that the code under evaluation has tampered with the critical evaluation function during the import process. The evaluation is terminated and a runtime tampering exception is reported. Simultaneously, the current active thread count is compared with the thread count saved in step S301. If the thread count increases, it indicates that the code under evaluation started a background thread during the import process; this exception information is recorded.
[0050] In a specific application example, in step S4, the evaluation process generates K sets of test inputs for correctness verification. Each set of test inputs is a newly allocated data tensor on the GPU, generated using a random number generator, according to the shape and data type specified by the current test parameter t. The value of K should be no less than 3 to ensure the coverage of correctness verification. The evaluation process sends the K sets of test inputs sequentially to the executing subprocess through inter-process communication mechanisms (such as shared memory or pipes).
[0051] Specifically, step S5 may include the following process: Step S501: Dynamically determine the number of iterations M and the number of warm-up iterations W, and generate M sets of new random inputs; Step S502: Subprocess execution; Receive new input, clear GPU L2 cache; Create GPU timing event using pre-saved function, call kernel function → full device synchronization → record end event → calculate time consumption; Random probability correctness check; Step S503: Return the time and output, and perform double-layer cross-validation in the evaluation process (GPU timing + external wall clock timing). Step S504: Take the median as the representative execution time and calculate the speedup ratio.
[0052] In a specific application example, in step S4, for each set of correctness verification test inputs, the opaque evaluation runner in the subprocess performs the following operations: Step S401: Receive a set of test inputs from the evaluation process.
[0053] Step S402: Call the kernel function to be evaluated, pass in the test input, and obtain the output tensor of the kernel function to be evaluated.
[0054] Step S403: Perform a full device synchronization operation using the GPU full device synchronization function reference saved in step S3. The full device synchronization operation waits for all computational operations on all GPU command streams in the current GPU device context to complete, including operations on non-default GPU command streams that the code under evaluation may create. This ensures that even if the kernel function under evaluation uses a stream injection strategy, computations initiated on non-default streams must be completed before execution continues.
[0055] Step S404: Verify that the output of the kernel function to be evaluated meets the following conditions: the output type is a standard tensor type and not a subclass of it; the size of the output storage space is greater than zero; the output data pointer is non-zero. The above verification is used to defend against lazy evaluation attacks, that is, the kernel function to be evaluated returns a spurious tensor object that has not actually allocated storage space or has not actually been computed.
[0056] Step S405: Send the output tensor of the kernel function to be evaluated to the evaluation process through an inter-process communication mechanism.
[0057] Step S406: The evaluation process calls the reference implementation's kernel function within the evaluation process itself (not in the execution subprocess), passing in the same test input as in step S401, and calculates the expected output. The evaluation process uses a numerical comparison method to compare the output of the kernel function under evaluation with the expected output. The numerical comparison method should set hierarchical tolerance parameters according to the operator type and data precision of the kernel function under evaluation. For linear operators (such as matrix multiplication), a binary input exact matching strategy can be adopted, that is, using an input tensor containing only 0s and 1s to ensure that all intermediate calculation results can be accurately represented within the floating-point representation range, thereby achieving accurate output matching.
[0058] If the correctness verification of any of the K sets of test inputs fails, the evaluation will terminate and a correctness verification failure will be reported.
[0059] In a specific application example, in step S4, the evaluation process further generates test inputs for M iterations of performance measurement. The test input for each iteration is a newly allocated data tensor on the GPU, generated using a random number generator, according to the shape and data type specified by the current test parameter t. The value of M is dynamically determined based on the estimated single execution time. Specifically, a small number (e.g., 5) trial runs are first performed to estimate the single execution time e (in milliseconds), and then M = max(1, int(R / e)), where R is the preset total measurement time (e.g., 100 milliseconds). Before sending the test inputs, the evaluation process performs a warmup: sending W test inputs (W = max(1, int(W_target / e)), where W_target is the preset warmup time, e.g., 25 milliseconds), and the warmup timing results are discarded. After warmup, the evaluation process starts a wall-clock timer and then sends the M iterations of test inputs sequentially to the execution subprocess.
[0060] In a specific application example, in step S4, for each iteration of performance measurement, the opaque evaluation runner in the subprocess performs the following operations: The evaluation process receives a set of test inputs. These test inputs are newly generated random data tensors in the current iteration, whose memory addresses and data contents are different from those of any previous call.
[0061] Clear the GPU's on-chip cache. The method is to write a zero value to a pre-allocated GPU buffer, the size of which is not less than the capacity of the current GPU's last-level on-chip cache. On GPUs with a Level 2 cache (L2 cache), this last-level on-chip cache is the L2 cache. The reason for clearing only the last-level cache and not the Level 1 cache is that the L1 cache is managed independently by each streaming multiprocessor and expires naturally with the workgroup scheduling when the kernel function starts; while the L2 cache is a globally shared cache, its contents are continuously retained between kernel function calls. If it is not actively cleared, subsequent iterations will achieve shorter execution times due to cache hits than a cold start, making the measurement results unreproducible. For GPU architectures without an L2 cache, the buffer size should be set to be not less than the capacity of the architecture's last-level shared cache; if the GPU architecture does not have any shared cache that continuously retains data across kernel function calls, this step can be skipped. The purpose of clearing the on-chip cache is to eliminate the impact of residual cache data from the previous iteration on the execution time of the current iteration, ensuring that the cache state is consistent with each iteration.
[0062] Create a start event and an end event using the GPU timing event constructor reference saved in step S3. Both events enable timing functionality. Record the start event on the default GPU command stream.
[0063] Call the kernel function to be evaluated, pass in the test input received in the previous step, and obtain the output tensor.
[0064] Perform a full device synchronization operation using the GPU full device synchronization function reference saved in step S3. This synchronization operation ensures that all GPU computations, including non-default GPU command streams, are completed before the end-of-life event is recorded.
[0065] Record the end event on the default GPU command stream. Perform a full device synchronization operation again to ensure the end event has been processed.
[0066] The timing function of the start and end events is used to calculate the time difference (in milliseconds) between the two events, which is used as the execution time of this iteration.
[0067] A correctness check is performed on the output of this iteration with a random probability p (e.g., p = 0.1). The purpose of the correctness check is to prevent the kernel function under evaluation from returning incorrect results in the performance measurement phase (the phase without full correctness verification) in exchange for faster execution speed. Since the code under evaluation does not know which iterations will be checked, it must produce correct output in every iteration.
[0068] The execution time and output tensor of this iteration are sent to the evaluation process via inter-process communication.
[0069] After all M performance measurement iterations are completed, the evaluation process stops the wall-clock timer and obtains the total wall-clock time T_wall. The evaluation process calculates the total GPU timing event time T_gpu for the M iterations. The evaluation process compares T_gpu with T_wall: Let T_ipc be the estimated total inter-process communication overhead (e.g., T_ipc = M × c, where c is the average overhead of a single communication). If T_gpu is significantly less than T_wall - T_ipc (e.g., T_gpu < 0.5 × (T_wall - T_ipc)), then the timing result is considered abnormal, the evaluation is terminated, and a timing abnormality is reported. The cross-validation mechanism between the GPU timing events and the wall-clock timing utilizes the following facts: GPU timing events are completed within the execution subprocess and can theoretically be affected by runtime tampering within the subprocess; while wall-clock timing is completed within the evaluation process and is not affected by any code within the execution subprocess. A significant inconsistency between the two timing layers indicates that the GPU timing events within the subprocess have been tampered with.
[0070] If the timing results are normal, the evaluation process takes the median of the execution times of M iterations as the representative execution time of the kernel function under the current test parameter t. The median is chosen instead of the mean because the median is not sensitive to outliers and can more stably reflect the typical execution time.
[0071] The evaluation process uses the same method as the steps described above (starting a separate subprocess, using an opaque evaluation runner, regenerating random input each time, clearing the L2 cache, and timing GPU timing events) to measure the execution time of the reference implementation under the current test parameters t and calculate the speedup.
[0072] The evaluation process kills the currently executing child process. This process of killing the child process ensures that all Python-level state (global variables, cached dictionaries, background threads, etc.) and GPU-level state (allocated GPU memory, loaded kernel functions, etc.) accumulated by the code under evaluation during the current evaluation of test parameter t are completely destroyed and will not leak into the evaluation of the next set of test parameters.
[0073] In a specific application example, step S7 outputs the correctness verification results, performance measurement results, and speedup ratio of the kernel function to be evaluated under all test parameters.
[0074] In a specific application example, see Figure 2 The present invention specifically includes the following detailed process: The evaluation task in this embodiment is an attention mechanism kernel function. The kernel function to be evaluated receives three input tensors: a query matrix Q, a key matrix K, and a value matrix V, and outputs a tensor representing the result of the attention calculation. The reference implementation is the standard scaled dot product attention function provided by the deep learning framework. The hardware environment is an NVIDIA H100 GPU equipped with 80GB of high-bandwidth memory and 50MB of L2 cache.
[0075] The test parameter set T contains 3 sets of test parameters: the first set of parameters t1 has an input shape of batch size 2, number of attention heads 8, sequence length 512, head dimension 64, and data type is half-precision floating-point number; the second set of parameters t2 has an input shape of batch size 4, number of attention heads 16, sequence length 1024, head dimension 128, and data type is half-precision floating-point number; the third set of parameters t3 has an input shape of batch size 1, number of attention heads 32, sequence length 2048, head dimension 64, and data type is half-precision floating-point number.
[0076] The number of input groups for correctness verification is K = 5. The preset warm-up time is 25 milliseconds. The preset total measurement time is R = 100 milliseconds. The probability of correctness sampling is p = 0.1.
[0077] The first step in the evaluation process is to load the scaling dot product attention function of the reference implementation and read the test parameter set T.
[0078] The second step is to determine if there is an untested parameter t1 in T, and then set the current test parameter t = t1.
[0079] Third, the evaluation process launches and executes a child process through the operating system's process forking mechanism. The child process has its own independent Python interpreter instance and independent CUDA context. The child process runs an opaque evaluation runner.
[0080] The fourth step involves the opaque evaluation runner completing its initialization in a subprocess.
[0081] Step 4.1: Before loading the code to be evaluated, the opaque evaluation runner saves the current references of the GPU full device synchronization function, GPU timing event constructor, numerical comparison function, and thread counting function into local variables.
[0082] Step 4.2: The opaque benchmark runner loads the code to be evaluated through a module import mechanism and obtains the kernel functions to be evaluated. All top-level code in the code to be evaluated is executed in this step. If the code to be evaluated has modified critical functions such as the GPU timing event constructor in its top-level code, these modifications have taken effect after this step.
[0083] Step 4.3: The opaque evaluation runner compares the current GPU full-device synchronization function, GPU timing event constructor, and numerical comparison function with the references saved in Step 4.1. If the current reference of any function is inconsistent with the saved reference, it indicates that the code to be evaluated has tampered with the critical evaluation function during the import process, the evaluation is terminated, and an exception is reported. Simultaneously, it checks whether the current thread count has increased.
[0084] All checks passed in this embodiment, so execution continues.
[0085] The fifth step involves the evaluation process generating five sets of test inputs. Each set of inputs consists of three tensors: Q, K, and V. Each tensor has a batch size of 2, an attention head count of 8, a sequence length of 512, and a head dimension of 64. The data type is half-precision floating-point numbers, and random data is allocated and filled on the GPU. The evaluation process sends the five sets of inputs to the execution subprocess via shared memory.
[0086] Step 6: Perform correctness verification on each of the 5 input groups. Taking the first input group as an example: Step 6.1: The opaque evaluation runner receives the first set of input tensors Q, K, and V from shared memory.
[0087] Step 6.2: The opaque evaluation runner calls the kernel function to be evaluated, passes in Q, K, and V, and obtains the output tensor.
[0088] Step 6.3: The opaque evaluation runner uses the GPU full device synchronization function saved in step 4.1 to perform full device synchronization, ensuring that computations on all GPU command streams are completed.
[0089] Step 6.4: The opaque evaluation runner verifies that the output tensor is of the standard tensor type and is not a subclass of it, that the storage space size is greater than zero, and that the data pointer is non-zero.
[0090] Step 6.5: The opaque evaluation runner sends the output tensor to the evaluation process via shared memory.
[0091] Step 6.6: Within the evaluation process, the scaled dot product attention function of the reference implementation is called to calculate the expected output. The evaluation process uses a numerical comparison method to compare the output of the kernel function under evaluation with the expected output, with an absolute tolerance of 0.01 and a relative tolerance of 0.01. For attention calculations of half-precision floating-point numbers, this tolerance setting can accommodate normal numerical differences caused by floating-point non-associativity, while being strict enough to detect calculation errors.
[0092] All 5 sets of inputs passed the correctness verification.
[0093] Step 7: The evaluation process estimates the single execution time to be approximately 0.5 milliseconds through 5 trial runs. Based on the formula, the number of warm-up cycles W = 50, and the number of performance measurement iterations M = 200. The evaluation process generates a total of 250 test inputs. After 50 warm-up cycles, an external timer is started, initiating 200 performance measurement iterations.
[0094] Step 8, taking the first performance measurement iteration as an example: Step 8.1: The opaque evaluation runner receives the newly generated test input tensors Q, K, and V. This input is newly allocated random data for this iteration, and its memory address and data content are different from all previous input calls.
[0095] Step 8.2: The opaque benchmark runner writes a zero value to the pre-allocated 50MB GPU buffer to clear the GPU's L2 cache.
[0096] Step 8.3: The opaque benchmark runner uses the GPU timing event constructor saved in Step 4.1 to create a start event and an end event, both of which enable timing. The start event is recorded on the default GPU command stream.
[0097] Step 8.4: The opaque evaluation runner calls the kernel function to be evaluated, passes in the test input, and obtains the output tensor.
[0098] Step 8.5: The opaque evaluation runner uses the GPU full device synchronization function saved in step 4.1 to perform full device synchronization.
[0099] Step 8.6: Record the end event on the default GPU command stream and perform full device synchronization again.
[0100] Step 8.7: Calculate the time difference between the start event and the end event. The execution time of this iteration is 0.487 milliseconds.
[0101] Step 8.8: Generate a random number between 0 and 1. In this case, it is 0.73, which is greater than the sampling probability of 0.1. Therefore, the correctness sampling is not performed.
[0102] Step 8.9: Send 0.487 milliseconds to the evaluation process.
[0103] After 200 iterations, the evaluation process stopped the external timer. The total time for the 200 GPU timing events, T_gpu = 98.2 milliseconds, and the total time for the external timer, T_wall = 105.7 milliseconds. The estimated total inter-process communication overhead, T_ipc = 200 × 0.02 milliseconds = 4.0 milliseconds. T_gpu = 98.2 milliseconds is greater than 0.5 × (105.7 - 4.0) = 50.85 milliseconds, so the timing result is normal.
[0104] The median execution time of 200 iterations, 0.491 milliseconds, was taken as the representative execution time of the kernel function under test at time t1. The evaluation process used the same method to measure the execution time of the reference implementation at time t1, with a median of 0.623 milliseconds. Speedup = 0.623 / 0.491 = 1.27 times.
[0105] The evaluation process kills the current child process, proceeds to step 2 to process t2, starts a new child process, and repeats the above process. After all three sets of test parameters have been evaluated, proceed to step 9 to output the correctness verification results, performance measurement results, and speedup ratio for all test parameters.
[0106] As can be seen from the above, the most crucial feature of the method of the present invention is that it adopts a dual-process isolation architecture, uses a defense method of "opaque runner + pre-saved reference + post-load check", creates a new sub-process for each set of parameters with each new random input, and completely clears the state; it uses GPU timing + wall clock timing for dual-layer cross-verification, and actively clears the L2 cache.
[0107] By employing the methods described above in this invention, shared environment attacks can be eradicated, process isolation completely blocks runtime tampering, data theft, and reference implementation pollution; it can defend against stream injection and delayed evaluation, with full device synchronization and output validity verification to capture all stream computations; it can eliminate cross-call cheating by creating new child processes and entirely new random inputs, preventing caching and merging of tasks; it can prevent timing tampering through double-layer verification, and median statistics resist outliers, making timing results reliable; it can achieve high compatibility, i.e., it does not restrict programming languages and libraries and supports multi-vendor GPU architectures; it can ensure reproducible results, with cache clearing and state zeroing, guaranteeing stable and repeatable evaluation.
[0108] The present invention also provides a storage medium that can be read by a computer or processor, wherein the storage medium stores a computer program for executing any of the above methods.
[0109] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0110] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.
Claims
1. A method for anti-spoofing evaluation based on automatic generation of GPU kernel functions, characterized in that, include: Step S1: Initialize the evaluation process; Load the trusted reference kernel function and store it in the evaluation process memory; read the test parameter set T; Step S2: Traverse the test parameters and start independent subprocesses; that is, take the unevaluated parameter t and create a new independent execution subprocess; the subprocess runs the opaque evaluation runner; Step S3: Subprocess initialization; pre-save key function references, synchronization, timing, numerical comparison, thread counting; load the kernel function code to be evaluated, and check it after loading; compare the current function with the pre-saved reference, and if they are inconsistent, it is determined to be tampered with, and the evaluation is terminated; Step S4: Correctness verification; The evaluation process generates K sets of new random inputs and sends them through inter-process communication; The evaluation process calculates the expected results using the reference implementation, performs strict numerical comparison, and terminates if it fails; Step S5: Performance measurement, used to prevent caching and fraud; Step S6: Cleanup and Looping; Kill the current child process to completely destroy all state and avoid cross-parameter pollution; Back Step S2: Process the next test parameter until all are completed; Step S7: Output the results; output the correctness of all parameters, execution time, and speedup ratio.
2. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to claim 1, characterized in that, Step S1 includes: The evaluation process loads the kernel function of the reference implementation and stores its reference in the evaluation process's memory space; The evaluation process reads the test parameter set T. Each element t in the test parameter set T represents a set of test parameters, including the shape and data type of the input tensor.
3. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to claim 1, characterized in that, In step S2, it is determined whether there are any unevaluated test parameters in the test parameter set T; If it exists, let the current test parameter be the first unevaluated test parameter t in T, and go to step S3; If it does not exist, proceed to step S7.
4. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to claim 1, characterized in that, In step S2, the evaluation process starts a new execution subprocess for the current test parameter t. The evaluation process creates the execution subprocess through the operating system's process creation mechanism to ensure that the subprocess has an independent Python runtime environment and an independent GPU device context.
5. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to any one of claims 1-4, characterized in that, In step S3, the opaque evaluation runner performs initialization operations in the execution subprocess, including: Step S301: Save references to key evaluation functions; Before loading the code to be evaluated, save the current references of the following functions to local variables respectively: GPU full device synchronization function, GPU timing event constructor, numerical comparison function, and current active thread count function; Step S302: Load the code to be evaluated; The kernel function to be evaluated is loaded from the code to be evaluated using Python's module import mechanism; Step S303: Perform post-load checks; After step S302, the current GPU full-device synchronization function, GPU timing event constructor, and numerical comparison function are compared with the references saved in step S301. If the current reference of any function is inconsistent with the saved reference, it indicates that the code under test has tampered with the key evaluation function during the import process. The evaluation is terminated and a runtime tampering exception is reported. At the same time, the current active thread count is compared with the thread count saved in step S301. If the thread count increases, it indicates that the code under test has started a background thread during the import process. This exception information is recorded.
6. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to any one of claims 1-4, characterized in that, Step S5 includes: Step S501: Dynamically determine the number of iterations M and the number of warm-up iterations W, and generate M sets of new random inputs; Step S502: Subprocess execution; Receive new input, clear GPU L2 cache; Create GPU timing event using pre-saved function, call kernel function → full device synchronization → record end event → calculate time consumption; Random probability correctness check; Step S503: Return the time consumption and output, and perform double-layer cross-validation in the evaluation process; Step S504: Take the median as the representative execution time and calculate the speedup ratio.
7. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to any one of claims 1-4, characterized in that, In step S4, for each set of correctness verification test inputs, the opaque evaluation runner in the subprocess performs the following operations: Step S4001: Receive a set of test inputs from the evaluation process; Step S4002: Call the kernel function to be evaluated, pass in the test input, and obtain the output tensor of the kernel function to be evaluated; Step S4003: Perform a full device synchronization operation using the saved GPU full device synchronization function reference; Step S4004: Verify that the output of the kernel function to be evaluated meets the following conditions: the output type is a standard tensor type and not a subclass of it; the size of the output storage space is greater than zero; the output data pointer is non-zero; the above verification is used to defend against lazy evaluation attacks, that is, the kernel function to be evaluated returns a fake tensor object that has not actually allocated storage space or has not actually been calculated. Step S4005: Send the output tensor of the kernel function to be evaluated to the evaluation process through an inter-process communication mechanism; Step S4006: The evaluation process calls the kernel function of the reference implementation within the evaluation process, passes in the same test input as in step S4001, and calculates the expected output; if the correctness verification of any of the K sets of test inputs fails, the evaluation is terminated and a correctness verification failure is reported.
8. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to claim 7, characterized in that, In step S5, the evaluation process generates test inputs for M iterations of performance measurement; The test input for each iteration is a newly allocated data tensor on the GPU, generated using a random number generator, according to the shape and data type specified by the current test parameter t; the value of M is dynamically determined based on the estimated single execution time.
9. The anti-spoofing evaluation method for automatic generation of GPU kernel functions according to claim 7, characterized in that, In step S4, for each iteration of the performance measurement, the opaque evaluation runner in the subprocess performs the following operations: Receive a set of test inputs from the evaluation process; clear the GPU's on-chip cache; The start and end events are created using the saved GPU timing event constructor reference, both of which enable timing functionality; the start event is recorded on the default GPU command stream; Call the kernel function to be evaluated, pass in the received test input, and obtain the output tensor; use the saved GPU full-device synchronization function reference to perform a full-device synchronization operation; record the end event on the default GPU command stream; perform the full-device synchronization operation again to ensure that the end event has been processed; The time difference between the start and end events is calculated using the timing functions of the start and end events, and is used as the execution time for this iteration. Perform a correctness check on the output of this iteration with a random probability p; The execution time and output tensor of this iteration are sent to the evaluation process via inter-process communication. The evaluation process uses the same method as the above steps to measure the execution time of the reference implementation under the current test parameter t and calculates the speedup. The evaluation process kills the currently executing child process.
10. A storage medium capable of being read by a computer or processor, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-9.