Methods, apparatus, computer devices, storage media, and computer program products for automatically generating parallel computing kernels.

Abstract

This application relates to a method, apparatus, computer device, storage medium, and computer program product for automatically generating a parallel computing kernel. The method includes: generating an action sequence based on the current iteration state, the action sequence including code editing operations on the current kernel code and instructions for calling external tools; the current iteration state including the current kernel code, historical action sequences, and their corresponding execution results; executing the action sequence to update the current kernel code based on the code editing operations, and calling external tools to process the updated current kernel code to obtain new execution results; updating the current iteration state based on the new execution results, and iteratively generating the next action sequence based on the updated current iteration state; and outputting the finally generated parallel computing kernel when a preset termination condition is met. This method can meet engineering deployment requirements in terms of reliability, versatility, and practical performance.

Description

Technical Field

[0001] This application relates to the field of software technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for automatically generating parallel computing kernels. Background Technology

[0002] With the development of parallel computing and deep learning compilation technologies in Graphics Processing Units (GPUs), the demand for the development and optimization of parallel computing kernels is increasing. Traditional development methods, relying on manual coding and expert tuning, are inefficient, time-consuming, and costly, making it difficult to meet the iterative needs of diverse operators and complex computing scenarios. Therefore, the industry is gradually adopting methods based on compiler-driven automatic generation, scheduling search, and large model code generation to automate the construction of parallel computing kernels, aiming to reduce human intervention and improve development efficiency and performance.

[0003] However, most existing parallel computing kernel generation methods only use the correctness of the final code or the performance of a single run as optimization objectives. The feedback signals are sparse and there is a lack of effective supervision over the generation process, making it difficult for the model to stably learn the complete debugging logic from compilation and verification to performance tuning. At the same time, in large-scale search spaces such as complex fusion kernels, the generation process is prone to problems such as reward speculation and insufficient evaluation, resulting in generated kernels that fail to meet engineering deployment requirements in terms of reliability, versatility, and practical performance. Summary of the Invention

[0004] Therefore, it is necessary to provide an automatic generation method, apparatus, computer equipment, storage medium, and computer program product for parallel computing kernels that can meet engineering deployment requirements in terms of reliability, versatility, and practical performance, in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for automatically generating parallel computing kernels, including:

[0006] An action sequence is generated based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process.

[0007] The action sequence is executed to update the current kernel code based on the code editing operation, and an external tool is invoked based on the invocation instruction to process the updated current kernel code and obtain a new execution result;

[0008] The current iteration state is updated based on the new execution result, and the next action sequence is generated iteratively based on the updated current iteration state; when the preset termination condition is met, the finally generated parallel computing kernel is output.

[0009] In one embodiment, the method further includes:

[0010] Obtain a high-level computational description for generating the parallel computing kernel, wherein the high-level computational description includes at least one of operator sequence, computation graph, tensor expression or intermediate representation;

[0011] Obtain a set of verification conditions, which includes tensor constraint information, numerical verification conditions, a set of test cases, and performance targets;

[0012] Based on the high-level computation description and the verification condition set, the current iteration state is initialized, so that the current iteration state includes the initial kernel code, the preset historical action sequence and the preset execution result.

[0013] In one embodiment, the test case set includes: a public test case set and a hidden test case set;

[0014] The hidden test case set is invisible to the proxy model during the generation and iteration process, and is used to verify the correctness of the current kernel code or evaluate rewards.

[0015] In one embodiment, the external tool includes at least one of the following types of tools:

[0016] Compiler, static analyzer, unit test runner, numerical verifier, performance profiler, memory error detection tool.

[0017] In one embodiment, updating the current iteration state based on the new execution result includes:

[0018] The new execution results returned by the external tool are subjected to structured parsing to extract key information and form structured status data; wherein, the key information includes at least one of the following: compilation error type and location, test failure case identifier and failure reason, numerical error distribution and statistics, performance index value and fluctuation range, and memory error type and occurrence location;

[0019] The current iteration state is updated based on the structured state data.

[0020] In one embodiment, the action sequence is generated by a preset proxy model, which is a large language model trained using reinforcement learning. The reward function used during the reinforcement learning training process includes a hierarchical verifiable reward; the hierarchical verifiable reward includes at least one of the following:

[0021] The tiered verifiable rewards include at least one of the following:

[0022] Compilation reward, which is a reward quantitatively evaluated based on the current compilation status of the kernel code;

[0023] The correctness reward is a reward calculated based on the numerical error between the execution result of the current kernel code on the test case set and the reference implementation;

[0024] Performance reward, which is a reward determined based on the degree of improvement of the current kernel code's performance metrics relative to a preset performance target;

[0025] Security constraint penalty, which is a negative reward imposed when abnormal behavior is detected in the current kernel code.

[0026] In one embodiment, an anti-speculation mechanism is introduced during the calculation of the correctness reward;

[0027] The anti-speculation mechanism includes at least one of the following:

[0028] Correctness verification is performed using a hidden test case set, which is not visible to the proxy model during training.

[0029] The test input data is subjected to randomization transformation, which includes one or more of the following: tensor shape randomization, numerical distribution randomization, and boundary condition randomization.

[0030] Perform abnormal behavior detection on the current kernel code;

[0031] Additional penalties are imposed when abnormal behavior is detected in the current kernel code.

[0032] In one embodiment, the sequence of actions is performed in an isolated sandbox environment, which includes at least one of the following security controls:

[0033] Restrict file system access permissions during compilation and execution;

[0034] Network communication and external system calls are prohibited;

[0035] Limit GPU resource usage;

[0036] Perform static analysis and runtime monitoring on the compiled executable file to detect potential malicious code behavior or resource abuse;

[0037] Record a complete execution log.

[0038] In one embodiment, the preset termination condition includes at least one of the following:

[0039] The current kernel code has successfully passed all test cases;

[0040] The numerical error between the current kernel code and the reference implementation is within a preset tolerance range;

[0041] The performance metrics of the current kernel code have reached or exceeded the preset performance targets;

[0042] The current kernel code meets the correctness verification conditions;

[0043] No further improvement was achieved after multiple rounds of iteration.

[0044] The number of iterations reaches the preset maximum iteration threshold, or the cumulative execution time exceeds the preset time budget limit.

[0045] Secondly, this application also provides an automatic generation device for parallel computing kernels, comprising:

[0046] The sequence generation module is used to generate an action sequence based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that performs parallel computation during the iteration process.

[0047] The sequence execution module is used to execute the action sequence to update the current kernel code based on the code editing operation, and to call external tools to process the updated current kernel code to obtain new execution results;

[0048] The kernel generation module is used to update the current iteration state based on the new execution result, and to iteratively generate the next action sequence based on the updated current iteration state; when the preset termination condition is met, the final generated parallel computing kernel is output.

[0049] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0050] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0051] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0052] The aforementioned automatic generation method for parallel computing kernels can generate action sequences based on the current iteration state. These action sequences include code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes the current kernel code, historical action sequences, and their corresponding execution results. The current kernel code is the code version of the parallel computing kernel during the iteration process. The action sequences are executed to update the current kernel code based on code editing operations and to call external tools to process the updated kernel code and obtain new execution results. The current iteration state is updated based on the new execution results, and the next action sequence is generated iteratively based on the updated state. When a preset termination condition is met, the final generated parallel computing kernel is output. This scheme can iteratively generate action sequences containing code editing and tool calls based on the current iteration state. During the generation process, the state is continuously updated and iteratively optimized based on the execution results. This transforms the generation of parallel computing kernels from a traditional process relying on manual writing and expert tuning into an automated process that can autonomously iterate and optimize. This significantly improves development efficiency, shortens the iteration cycle, reduces labor costs, and can adapt to diverse operators and complex fusion kernel scenarios, meeting the needs of rapid iteration.

[0053] Furthermore, the above method provides the model with dense and traceable process feedback signals by executing external tools multiple times and continuously obtaining new execution results, thus achieving effective supervision of the entire process. This enables the model to stably learn complete debugging logic, solving the problems of sparse feedback signals, lack of process supervision, and difficulty in learning complete debugging logic in existing methods. In addition, through iterative optimization based on the current iteration state and evaluation by external tools, the kernel code can be fully verified and continuously improved in a complex search space. This effectively alleviates the problems of insufficient kernel reliability, poor versatility, and substandard actual performance caused by reward speculation and insufficient evaluation, making the final generated parallel computing kernel more in line with the actual requirements of engineering deployment. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating an automatic generation method for a parallel computing kernel in one embodiment;

[0056] Figure 2 This is a flowchart illustrating the process of initializing the current iteration state in one embodiment;

[0057] Figure 3 This is a flowchart illustrating the process of updating the current iteration state in one embodiment;

[0058] Figure 4 This is a flowchart illustrating a kernel generation method in one embodiment;

[0059] Figure 5 This is a structural block diagram of an automatic generation device for parallel computing kernels in one embodiment;

[0060] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] This application involves parallel programming of graphics processing units (GPUs), kernel generation and fusion optimization of Compute Unified Device Architecture (CUDA), compilation and performance analysis, and code generation and automated tuning techniques based on reinforcement learning (RL).

[0063] With the rapid development of parallel computing and deep learning compilation technologies for graphics processing units (GPUs), the demand for the development and optimization of parallel computing kernels is increasing. Traditional methods, relying on manual coding and expert tuning, require engineers to manually write CUDA C++ or Parallel Thread Execution (PTX) kernels based on operator semantics and GPU hardware characteristics (thread blocks, shared memory, memory merging, etc.), and then iteratively optimize through repeated compilation, running, and performance profiling tools. In complex fusion kernel scenarios, this approach suffers from high development costs, long development cycles, and low efficiency, making it difficult to adapt to rapidly iterating engineering needs.

[0064] To reduce reliance on manual intervention, the industry is gradually adopting automated methods to build parallel computing kernels. Deep learning frameworks and compilers, such as NVIDIA's nvFuser, Accelerated Linear Algebra (XLA), and Tensor Virtual Machine (TVM), can achieve operator fusion and kernel generation based on computation graphs. However, these methods typically rely on fixed intermediate representations (IRs) and scheduling strategies. In cases involving new operator forms, complex control flows, specific hardware microarchitectures, and atypical fusion scenarios, expert intervention is still required.

[0065] Further, automatic scheduling and automatic tuning techniques have emerged, which generate scheduling strategies and parameter combinations through random search, genetic search, or cost models, and iteratively optimize performance based on actual operation measurements. However, such methods have low search efficiency and high measurement costs in large-scale search spaces, and it is difficult to achieve graph-level collaborative optimization across operators, resulting in a significant bottleneck in overall optimization efficiency.

[0066] In recent years, the industry has begun to use large language models to generate kernels and combine testing and evaluation for iterative improvement. However, the relevant methods generally suffer from problems such as insufficient training data and inadequate evaluation coverage, which can easily lead to rewarding speculation or cheating. Furthermore, the models cannot stably learn the step-by-step debugging and iterative optimization process of human experts, resulting in the generated kernels failing to meet the requirements in terms of reliability, versatility, and actual deployment performance.

[0067] To address the aforementioned issues, this application provides an automatic method for generating a parallel computing kernel. This method can be applied to an automatic generation device for parallel computing kernels or a computer device. The automatic generation device for parallel computing kernels can be a functional module or functional entity within the computer device.

[0068] For example, the aforementioned computer equipment can be a server, desktop computer, laptop computer, embedded development device, or cloud computing node, etc., that possesses data processing and program execution capabilities. The aforementioned automatic generation method for parallel computing kernels can be implemented through software programs, hardware modules, or a combination of software and hardware, and can adapt to application scenarios related to parallel computing of graphics processors, development of deep learning operators, compilation optimization, and automated kernel tuning. The aforementioned automatic generation method can be executed entirely by an automatic generation device for parallel computing kernels or a pre-set proxy model in the computer equipment. This proxy model is a model trained based on reinforcement learning (RL) and can autonomously complete steps such as generating action sequences, parsing execution results, updating the current iteration state, and iterative optimization.

[0069] In one exemplary embodiment, such as Figure 1 As shown, an automatic method for generating parallel computing kernels is provided, which includes the following steps 101 to 104.

[0070] 101. Generate an action sequence based on the current iteration state.

[0071] The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes the current kernel code, the historical action sequence and its corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process.

[0072] The aforementioned current iteration state serves as the basis for the agent model's decision-making. It includes the current kernel code, historical action sequences, and the execution results of external tools corresponding to each historical action sequence, used to record code iteration trajectories and debugging information. The current iteration state is used to record information throughout the entire process of code generation, debugging, and optimization.

[0073] The above action sequence can be generated by the proxy model, including code editing operations such as adding, deleting, and modifying the current kernel code, as well as calling instructions for external tools such as compilers, testers, verifiers, and performance profilers, to achieve code modification and automated verification.

[0074] The code editing operations include adding, deleting, modifying, and replacing kernel code, and the external tool call instructions are used to trigger automated verification and analysis processes such as compilation, testing, verification, and performance profiling.

[0075] In some implementations, the action sequence is generated by a pre-defined proxy model, which is a large language model trained using reinforcement learning. The reward function used during reinforcement learning training includes hierarchical verifiable rewards; wherein, the hierarchical verifiable rewards include at least one of the following:

[0076] Compilation bonus is a reward that is quantitatively evaluated based on the current compilation status of the kernel code;

[0077] The correctness reward is calculated based on the numerical error between the execution result of the current kernel code on the test case set and the reference implementation.

[0078] Performance bonuses are rewards determined based on the degree to which the current kernel code's performance metrics improve relative to preset performance targets.

[0079] Security constraint penalties are negative rewards imposed when abnormal behavior is detected in the current kernel code.

[0080] The aforementioned compilation reward is used to quantitatively evaluate whether the current kernel code was successfully compiled, the number of compilation warnings, and the types of compilation errors, guiding the agent model to prioritize the generation of syntactically valid and buildable kernel code, thereby improving the effectiveness of code generation.

[0081] The aforementioned correctness reward can be calculated based on the execution results of the current kernel code on the test case set and the numerical error between it and the reference implementation. This is used to ensure that the generated kernel code meets the requirements of computational semantics and numerical precision, thereby improving the reliability and versatility of the kernel.

[0082] The aforementioned performance rewards can be determined based on the improvement of the current kernel code's performance metrics relative to preset performance targets or historical versions. These rewards are used to drive the proxy model to continuously optimize the kernel's operating efficiency, enabling the generated kernel to gradually reach or exceed preset performance targets.

[0083] The aforementioned security constraints and penalties are used to impose penalties when abnormal behaviors such as unauthorized API calls, unauthorized system operations, and resource overruns are detected in the current kernel code. These penalties constrain the proxy model to generate code that conforms to security specifications, ensuring that the kernel generation and execution process is secure and controllable.

[0084] In the above implementation method, the layered verifiable reward provides dense and reliable feedback signals to the agent model from multiple dimensions such as compilation legality, functional correctness, running performance and execution security. This effectively solves the problems of sparse feedback and speculative rewards in existing technologies, enabling the model to stably learn complete debugging and optimization logic, and can improve the quality, reliability and engineering practicality of automatically generated kernels.

[0085] In some implementations, an anti-speculation mechanism is introduced during the calculation of the correctness reward, and the anti-speculation mechanism includes at least one of the following:

[0086] Correctness verification is performed using a hidden test case set, which is not visible to the proxy model during training.

[0087] The test input data is subjected to randomization transformation, which includes one or more of the following: tensor shape randomization, numerical distribution randomization, and boundary condition randomization.

[0088] Perform abnormal behavior detection on the current kernel code;

[0089] Additional penalties are imposed when abnormal behavior is detected in the current kernel code.

[0090] Abnormal behavior refers to non-compliant, non-general, and unreliable computational behaviors adopted by the current kernel code in order to obtain higher rewards. These behaviors mainly include hard-coding fixed outputs, bypassing normal verification logic, relying on specific operating environments, performing feature fitting only for test cases, and forging execution results, which do not conform to the real computational semantics and engineering deployment requirements. Such behaviors will cause the kernel to fail to run normally in general scenarios, affecting the reliability and generalization of the generated results.

[0091] The aforementioned tensor shape randomization refers to randomly transforming shape-related parameters such as tensor dimension, length, batch size, and number of channels of the input data during the testing and verification phase. This allows the proxy model to generate and verify kernel code under diverse tensor sizes, avoiding the model's inability to generalize to scenarios with inputs of different sizes when it only adapts to fixed shapes, thus improving the universality and robustness of the generated kernel.

[0092] The aforementioned randomization of numerical distribution refers to the random adjustment of the numerical range, numerical type, random seed, and data distribution pattern of the input tensor, so that the verification process covers different numerical ranges and data distribution scenarios, ensuring that the generated kernel code can maintain computational stability and numerical accuracy under various numerical inputs, and will not experience abnormalities or precision failures due to changes in input values.

[0093] The aforementioned boundary condition randomization refers to randomly introducing boundary scenario data into test cases, including extreme value data, zero value data, tail-disaligned data, boundary coordinates and boundary masks, etc., to test the execution reliability of kernel code under extreme and boundary scenarios, avoid numerical errors, memory overflows or runtime crashes due to imperfect boundary handling, and improve the stability of the generated kernel in actual deployment.

[0094] In the above implementation method, the use of a hidden test case set for correctness verification can prevent the proxy model from obtaining unreasonable rewards by memorizing the characteristics of public test cases or specifically fitting test data, and ensure that the correctness reward can truly reflect the generalization ability of the kernel code.

[0095] In the above implementation method, the test input data is randomized in terms of tensor shape, numerical distribution, and boundary conditions. This can expand the test coverage and force the proxy model to generate kernel code that can adapt to different input scenarios and has stable computational logic, thus avoiding the situation where the test only passes the verification under specific inputs.

[0096] In the above implementation method, abnormal behavior detection is performed on the current kernel code to identify abnormal behaviors such as hard-coded output, bypassing verification logic, and illegal dependency on execution environment, so as to ensure that the generated code conforms to normal computing logic and engineering specifications.

[0097] In the above implementation, applying additional penalties when the above-mentioned abnormal behavior is detected can directly suppress the reward speculation and cheating behavior of the proxy model, ensure the authenticity and effectiveness of the reward signal, and improve the credibility and stability of the iterative optimization process.

[0098] In the above implementation method, by introducing a mechanism to prevent speculation and cheating in the proxy model by introducing hidden test cases, randomized transformation of input data, and abnormal behavior detection and punishment mechanism in the correctness reward calculation, the reward signal can be effectively prevented from being speculated and cheated. This ensures that the reward signal is true and reliable. At the same time, it expands the verification coverage, improves the universality, robustness and stability of the kernel under different input and boundary scenarios, and makes the final generated parallel computing kernel meet the actual requirements of engineering deployment.

[0099] It should be noted that after each round of action sequence is completed and a new execution result is obtained, the agent model can immediately calculate the hierarchical verifiable reward and the corresponding anti-speculation penalty based on the current execution result. The reward signal is used as the key feedback for iterative optimization to guide the generation of the next round of action sequence. When the preset termination condition is met, no new reward is calculated, and the final kernel is directly output, thus forming a complete closed-loop reinforcement learning optimization process.

[0100] 102. Execute the action sequence to update the current kernel code based on code editing operations, and call external tools based on the call instructions to process the updated current kernel code and obtain new execution results.

[0101] Among them, the kernel code can be iteratively modified by performing code editing operations, and the updated kernel code can be input into external tools such as the compiler, unit test runner, numerical verifier, and performance profiler to obtain execution results such as compilation status, test results, numerical errors, running performance and exception information.

[0102] In some embodiments, external tools can be invoked in an isolated sandbox environment based on invocation instructions to compile, verify, test, or perform performance acquisition on the updated current kernel code, and obtain the new execution results returned by the tools.

[0103] The sandbox environment is used to restrict system permissions, resource access, and network calls to ensure that code execution is secure and controllable; the execution results include compilation information, test results, numerical errors, runtime performance, and exception information.

[0104] In some implementations, the action sequence is performed in an isolated sandbox environment, which includes at least one of the following security controls:

[0105] (1) Restrict file system access permissions during compilation and execution.

[0106] The sandbox environment only allocates the necessary file system access permissions for the execution of action sequences. It only allows reading and writing of temporary files and output files required for kernel code compilation and execution, and prohibits access to sensitive system directories (such as system configuration directories and user data directories). This can prevent malicious code or abnormal compilation processes from tampering with or stealing system files and ensure the integrity of the host file system.

[0107] (2) Network communication and external system calls are prohibited.

[0108] The sandbox environment can block network communication during the compilation and execution process, and disable operations such as process calls and service calls to external systems. This prevents the generated kernel code or temporary files during the compilation process from being illegally transmitted to the outside, and also prevents external malicious programs from interfering with the kernel generation process by accessing the sandbox environment through the network, thus eliminating network-level security risks.

[0109] (3) Limit GPU resource usage.

[0110] Specifically, for parallel computing kernels, the sandbox environment will preset resource usage limits for the GPU resources that compilation and execution depend on, such as limiting the proportion of GPU computing power, the amount of video memory used, and the kernel runtime. This will prevent excessive GPU resource consumption during a single round of action sequence execution, which could lead to the exhaustion of the overall system computing power. At the same time, it will prevent hardware overload and system lag caused by malicious code or abnormal kernels infinitely occupying GPU resources, thus ensuring the fair allocation and stable scheduling of GPU resources.

[0111] For example, the parallel computing kernel involved in the embodiments of this application can be a CUDA kernel, or it can be a ROCm kernel, an OpenCL kernel, etc. Among them, the ROCm kernel is a parallel computing kernel adapted to mainstream discrete graphics cards; the OpenCL kernel is a general-purpose parallel computing kernel that can be adapted to various hardware devices.

[0112] (4) Perform static analysis and runtime monitoring on the compiled executable file to detect potential malicious code behavior or resource abuse.

[0113] For the compiled executable file, the sandbox environment first scans the code logic with static analysis tools to identify potential malicious instructions and resource abuse (such as infinite loops and illegal memory access). During the runtime phase, it monitors the kernel's execution behavior in real time, tracks the GPU instruction execution trajectory, memory access range, and resource usage changes. Once abnormal behavior is detected (such as unauthorized access to video memory or malicious consumption of computing power), execution is immediately terminated and an alarm is triggered to intercept risky behavior in advance.

[0114] (5) Record a complete execution log.

[0115] The sandbox environment records a full log of the entire action sequence, including compilation commands, GPU resource usage, kernel runtime output, and exception triggering information. The log content is traceable and auditable. This facilitates subsequent troubleshooting of faults in the kernel generation process (such as compilation failures or runtime errors) and provides a basis for tracing security incidents, allowing for rapid identification of problematic aspects and causes if security risks arise.

[0116] In the above implementation, the execution of the action sequence is placed in an isolated sandbox environment containing multi-dimensional security control measures, which not only ensures the security and stability of the kernel generation process, but also takes into account the rationality of resource utilization and the traceability of the process, significantly improving the engineering practicality and security reliability of the parallel computing kernel automatic generation method.

[0117] In some implementations, the external tool includes at least one of the following types of tools:

[0118] Compiler, static analyzer, unit test runner, numerical verifier, performance profiler, memory error detection tool.

[0119] The compiler mentioned above is used to perform syntax checks, compile and link the updated current kernel code, and return compilation logs, compilation error messages or compilation success status, which serve as a direct basis for whether the kernel code can be built normally.

[0120] The static inspector described above is used to detect potential risks, undefined behaviors, memory access vulnerabilities, or syntax errors in the code without running the current kernel code, thus identifying code defects that are difficult to expose during runtime.

[0121] The aforementioned unit test runner is used to execute preset test cases, drive the updated current kernel code to run, and return test execution results, including test case pass rate, failed test case identifiers, and corresponding error stack information.

[0122] The aforementioned numerical verifier is used to compare the actual output of the current kernel code with the standard output, calculate the numerical error and statistically analyze the error distribution to verify whether the kernel calculation result meets the preset numerical verification conditions.

[0123] The aforementioned performance profiler is used to collect performance metrics of the updated current kernel code during runtime, including runtime latency, throughput, video memory usage, GPU utilization, etc., providing quantitative basis for performance optimization and iterative improvement.

[0124] The aforementioned memory error detection tool is used to detect illegal memory operations such as memory out-of-bounds access, uninitialized access, and memory leaks in the current kernel code during runtime, ensuring the security and stability of the generated kernel.

[0125] In the above implementation method, through the coordinated invocation of compiler, static checker, unit test runner, numerical verifier, performance profiler and memory error detection tool, the kernel code can be comprehensively detected and evaluated from multiple dimensions such as code building, functional verification, numerical accuracy, running performance and memory safety. This provides complete and reliable feedback information for iterative optimization, effectively improves the kernel generation quality and execution stability, and meets the reliability requirements of engineering deployment.

[0126] 103. Update the current iteration state based on the new execution result, and iteratively generate the next action sequence based on the updated current iteration state.

[0127] In this process, the new execution results returned by the tool are integrated into the current iteration state, forming a complete state that includes the latest kernel code, historical actions, and the latest execution results. The proxy model continues to generate the next round of action sequences based on the updated state, thus achieving a closed-loop iteration from code modification and tool verification to state updates.

[0128] 104. When the preset termination condition is met, output the final generated parallel computing kernel.

[0129] The preset termination conditions include successful compilation, successful execution of all test cases, numerical error meeting requirements, performance indicators reaching preset targets, and the number of iterations reaching the upper limit. The iteration will terminate and a usable kernel will be output once any one of these conditions is met.

[0130] In some implementations, the aforementioned preset termination condition includes at least one of the following:

[0131] (1) The current kernel code has successfully passed all test cases;

[0132] (2) The numerical error between the current kernel code and the reference implementation is within the preset tolerance range;

[0133] (3) The performance metrics of the current kernel code have reached or exceeded the preset performance targets;

[0134] (4) The current kernel code meets the correctness verification conditions;

[0135] (5) No further improvement was achieved after multiple iterations;

[0136] (6) The number of iterations reaches the preset maximum number of iterations threshold, or the cumulative execution time exceeds the preset time budget limit.

[0137] The aforementioned preset termination conditions comprehensively constrain the automatic generation and iterative optimization process of the parallel computing kernel from multiple dimensions, including functional correctness, numerical accuracy, running performance, iterative convergence, and resource constraints.

[0138] Among them, passing all test cases, meeting the numerical error tolerance range, and meeting the correctness verification conditions are used to ensure that the generated parallel computing kernel is consistent with the expected semantics in terms of computational logic, and to avoid functional errors or numerical anomalies.

[0139] The above performance indicators meet or exceed the preset targets to ensure that the generated kernel meets the operational efficiency requirements of actual deployment.

[0140] The above-mentioned continuous iterations without further improvement are used to identify the convergence state of the iteration and avoid invalid loops.

[0141] The above-mentioned number of iterations or execution time reaches the preset upper limit, which is used to control the computing resources and time overhead of the overall generation process, and ensure that the automatic generation process is efficient and controllable.

[0142] In the above implementation method, by combining and setting the above-mentioned multiple preset termination conditions, the stability and practicality of the automatic generation process can be improved while ensuring kernel quality, so that the final output parallel computing kernel can simultaneously meet the requirements of correctness, reliability, performance and engineering deployment.

[0143] In the above embodiments, an action sequence including code editing and tool calls can be generated iteratively based on the current iteration state. During the generation process, the state is continuously updated and iteratively optimized according to the execution results. This transforms the generation of parallel computing kernels from a traditional process that relies on manual writing and expert tuning to an automated process that can be autonomously iteratively optimized. This significantly improves development efficiency, shortens the iteration cycle, and reduces labor costs. It can adapt to diverse operators and complex fusion kernel scenarios and meet the needs of rapid iteration.

[0144] Furthermore, the above method provides the model with dense and traceable process feedback signals by executing external tools multiple times and continuously obtaining new execution results, thus achieving effective supervision of the entire process. This enables the model to stably learn complete debugging logic, solving the problems of sparse feedback signals, lack of process supervision, and difficulty in learning complete debugging logic in existing methods. In addition, through iterative optimization based on the current iteration state and evaluation by external tools, the kernel code can be fully verified and continuously improved in a complex search space. This effectively alleviates the problems of insufficient kernel reliability, poor versatility, and substandard actual performance caused by reward speculation and insufficient evaluation, making the final generated parallel computing kernel more in line with the actual requirements of engineering deployment.

[0145] In one exemplary embodiment, such as Figure 2 As shown, a flowchart for initializing the current iteration state is provided, which includes the following steps 201 to 203.

[0146] 201. Obtain the high-level computation description used to generate the parallel computing kernel.

[0147] The high-level computation description includes at least one of operator sequences, computation graphs, tensor expressions, or intermediate representations.

[0148] The aforementioned high-level computation description is an abstract expression of the target computation logic. It is used to clarify the operator functions, data dependencies, and overall computational semantics that the kernel to be generated needs to implement, and can adapt to the description requirements of diverse operators and complex fusion scenarios.

[0149] 202. Obtain the verification condition set.

[0150] The verification condition set includes tensor constraint information, numerical verification conditions, test case set, and performance target.

[0151] The aforementioned set of verification conditions provides quantitative constraints for this iterative process. Tensor constraints limit the shape and data type of input and output tensors, numerical verification conditions limit the allowable error range of the calculation results, the test case set verifies the integrity of kernel functionality, and performance targets constrain metrics such as runtime latency, throughput, and video memory usage.

[0152] In some implementations, the aforementioned test case set includes: a public test case set and a hidden test case set.

[0153] The hidden test case set is invisible to the proxy model during the generation and iteration process, and is used to verify the correctness of the current kernel code or evaluate rewards.

[0154] In the above implementation, the public test case set is used for routine debugging and repair of the proxy model during iterative optimization, while the hidden test case set remains invisible to the proxy model during model generation and iteration, and is only used for independent correctness verification or reward evaluation of the current kernel code.

[0155] The use of the aforementioned public and hidden test case sets can effectively avoid problems such as overfitting, targeted fitting, or reward speculation in the proxy model based solely on public test cases, thereby further improving the generalization ability and actual deployment reliability of the generated kernel.

[0156] 203. Based on the high-level computation description and verification condition set, initialize the current iteration state so that the current iteration state includes the initial kernel code, the preset historical action sequence and the preset execution result.

[0157] Among them, the initial kernel code is the starting point for iterative optimization, the preset historical action sequence can be an empty sequence by default, the preset execution result can be the initial unexecuted state, and the current iteration state as a whole can provide a standardized and reproducible initial decision-making environment for the agent model.

[0158] In one exemplary embodiment, such as Figure 3 As shown, a flowchart for updating the current iteration state is provided, which includes the following steps 301 and 302.

[0159] 301. Perform structured parsing on the new execution results returned by external tools, extract key information to form structured state data.

[0160] The key information includes at least one of the following: compilation error type and location, test failure case identifier and failure reason, numerical error distribution and statistics, performance indicator values ​​and fluctuation range, and memory error type and location. This key information can provide clear and attributable feedback signals for the proxy model.

[0161] The structured parsing described above is used to organize and quantify unstructured logs, error stacks, output data, and other information returned by compilers, testers, performance profilers, etc.

[0162] 302. Update the current iteration state based on the structured state data.

[0163] By incorporating structured state data into the current iteration state and replacing the original execution result information, the current iteration state can fully and accurately reflect the latest iteration of the kernel code, providing a reliable basis for generating the next round of action sequences for the proxy model, thereby improving the efficiency and accuracy of debugging and optimization.

[0164] In one exemplary embodiment, such as Figure 4As shown, a flowchart of a kernel generation method is provided. This process involves the generation of a fusion kernel for an element-wise operator chain (bias+gelu+dropout). When generating a kernel using the automatic generation method of parallel computing kernel provided in this application embodiment, the process may include the following steps 401 to 406.

[0165] 401. Receive operator chain, tensor shape, tensor type, random seed and numerical tolerance, and generate reference implementation and test cases.

[0166] The aforementioned "element-wise operator chain (bias + gelu + dropout)" is a combination of continuous computational logic in the field of deep learning. Here, bias refers to the bias addition operator, used to add a fixed bias value to each element of the input tensor; gelu refers to the Gaussian error linear unit operator, an element-wise activation function used to perform a non-linear transformation on each element of the tensor; and dropout refers to the random deactivation operator, used to randomly zero out each element of the tensor during training to prevent overfitting. The core feature of this operator chain is element-wise computation (i.e., the computation of each element depends only on itself, with no cross-element dependencies). Integrating these operators into a single kernel can significantly reduce GPU memory data transfer and kernel startup overhead, improving overall computational efficiency.

[0167] For example, the tensor shape can take the form of batch and hidden layer dimensions such as [B,H], and the tensor type can include deep learning data types such as FP16 or BF16.

[0168] The above reference implementation can be used to provide standard output truth values ​​to verify numerical correctness. Test cases can include random tensors and boundary cases to cover normal and extreme computing scenarios.

[0169] In step 401 above, a reference implementation can first be obtained by combining the input operator chain and high-level computational semantics through a deep learning framework or operator library. This reference implementation can serve as a source of numerical truth for subsequent verification of the computational correctness of the generated kernel. At the same time, based on tensor shape, tensor type, and random seed, test cases containing regular data, boundary data, and random data are automatically constructed, and numerical tolerance is used as the numerical error judgment standard to ensure that the test cases can cover different input scenarios and precision constraints, providing a unified verification benchmark for subsequent compilation, testing, verification, and performance tuning.

[0170] 402. Generate the initial kernel.

[0171] The initial kernel includes grid and thread block configurations adapted to GPU hardware, vectorized loading methods, register caching mechanisms, and result write-back logic, enabling it to initially realize the computational functions of the target operator chain.

[0172] In step 402 above, the proxy model can generate initial kernel code with basic computational logic based on task information such as high-level computational description, tensor shape, and tensor type. This initial kernel code configures the grid and thread block layout according to the GPU parallel execution paradigm, uses vectorized memory access for data loading, utilizes registers for temporary storage and computation of intermediate results, and performs element-by-element computation and result writing back according to operator chain logic, thus forming an initial code version that can be used for subsequent compilation, verification, and iterative optimization.

[0173] 403. Call the compilation tool to compile. If it fails, parse the error stack and generate a patch.

[0174] For example, the compilation tools mentioned above may include nvcc or clang.

[0175] For example, the patches mentioned above include modifications to type conversion, inclusion of header files, and correction of template instantiation.

[0176] In step 403 above, a specified compilation tool (such as nvcc or clang) can be called in an isolated sandbox environment to perform a compilation operation on the initial kernel code. If the compilation is successful, an executable binary file is output, and the subsequent unit testing and numerical verification stages are entered. If the compilation fails, the compilation error stack information is extracted, the error type (such as syntax error, type mismatch, missing header file, template instantiation exception, etc.) and the location of the error are analyzed, and the proxy model automatically generates a targeted repair patch based on the error characteristics. After the patch is applied, the compilation process is retried until the kernel code is successfully compiled.

[0177] 404. Call the unit test runner, compare the output with the reference implementation, and if the error exceeds the threshold, locate and fix the numerical stability or boundary conditions.

[0178] During the troubleshooting and repair process, boundary-related issues such as tensor tail alignment anomalies and mask processing errors can be corrected. Tail alignment refers to anomalies caused by tensor end data not conforming to hardware memory access alignment specifications; a mask is identification data used during computation to shield invalid computation locations and constrain the computation range.

[0179] In step 404 above, the unit test runner can be invoked first to load the compiled kernel binary file, drive the execution of the preset test case set, and collect the output calculation results. The results are then compared element by element with the standard output of the reference implementation to calculate the numerical error and compare it with the preset numerical tolerance threshold. If the error does not exceed the threshold, it indicates that the kernel numerical correctness meets the standard, and the subsequent performance evaluation stage begins. If the error exceeds the threshold, the source of the error is located, and the focus is on investigating numerical stability issues or boundary condition handling defects. The kernel code logic is adjusted according to the problem type to complete the repair. After the repair, the test is re-executed until the numerical error meets the tolerance requirements.

[0180] For example, the aforementioned numerical stability problems may include loss of floating-point precision, overflow of exponential or logarithmic functions, etc.

[0181] For example, the aforementioned defects in boundary condition processing may include tensor tail data alignment errors, mask logic errors, and extreme value data calculation errors.

[0182] 405. Perform multiple rounds of performance evaluation through the performance profiler, collecting latency, bandwidth, and utilization data, and the agent adjusts its strategy based on the bottleneck.

[0183] Bottlenecks refer to issues that restrict kernel operating efficiency, specifically including:

[0184] Memory access bottleneck, where kernel operating efficiency is limited by memory data read and write speed (such as excessive global memory access latency or insufficient bandwidth), is the most common performance bottleneck in GPU parallel computing.

[0185] Arithmetic bottleneck, namely, the utilization of the kernel computing unit (CUDA Core) is not up to the limit, resulting in low execution efficiency of computing instructions;

[0186] Branch bottleneck refers to the inconsistency in thread execution paths within a warp caused by conditional branches in the kernel code, leading to thread idleness.

[0187] The agent mentioned above can refer to an agent model.

[0188] Adjustment strategies to address the aforementioned bottlenecks may include, but are not limited to: enhancing Instruction-Level Parallelism (ILP), which involves optimizing instruction scheduling and merging computational operations to enable a single thread to execute multiple independent instructions simultaneously, thereby improving the utilization of computational units; reducing global memory accesses, which involves caching data using registers and shared memory to reduce the frequency of access to high-latency global memory; and using shared memory chunking, which involves dividing large tensors into smaller chunks and loading them into shared memory to reduce the number of global memory accesses and improve data reuse.

[0189] 406. After reaching the correctness and performance thresholds, output the final kernel and integrated code.

[0190] First, it can be verified whether the current kernel simultaneously meets the preset correctness threshold and performance threshold. The correctness threshold includes passing all test cases and numerical errors within the preset tolerance range. The performance threshold includes indicators such as runtime latency, video memory bandwidth, and GPU utilization reaching or exceeding preset targets.

[0191] If all conditions are met, the iterative optimization process terminates, the final kernel code after multiple rounds of compilation, testing, and performance tuning is extracted, and the corresponding integration code is automatically generated. The integration code includes kernel call interfaces, data preprocessing or postprocessing logic, and adaptation code with deep learning frameworks, so that the output results can be directly integrated into the actual engineering system and meet the deployment and usage requirements. If the conditions are not met, the process returns to the proxy model to generate a new action sequence and continues iterative optimization until the correctness and performance thresholds are reached.

[0192] In this embodiment, an element-wise operator chain fusion kernel can be generated quickly and reliably through an automated generation, compilation, testing, debugging, and performance tuning closed loop. This significantly improves computational performance and reduces memory overhead while ensuring numerical accuracy and stability, effectively reducing manual development costs and meeting the engineering requirements for efficient deployment of deep learning operators.

[0193] For example, for attention subgraphs ( The generation of the fusion kernel can also adopt the automatic generation method of the parallel computing kernel provided in the embodiments of this application.

[0194] Among them, “attention subgraph” ")" refers to continuous computation in the deep learning Transformer model, which involves multiplying the query by the transpose of the key. It consists of three consecutive operators: normalized exponential function (softmax), and multiplication of the value matrix (PV). By integrating these three steps into a kernel, the memory read and write speed can be greatly reduced and the calculation speed can be improved.

[0195] When generating a kernel using the automatic generation method for parallel computing kernels provided in this application embodiment, the following steps are included:

[0196] Step A: Provide a stable softmax reference implementation, different sequence length distributions, and boundary use cases.

[0197] Firstly, during the task construction phase, a numerically stable reference implementation of the normalized exponential function, test data covering different sequence length distributions, and various boundary test cases are provided to provide a standard benchmark for kernel generation and verification.

[0198] The task construction phase described above establishes a unified and comprehensive benchmark for subsequent kernel generation and verification. First, a stable softmax (normalized exponential function) reference implementation based on high-precision floating-point computation (such as FP32) is provided as the truth base for numerical correctness verification, avoiding reward distortion caused by the reference implementation's own precision issues. Simultaneously, test cases covering different sequence length distributions (short, medium, long) are constructed, supplemented with boundary cases such as sequence length extrema, zero-value matrices, and extreme value values. This ensures the kernel's adaptability and stability to various input scenarios can be verified, providing a complete verification standard for subsequent compilation, testing, and optimization stages.

[0199] Step B: Use tiling and shared memory caching, and apply numerically stable max-reduction + exp-sum to softmax; and perform subsequent matrix multiplication with V in the same kernel.

[0200] The above-mentioned "max-reduction+exp-sum" is a numerically stable Softmax calculation method. It first performs maximum value reduction (max-reduction) on each row or each attention head, subtracting the maximum value from all elements to avoid exponential overflow; then it performs exponential summation (exp-sum) to complete the normalization calculation, which can ensure the numerical stability under low-precision floating-point operations and prevent the calculation results from overflowing or becoming infinite.

[0201] Specifically, during the proxy strategy phase, data access is optimized using block and shared memory caching. The normalized exponential function is calculated using a numerically stable maximum reduction and exponential summation method. The entire calculation process, including query key multiplication, normalized exponential function processing, and multiplication of the value matrix, is completed continuously within the same kernel, ensuring computational efficiency and numerical stability.

[0202] The aforementioned proxy strategy phase focuses on targeted optimization of the computational characteristics of the attention subgraph in order to generate an efficient and stable fusion kernel.

[0203] This can be achieved by employing a tiling and shared memory caching strategy. Large Q, K, and V matrices can be divided into fixed-size sub-blocks and loaded into GPU shared memory for computation, significantly reducing global memory access frequency and improving data reuse. For softmax calculation, a numerically stable maximum reduction and exponential summation method is used. By subtracting the maximum value of each attention head before calculating the exponent, numerical overflow issues are avoided. Simultaneously, The functions of softmax and PV are integrated into a single kernel, eliminating the overhead of memory transfer and startup caused by multi-kernel calls and improving overall computing efficiency.

[0204] The Q (Query), K (Key), and V (Value) matrices mentioned above are the core input matrices for the self-attention mechanism of the Transformer architecture:

[0205] Among them, the Q matrix represents the query information of the current position to be calculated, and is used to retrieve the context information associated with it; the K matrix represents the key information of all positions, and is used to match with the Q matrix to calculate the attention weights; the V matrix represents the value information of all positions, and is the object of the weighted summation of attention weights; all three are obtained by linear transformation of the input sequence, and the dimensions are usually [sequence length, feature dimension], which are the data carriers for the calculation of attention subgraphs.

[0206] Step C: The correctness reward is based on the error between the implementation and the reference implementation in half-precision floating-point (FP16) / bfloat16 floating-point (BF16) mode; the performance reward uses the median / quantile of multiple measurements with a variance penalty to avoid unstable optimization. Both half-precision floating-point (FP16) and bfloat16 floating-point (BF16) are 16-bit floating-point data types widely used in deep learning, and both use 16 bits for numerical storage and computation.

[0207] During the reward evaluation phase, reward rules can be customized for the application scenarios of the attention subgraph kernel to ensure the rationality of the optimization direction. Specifically, the correctness reward is based on the element-wise error calculation between the kernel output and the reference implementation in low-precision FP16 / BF16 scenarios; the smaller the error, the higher the reward value, ensuring that the accuracy under low-precision calculations meets deployment requirements. The performance reward involves multiple rounds of repeated measurements of kernel latency, bandwidth, and other metrics, using the median or quantile as the performance benchmark to avoid single-measurement errors. A variance penalty term is also added to deduct rewards for kernels with excessive performance fluctuations, preventing unstable optimizations where the proxy model only pursues a single optimal result, and ensuring that the generated kernel has both accuracy and stable performance.

[0208] In the above example, for the attention subgraph fusion kernel, through standardized task construction, targeted agent optimization and anti-speculative reward mechanism, a kernel with stable values, excellent performance and reliable deployment can be automatically generated. This can effectively reduce memory read and write overhead, improve the computational efficiency of Transformer model, and avoid optimization fluctuations and reward speculation problems.

[0209] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0210] Based on the same inventive concept, this application also provides an automatic parallel computing kernel generation apparatus for implementing the automatic parallel computing kernel generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the automatic parallel computing kernel generation apparatus provided below can be found in the limitations of the automatic parallel computing kernel generation method described above, and will not be repeated here.

[0211] In one exemplary embodiment, such as Figure 5 As shown, an automatic generation device for parallel computing kernels is provided, comprising:

[0212] The sequence generation module 501 is used to generate an action sequence based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process.

[0213] The sequence execution module 502 is used to execute the action sequence to update the current kernel code based on the code editing operation, and to call an external tool based on the call instruction to process the updated current kernel code and obtain a new execution result;

[0214] The kernel generation module 503 is used to update the current iteration state according to the new execution result, and iteratively generate the next action sequence based on the updated current iteration state; when the preset termination condition is met, the final generated parallel computing kernel is output.

[0215] In some embodiments, the apparatus further includes: an initialization module, configured to:

[0216] Obtain a high-level computational description for generating the parallel computing kernel, wherein the high-level computational description includes at least one of operator sequence, computation graph, tensor expression or intermediate representation;

[0217] Obtain a set of verification conditions, which includes tensor constraint information, numerical verification conditions, a set of test cases, and performance targets;

[0218] Based on the high-level computation description and the verification condition set, the current iteration state is initialized, so that the current iteration state includes the initial kernel code, the preset historical action sequence and the preset execution result.

[0219] In some embodiments, the test case set includes: a public test case set and a hidden test case set;

[0220] The hidden test case set is invisible to the proxy model during the generation and iteration process, and is used to verify the correctness of the current kernel code or evaluate rewards.

[0221] In some embodiments, the external tool includes at least one of the following types of tools:

[0222] Compiler, static analyzer, unit test runner, numerical verifier, performance profiler, memory error detection tool.

[0223] In some embodiments, the kernel generation module 503 is specifically used for:

[0224] The new execution results returned by the external tool are subjected to structured parsing to extract key information and form structured state data. The key information includes at least one of the following: compilation error type and location, test failure case identifier and failure reason, numerical error distribution and statistics, performance index value and fluctuation range, and memory error type and occurrence location. Based on the structured state data, the current iteration state is updated.

[0225] In some embodiments, the action sequence is generated by a preset proxy model, which is a large language model obtained based on reinforcement learning training, and the reward function used in the reinforcement learning training process includes hierarchical verifiable rewards.

[0226] The tiered verifiable rewards include at least one of the following:

[0227] Compilation reward, which is a reward quantitatively evaluated based on the current compilation status of the kernel code;

[0228] The correctness reward is a reward calculated based on the numerical error between the execution result of the current kernel code on the test case set and the reference implementation;

[0229] Performance reward, which is a reward determined based on the degree of improvement of the current kernel code's performance metrics relative to a preset performance target;

[0230] Security constraint penalty, which is a negative reward imposed when abnormal behavior is detected in the current kernel code.

[0231] In some embodiments, an anti-speculation mechanism is introduced during the calculation of the correctness reward, the anti-speculation mechanism including at least one of the following:

[0232] Correctness verification is performed using a hidden test case set, which is not visible to the proxy model during training.

[0233] The test input data is subjected to randomization transformation, which includes one or more of the following: tensor shape randomization, numerical distribution randomization, and boundary condition randomization.

[0234] Perform abnormal behavior detection on the current kernel code;

[0235] Additional penalties are imposed when abnormal behavior is detected in the current kernel code.

[0236] In some embodiments, the action sequence is performed in an isolated sandbox environment, which includes at least one of the following security controls:

[0237] Restrict file system access permissions during compilation and execution;

[0238] Network communication and external system calls are prohibited;

[0239] Limit GPU resource usage;

[0240] Perform static analysis and runtime monitoring on the compiled executable file to detect potential malicious code behavior or resource abuse;

[0241] Record a complete execution log.

[0242] In some embodiments, the preset termination condition includes at least one of the following:

[0243] The current kernel code has successfully passed all test cases;

[0244] The numerical error between the current kernel code and the reference implementation is within a preset tolerance range;

[0245] The performance metrics of the current kernel code have reached or exceeded the preset performance targets;

[0246] The current kernel code meets the correctness verification conditions;

[0247] No further improvement was achieved after multiple rounds of iteration.

[0248] The number of iterations reaches the preset maximum iteration threshold, or the cumulative execution time exceeds the preset time budget limit.

[0249] Each module in the aforementioned automatic generation device for parallel computing kernels can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can invoke and execute the operations corresponding to each module.

[0250] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 6 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the various processes involved in the above method embodiments.

[0251] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0252] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0253] An action sequence is generated based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process.

[0254] The action sequence is executed to update the current kernel code based on the code editing operation, and an external tool is invoked based on the invocation instruction to process the updated current kernel code and obtain a new execution result;

[0255] The current iteration state is updated based on the new execution result, and the next action sequence is generated iteratively based on the updated current iteration state; when the preset termination condition is met, the finally generated parallel computing kernel is output.

[0256] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0257] Obtain a high-level computational description for generating the parallel computing kernel, wherein the high-level computational description includes at least one of operator sequence, computation graph, tensor expression or intermediate representation;

[0258] Obtain a set of verification conditions, which includes tensor constraint information, numerical verification conditions, a set of test cases, and performance targets;

[0259] Based on the high-level computation description and the verification condition set, the current iteration state is initialized, so that the current iteration state includes the initial kernel code, the preset historical action sequence and the preset execution result.

[0260] In one embodiment, the test case set includes: a public test case set and a hidden test case set;

[0261] The hidden test case set is invisible to the proxy model during the generation and iteration process, and is used to verify the correctness of the current kernel code or evaluate rewards.

[0262] In one embodiment, the external tool includes at least one of the following types of tools:

[0263] Compiler, static analyzer, unit test runner, numerical verifier, performance profiler, memory error detection tool.

[0264] In one embodiment, updating the current iteration state based on the new execution result includes:

[0265] The new execution results returned by the external tool are subjected to structured parsing to extract key information and form structured status data; wherein, the key information includes at least one of the following: compilation error type and location, test failure case identifier and failure reason, numerical error distribution and statistics, performance index value and fluctuation range, and memory error type and occurrence location;

[0266] The current iteration state is updated based on the structured state data.

[0267] In one embodiment, the action sequence is generated by a preset agent model, which is a large language model obtained based on reinforcement learning training, and the reward function used in the reinforcement learning training process includes hierarchical verifiable rewards.

[0268] The tiered verifiable rewards include at least one of the following:

[0269] Compilation reward, which is a reward quantitatively evaluated based on the current compilation status of the kernel code;

[0270] The correctness reward is a reward calculated based on the numerical error between the execution result of the current kernel code on the test case set and the reference implementation;

[0271] Performance reward, which is a reward determined based on the degree of improvement of the current kernel code's performance metrics relative to a preset performance target;

[0272] Security constraint penalty, which is a negative reward imposed when abnormal behavior is detected in the current kernel code.

[0273] In one embodiment, an anti-speculation mechanism is introduced during the calculation of the correctness reward, the anti-speculation mechanism including at least one of the following:

[0274] Correctness verification is performed using a hidden test case set, which is not visible to the proxy model during training.

[0275] The test input data is subjected to randomization transformation, which includes one or more of the following: tensor shape randomization, numerical distribution randomization, and boundary condition randomization.

[0276] Perform abnormal behavior detection on the current kernel code;

[0277] Additional penalties are imposed when abnormal behavior is detected in the current kernel code.

[0278] In one embodiment, the action sequence is performed in an isolated sandbox environment, which includes at least one of the following security controls:

[0279] Restrict file system access permissions during compilation and execution;

[0280] Network communication and external system calls are prohibited;

[0281] Limit GPU resource usage;

[0282] Perform static analysis and runtime monitoring on the compiled executable file to detect potential malicious code behavior or resource abuse;

[0283] Record a complete execution log.

[0284] In one embodiment, the preset termination condition includes at least one of the following:

[0285] The current kernel code has successfully passed all test cases;

[0286] The numerical error between the current kernel code and the reference implementation is within a preset tolerance range;

[0287] The performance metrics of the current kernel code have reached or exceeded the preset performance targets;

[0288] The current kernel code meets the correctness verification conditions;

[0289] No further improvement was achieved after multiple rounds of iteration.

[0290] The number of iterations reaches the preset maximum iteration threshold, or the cumulative execution time exceeds the preset time budget limit.

[0291] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the various processes involved in the above method embodiments.

[0292] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the various processes involved in the above method embodiments.

[0293] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0294] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0295] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for automatically generating parallel computing kernels, characterized in that, include: An action sequence is generated based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process. The action sequence is executed to update the current kernel code based on the code editing operation, and an external tool is invoked based on the invocation instruction to process the updated current kernel code and obtain a new execution result; The current iteration state is updated based on the new execution result, and the next action sequence is generated iteratively based on the updated current iteration state; when the preset termination condition is met, the finally generated parallel computing kernel is output.

2. The method according to claim 1, characterized in that, The method further includes: Obtain a high-level computational description for generating the parallel computing kernel, wherein the high-level computational description includes at least one of operator sequence, computation graph, tensor expression or intermediate representation; Obtain a set of verification conditions, which includes tensor constraint information, numerical verification conditions, a set of test cases, and performance targets; Based on the high-level computation description and the verification condition set, the current iteration state is initialized, so that the current iteration state includes the initial kernel code, the preset historical action sequence and the preset execution result.

3. The method according to claim 2, characterized in that, The test case set includes: a public test case set and a hidden test case set; The hidden test case set is invisible to the proxy model during the generation and iteration process, and is used to verify the correctness of the current kernel code or evaluate rewards.

4. The method according to claim 1, characterized in that, The external tools include at least one of the following types of tools: Compiler, static analyzer, unit test runner, numerical verifier, performance profiler, memory error detection tool.

5. The method according to claim 1, characterized in that, Updating the current iteration state based on the new execution result includes: The new execution results returned by the external tool are subjected to structured parsing to extract key information and form structured status data; wherein, the key information includes at least one of the following: compilation error type and location, test failure case identifier and failure reason, numerical error distribution and statistics, performance index value and fluctuation range, and memory error type and occurrence location; The current iteration state is updated based on the structured state data.

6. The method according to claim 1, characterized in that, The action sequence is generated by a preset agent model, which is a large language model obtained by reinforcement learning training. The reward function used in the reinforcement learning training process includes hierarchical verifiable rewards. The tiered verifiable rewards include at least one of the following: Compilation reward, which is a reward quantitatively evaluated based on the current compilation status of the kernel code; The correctness reward is a reward calculated based on the numerical error between the execution result of the current kernel code on the test case set and the reference implementation; Performance reward, which is a reward determined based on the degree of improvement of the current kernel code's performance metrics relative to a preset performance target; Security constraint penalty, which is a negative reward imposed when abnormal behavior is detected in the current kernel code.

7. The method according to claim 6, characterized in that, The calculation of the correctness reward incorporates an anti-speculation mechanism, which includes at least one of the following: Correctness verification is performed using a hidden test case set, which is not visible to the proxy model during training. The test input data is subjected to randomization transformation, which includes one or more of the following: tensor shape randomization, numerical distribution randomization, and boundary condition randomization. Perform abnormal behavior detection on the current kernel code; Additional penalties are imposed when abnormal behavior is detected in the current kernel code.

8. The method according to any one of claims 1 to 7, characterized in that, The action sequence is performed in an isolated sandbox environment, which includes at least one of the following security controls: Restrict file system access permissions during compilation and execution; Network communication and external system calls are prohibited; Limit GPU resource usage; Perform static analysis and runtime monitoring on the compiled executable file to detect potential malicious code behavior or resource abuse; Record a complete execution log.

9. The method according to any one of claims 1 to 7, characterized in that, The preset termination condition includes at least one of the following: The current kernel code has successfully passed all test cases; The numerical error between the current kernel code and the reference implementation is within a preset tolerance range; The performance metrics of the current kernel code have reached or exceeded the preset performance targets; The current kernel code meets the correctness verification conditions; No further improvement was achieved after multiple rounds of iteration. The number of iterations reaches the preset maximum iteration threshold, or the cumulative execution time exceeds the preset time budget limit.

10. An automatic generation device for parallel computing kernels, characterized in that, The device includes: The sequence generation module is used to generate an action sequence based on the current iteration state. The action sequence includes code editing operations on the current kernel code and instructions for calling external tools. The current iteration state includes: the current kernel code, historical action sequences and their corresponding execution results. The current kernel code is the code version of the kernel that is computed in parallel during the iteration process. The sequence execution module is used to execute the action sequence to update the current kernel code based on the code editing operation, and to call external tools to process the updated current kernel code to obtain new execution results; The kernel generation module is used to update the current iteration state based on the new execution result, and to iteratively generate the next action sequence based on the updated current iteration state; when the preset termination condition is met, the final generated parallel computing kernel is output.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images