Code generation and tuning method and apparatus

Abstract

This invention provides a method and apparatus for automatic code generation and optimization, relating to the field of data processing technology. The method includes: acquiring natural language prompts input by a user, and parsing the prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters; analyzing the technical manual corresponding to the target hardware platform information using a large language model to extract target hardware factor information; generating skeleton code using the large language model; using script code for each kernel function call placeholder; executing the script code to obtain hardware primitive-level kernel code, and jointly compiling and functionally verifying the skeleton code and the hardware primitive-level kernel code; after successful functional verification, iteratively optimizing the parameter configuration of the skeleton code and the hardware primitive-level kernel code using a tree search algorithm; and outputting the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after iterative optimization.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for automatic code generation and optimization. Background Technology

[0002] Tensor operators are core computationally intensive operations in deep learning and large language models, including matrix multiplication and convolution, accounting for the majority of computation during model inference and training. With the continuous growth in the size of model parameters, there is an urgent need for efficient implementations of high-performance tensor operators on different hardware platforms.

[0003] Currently, high-performance implementations of tensor operators mainly rely on the following approaches: The first is for hardware manufacturers or developers to manually write assembly-level or hardware primitive-level optimization code for specific hardware platforms. While this method can fully utilize hardware characteristics to achieve high performance, it has a long development cycle, high labor costs, and the written code is deeply bound to specific hardware, making it unusable across platforms. The second is to use AI compiler frameworks to generate target code through predefined optimization rules and scheduling templates. However, such frameworks still require experts to manually write backend implementations and optimization rules for each new hardware platform, resulting in a long support cycle for emerging hardware architectures. The third is to directly generate code using large language models. However, existing methods only allow large language models to generate hardware-independent high-level language code, and the performance of the generated code is significantly lower than that of manually optimized libraries, failing to meet actual deployment requirements. The root cause of this problem is that large language models lack the ability to understand the underlying characteristics of the target hardware architecture, making it difficult to directly generate correct hardware primitive-level instruction code, and the generated code lacks effective automated parameter tuning methods. Summary of the Invention

[0004] This invention provides a method and apparatus for automatic code generation and optimization, which solves the problem of how to more effectively generate and optimize code automatically in the prior art.

[0005] This invention provides a method for automatic code generation and optimization, comprising: Obtain natural language prompts input by the user, and parse the natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters; The technical manual corresponding to the target hardware platform information is analyzed using a large language model to extract target hardware factor information; Based on the target hardware factor information, the target operator type, and the operator dimension parameters, skeleton code is generated using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computational operations or data movement operations; For each kernel function call placeholder in the skeleton code, the large language model is used to generate corresponding script code; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; The script code is executed to obtain the hardware primitive-level kernel code, and the skeleton code and the hardware primitive-level kernel code are jointly compiled and functionally verified. After the functional verification is passed, the parameter configuration of the skeleton code and the hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses the large language model to infer the optimization action based on the current search state information; Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0006] According to the code automatic generation and optimization method provided by the present invention, the step of analyzing the technical manual corresponding to the target hardware platform information using a large language model to extract target hardware factor information includes: Load pre-set hardware inherent optimization prompts; wherein, the hardware inherent optimization prompts include mapping relationships between various optimization techniques and various hardware influencing factors described in natural language; Input the hardware inherent optimization prompts, the target hardware platform information, and the technical manual into the large language model; The system receives the target hardware factor information output by the large language model; wherein the target hardware factor information includes at least one of memory hierarchy parameters, instruction set characteristic parameters, and register configuration parameters.

[0007] According to the code automatic generation and optimization method provided by the present invention, the various optimization techniques include block optimization, loop reordering optimization, vectorization optimization, data layout optimization and pipeline optimization. The memory hierarchy parameters specifically include the capacity and bandwidth data of each cache level; The instruction set characteristic parameters specifically include the mnemonic names, operand formats, and usage constraints of vector or matrix instructions; The register configuration parameters specifically include the number of vector registers, their bit width, and the data type support.

[0008] According to the present invention, an automatic code generation and optimization method is provided, wherein the step of generating skeleton code using the large language model based on the target hardware factor information, the target operator type, and the operator dimension parameters includes: Construct a first prompt word; wherein the first prompt word includes the target hardware factor information, the target operator type, the operator dimension parameter, and a few-sample code example; Input the first prompt word into the large language model to obtain the skeleton code; The skeleton code implements matrix or tensor traversal through a multi-level nested loop structure and defines block parameters for controlling the loop step size; the kernel function call placeholder is located in the inner layer of the multi-level nested loop structure and is used to call indivisible computation tasks.

[0009] According to the present invention, an automatic code generation and optimization method is provided, wherein generating corresponding script code using the large language model includes: Construct a second prompt word; wherein the second prompt word includes the target hardware factor information, the functional description of the kernel function call placeholder, and the corresponding hardware primitive specification; The second prompt word is input into the large language model, and the script code written in a high-level programming language is generated based on the second prompt word by reasoning using the large language model. The script code defines a print function, which is configured to output assembly instructions or hardware built-in function call instructions containing specific register allocations, in the order of load instructions, compute instructions, and store instructions.

[0010] According to the code automatic generation and optimization method provided by the present invention, the step of jointly compiling and functionally verifying the skeleton code and the hardware primitive-level kernel code includes: If the joint compilation fails or the result of the functional verification does not match the expectation, obtain the compiler error information or the test result difference information. The skeleton code, the script code, and the compiler error information or the test result difference information are fed back to the large language model; The modified skeleton code or modified script code is regenerated using the large language model, and the joint compilation and functional verification are re-executed until the functional verification passes.

[0011] According to the code automatic generation and optimization method provided by the present invention, the step of iteratively optimizing the parameter configuration of the skeleton code and the hardware primitive-level kernel code using a tree search algorithm includes: Initialize the root node of the Monte Carlo tree search; wherein the root node contains the initial parameter configuration and the corresponding performance benchmark value; Multiple rounds of search iterations are performed, each round of search iterations includes a selection phase, an expansion phase, a simulation phase, and a backpropagation phase; In the expansion phase, optimization prompts are constructed; wherein, the optimization prompts include path history information from the root node to the current leaf node and global node statistics. The optimization prompts are input into the large language model, and the set of candidate optimization actions output by the large language model is received. New child nodes are expanded in the search tree based on the set of candidate optimization actions.

[0012] According to the code automatic generation and optimization method provided by the present invention, the path history information includes all optimization action sequences executed on the search path and the performance change magnitude brought about by each optimization action; the optimization actions in the candidate optimization action set include at least one of the following: Adjust the size of the block parameters in the skeleton code; Adjust the microkernel size parameters of the hardware primitive-level kernel code; Adjust the execution order of independent instructions in the hardware primitive-level kernel code to optimize pipeline parallelism.

[0013] According to the code automatic generation and optimization method provided by the present invention, the output includes skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization, comprising: After reaching the preset number of iterations, select the node with the highest cumulative reward or the best average performance index from the search tree; The parameter configuration corresponding to the node is parsed, and the parameter configuration is used as input parameters to execute the script code to regenerate the final hardware primitive-level kernel code.

[0014] The present invention also provides a code automatic generation and optimization device, comprising: The acquisition module is used to acquire natural language prompts input by the user, and parse the natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters. The extraction module is used to analyze the technical manual corresponding to the target hardware platform information using a large language model, and extract the target hardware factor information. The first generation module is used to generate skeleton code based on the target hardware factor information, the target operator type, and the operator dimension parameters using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computation operations or data movement operations; The second generation module is used to generate corresponding script code for each kernel function call placeholder in the skeleton code using the large language model; wherein the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed. An execution module is used to execute the script code to obtain the hardware primitive-level kernel code, and to jointly compile and verify the skeleton code with the hardware primitive-level kernel code. The tuning module is used to iteratively tune the parameter configuration of the skeleton code and the hardware primitive-level kernel code using a tree search algorithm after the function verification is passed; wherein, the iterative tuning process uses the large language model to infer the tuning action based on the current search state information; The output module is used to output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the code automatic generation and tuning method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the code automatic generation and tuning method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the code automatic generation and optimization method as described above.

[0018] This invention provides a method and apparatus for automatic code generation and optimization. It acquires natural language prompts and parses them to obtain target operator type, target hardware platform information, and operator dimension parameters. It then utilizes a large language model to automatically extract target hardware factor information from technical manuals, enabling the large language model to understand the underlying architecture characteristics of the target hardware. Based on this, a layered code generation strategy is employed. First, the large language model generates skeleton code that implements the control flow logic and includes kernel function call placeholders. Then, for each placeholder, executable script code is generated that outputs hardware primitive-level kernel code. This script code acts as a bridge to indirectly implement... This invention enables the automatic generation of hardware primitive-level code, overcoming the difficulty of directly generating correct assembly-level code from large language models. The correctness of the generated code is ensured through joint compilation and functional verification. Furthermore, after successful functional verification, a tree search algorithm, which uses the large language model to infer and optimize actions based on search state information, is employed to iteratively optimize parameter configurations. This achieves automated generation and intelligent optimization of high-performance tensor operator code at the hardware primitive level for different hardware platforms without the need for manually writing hardware-specific optimization rules. This significantly reduces the development cost of cross-platform tensor operators and improves the execution performance of the generated code. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the overall process of the automatic code generation and optimization method provided by the present invention; Figure 2 This is a schematic diagram of the system framework of the hardware architecture understanding module provided by the present invention; Figure 3 This is a schematic diagram of the system framework for skeleton code generation, kernel code generation, and automatic tuning provided by the present invention; Figure 4 A schematic diagram of the structure of the automatic code generation and optimization device provided by the present invention.

[0020] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The code automatic generation and optimization method provided in this invention can be executed by an electronic device with data processing capabilities, such as a server, workstation, personal computer, or cloud computing platform. This electronic device includes a processor and a memory. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), or a combination of both. The memory stores executable instructions, and the processor executes these executable instructions to implement the code automatic generation and optimization method described in this invention.

[0023] In one specific embodiment of the present invention, a processor is described as the execution subject, on which a large language model (LLM) is deployed, and the processor is able to access the target hardware platform or perform compilation and execution operations on the target hardware platform via a remote connection.

[0024] It should be noted that the tensor operators involved in this invention refer to mathematical operations used to process multidimensional arrays (i.e., tensors), including but not limited to matrix multiplication (GEMM) and convolution (Conv) operations. These operations account for the majority of computational load in the inference and training processes of deep learning and large language models. The hardware primitives involved in this invention refer to low-level instructions or interfaces that directly manipulate hardware resources, including but not limited to CPU assembly instructions (such as RISC-V vector extension instructions and ARMNEON instructions) and GPU hardware built-in functions (such as the relevant template functions of CUDA Tensor Cores). These hardware primitives can precisely control hardware resources such as computing units, registers, and memory.

[0025] The large language model involved in this invention is an artificial intelligence model based on deep learning. It learns the statistical relationships of language by training on massive amounts of text data and can perform various natural language processing tasks such as code generation and text understanding.

[0026] Figure 1 This is a schematic diagram of the overall process of the automatic code generation and optimization method provided by the present invention, such as... Figure 1 As shown, the code automatic generation and optimization method provided in this embodiment of the invention includes the following steps: Step S110: Obtain the natural language prompt information input by the user, and parse the target operator type, target hardware platform information, and operator dimension parameters based on the natural language prompt information.

[0027] In this step, the processor obtains a natural language prompt message input by the user through the human-computer interaction interface.

[0028] Natural language prompts refer to code generation requirements described by users in natural language, which contain key information elements for specifying the code generation task.

[0029] The processor parses the natural language prompts and extracts three key types of information: target operator type, target hardware platform information, and operator dimension parameters.

[0030] The target operator type refers to the type of operation of the tensor operator that the user expects to generate, which represents the type of mathematical operation to be implemented. The target operator type can include, but is not limited to, general matrix multiplication (GEMM), convolution operation (Conv), matrix transpose, element-wise operation, etc.

[0031] Target hardware platform information refers to the identification information of the hardware platform on which the generated code is expected to run. It is used to determine the specific hardware architecture that needs to be adapted for subsequent code generation.

[0032] The target hardware platform information may include the name, model, and architecture type of the hardware platform, such as a RISC-V architecture processor, an ARM architecture processor, an NVIDIA GPU, or other types of processors.

[0033] Operator dimension parameters refer to the dimensional information of the data processed by the tensor operator to be generated. They are used to determine the specific size of the matrix or tensor operated on by the operator. For example, for the GEMM operation, operator dimension parameters may include the number of rows m, the number of columns n, and the common dimension k of the matrix.

[0034] Specifically, the processor can perform structured parsing of natural language prompts through methods such as text parsing, keyword extraction, or regular expression matching. It can also utilize large language models to perform semantic understanding and information extraction from the natural language prompts. For example, if a user inputs "RISCV C910 GEMM 102410241024", the processor can parse it to obtain the target operator type as GEMM, the target hardware platform information as RISCV C910, and the operator dimension parameters as m=1024, k=1024, and n=1024.

[0035] Step S120: Analyze the technical manual corresponding to the target hardware platform information using a large language model to extract target hardware factor information.

[0036] After obtaining the target hardware platform information, the processor loads the technical manual corresponding to the target hardware platform information.

[0037] A technical manual is a document that describes the technical characteristics of a target hardware platform, such as its hardware architecture, instruction set, and performance parameters. It can be a technical document that has been collected and stored locally in advance, or it can be a technical document provided by the user.

[0038] The processor uses a large language model to analyze and process the technical manual, extracting information about the target hardware factors.

[0039] Target hardware factor information refers to a set of hardware characteristic parameters extracted from technical manuals that are closely related to code generation and optimization. Target hardware factor information characterizes the key characteristics of the target hardware platform in terms of memory architecture, computing power, register resources, etc., and these characteristics directly affect the selection of code optimization strategies.

[0040] Target hardware factor information may include, but is not limited to: parameter information describing the hardware storage system, information describing the computing instructions supported by the hardware, information describing the hardware register resources, and information describing the hardware computing unit configuration.

[0041] Figure 2 This is a schematic diagram of the system framework of the hardware architecture understanding module provided by the present invention, such as... Figure 2 As shown, when extracting target hardware factor information, the processor can leverage pre-defined background knowledge to guide the large language model to more accurately understand hardware optimization-related concepts and their interrelationships. This allows for the targeted extraction of hardware factor information from technical manuals that is relevant to code generation. The extracted target hardware factor information will serve as crucial input for subsequent skeleton code generation and kernel code generation, guiding code optimization strategies.

[0042] Step S130: Based on the target hardware factor information, target operator type and operator dimension parameters, generate skeleton code using a large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and contains kernel function call placeholders for identifying computation operations or data movement operations.

[0043] After obtaining the target hardware factor information, the processor uses the target hardware factor information, target operator type, and operator dimension parameters as input conditions to generate the skeleton code corresponding to the target operator using a large language model.

[0044] Sketch code refers to the program code that implements the high-level control flow logic of tensor operators, and it is written in a high-level programming language.

[0045] The skeleton code defines the overall computational framework and data flow of the operator, including but not limited to loop structures, data partitioning strategies, loop traversal order, and data layout methods.

[0046] Skeleton code can be written in different programming languages ​​depending on the type of target hardware platform. For example, it can be written in C for CPU platforms and in CUDA C++ for GPU platforms.

[0047] The skeleton code contains kernel function call placeholders. Kernel function call placeholders are reserved positions in the skeleton code to identify computational operations or data movement operations that require calling underlying hardware primitive-level code.

[0048] Each kernel function call placeholder corresponds to an indivisible atomic computation or data movement task, which needs to be implemented efficiently through hardware primitive-level code. For example, kernel function call placeholders can identify data packing operations or core computation operations.

[0049] Figure 3 This is a schematic diagram of the system framework for skeleton code generation, kernel code generation, and automatic tuning provided by the present invention, as shown below. Figure 3As shown, skeleton code generation is located at the upper layer of the layered generation architecture, and it is mainly responsible for implementing high-level optimization strategies such as tiling, reordering, and data layout.

[0050] The skeleton code traverses matrices or tensors in blocks using loop structures and controls the amount of data processed each time through block parameters. Meanwhile, computational cores and data transfer operations that require deep optimization are reserved as placeholders for kernel function calls, which are then replaced by high-performance kernel code at the hardware primitive level generated in subsequent steps.

[0051] Step S140: For each kernel function call placeholder in the skeleton code, generate corresponding script code using the large language model; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed.

[0052] After the skeleton code is generated, the processor uses the large language model to generate script code corresponding to each kernel function call placeholder contained in the skeleton code.

[0053] Script code refers to program code written in a high-level scripting language (such as Python or other interpreted programming languages) that is configured to automatically output hardware primitive-level kernel code for the target hardware platform when executed.

[0054] In this application, the script code is not directly hardware primitive-level code, but a code generator whose execution result is the hardware primitive-level kernel code.

[0055] Hardware primitive-level kernel code refers to code written directly using the low-level instructions or built-in functions of the target hardware platform, which can precisely control hardware resources such as computing units, registers, and memory access.

[0056] For CPU platforms, hardware primitive-level kernel code can be assembly language code or code containing calls to hardware built-in functions (intrinsic functions); for GPU platforms, hardware primitive-level kernel code can be code containing calls to dedicated hardware acceleration units.

[0057] The design of using script code as an intermediate bridge to indirectly generate hardware primitive-level kernel code fully utilizes the ability of large language models to generate high-level programming language code, while avoiding the error problems that are prone to occur when large language models directly generate complex low-level hardware primitive code.

[0058] like Figure 3As shown, kernel code generation is located at the lower layer of the layered generation architecture, and is mainly responsible for implementing instruction-level deep optimizations such as vectorization and pipeline.

[0059] Step S150: Execute the script code to obtain the hardware primitive-level kernel code, and then jointly compile and verify the skeleton code with the hardware primitive-level kernel code.

[0060] The processor executes all the script code generated in the preceding steps, and each script code outputs the corresponding hardware primitive-level kernel code file after execution.

[0061] Subsequently, the processor jointly compiles the skeleton code with all generated hardware primitive-level kernel code files, that is, using a compiler and compilation options adapted to the target hardware platform, to compile and link the skeleton code and all kernel code files into an executable program.

[0062] After successful joint compilation, the processor executes the compiled executable program on the target hardware platform to perform functional verification. Functional verification refers to the process of verifying whether the calculation results of the generated code are consistent with the expected correct results by running preset test cases. Functional verification may include comparing the output results of the generated code with the calculation results of a reference implementation (such as the calculation results of a standard mathematical library) and determining whether the numerical error between the two is within an acceptable range.

[0063] If the joint compilation fails or the functional verification results do not meet expectations, the processor can obtain the corresponding error information or difference information, feed it back to the large language model, drive the large language model to regenerate the corrected code, and repeat the joint compilation and functional verification process until the functional verification passes.

[0064] Step S160: After the functional verification is passed, the parameter configuration of the skeleton code and hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses a large language model to infer the optimization action based on the current search state information.

[0065] After the functional verification is passed, the processor uses a tree search algorithm to iteratively optimize the parameter configurations in the skeleton code and hardware primitive-level kernel code to further improve the code execution performance.

[0066] Parameter configuration refers to the set of adjustable optimization parameters in the skeleton code and hardware primitive-level kernel code. Different combinations of these parameter values ​​can lead to differences in code execution performance. Parameter configuration may include, but is not limited to, block size parameters in the skeleton code, microkernel size parameters in the kernel code, and instruction order.

[0067] Tree search algorithms are algorithms that organize the search process in a tree structure within a decision space. They systematically explore different combinations of parameter configurations to find the optimal configuration scheme for code execution performance. Tree search algorithms can employ Monte Carlo Tree Search (MCTS) or other tree-based search algorithms.

[0068] During iterative tuning, the processor uses a large language model to infer tuning actions based on the current search state information. Search state information refers to information accumulated during the search process that describes the current search progress and historical exploration, and may include attempted parameter configurations and their corresponding performance results, historical records of search paths, etc.

[0069] Tuning refers to a specific modification to the current parameter configuration, such as increasing or decreasing the value of a block parameter or adjusting the execution order of a segment of instructions. Utilizing the semantic understanding and reasoning capabilities of large language models to guide decision-making during the search process can enable the search to converge more efficiently towards optimal performance. Figure 3 The automatic tuning search tree structure is shown on the right.

[0070] In each iteration of optimization, the processor modifies the parameter configuration based on the optimization actions obtained from the large language model inference, regenerates the code based on the modified parameter configuration, and runs the test on the target hardware platform to obtain performance feedback data. Then, based on the performance feedback data, the state information of the search tree is updated to provide a basis for the next iteration.

[0071] Step S170: Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after iterative tuning.

[0072] After iterative tuning is completed, the processor determines the optimal parameter configuration from the search tree, extracts the optimal parameter configuration, and generates the final skeleton code and hardware primitive-level kernel code based on the optimal parameter configuration, which are then output as the final result.

[0073] The output can include the final skeleton code file, hardware primitive-level kernel code file, and performance test report.

[0074] Performance test reports can include performance metrics of the code running on the target hardware platform and the percentage improvement compared to the baseline performance. Users can directly integrate the output code files into their applications.

[0075] This application achieves fully automated generation and optimization of high-performance tensor operator code at the hardware primitive level, from a single line of natural language prompts for the user. This method automatically analyzes hardware technical manuals using a large language model to extract hardware factor information, eliminating the tedious process of manual manual analysis. By dividing code generation into two levels—skeleton code and kernel code—it fully leverages the ability of large language models to generate high-level language code while indirectly achieving deep optimization at the hardware primitive level through script code. Furthermore, by employing a tree search algorithm guided by a large language model for parameter tuning, it achieves intelligent and automated optimization of code performance. The entire process eliminates the need for hardware experts to manually write low-level code, significantly reducing the development threshold and cycle time for high-performance tensor operators, while generating code that approaches or even surpasses the performance level of manually optimized libraries.

[0076] Optionally, a large language model can be used to analyze the technical manual corresponding to the target hardware platform information to extract target hardware factor information, including: Load pre-defined hardware optimization tips; these tips include mappings between various optimization techniques described in natural language and various hardware influencing factors. Input the hardware's inherent optimization prompts, target hardware platform information, and technical manuals into the large language model; Receive target hardware factor information output by the large language model; wherein, the target hardware factor information includes at least one of memory hierarchy parameters, instruction set characteristic parameters, and register configuration parameters.

[0077] In this application, the processor first loads pre-defined hardware-inherent optimization hints from a predefined knowledge base.

[0078] Hardware-intrinsic optimization hints are a set of pre-written and stored hint texts described in natural language. Their core content is to establish a mapping relationship between various optimization techniques and various hardware influencing factors, that is, to explain in natural language which hardware characteristics need to be considered when performing various optimizations on tensor operators.

[0079] In this application, "multiple optimization techniques" refers to various performance optimization strategies commonly used in the implementation and optimization of tensor operators. These strategies improve code execution efficiency from different levels. "Multiple hardware influencing factors" refers to various hardware characteristic dimensions on the hardware platform that are closely related to the code optimization effect. Different hardware characteristic parameters in different dimensions affect different optimization strategies.

[0080] The mapping relationship describes the correspondence between each optimization technique and the hardware factors it depends on in the form of natural language.

[0081] For example, this mapping relationship can be described as follows: the implementation of a certain optimization strategy requires reference to the hardware parameters related to the storage system; Another optimization strategy requires reference to hardware parameters related to instruction sets and register resources. The role of inherent hardware optimization hints is to serve as background knowledge input to the large language model, activating its ability to understand hardware optimization principles and guiding it to selectively extract the most relevant hardware information for code optimization when analyzing technical manuals.

[0082] like Figure 2 As shown, the hardware inherent optimization hints are located on the input side of the hardware architecture understanding module. The left side displays various optimization techniques: block partitioning, loop rearrangement, vectorization, data rearrangement, and pipelined; the middle displays various hardware factors: stream processors, memory architecture, memory access / computation instructions, and vector registers.

[0083] After loading the hardware inherent optimization hints, the processor takes the hardware inherent optimization hints, the target hardware platform information parsed in step S110, and the technical manual corresponding to the target hardware platform as inputs and passes them into the large language model for processing.

[0084] The hardware-inherent optimization hints provide directional guidance for the analysis technical manual of the large language model, the target hardware platform information clarifies the types of hardware platforms that need to be focused on for the large language model, and the technical manual provides the original technical data to be analyzed for the large language model.

[0085] Subsequently, the processor receives the target hardware factor information output by the large language model after analysis and processing. The target hardware factor information is a set of hardware characteristic parameters extracted and structured by the large language model from the technical manual, which includes at least one of the following: memory hierarchy parameters, instruction set characteristic parameters, and register configuration parameters.

[0086] Memory hierarchy parameters refer to parameters that describe the storage architecture of the target hardware platform, including information such as the capacity and bandwidth of each level of cache and memory. These parameters are used to guide the formulation of data partitioning strategies, so that the amount of data processed in each partition can be adapted to the corresponding level of cache capacity, thereby maximizing the cache hit rate.

[0087] Instruction set characteristic parameters are parameters that describe the types, formats, and usage of low-level instructions supported by the target hardware platform. These include information such as the names of vector or matrix instructions, operand formats, and usage constraints. These parameters are used to guide the selection and use of hardware primitive instructions in the kernel code.

[0088] Register configuration parameters are parameters that describe the register resources of a target hardware platform, including the number of registers, bit width, and supported data types. These parameters are used to guide the determination of the microkernel size and the formulation of register allocation strategies.

[0089] like Figure 2 As shown on the right, taking the RISC-V C910 processor as an example, the extracted target hardware information includes: information such as 32KB L1 cache in terms of memory architecture, information such as vle32.v (vector loading instruction) and vfmacc.vv ​​(vector multiplication and accumulation instruction) in terms of memory access / computation instructions, and information such as 32 128-bit long vector registers.

[0090] In this application, the pre-built hardware-inherent optimization hints guide the large language model analysis technical manual, which can selectively extract key hardware characteristic parameters directly related to code optimization. This avoids the tedious process of manually sifting through complex technical manuals for key information, while ensuring the comprehensiveness and accuracy of the extracted hardware factor information, providing a reliable hardware knowledge foundation for subsequent code generation.

[0091] Optionally, various optimization techniques include block optimization, loop reordering optimization, vectorization optimization, data layout optimization, and pipeline optimization; The memory hierarchy parameters specifically include the capacity and bandwidth data of each cache level; The instruction set characteristic parameters specifically include the mnemonic name, operand format, and usage constraints of vector or matrix instructions; The register configuration parameters specifically include the number of vector registers, their bit width, and the data types they support.

[0092] Specifically, various optimization techniques include block optimization, loop reordering optimization, vectorization optimization, data layout optimization, and pipeline optimization.

[0093] Block decomposition optimization refers to decomposing matrices or tensors into smaller data blocks to improve memory access patterns, ensuring that the amount of data processed each time is compatible with cache capacity. Loop reordering optimization refers to swapping the iteration order of multi-level nested loops to improve the spatial and temporal locality of data access, thereby increasing memory access efficiency.

[0094] Vectorization optimization refers to an optimization strategy that packages scalar data into vector data and uses hardware-supported vector instructions to perform simultaneous computation and memory access on multiple data elements.

[0095] Data layout optimization refers to the strategy of rearranging the way matrix or tensor data is stored in memory to better adapt to the hardware's memory access patterns and instruction requirements. Pipeline optimization refers to the strategy of overlapping computational operations and memory access operations to minimize the impact of memory access latency on computational performance.

[0096] The five optimization techniques mentioned above optimize tensor operators from different dimensions, such as data block granularity, loop access order, instruction-level parallelism, data storage arrangement, and operation overlap, covering the complete optimization chain from high-level algorithm logic to low-level instruction orchestration.

[0097] Regarding target hardware factors, the memory hierarchy parameters specifically include the capacity and bandwidth data of each level of cache.

[0098] Caches at various levels include, but are not limited to, L1 cache, L2 cache, and L3 cache.

[0099] Capacity data represents the amount of data that each level of cache can store, while bandwidth data represents the data transfer rate between each level of cache and the processing core or the next level of storage. For example, for a RISC-V C910 processor, the extracted memory hierarchy parameters may include information such as an L1 cache capacity of 32KB and an L2 cache capacity of 1MB.

[0100] Instruction set characteristic parameters specifically include the mnemonic names, operand formats, and usage constraints of vector or matrix instructions. The mnemonic name refers to the textual identifier of the instruction in the assembly language, such as vfmacc.vv ​​(vector floating-point multiply-accumulate instruction in the RISC-V Vector Extensions instruction set) and fmla (floating-point multiply-accumulate instruction in the ARM NEON instruction set).

[0101] Operand format refers to the type, number, and arrangement rules of the operands in an instruction, such as the register types and numbers of the source and destination operands. Usage constraints refer to the restrictions that must be followed when using the instruction, such as operand alignment requirements and rules for using specific registers.

[0102] Register configuration parameters specifically include the number of vector registers, their bit width, and supported data types. The number of vector registers determines the amount of data that can be held in a register simultaneously, directly affecting the microkernel's expansion capability. The bit width determines the number of data elements a single vector register can hold. Data type support indicates the data precision types supported by the register (such as single-precision floating-point numbers FP32, half-precision floating-point numbers FP16, etc.).

[0103] For example, for a RISC-V C910 processor, the extracted register configuration parameters may include information such as 32 128-bit vector registers and support for FP32 and FP64 data types.

[0104] In this application, by refining the definitions of optimization techniques and hardware factor parameters, the large language model can accurately focus on the hardware characteristic dimensions closely related to the implementation of various optimization strategies when analyzing technical manuals, and comprehensively extract hardware factor information covering multiple dimensions such as memory hierarchy, instruction set and registers, providing complete hardware knowledge support for the subsequent formulation of multi-level optimization strategies.

[0105] Optionally, based on target hardware factor information, target operator type, and operator dimension parameters, a large language model is used to generate skeleton code, including: Construct the first prompt word; where the first prompt word includes target hardware factor information, target operator type, operator dimension parameters, and a few-sample code example; Input the first prompt word into the large language model to obtain the skeleton code; The skeleton code implements matrix or tensor traversal through a multi-level nested loop structure and defines block parameters to control the loop step size; the kernel function call placeholder is located in the inner layer of the multi-level nested loop structure and is used to call indivisible computation tasks.

[0106] In this application, the processor first constructs a first cue word for driving the generation of skeleton code for a large language model.

[0107] The first prompt is a structured prompt text that integrates the following information elements: the target hardware factor information extracted in step S120, which informs the large language model of the key characteristics of the target hardware platform so that the large language model can select an appropriate optimization strategy accordingly; the target operator type parsed in step S110, which clarifies the computational logic that the large language model needs to implement; the operator dimension parameters parsed in step S110, which determine information such as loop boundaries and data scale in the skeleton code; and few-shot code examples.

[0108] Few-shot code examples are one or more pre-prepared reference code snippets that demonstrate the writing style, structural organization, and use of kernel function call placeholders for operator-like skeleton code. These snippets are used to guide large language models in generating skeleton code in the desired format and style.

[0109] Few-shot code examples demonstrate the structural paradigm of skeleton code to large language models, enabling them to generate skeleton code with the expected structure through analogical reasoning.

[0110] After constructing the first prompt, the processor inputs the first prompt into the large language model. Based on the instruction requirements, hardware factor information, operator information, and few-shot code examples in the first prompt, the large language model generates skeletal code through reasoning.

[0111] The generated skeletal code has the following structural features: The skeletal code traverses matrices or tensors through a multi-layer nested loop structure. The multi-layer nested loop structure refers to a program structure composed of nested loop statements at multiple levels. The outer loop realizes the macroscopic traversal of the entire matrix or tensor, and the inner loop realizes the microscopic processing of the data blocks after partitioning.

[0112] The skeletal code defines chunk parameters for controlling the loop step size. The chunk parameters are parameter variables that control the step size of each layer of loops in the multi-layer nested loop structure. For example, in the GEMM operator, parameters such as BM, BN, and BK can be defined to control the chunk sizes of the matrix M dimension, N dimension, and K dimension respectively.

[0113] The values of the chunk parameters determine the size of the data blocks processed in each iteration, and their reasonable values directly affect cache utilization efficiency and computing performance.

[0114] The kernel function call placeholder is located inside the multi-layer nested loop structure and is used to call indivisible computing tasks.

[0115] The setting position of the kernel function call placeholder determines the execution timing of the hardware primitive-level kernel code in the entire computing process. Placing it inside the multi-layer nested loop structure enables the kernel code to be called in each loop iteration to perform efficient low-level computing or data transfer operations on the current chunk data.

[0116] As Figure 3 Shown on the left, an example structure of the skeletal code can be: Define the chunk parameter BM = 64, traverse the matrix through a for loop (such as for(int i = 0; i < M; i += BM)), and call kernel function call placeholders such as PACK(A) and COMPUTE(A, B) inside the loop.

[0117] In this application, by constructing the first prompt containing hardware factor information, operator information, and few-shot code examples to guide the large language model to generate skeletal code, the generated skeletal code can fully reflect high-level optimization strategies. At the same time, through the design of the multi-layer nested loop structure and the kernel function call placeholder, a clear separation between high-level control logic and low-level hardware primitive implementation is achieved, providing a structured framework foundation for the subsequent independent generation and parameter tuning of kernel code.

[0118] Optionally, use the large language model to generate corresponding script code, including: Construct a second prompt word; wherein the second prompt word includes target hardware factor information, functional description of kernel function call placeholders, and corresponding hardware primitive specifications; The second cue word is input into the large language model, and the large language model is used to generate script code written in a high-level programming language based on the reasoning of the second cue word. The script code defines a print function, which is configured to output assembly instructions or hardware built-in function call instructions containing specific register allocations, in the order of load instructions, compute instructions, and store instructions.

[0119] In this application, the processor constructs a second prompt word for each kernel function call placeholder in the skeleton code to drive the large language model to generate the corresponding script code.

[0120] The second prompt is a structured prompt text that integrates the following information elements: target hardware factor information extracted in step S120, especially instruction set characteristic parameters and register configuration parameters related to instruction-level optimization, used to inform the large language model of the hardware primitive instructions and register resources that can be used; the functional description of the currently targeted kernel function call placeholder, that is, the natural language description of the computation operation or data movement operation represented by the kernel function call placeholder, including the input data, output data and the functional logic to be implemented; and the corresponding hardware primitive specification, that is, the detailed specification of the low-level instructions available on the target hardware platform, including the instruction syntax, operand requirements, execution effect and other information.

[0121] For example, if the kernel function call placeholder is the `PACK` function, the function description indicates that the function needs to pack matrix data of a specific dimension into a contiguous memory region according to a specified layout; if the kernel function call placeholder is the `COMPUTE` function, the function description indicates that the function needs to perform multiplication and addition operations on two input matrix blocks. The hardware primitive specification provides the instruction reference information needed to generate correct hardware primitive-level code for the large language model.

[0122] After constructing the second cue word, the processor inputs the second cue word into the large language model, which then infers and generates script code written in a high-level programming language (such as Python) based on the hardware factor information, functional description, and hardware primitive specifications in the second cue word.

[0123] The generated script code defines a print function. A print function is a function in the script code used to output hardware primitive-level instruction text in a predetermined format and order.

[0124] The print function is configured to output assembly instructions or hardware built-in function call instructions, including specific register allocations, in the order of load instructions, compute instructions, and store instructions.

[0125] Load instructions refer to low-level instructions that read data from memory or shared memory into registers, such as the vle32.v instruction on the RISC-V platform.

[0126] Computation instructions are low-level instructions that perform operations on data in registers, such as the vfmacc.vv ​​instruction on the RISC-V platform.

[0127] Store instructions are low-level instructions that write the calculation results in a register back to memory, such as the vse32.v instruction on the RISC-V platform.

[0128] The script code outputs the instruction sequence according to the standard load-compute-store structure through the print function, and performs specific register number allocation and instruction parameter filling during the output process.

[0129] like Figure 3 As shown in the middle section, the script code is written in Python (for example, it contains a for c in range(0,nr) loop and print statements such as f_asm.write("vle32.v C0[c], ..."). The kernel code output after execution includes loading instructions such as vle32.v in the PACK section and calculation instructions such as vfmacc.vv ​​in the COMPUTE section.

[0130] The print function in the script code can accept microkernel parameters as input parameters, and dynamically control the number of output instructions and register allocation scheme based on these parameters.

[0131] This parameterized design allows the same script code to output kernel code of different sizes under different parameter configurations, providing flexibility for subsequent parameter tuning.

[0132] In this application, a second prompt word containing functional descriptions and hardware primitive specifications is constructed to guide a large language model to generate script code. The script code defines a print function that outputs low-level instructions with register allocation in the order of load-compute-store. This cleverly utilizes the advantage of large language models in generating high-level language code. By using the script code as an intermediate bridge, deep optimization at the hardware primitive level is indirectly achieved, effectively solving the technical problem that large language models are prone to errors when directly generating low-level assembly code.

[0133] Optionally, the skeleton code and hardware primitive-level kernel code are jointly compiled and functionally verified, including: If the joint compilation fails or the results of the functional verification do not match expectations, obtain the compiler error message or the test result difference information. The skeleton code, script code, compiler error messages, or test result discrepancies are fed back to the large language model. The corrected skeleton code or script code is regenerated using a large language model, and joint compilation and functional verification are re-executed until the functional verification passes.

[0134] Specifically, during the joint compilation process, if the compiler detects syntax errors, type mismatches, linking errors, or other issues leading to compilation failure, or if during functional verification after successful joint compilation, the processor finds that the calculation results obtained from running test cases are inconsistent with the expected correct results (e.g., numerical errors exceed the allowable range), then the processor obtains the corresponding error feedback information. When joint compilation fails, the processor obtains compiler error information, including the error type, error location (source file name and line number), error description, and error message provided by the compiler. When the functional verification results do not match expectations, the processor obtains test result difference information, including specific differences between the expected and actual outputs, such as the location of mismatched data elements, the magnitude of numerical deviations, and the overall calculation accuracy.

[0135] After receiving the error feedback information, the processor inputs the current skeleton code, the corresponding script code, and the compiler error information or test result difference information into the large language model as feedback information.

[0136] By providing the complete code context and specific error information to the large language model simultaneously, the large language model can perform targeted analysis and correction based on a full understanding of the code intent and the cause of the error.

[0137] Subsequently, based on the received code and error feedback information, the large language model analyzes the cause of the error and regenerates the corrected skeleton code or the corrected script code. The correction can be to fix logical errors, data indexing errors, etc. in the skeleton code, or to fix instruction usage errors, register allocation conflicts, etc. in the script code.

[0138] The processor re-executes the joint compilation and functional verification on the corrected code. If compilation errors or functional verification failures occur again, the above feedback and correction process is repeated, forming an iterative correction loop, until joint compilation succeeds and functional verification passes. Figure 1 As shown, after judging whether the code is correct in step 4, if the result is "no", the process returns to step 3 to regenerate the skeleton code and kernel code.

[0139] In this application, an automatic error correction capability is achieved in the code generation process through the feedback correction mechanism of the compiler error information and test result difference information, ensuring the functional correctness of the final output code. This feedback correction mechanism makes full use of the code understanding and error diagnosis capabilities of the large language model, which can automatically locate and fix defects in the code, avoiding the tedious process of manual debugging.

[0140] Optionally, a tree search algorithm is used to iteratively optimize the parameter configurations of the skeleton code and the hardware primitive-level kernel code, including: Initialize the root node of the Monte Carlo tree search; the root node contains the initial parameter configuration and the corresponding performance baseline value; Multiple rounds of search iterations are performed, each round of search iterations includes a selection phase, an expansion phase, a simulation phase, and a backpropagation phase; During the expansion phase, optimization prompts are constructed; these prompts include historical path information from the root node to the current leaf node and global node statistics. Input the tuning prompts into the large language model, receive the set of candidate tuning actions output by the large language model, and expand new child nodes in the search tree based on the set of candidate tuning actions.

[0141] In this application, specifically, the processor first initializes the root node of the Monte Carlo Tree Search (MCTS). Monte Carlo Tree Search is a heuristic search algorithm that explores the decision space by repeatedly performing four steps—selection, expansion, simulation, and backpropagation—on the search tree. It is suitable for finding optimal or near-optimal solutions in large-scale discrete search spaces.

[0142] The root node is the starting node of the search tree, representing the initial state of the current skeleton code and hardware primitive-level kernel code before tuning. The root node contains the initial parameter configuration, that is, the current set of values ​​for all adjustable parameters in the skeleton code and hardware primitive-level kernel code, such as the initial values ​​of each block parameter (e.g., BM, BN, BK) in the skeleton code and the initial values ​​of microkernel size parameters (e.g., mr, nr, vl) in the kernel code.

[0143] The root node also contains a performance benchmark value corresponding to the initial parameter configuration. This benchmark value is the performance metric (such as GFLOPS or TFLOPS) measured by actually running the compiled code on the target hardware platform under this initial parameter configuration. This performance benchmark value serves as a reference for evaluating the performance improvement during subsequent tuning. The root node's access count is initialized to a preset value (e.g., 1), and the cumulative reward is initialized to a preset value (e.g., 0), which is used for node selection calculations during the subsequent search process.

[0144] After the root node is initialized, the processor executes multiple rounds of search iterations. Each round of search iterations includes a selection phase, an expansion phase, a simulation phase, and a backpropagation phase in sequence.

[0145] During the selection phase, the processor starts from the root node and selects child nodes layer by layer down the search tree from the existing nodes until it reaches a leaf node (i.e., a node that has not yet been fully expanded). During the selection process, the processor can use the Upper Confidence Bound (UCB) formula or other node selection strategies to balance the utilization of existing high-reward paths with the exploration of unexplored paths.

[0146] During the expansion phase, the processor constructs tuning prompts. Tuning prompts are structured text messages used to convey the current search state information to the large language model, driving it to infer tuning actions.

[0147] The tuning prompts contain path history information from the root node to the current leaf node, that is, information records on the complete path from the root node of the search tree to the current leaf node. The path history information reflects the sequence of tuning operations and their effects in order to reach the current state, providing context for causal reasoning for large language models.

[0148] The optimization suggestions also include global node statistics, which is a summary of statistical data for all expanded nodes in the search tree. Global node statistics reflect the overall exploration of the search space, providing reference information from a global perspective for large language models and helping them discover optimization patterns and trends across paths.

[0149] like Figure 3 As shown on the right, the search tree starts from the root node S0 and forms multiple search paths (S0→S1→S6→S7, S0→S2→S3, S0→S2→S4→S5, etc.) through multiple iterations. The state space can be represented by the current values ​​of each parameter, and the action space can be represented by the adjustment operations on the parameters. The tuning prompts encode the path history and global statistics in these search trees into text forms that can be understood by the large language model.

[0150] The processor inputs tuning prompts into the large language model, which analyzes and infers based on path history information and global node statistics, and outputs a set of candidate tuning actions.

[0151] The candidate tuning action set is a set of tuning actions with optimization potential recommended by the large language model based on the current search state. Each tuning action corresponds to a specific modification scheme for the current parameter configuration.

[0152] The processor selects one or more tuning actions from the set of candidate tuning actions, and creates a new child node at the current leaf node of the search tree based on the selected tuning action. Each newly created child node contains the new parameter configuration obtained after executing the corresponding tuning action.

[0153] During the simulation phase, the processor regenerates the code based on the parameter configuration corresponding to the newly created child nodes in the extension phase (i.e., it uses the parameter configuration to execute script code to generate kernel code and update the parameters in the skeleton code), and actually compiles and runs it on the target hardware platform to obtain the actual performance indicators under the parameter configuration.

[0154] During the backpropagation phase, the processor propagates the performance feedback information obtained in the simulation phase (such as the performance improvement as a reward value) layer by layer along the search path from the child node to the root node, updating the cumulative reward and access count of all nodes on the path.

[0155] Through the above implementation methods, a systematic search framework utilizing the Monte Carlo tree search algorithm, combined with the intelligent decision-making capabilities of the large language model in the expansion phase, achieves efficient and automatic tuning of parameter configurations. The large language model can perform deep reasoning based on path history information and global node statistics, understanding the interrelationships between different parameter adjustments, thereby recommending tuning actions with high optimization potential. This significantly improves search efficiency, enabling the finding of near-optimal parameter configurations within a limited number of search iterations.

[0156] Optionally, the path history information includes all tuning action sequences executed on the search path and the performance change magnitude of each tuning action; the tuning actions in the candidate tuning action set include at least one of the following: Adjust the size of the block parameters in the skeleton code; Adjust the microkernel size parameters of the hardware primitive-level kernel code; Adjust the execution order of independent instructions in the hardware primitive-level kernel code to optimize pipeline parallelism.

[0157] Specifically, path history information includes all optimization action sequences executed along the search path and the performance change magnitude of each optimization action. All optimization action sequences record the complete operation trajectory from the root node to the current leaf node, and the performance change magnitude of each optimization action quantifies the actual impact of each optimization action on code performance. By correlating these two, the large language model can identify the parameters with the most significant performance impact, thus prioritizing the adjustment of key parameters in subsequent decisions.

[0158] The tuning actions in the candidate tuning action set include at least one of the following: adjusting the size of the block parameters in the skeleton code (such as increasing or decreasing the granularity of BM, BN, or BK); adjusting the microkernel size parameters of the hardware primitive-level kernel code (such as changing the values ​​of mr, nr, or vl); and adjusting the execution order of independent instructions in the hardware primitive-level kernel code to optimize pipeline parallelism (such as interleaving load instructions and compute instructions to achieve parallel execution in the processor pipeline).

[0159] The three types of tuning actions mentioned above cover high-level optimization parameters at the skeleton code level and low-level optimization parameters at the kernel code level, respectively, enabling the tuning process to be carried out collaboratively in multiple dimensions.

[0160] By refining path history information into a record of the correlation between optimization actions and performance changes, and clarifying optimization actions into three levels: block parameter adjustment, microkernel size adjustment, and instruction sequence adjustment, the large language model can make refined inference decisions on multiple optimization dimensions, achieving collaborative optimization of the skeleton code layer and the kernel code layer.

[0161] Optionally, the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after iterative tuning are output, including: After reaching the preset number of iterations, select the node with the highest cumulative reward or the best average performance index from the search tree; The parameter configuration corresponding to the node is parsed, and the parameter configuration is used as input parameters to execute script code to regenerate the final hardware primitive-level kernel code.

[0162] In an optional implementation, the process of configuring the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameters after iterative tuning of the output in step S170 can be further refined.

[0163] Specifically, after reaching a preset number of iterations, the processor traverses all nodes in the search tree, selecting the node with the highest cumulative reward or the best average performance metric as the final result. The processor parses the complete parameter configuration information from the selected optimal node, uses the parameters related to kernel code generation as input, and executes the script code to regenerate the final hardware primitive-level kernel code. Since the script code is a parameterized code generator, passing in the optimal parameter configuration will deterministically generate the kernel code version with the best performance.

[0164] The processor synchronously updates the parameter values ​​in the skeleton code, and after compilation and verification, outputs the final skeleton code and hardware primitive-level kernel code as the result.

[0165] By selecting the optimal node from the search tree and re-executing the script code based on its parameter configuration, the final output code is ensured to correspond to the globally optimal parameter configuration found during the search process, transforming the tuning results into high-performance code artifacts that can be used directly.

[0166] The automatic code generation and optimization apparatus provided by the present invention is described below. The automatic code generation and optimization apparatus described below can be referred to in correspondence with the automatic code generation and optimization method described above.

[0167] Figure 4 This is a schematic diagram of the structure of the automatic code generation and optimization device provided by the present invention, such as... Figure 4 As shown, it includes: The acquisition module 410 is used to acquire natural language prompt information input by the user, and parse the natural language prompt information to obtain the target operator type, target hardware platform information and operator dimension parameters. Extraction module 420 is used to analyze the technical manual corresponding to the target hardware platform information using a large language model to extract target hardware factor information; The first generation module 430 is used to generate skeleton code based on the target hardware factor information, the target operator type, and the operator dimension parameters using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computation operations or data movement operations; The second generation module 440 is used to generate corresponding script code for each kernel function call placeholder in the skeleton code using the large language model; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; Execution module 450 is used to execute the script code to obtain the hardware primitive-level kernel code, and to jointly compile and verify the skeleton code with the hardware primitive-level kernel code. The tuning module 460 is used to iteratively tune the parameter configuration of the skeleton code and the hardware primitive-level kernel code using a tree search algorithm after the function verification is passed; wherein, the iterative tuning process uses the large language model to infer the tuning action based on the current search state information; The output module 470 is used to output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0168] In this application, by acquiring and parsing natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters, a large language model is used to automatically extract target hardware factor information from technical manuals. This enables the large language model to understand the underlying architecture characteristics of the target hardware. Based on this, a layered code generation strategy is adopted. First, the large language model generates skeleton code that implements the control flow logic and includes kernel function call placeholders. Then, for each placeholder, executable script code is generated that outputs hardware primitive-level kernel code. The script code acts as a bridge to indirectly achieve the automatic generation of hardware primitive-level code, solving the difficulty of the large language model directly generating correct assembly-level code. The correctness of the generated code is ensured through joint compilation and functional verification. After the functional verification is passed, a tree search algorithm, inferring optimization actions based on the search state information by the large language model, is used to iteratively optimize the parameter configuration. Thus, without the need for manually writing hardware-specific optimization rules, the automatic generation and intelligent optimization of high-performance tensor operator code at the hardware primitive level for different hardware platforms are achieved. This significantly reduces the development cost of cross-platform tensor operators and improves the execution performance of the generated code.

[0169] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an automatic code generation and optimization method, which includes: acquiring natural language prompts input by the user, and parsing the natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters. The technical manual corresponding to the target hardware platform information is analyzed using a large language model to extract target hardware factor information; Based on the target hardware factor information, the target operator type, and the operator dimension parameters, skeleton code is generated using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computational operations or data movement operations; For each kernel function call placeholder in the skeleton code, the large language model is used to generate corresponding script code; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; The script code is executed to obtain the hardware primitive-level kernel code, and the skeleton code and the hardware primitive-level kernel code are jointly compiled and functionally verified. After the functional verification is passed, the parameter configuration of the skeleton code and the hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses the large language model to infer the optimization action based on the current search state information; Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0170] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the code automatic generation and optimization methods provided by the above methods, the method including: obtaining natural language prompt information input by the user, and parsing the natural language prompt information to obtain the target operator type, target hardware platform information and operator dimension parameters; The technical manual corresponding to the target hardware platform information is analyzed using a large language model to extract target hardware factor information; Based on the target hardware factor information, the target operator type, and the operator dimension parameters, skeleton code is generated using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computational operations or data movement operations; For each kernel function call placeholder in the skeleton code, the large language model is used to generate corresponding script code; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; The script code is executed to obtain the hardware primitive-level kernel code, and the skeleton code and the hardware primitive-level kernel code are jointly compiled and functionally verified. After the functional verification is passed, the parameter configuration of the skeleton code and the hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses the large language model to infer the optimization action based on the current search state information; Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0172] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the code automatic generation and optimization methods provided by the above methods, the method comprising: acquiring natural language prompt information input by a user, and parsing the natural language prompt information to obtain target operator type, target hardware platform information, and operator dimension parameters; The technical manual corresponding to the target hardware platform information is analyzed using a large language model to extract target hardware factor information; Based on the target hardware factor information, the target operator type, and the operator dimension parameters, skeleton code is generated using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computational operations or data movement operations; For each kernel function call placeholder in the skeleton code, the large language model is used to generate corresponding script code; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; The script code is executed to obtain the hardware primitive-level kernel code, and the skeleton code and the hardware primitive-level kernel code are jointly compiled and functionally verified. After the functional verification is passed, the parameter configuration of the skeleton code and the hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses the large language model to infer the optimization action based on the current search state information; Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic code generation and optimization, characterized in that, include: Obtain natural language prompts input by the user, and parse the natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters; The technical manual corresponding to the target hardware platform information is analyzed using a large language model to extract target hardware factor information; Based on the target hardware factor information, the target operator type, and the operator dimension parameters, skeleton code is generated using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computational operations or data movement operations; For each kernel function call placeholder in the skeleton code, the corresponding script code is generated using the large language model; wherein, the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed; The script code is executed to obtain the hardware primitive-level kernel code, and the skeleton code and the hardware primitive-level kernel code are jointly compiled and functionally verified. After the functional verification is passed, the parameter configuration of the skeleton code and the hardware primitive-level kernel code is iteratively optimized using a tree search algorithm; wherein, the iterative optimization process uses the large language model to infer the optimization action based on the current search state information; Output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

2. The method according to claim 1, characterized in that, The analysis of the technical manual corresponding to the target hardware platform information using a large language model extracts target hardware factor information, including: Load pre-set hardware inherent optimization prompts; wherein, the hardware inherent optimization prompts include mapping relationships between various optimization techniques and various hardware influencing factors described in natural language; Input the hardware inherent optimization prompts, the target hardware platform information, and the technical manual into the large language model; The system receives the target hardware factor information output by the large language model; wherein the target hardware factor information includes at least one of memory hierarchy parameters, instruction set characteristic parameters, and register configuration parameters.

3. The method according to claim 2, characterized in that, The various optimization techniques include block optimization, loop reordering optimization, vectorization optimization, data layout optimization, and pipeline optimization; The memory hierarchy parameters specifically include the capacity and bandwidth data of each cache level; The instruction set characteristic parameters specifically include the mnemonic names, operand formats, and usage constraints of vector or matrix instructions; The register configuration parameters specifically include the number of vector registers, their bit width, and the data type support.

4. The method according to claim 1, characterized in that, The step of generating skeleton code using the large language model based on the target hardware factor information, the target operator type, and the operator dimension parameters includes: Construct a first prompt word; wherein the first prompt word includes the target hardware factor information, the target operator type, the operator dimension parameter, and a few-sample code example; Input the first prompt word into the large language model to obtain the skeleton code; The skeleton code implements matrix or tensor traversal through a multi-level nested loop structure and defines block parameters for controlling the loop step size; the kernel function call placeholder is located in the inner layer of the multi-level nested loop structure and is used to call indivisible computation tasks.

5. The method according to claim 1, characterized in that, The process of generating corresponding script code using the large language model includes: Construct a second prompt word; wherein the second prompt word includes the target hardware factor information, the functional description of the kernel function call placeholder, and the corresponding hardware primitive specification; The second prompt word is input into the large language model, and the script code written in a high-level programming language is generated based on the second prompt word by reasoning using the large language model. The script code defines a print function, which is configured to output assembly instructions or hardware built-in function call instructions containing specific register allocations, in the order of load instructions, compute instructions, and store instructions.

6. The method according to claim 1, characterized in that, The joint compilation and functional verification of the skeleton code and the hardware primitive-level kernel code includes: If the joint compilation fails or the result of the functional verification does not match the expectation, obtain the compiler error information or the test result difference information. The skeleton code, the script code, and the compiler error information or the test result difference information are fed back to the large language model; The modified skeleton code or modified script code is regenerated using the large language model, and the joint compilation and functional verification are re-executed until the functional verification passes.

7. The method according to claim 1, characterized in that, The iterative optimization of parameter configurations for the skeleton code and the hardware primitive-level kernel code using a tree search algorithm includes: Initialize the root node of the Monte Carlo tree search; wherein the root node contains the initial parameter configuration and the corresponding performance benchmark value; Multiple rounds of search iterations are performed, each round of search iterations includes a selection phase, an expansion phase, a simulation phase, and a backpropagation phase; In the expansion phase, optimization prompts are constructed; wherein, the optimization prompts include path history information from the root node to the current leaf node and global node statistics. The optimization prompts are input into the large language model, and the set of candidate optimization actions output by the large language model is received. New child nodes are expanded in the search tree based on the set of candidate optimization actions.

8. The method according to claim 7, characterized in that, The path history information includes all tuning action sequences executed on the search path and the performance change magnitude of each tuning action; the tuning actions in the candidate tuning action set include at least one of the following: Adjust the size of the block parameters in the skeleton code; Adjust the microkernel size parameters of the hardware primitive-level kernel code; Adjust the execution order of independent instructions in the hardware primitive-level kernel code to optimize pipeline parallelism.

9. The method according to claim 1, characterized in that, The output, after the iterative optimization, corresponds to the skeleton code and hardware primitive-level kernel code for the optimal parameter configuration, including: After reaching the preset number of iterations, select the node with the highest cumulative reward or the best average performance index from the search tree; The parameter configuration corresponding to the node is parsed, and the parameter configuration is used as input parameters to execute the script code to regenerate the final hardware primitive-level kernel code.

10. A code automatic generation and optimization device, characterized in that, include: The acquisition module is used to acquire natural language prompts input by the user, and parse the natural language prompts to obtain the target operator type, target hardware platform information, and operator dimension parameters. The extraction module is used to analyze the technical manual corresponding to the target hardware platform information using a large language model, and extract the target hardware factor information. The first generation module is used to generate skeleton code based on the target hardware factor information, the target operator type, and the operator dimension parameters using the large language model; wherein, the skeleton code is used to implement the control flow logic of the operator and includes kernel function call placeholders for identifying computation operations or data movement operations; The second generation module is used to generate corresponding script code for each kernel function call placeholder in the skeleton code using the large language model; wherein the script code is configured to output hardware primitive-level kernel code for the target hardware platform when executed. An execution module is used to execute the script code to obtain the hardware primitive-level kernel code, and to jointly compile and verify the skeleton code with the hardware primitive-level kernel code. The tuning module is used to iteratively tune the parameter configuration of the skeleton code and the hardware primitive-level kernel code using a tree search algorithm after the function verification is passed; wherein, the iterative tuning process uses the large language model to infer the tuning action based on the current search state information; The output module is used to output the skeleton code and hardware primitive-level kernel code corresponding to the optimal parameter configuration after the iterative optimization.

Images