Code generation method and device and related product

Abstract

The invention provides a code generation method, and the method can obtain a first program code, and the first program code is a code facing a first accelerator; converting the first program code into a second program code based on a generative pre-training model; the second program code is a code oriented to a second accelerator.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a code generation method, device and related products. Background Art

[0002] As computing demands grow, deep learning accelerators are increasingly being used. However, deep learning accelerators typically feature high parallelism, complex memory structures, and specialized instruction set architectures. Programmers often struggle to grasp the hardware architecture and instruction set characteristics of each deep learning accelerator, making programming for them challenging. Summary of the Invention

[0003] Based on existing technologies, this application provides a code generation method, device, and related products that can assist programmers in programming deep learning accelerators, thereby simplifying programming difficulty and improving programming efficiency.

[0004] The present disclosure provides a code generation method, the method comprising:

[0005] Obtaining a first program code, where the first program code is a code for a first accelerator;

[0006] Based on the generative pre-trained model, the first program code is converted into a second program code; the second program code is a code for a second accelerator.

[0007] The present disclosure also provides a code generation device, which includes a memory and a processor. The memory is used to store a computer program, and the processor implements the method when executing the computer program in the memory.

[0008] The present disclosure also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the method described above is implemented.

[0009] The present disclosure also provides a computer-readable storage medium for storing a computer program or instruction. When the computer program or instruction is executed by a processor, the method described above is implemented.

[0010] The code generation method, device, and related products provided by the present disclosure automatically convert a first program code into a functionally equivalent second program code by using a generative pre-training model (such as GPT), so that the first program code can be transplanted to a second accelerator for execution. Programmers no longer need to rewrite the second program code based on the programming model and instruction set of the second accelerator, thereby reducing programming difficulty and greatly improving programming efficiency. Based on the code generation method provided by the embodiments of the present disclosure, even if programmers do not fully understand and master the hardware architecture and instruction set of all processors, especially in the case where accelerators generally have high parallelism, complex storage structures, and special instruction set architectures, programmers can also complete accelerator-oriented programming. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a programming model of a first accelerator provided in one embodiment;

[0012] Figure 2 A schematic diagram of a programming model for a second accelerator provided in one embodiment;

[0013] Figure 3 1 is a flow chart of a code generation method in one embodiment;

[0014] Figure 4 Schematic diagram of a flow chart of a code correction method in one embodiment;

[0015] Figure 5 A schematic flow chart of a code generation method according to another embodiment;

[0016] Figure 6 To adopt Figure 5 A schematic diagram of a code conversion process of the code generation method of the illustrated embodiment;

[0017] Figure 7 1 is a flow chart of an automatic tuning method according to an embodiment;

[0018] Figure 8 FIG. 4 is a flow chart of an automatic tuning method in another embodiment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0020] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a," "an," "the," and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should be understood that the term "and / or" as used herein is simply a term used to describe the existence of three possible relationships between related objects. For example, "A and / or B" can represent the existence of A alone, the existence of both A and B, and the existence of B alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0022] It should be understood that although the terms "first," "second," and "third" may be used in the embodiments of the present disclosure to describe preset ranges, etc., these preset ranges should not be limited to these terms. These terms are merely used to distinguish one preset range from another. For example, without departing from the scope of the embodiments of the present disclosure, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0023] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0024] As computing demands grow, deep learning accelerators are increasingly being used. However, deep learning accelerators typically feature high parallelism, complex memory structures, and specialized instruction set architectures. Programmers often struggle to grasp the hardware architecture and instruction set characteristics of each deep learning accelerator, making programming for them challenging.

[0025] To reduce the burden on programmers in heterogeneous programming models, source-to-source compilation offers a viable solution. Existing source-to-source compilation methods typically include rule-based and symbolic synthesis. The rule-based approach requires programming experts to manually set a series of code conversion rules applied to abstract syntax trees between different programming languages. The system then uses pattern matching to parse and compile the input source program. For example, a translator used to convert CUDA (Compute Unified Device Architecture) code to FPGA (Field Programmable Gate Array) code defines a series of conversion rules for data communication, computational optimization, and parallel mapping. Symbolic synthesis can generate semantically preserved target code from domain-specific languages ​​(DSLs). However, because symbolic synthesis relies on a search-based SMT (Satisfiability Modulo Theories) solver, which is slow to run and has high code conversion costs, symbolic synthesis is difficult to apply to large-scale program code compilation. However, since deep learning accelerators (DLA) usually have complex hardware architectures and programming models, the above-mentioned compilation technology is difficult to apply to deep learning accelerators. Among them, the programming model is an abstract description of the processor hardware architecture, which may include the processor's control mode, computing resources, and storage structure, etc., which helps programmers to develop and utilize the processor and write program code, and maximize the use of the processor's hardware resources. The storage structure consists of storage units with different speeds and sizes. If programmers want to use a processor to implement specific operations, they need to have a certain understanding of the processor's hardware architecture and its programming method. For example, when programmers program for the first accelerator, they need to be familiar with the programming model and programming rules of the first accelerator; and when programmers program for the second accelerator, they need to understand the programming model and programming rules of the second accelerator, which leads to lower programming efficiency.

[0026] Based on this, the present disclosure aims to provide a code generation method for deep learning accelerators, which can realize program code conversion between different accelerators, thereby reducing the programming difficulty of the accelerator and improving the programming efficiency of programmers. Among them, deep learning accelerators are accelerator engines for deep learning operations, which include but are not limited to graphics processing units (GPUs) containing tensor processor cores, tensor processing units (TPUs), and various intelligent processors such as IPUs (Intelligence Processing Units) or NPUs (Neural-network Processing Units).

[0027] For example, the first accelerator can be a GPU. At the software level, the GPU is a multi-threaded programming model, such as Figure 1 As shown, the first accelerator can realize thread parallelism (Thread) and thread block parallelism (Thread Block), multiple threads can form a thread block, and multiple thread blocks can form a grid. The storage structure of the first accelerator may include multiple storage units (not shown) such as registers, shared memory, and global memory; wherein each thread can access registers or local storage, each thread block can access shared memory, and each grid can access global memory. The instruction set of the first accelerator may include matrix multiplication and accumulation instructions, synchronization instructions, data access instructions, etc. The program code obtained based on the programming model and programming rules of the first accelerator is recorded as the first program code. For example, the first program code may be CUDAC code.

[0028] The second accelerator may be an IPU having multiple processor cores, and the programming model of the second accelerator may be as follows: Figure 2 As shown. The second accelerator includes multiple processor cores, and the multiple processor cores can constitute at least one cluster (Cluster). The multiple processor cores in each cluster can execute tasks in parallel, and the multiple clusters can also execute tasks in parallel. Therefore, the second accelerator can achieve inter-core parallelism and also achieve inter-cluster parallelism. The storage structure of the second accelerator may include on-chip storage units of each processor core (including neuron storage units and weight storage units, not shown in the figure), shared storage units SRAM and global storage units DRAM and other storage units. The instruction set of the second accelerator may include matrix operation instructions, convolution instructions, fully connected instructions, data access instructions and synchronization instructions, etc. The program code obtained based on the programming model and programming rules of the second accelerator can be recorded as the second program code.

[0029] Since the first accelerator and the second accelerator have completely different programming models and instruction sets, the first program code written for the first accelerator is difficult to apply to the second accelerator. For example, the program code written for the GPU cannot be applied to the second accelerator (such as IPU).

[0030] like Figure 3 As shown, the present disclosure provides a code generation method that can realize automatic conversion of program code between a first accelerator and a second accelerator, thereby assisting programmers in program writing and improving programming efficiency. The method may include:

[0031] S310: Obtain a first program code, where the first program code is a code for a first accelerator;

[0032] The first accelerator may be a graphics processing unit (GPU). The program code for the first processor may be program code written based on the programming model and instruction set of the first accelerator. The programming model and programming rules of the first accelerator may be recorded in documents such as its programming manual and instruction set manual.

[0033] The first program code can be a program code written in a high-level programming language (such as C language) based on a programming model of the first accelerator. For example, the first program code is CUDAC code. The first program code may include built-in parallel variables constructed according to the parallel characteristics of the first accelerator, tensor functions constructed according to the storage structure and instruction set of the first accelerator, and storage units. Among them, built-in variables refer to standard data types and attributes for program execution provided by the compiler. For example, the first accelerator can implement thread block parallelism and thread parallelism, and the first program code may include built-in parallel variables blockIdx and threadIdx. The first program code may include tensor operation primitives such as nvcuda::wmma, and may also include storage units such as registers (__register__), shared memory (__shared__), and global memory (__global__).

[0034] S320: Based on the generative pre-trained model, convert the first program code into a second program code; the second program code is a code for the second accelerator.

[0035] The second program code is a code for the second accelerator, which can be a tensor processing unit (TPU) and various intelligent processors, such as IPU (Intelligence Processing Unit) or NPU (Neural-network Processing Unit). Figure 2Taking the IPU shown as an example, the second program code may include built-in parallel variables constructed according to the parallel characteristics of the second accelerator, tensor functions constructed according to the storage structure and instruction set of the second accelerator, and storage units, etc. Among them, built-in variables refer to standard data types and attributes provided by the compiler for program execution. For example, the second accelerator can realize inter-core parallelism and inter-cluster parallelism, and the second program code may include built-in parallel variables coreId, clusterId. The second program code may include tensor operation primitives such as __bang_conv_, __bang_mlp_, and may also include storage units such as neuron storage unit (__nram__), weight storage unit (__wram__), shared storage unit (__shared__) and global storage unit (__device__).

[0036] A generative pre-trained transformer (GPT) model is a large language model (LLM), an artificial neural network based on the Transformer model. This model is pre-trained using a dataset and can be used for inference operations.

[0037] In the embodiment of the present disclosure, the generative pre-training model can be used for the automatic conversion of program code to realize source code to source code compilation. Wherein, the first program code can be used as the input of the generative pre-training model, and the generative pre-training model automatically converts the first program code into the second program code, the first program code and the second program code have different representations, and the first program code and the second program code have the same function. Specifically, the generative pre-training model can automatically convert the representation of functions, variables (such as storage units, parallel variables), control flows and operation instructions (including tensor operation primitives) in the first program code into corresponding representations in the second program code.

[0038] The code generation method of the embodiment of the present disclosure automatically converts the first program code into a functionally equivalent second program code by using a generative pre-training model (such as GPT), so that the first program code can be transplanted to the second accelerator for execution. The programmer no longer needs to rewrite the second program code according to the programming model and instruction set of the second accelerator, thereby reducing the programming difficulty and greatly improving the programming efficiency. Based on the code generation method provided by the embodiment of the present disclosure, even if the programmer does not fully understand and master the hardware architecture and instruction set of all processors, especially in the case where accelerators generally have high parallelism, complex storage structures and special instruction set architectures, the programmer can also complete accelerator-oriented programming.

[0039] Optionally, in order to improve the conversion accuracy from the first program code to the second program code, the method of the embodiment of the present disclosure can also train an open source generative pre-training model through programming rules such as the programming model and instruction set of the first program code and the second accelerator, so as to use the generative pre-training model to convert the first program code into the second program code.

[0040] Optionally, the method of the embodiment of the present disclosure may first convert the first program code into an intermediate program code, and then convert the intermediate program code into a second program code according to the programming model and programming rules of the second accelerator. The intermediate program code has a higher level of abstraction than the first program code or the second program code. For example, the intermediate program code may be a program code written in standard C or Python. The specific code conversion process can be described below.

[0041] Further optionally, due to the large differences in the programming models and programming rules of the first accelerator and the second accelerator, the first program code and the second program code have large differences in representation and programming rules. Therefore, in the process of converting the above-mentioned first program code to the second program code based on the generative pre-training model, there may be errors, resulting in a low accuracy rate of conversion between program codes. Based on this, the code generation method of the embodiment of the present disclosure can also detect and correct the second program code output by the above-mentioned generative pre-training model to ensure that the generated second program code can run normally, improve the accuracy rate of program code conversion and compilation efficiency, etc.

[0042] like Figure 4 As shown, the code generation method disclosed in the present invention may also include the following steps:

[0043] S410. Detecting whether the second program code output by the generative pre-training model is correct through a detection module; wherein the detection module is used to detect whether the semantics in the second program code are correct.

[0044] If the detection module determines that the second program code output by the generative pre-trained model is incorrect, for example, if the detection module determines that the second program code contains semantically incorrect program statements, step S420 is executed to correct the second program code, and then the process returns to the above detection step until the detection module determines that the second program code is correct. If the detection module determines that the second program code is correct, there is no need to correct the second program code output by the code generation module.

[0045] Optionally, when the detection module determines that there is a semantic error in the second program code, the programmer can manually check and modify the second program code. Optionally, when the detection module determines that there is a semantic error in the second program code, the code generation method of the present disclosure can also automatically correct the second program code through a code correction module.

[0046] For example, the code correction module may include an SMT solver and a code modification module. The SMT solver may be used to determine the constraints during the conversion process from the first program code to the second program code, so as to ensure the semantic correctness of the program code conversion process through the constraints, so that the semantics of the first program code is equivalent to the semantics of the second program code. The code modification module may be used to automatically modify the second program code based on the optimal constraints output by the SMT solver.

[0047] In one embodiment, the code generation method of the present disclosure first converts the first program code into an intermediate program code, and then converts the intermediate program code into a second program code. Figure 5 As shown, the code generation method of the embodiment of the present disclosure may include:

[0048] S510: Obtain a first program code. The first program code is a code for a first accelerator. For details, please refer to the description of the above embodiment.

[0049] S520: Based on the generative pre-trained model, convert the first program code into an intermediate program code; wherein the intermediate program code may be a program code written in a high-level programming language (such as standard C or Python), and the intermediate program code may be a serial code that supports scalar operations. According to the level of abstraction of the processor hardware, the intermediate program code has a higher level of abstraction than the first program code and the second program code.

[0050] During the code generation process, the first program code can be input into the generative pre-training model as input data, and the generative pre-training model can automatically output the intermediate program code corresponding to the first program code. The intermediate program code implements the same computing function as the first program code, and the intermediate program code is a code that does not contain any accelerator hardware information.

[0051] S530: Convert the intermediate program code into a second program code. The second program code includes programming rules such as parallel variables that match the parallel characteristics of the second accelerator, storage units that are compatible with the storage structure of the second accelerator, and operation primitives (such as tensor operation primitives) that are compatible with the instruction set of the second accelerator. The code generation method disclosed herein can convert the intermediate program code into the second program code based on the programming rules of the second program code.

[0052] Optionally, when converting the first program code into the intermediate program code, issues such as the parallelism, storage structure, and instruction set of different processors need to be considered. The embodiment of the present application can convert the first program code into the intermediate program code based on the generative pre-trained model and the programming manual related to the first program code. Specifically, the step of converting the first program code into the intermediate program code can include at least one of the following:

[0053] Serialization step: Based on the generative pre-training model, automatically identify the parallel variables in the first program code, convert the parallel variables of the first program code into serialized serial code (such as loop code) to reconstruct the parallel operation of the first program code through the loop code. Specifically, the generative pre-training model can identify the parallel variables in the first program code by querying the programming manual related to the first accelerator, and determine the meaning of the parallel variable in combination with the description of the parallel variable, thereby converting the parallel variables in the first program code into equivalent serial code. Among them, the programming manual of the first accelerator may include the programming model of the first accelerator, the programming rules of the first program code, and the programming rules of the first program code include but are not limited to: the built-in variables and functions of the first program code, the storage unit used by the first program code, the operation primitives of the first program code (such as tensor operation primitives, synchronization primitives, etc.) and the programming interface of the first program code. For example, the parallel variables in the first program code may include threadId and BlockId. The serialization step can convert the parallel variables threadId and BlockId in the first program code into equivalent for loop code based on the generative pre-training model.

[0054] The first storage conversion step: based on the generative pre-trained model, automatically identify the storage space in the first program code, and convert the storage space in the first program code into the storage space of the intermediate program code; wherein, the storage space in the first program code can be the storage space involved in at least one program instruction in the first program code. Specifically, the generative pre-trained model can identify the storage space in the first program code and the computing operations related to the corresponding storage space by querying the programming manual related to the first accelerator (for example, the instruction set manual of the first accelerator), and convert the storage space involved in the relevant computing operations (such as data read / write operations) in the first program code into the storage space of the intermediate program code, thereby realizing the storage conversion step.

[0055] De-tensorization: Based on the generative pre-trained model, tensor operation primitives in the first program code are automatically identified and converted into scalar operation statements in the intermediate program code. Primitives are the smallest components of a computer language, which may include operations and variables. Here, tensor operation primitives are used to represent tensor-related computer language primitives.

[0056] Since the first accelerator supports tensor operations, while the intermediate program code only supports scalar operations, in the process of converting the first program code into the intermediate program code, it is necessary to perform an inverse tensorization step to convert a single tensor operation primitive in the first program code into at least one scalar operation statement in the equivalent intermediate program code. Specifically, the generative pre-training model can automatically identify the tensor operation primitive in the first program code, and obtain the operation meaning of the tensor operation primitive according to the programming manual of the first program code, thereby converting the tensor operation primitive in the first program code into an equivalent scalar operation statement, and the intermediate program code has the same operation function as the tensor operation primitive. Among them, the tensor operation primitive refers to the virtual instruction for performing tensor operations in the first program code. Tensor operations include but are not limited to one-dimensional vector operations, two-dimensional matrix operations or higher-dimensional data operations. The tensor operation primitive uses the tensor data of the corresponding dimension as the smallest unit to read, write and operate data.

[0057] For example, the tensor operation primitive in the first program code is nvcuda::wmma, which is used to complete the matrix acceleration operation of D=A*B+C. The tensor calculation primitive includes three variables: matrix_a, matrix_b, and accumulator, wherein matrix_a is used to represent the first matrix A, matrix_b is used to represent the second matrix B, and accumulator is used to represent the source address (address of operand C) or destination address (address of operand D) of the addition operation. Based on the above-mentioned generative pre-training model, the above-mentioned tensor operation primitive nvcuda::wmma can be converted into an intermediate program code, wherein the intermediate program code implements the bitwise multiplication of each element in the first matrix A and the second matrix B respectively through a loop body (such as a for loop) to obtain a product matrix; then, the bitwise addition of each element in the product matrix and the operand C is implemented through the loop body, and the operation result is written into the operand D.

[0058] Further optionally, when converting the first program code into the intermediate program code, the code generation method of the embodiment of the present disclosure may sequentially perform the aforementioned serialization step, the first storage conversion step, and the inverse tensor quantization step. In other optional embodiments, the aforementioned serialization step, the first storage conversion step, and the inverse tensor quantization step may also be performed in any order, without specific limitation.

[0059] Furthermore, optionally, during the process of converting the first program code into the intermediate program code, the code generation method of the present disclosure may further include, through a detection module, detecting and correcting the program codes outputted by the serialization step, the storage conversion step, and the inverse tensor quantization step, respectively, to ensure the accuracy of the conversion from the first program code to the intermediate program code. Specifically, the detection module may be used to detect whether the codes outputted by each of the aforementioned steps contain errors. If the detection module determines that the code outputted by any of the aforementioned steps is incorrect, the code correction module may be used to correct the incorrect intermediate program code.

[0060] Optionally, the code generation method of the embodiment of the present disclosure can convert the intermediate program code into the second program code based on the generative pre-training model. The above-mentioned step of converting the intermediate program code into the second program code also includes at least one of a parallelization step, a second storage conversion step, and a tensorization step. Further, optionally, in the process of converting the intermediate program code into the second program code, the code generation method of the embodiment of the present disclosure can also detect and correct the above-mentioned parallelization step, the second storage conversion step, and the tensorization step respectively to ensure the accuracy of the conversion from the intermediate program code to the second program code.

[0061] in,

[0062] Parallelization step: Based on the generative pre-training model, according to the parallel characteristics of the second accelerator, the serial code in the intermediate program code is converted into parallel variables in the second program code, so as to maximize the utilization of the second accelerator. The parallel characteristics of the second accelerator may refer to the parallel mode of the second accelerator. For example, the parallel characteristics of the second accelerator include inter-core parallelism and inter-cluster parallelism. The parallel variables of the second program code are built-in variables of the second program code, which may include coreId and clusterId. The code generation method disclosed in the present invention can bind the code segments obtained by splitting to the corresponding parallel variables coreId or clusterId respectively, thereby realizing the parallel processing of the intermediate program code to the second program code. For ease of explanation, the embodiment of the present disclosure records the intermediate program code that has been parallelized as the third program code. Specifically, the parallelization step may include:

[0063] Loop recovery sub-step: Based on the generative pre-trained model, the serial code of the intermediate program code is converted into loop code (such as a for loop) to prepare for the subsequent loop reordering sub-step, loop fusion sub-step, and loop splitting sub-step.

[0064] Loop rearrangement sub-step: Based on the generative pre-trained model, the nesting order of the loop code or the iteration order of the loop variables in the loop code (for example, the iteration order of the loop variables i, j, and k in the loop code) is automatically changed. Loop rearrangement can reduce data access and improve computing efficiency.

[0065] The loop fusion substep automatically merges multiple independent loops into a single loop based on a generative pre-trained model. This reduces loop overhead, increases secondary accelerator utilization, or accelerates computations through vectorized instructions. To further improve program code efficiency, this loop fusion substep can also perform compilation optimizations such as redundant code removal and constant folding on the fused loop.

[0066] Loop Splitting Sub-Step: Based on the generative pre-trained model, the loop code that has undergone the loop rearrangement sub-step and / or loop fusion sub-step is split into at least one code segment, and each of the at least one code segments is bound to a parallel variable in the second program code. To further improve the program code's execution efficiency, the loop splitting sub-step may also perform compilation optimizations such as redundant code removal or constant folding on the split code segments, and then bind each code segment to a parallel variable in the second program code.

[0067] Furthermore, the detection module can be used to detect whether the third program code obtained through the parallelization step is correct. If the detection module determines that there is a semantic error in the third program code, the third program code can be modified through the code correction module. The code correction module may include an SMT solver and a code modification module. The SMT solver is used to ensure that each sub-step in the parallelization step meets the constraint conditions to ensure that the code semantics before and after the parallelization step are executed remain consistent. The code modification module is used to automatically correct the third program code. Specifically,

[0068] In the loop recovery sub-step, the SMT solver needs to ensure that the bounds of the loop code are equal to the bounds of the parallel variables in the second program code.

[0069] In the loop reordering sub-step, the SMT solver needs to ensure that the loop execution condition (Loop condition) of the loop code remains unchanged before and after the loop reordering sub-step is executed, and the order of statements in the loop code remains unchanged.

[0070] In the loop fusion sub-step, the SMT solver needs to ensure that there is no dependency between the loop codes to be fused, and the iteration space of a single loop code to be fused is the same as the iteration space of the loop code after fusion.

[0071] In the loop splitting sub-step, the SMT solver should at least ensure the following constraints:

[0072] The actual index value of the loop code before splitting can be obtained according to the loop variable of at least one code segment;

[0073] At least one code segment has the same semantics as the loop code before the split;

[0074] The range and stride of at least one code segment match the range and stride in the loop code before splitting.

[0075] The SMT solver can determine the corresponding optimal solution according to the above constraints, and the code modification module can modify the second program code output by the generative pre-training model based on the optimal solution.

[0076] In the embodiments of the present disclosure, the above-mentioned parallelization steps are implemented through a generative pre-training model, which not only ensures the correctness of the program conversion, but also takes into account the abstract characteristics of different accelerators, thereby improving the versatility of the above-mentioned code conversion.

[0077] The second storage conversion step: based on the generative pre-training model, convert the storage space in the intermediate program code into the storage space in the second program code. Specifically, the storage conversion step can automatically identify the storage space in the second program code based on the instruction set manual of the second accelerator and the generative pre-training model; identify the data access operations (such as read operations and / or write operations) in the intermediate program code based on the generative pre-training model, such as identifying the location of the data access operations in the program code in the intermediate program code based on the generative pre-training model, and convert the storage space involved in the data access operations into the storage space of the second program code. For ease of explanation, the embodiment of the present disclosure records the program code obtained after the second storage conversion step as the fourth program code.

[0078] Furthermore, in order to ensure the correctness of the storage conversion from the intermediate program code to the second program code, the detection module can be used to detect whether the fourth program code is correct, and when the fourth program code is incorrect, the fourth program code can be automatically corrected through the code correction module. For example, the data access operation in the program code can be processed through the code correction module to bind the variables involved in the data access operation to the storage unit of the second program code.

[0079] For example, the storage space of the second accelerator may include neuron storage units, weight storage units, shared storage units, and global storage units. The storage conversion step may be based on the generative pre-training model and the instruction set manual of the second accelerator to determine whether the neuron storage unit or the weight storage unit can be used as an input storage unit, and whether the neuron storage unit can be used as an output storage unit. Furthermore, based on the generative pre-training model, read operations and / or write operations in the intermediate program code may be searched, and the storage units involved in the read operation may be replaced with neuron storage units or weight storage units, and the storage units involved in the write operation may be replaced with neuron storage units.

[0080] Tensorization step: Based on the generative pre-trained model, the scalar operation statements in the intermediate program code are converted into tensor operation primitives in the second program code. Specifically, the tensorization step can automatically detect tensorizable code snippets in the intermediate program code based on the generative pre-trained model, where the tensorizable code snippets include at least one scalar operation statement; according to the programming manual related to the second program code and the generative pre-trained model, the tensorizable code snippets are automatically converted into corresponding tensor operation primitives. Primitives refer to the smallest component of a computer language, which can include operations and variables, etc.

[0081] For example, the tensor operation primitives of the second program code may include __bang_conv, __bang_mlp, etc., and the above tensor operation primitives can realize matrix multiplication and addition operations. In this way, the tensor quantization step can automatically divide the intermediate program code into at least one tensorizable code fragment based on the generative pre-training model, and at the same time, the semantics and functions of the tensor operation primitives of the second program code can be input into the generative pre-training model, and the generative pre-training model can convert at least one tensorizable code fragment in the intermediate program code into corresponding tensor operation primitives according to the semantics and functions of the tensorizable code fragments and the semantics and functions of the tensor operation primitives of the second program code. For ease of explanation, the embodiment of the present disclosure records the program code obtained after the second storage conversion step as the fifth program code.

[0082] Furthermore, the code generation method disclosed herein can also detect and correct the fifth program code. Specifically, the code generation method disclosed herein can determine whether the fifth program code is correct through a detection module; if the detection module determines that there are erroneous tensor operation primitives in the fifth program code, for example, the tensor operation primitives in the fifth program code contain erroneous parameters, the method disclosed herein can correct the erroneous tensor operation primitives in the fifth program code based on the grammar-guided synthesis paradigm to ensure the correctness of the tensor quantization step.

[0083] Optionally, the SMT solver of the code correction module can also convert the tensorizable code snippet into the tensor operation primitive of the second program code based on the syntax-guided synthesis paradigm. Among them, the SMT solver can regard the tensor operation primitive in the second program code as a domain-specific language (DSL) and regard the above-mentioned tensorization step as a small-scale program synthesis. The code translator can search for a target intermediate representation equivalent to the tensor operation primitive in the second program code in the intermediate representation it provides (such as METALIFT-IR(G)), so that the target intermediate representation is semantically equivalent to the tensorizable code snippet. In this way, the code modification module of the code correction module can realize the conversion of the candidate code snippet into the corresponding correct tensor operation primitive. If the tensorizable code snippet cannot be proved to be equivalent, it means that there is no suitable tensor instruction. At this time, the original scalar representation can be retained to ensure the correct function of the program.

[0084] Optionally, the multiple steps included in the above-mentioned code generation method can be divided into at least one Pass, wherein Pass is a structured technology developed by the compiler to complete functions such as conversion, analysis or optimization of compiled objects. The execution of Pass is the process of the compiler converting, analyzing and optimizing the compiled object. For example, the above-mentioned serialization step and parallelization step can be divided into one Pass to realize the serial-parallel conversion between different program codes. The above-mentioned first storage conversion step and the second storage conversion step can be divided into one Pass to realize the storage unit conversion between different program codes. The above-mentioned tensorization step and the inverse tensorization step can be divided into one Pass to realize the conversion of operation primitives between different program codes. The above-mentioned Passes can be executed in sequence to realize the conversion from the first program code to the second program code.

[0085] like Figure 6 As shown, the code generation method of the embodiment of the present disclosure can sequentially perform the above-mentioned parallelization steps ( Figure 6 The program code shown on the left), the second storage conversion step ( Figure 6 The program code shown in the middle) and the tensor quantization step ( Figure 6 In other optional embodiments, the parallelization step, the second storage conversion step, and the tensor quantization step may be performed in any order, which is not specifically limited here.

[0086] like Figure 6In the parallelization step of the program code shown on the left, the loop recovery sub-step and the loop splitting sub-step are first executed in sequence based on the generative pre-training model. Then, the generated program code is input into the code correction module based on the SMT solver for detection and correction. Finally, the corrected code segment is bound to the parallel variables coreId and clusterId of the second program code based on the generative pre-training model.

[0087] like Figure 6 The program code shown in the middle, in the second storage conversion step, first identifies the storage units involved in the second program code based on the generative pre-training model and the instruction set manual of the second accelerator (for example, data is read in using the neuron storage unit nram and the weight storage unit wram, and data is written out using the neuron storage unit nram). Then, the read operations and write operations in the intermediate program code can be automatically detected based on the generative pre-training model, and the read operations and write operations in the program code are processed through the code correction module. Finally, the storage units in the second program code are bound based on the generative pre-training model to implement the second storage conversion step.

[0088] like Figure 6 In the program code shown on the right, in the tensorization step, the tensor-quantifiable code snippets are first automatically detected based on the generative pre-training model. As shown in the figure, there are four tensor-quantifiable code snippets, namely, a code snippet for reading in B_wram, a code snippet for reading in A_wram, a code snippet for writing in C, and a tensor operation code snippet. Afterwards, based on the generative pre-training model and the programming manual of the second program code, the above-mentioned tensor-quantifiable code snippets are converted into tensor operation primitives in the second program code. For example, the code snippet for reading in B_wram is converted into float B_wram, the code snippet for reading in A_wram is converted into floatA_nram, the code snippet for writing in C is converted into float C_nram, and the tensor operation code snippet is converted into _bang_mlp(). Since there are errors in the above conversion process, the above-mentioned erroneous tensor operation primitives can also be corrected by the code correction module to obtain the converted second program code.

[0089] The present disclosure can also Figure 6 The code generation method shown in the figure is compared with other code generation methods, and the results are shown in the following table:

[0090]

[0091] Among them, the first program code (Source) is CUDAC, and the second program code can be BANGC, Hip or standard C code. It can be seen that when using Figure 6When the generative pre-training model and code correction module are combined (i.e., Mardus in the table), the compilation efficiency (Compilation Accuracy) and computational efficiency (Computation Accuracy) are both high. When code generation is performed based on the generative pre-training model (i.e., GPT-4Zero-shot, GPT-4Few-shot in the table), the compilation efficiency and computational efficiency are relatively high. Figure 6 As shown, it is relatively low.

[0092] In one embodiment, the above-mentioned first program code can also be a serial program code written in standard C language or Python language, and the code generation method can convert the first program code into a second program code for the second accelerator based on a generative pre-training model. The code conversion process may include one or more of the above-mentioned parallelization step, the second storage conversion step, and the tensorization step. Furthermore, the method disclosed in the present invention can also detect and correct the above-mentioned various conversion steps. The code conversion process can specifically refer to the conversion process from the intermediate program code to the second program code in the above-mentioned embodiment, which will not be repeated here.

[0093] In an optional embodiment, the code generation method of the present disclosure may also convert the intermediate program code into the second program code by a compiler adapted for the second accelerator. The compiler supports the intermediate program code and the second program code, and is capable of converting the intermediate program code into the second program code and further converting the second program code into binary instructions that can be run by the second accelerator.

[0094] As a further improvement, the code generation method of the embodiment of the present disclosure may further include an automatic tuning process to ensure the running performance of the second program code. Figure 7 As shown, the method of the embodiment of the present disclosure may further include:

[0095] S710: Obtain a first program code, wherein the first program code is a code for a first accelerator. As described in the above example, the first program code may be a program code written for a GPU.

[0096] S720: Based on the generative pre-trained model, convert the first program code into multiple candidate program codes; wherein the multiple candidate program codes are codes for the second accelerator. In the method of the disclosed embodiment, the process of converting the intermediate program code into the candidate program code based on the generative pre-trained model is substantially the same as the process of converting the intermediate program code into the second program code in the above embodiment. For details, please refer to the above embodiment.

[0097] S730. Select the optimal program code from multiple candidate program codes, and use the optimal program code as the second program code. Optionally, the automatic tuning method of the embodiment of the present disclosure can use a beam search method to find the optimal program code from multiple candidate program codes, and use the optimal program code as the second program code. In other embodiments, the automatic tuning method of the present disclosure can also use Xgboost (eXtreme gradient boosting, distributed gradient enhancement) search method, brute-force search method (brute force search method) and other search methods to find the optimal program code from multiple candidate program codes to determine the second program code. Among them, the search method can be determined based on the characteristics of the second program code and the number of candidate program codes, and is not specifically limited here.

[0098] Further optionally, the number of the above-mentioned candidate program codes may be too large, thereby affecting the efficiency of the above-mentioned automatic tuning. Based on this, the automatic tuning method of the embodiment of the present disclosure can also evaluate the performance of the above-mentioned multiple candidate program codes through a generative pre-training model, and sort them according to the performance corresponding to the multiple candidate program codes, so that relatively poor candidate program codes can be deleted, and only k (k is a positive integer greater than or equal to 1) candidate program codes with better performance rankings are retained. In this way, during the automatic tuning process, the method of the present disclosure can use a beam search method to find the optimal program code from the k candidate program codes, and use the optimal program code as the second program code.

[0099] Further optionally, the above automatic tuning method can be applied to various steps in the process of converting the first program code to the second program code, including but not limited to the process of converting the first program code to the intermediate program code and the process of converting the intermediate program code to the second program code.

[0100] In one embodiment, optionally, the automatic tuning process can be a separate Pass, which can interact with the above-mentioned Passes to achieve automatic tuning and optimization of each step in the code generation process. Figure 8 As shown, the automatic tuning method may further include:

[0101] S810: Obtain a first program code, wherein the first program code is a code for a first accelerator. As described in the above example, the first program code may be a program code written for a GPU.

[0102] S820: Based on the generative pre-trained model, convert the first program code into a plurality of first candidate program codes; wherein the plurality of first candidate program codes are represented in the same manner as the intermediate program code, for example, the plurality of first candidate program codes are program codes written in a high-level programming language (such as standard C or Python), and the first candidate program codes are serial codes including scalar operation statements. The plurality of first candidate program codes may have at least one different adjustment parameter, and the adjustment parameter may vary according to different stages of the program code conversion.

[0103] S830: Select an optimal program code from the plurality of first candidate program codes, and use the optimal program code as an intermediate program code.

[0104] Further optionally, the number of the above-mentioned first candidate program codes may be too large, thereby affecting the efficiency of the above-mentioned automatic tuning. Based on this, the automatic tuning method of the embodiment of the present disclosure can also evaluate the performance of the above-mentioned multiple first candidate program codes through a generative pre-training model, and sort them according to the performance corresponding to the multiple first candidate program codes. In this way, relatively poor candidate program codes can be deleted, and only m (m is a positive integer greater than or equal to 1) first candidate program codes with better performance rankings are retained. Furthermore, the above-mentioned method can select the optimal program code with the best performance from the m first candidate program codes, and use the optimal program code as the intermediate program code.

[0105] S840. Based on the generative pre-training model, convert the intermediate program code into a plurality of second candidate program codes; wherein the plurality of second candidate program codes are codes for the second accelerator. In the method of the embodiment of the present disclosure, the process of converting the intermediate program code into the candidate program code based on the generative pre-training model is basically the same as the process of converting the intermediate program code into the second program code in the above embodiment. For details, please refer to the above embodiment. Optionally, the plurality of second candidate program codes have at least one different adjustment parameter, and the adjustment parameter varies according to different stages of the program code conversion.

[0106] S850. Select the optimal program code from multiple second candidate program codes, and use the optimal program code as the second program code. The automatic tuning method of the embodiment of the present disclosure can use a beam search method to find the optimal program code from multiple second candidate program codes, and use the optimal program code as the second program code. In other embodiments, the automatic tuning method of the present disclosure can also use Xgboost (eXtreme gradient boosting, distributed gradient enhancement) search method, brute-force search method (brute force search method) and other search methods to find the optimal program code from multiple candidate program codes to determine the second program code. Among them, the search method can be determined based on the characteristics of the second program code and the number of candidate program codes, and is not specifically limited here.

[0107] Further optionally, the number of the above-mentioned second candidate program codes may be too large, thereby affecting the efficiency of the above-mentioned automatic tuning. Based on this, the automatic tuning method of the embodiment of the present disclosure can also evaluate the performance of the above-mentioned multiple second candidate program codes through a generative pre-training model, and sort them according to the performance corresponding to the multiple second candidate program codes, so that relatively poor candidate program codes can be deleted, and only n (n is a positive integer greater than or equal to 1) second candidate program codes with better performance rankings are retained. In this way, during the automatic tuning process, the method of the present disclosure can use a beam search method to find the optimal program code from the n second candidate program codes, and use the optimal program code as the second program code.

[0108] Optionally, the adjustment parameters of the above-mentioned multiple first candidate program codes and multiple second candidate program codes may at least include: one or more of: the number of basic blocks (Blocks) of each logical loop, the size (size) of each basic block, the identification of parallelizable loops, the order of logical loops, and the data movement method.

[0109] In one embodiment, the present disclosure further provides a code generation device, the code generation device including a memory and a processor, the memory being used to store a computer program, and the processor implementing the method of any of the above embodiments when executing the computer program in the memory. Optionally, the above computer program can be used to implement conversion between program codes, and the computer program is divided into at least one Pass according to its function, wherein a Pass is a structured technology developed by a compiler for completing functions such as conversion, analysis, or optimization of compiled objects. The execution of a Pass is the process by which the compiler converts, analyzes, and optimizes the compiled object.

[0110] Specifically, the code generation device can be used to execute the above-mentioned at least one Pass to implement the following method steps:

[0111] A first program code is obtained, where the first program code is a code for a first accelerator; for example, the first program code is a program code for a GPU.

[0112] Based on the generative pre-trained model, the first program code is converted into a second program code; the second program code is a code for a second accelerator, for example, the second program code is a program code for an IPU.

[0113] Because the first accelerator and the second accelerator have different programming models and programming rules, the first program code and the second program code have different code representations. The disclosed embodiment can implement the conversion between the first program code and the second program code using a generative pre-training model (such as GPT), reducing programming difficulty and improving programming efficiency.

[0114] In one embodiment, the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, implements the method of any of the above embodiments. For details, please refer to the above method embodiments.

[0115] In one embodiment, the present disclosure further provides a computer-readable storage medium for storing a computer program or instructions, which, when executed by a processor, implements the method of any of the above embodiments. For details, please refer to the above method embodiments.

[0116] Among them, the storage medium can be a computer-readable storage medium, such as ferroelectric random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or compact disk read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0117] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A code generation method, characterized in that: The method comprises: Obtaining a first program code, where the first program code is a code for a first accelerator; Based on the generative pre-trained model, the first program code is converted into a second program code; the second program code is a code for a second accelerator.

2. The method according to claim 1, characterized in that Based on the generative pre-trained model, converting the first program code into a second program code further includes: Based on the generative pre-trained model, converting the first program code into an intermediate program code; The intermediate program code is converted into the second program code.

3. The method according to claim 2, characterized in that The converting the first program code into an intermediate program code based on the generative pre-trained model includes at least one of the following: Automatically detecting parallel variables in the first program code based on a generative pre-trained model, and converting the parallel variables in the first program code into serial codes corresponding to the intermediate program code; Automatically detecting a storage space in the first program code based on a generative pre-trained model, and converting the storage space in the first program code into a storage space corresponding to the intermediate program code; Automatically detect tensor operation primitives in the first program code based on a generative pre-trained model, and convert the tensor operation primitives in the first program code into scalar operation statements corresponding to the intermediate program code.

4. The method according to claim 2 or 3, characterized in that The converting the intermediate program code into the second program code further includes at least one of the following: Based on the generative pre-trained model, converting serial codes in the intermediate program code into parallel variables in the second program code; Based on the generative pre-trained model, converting the storage space in the intermediate program code into the storage space in the second program code; Based on the generative pre-trained model, scalar operation statements in the intermediate program code are converted into tensor operation primitives in the second program code.

5. The method according to claim 4, characterized in that The serial code in the intermediate program code is represented by a loop; The converting the serial code in the intermediate program code into parallel variables in the second program code based on the generative pre-trained model further includes: Based on the generative pre-trained model, splitting the intermediate program code into at least one code segment according to the parallel characteristics of the second accelerator; The at least one code segment is respectively bound to the parallel variables of the second program code.

6. The method according to claim 4, characterized in that The converting, based on the generative pre-trained model, the scalar operation statements in the intermediate program code into tensor operation primitives in the second program code further includes: Detecting a tensorizable code snippet capable of performing tensorization in the intermediate program code based on the generative pre-trained model; wherein the tensorizable code snippet includes at least one scalar operation statement; According to a programming manual associated with the second program code, the tensorizable code snippet is converted into a corresponding tensor operation primitive.

7. The method according to claim 4, characterized in that The converting the storage space in the intermediate program code into the storage space of the second program code based on the generative pre-trained model further includes: automatically identifying storage space in the second program code based on an instruction set manual of the second accelerator and the generative pre-trained model; Based on the generative pre-trained model, computing operations in the intermediate program code are identified, and storage space involved in the computing operations is converted into storage space of the second program code.

8. The method according to any one of claims 4 to 7, characterized in that: The method further comprises: detecting whether the second program code is correct by a detection module; If the second program code is incorrect, the second program code is corrected.

9. The method according to claim 2, characterized in that The intermediate program code is a program code written in Python; and the converting the intermediate program code into the second program code further includes: The intermediate program code is compiled into the second program code by a compiler.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Based on a generative pre-trained model, converting the first program code into a plurality of candidate program codes; wherein the plurality of candidate program codes are codes for the second accelerator; An optimal program code is selected from the plurality of candidate program codes, and the optimal program code is used as the second program code.

11. A code generating device, characterized in that: The device includes a memory and a processor, the memory is used to store a computer program, and when the processor executes the computer program in the memory, the method according to any one of claims 1 to 10 is implemented.

12. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when the computer program or instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program or instructions, and when the computer program or instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images