Cross-platform program translation method and device

Abstract

The invention provides a cross-platform program translation method and device, and the method comprises the steps: obtaining a source program used for running on a source platform, converting a parallel built-in variable in the source program into a serial program based on a loop instruction through a large language model, and carrying out the deep learning of the hardware performance of an accelerator according to a target platform, cyclically segmenting the serial program and binding the serial program to a target platform through a built-in parallel variable of the target platform to obtain an input code; checking the storage use condition of the input code, and decorating the storage of the input code according to a preset storage level to obtain an input program; tensor operation existing in the input program is restored into scalar calculation, a tensor operation instruction of the target platform is used for expressing the scalar calculation again, a target program used for running on the target platform is obtained, the deep learning accelerator of the target platform runs the target program, and a running result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of computer software and compilers, and in particular to a cross-platform program translation method, device, storage medium, computer program product and electronic device. Background Art

[0002] As the computing demands of neural network workloads continue to grow, deep learning accelerators (DLAs) have received great attention recently. In fact, some common deep learning accelerator DLAs (e.g., NVIDIA GPUs with Tensor Cores, Google TPUs, GraphCore IPUs, and Cambricon MLUs) have been deployed in data centers of cloud and Internet service companies such as Microsoft, Google, and Amazon. However, despite the widespread deployment of these devices in the industry, there are two major problems with such heterogeneous data centers: (1) programming difficulty. DLAs are particularly difficult to program due to their extremely high parallelism, complex memory hierarchy, and special instruction set architecture (ISA); (2) low hardware utilization. The overall hardware utilization of heterogeneous data centers is usually low because programs (deep learning programs, intelligent applications, kernel functions, kernel functions) written for one platform (deep learning accelerator) cannot be directly run on another platform, even if the platform is not currently occupied. Ideally, if we only need to write a program once and run it on different DLAs, that is, to achieve "write once, run everywhere", then the programming difficulties and utilization issues of heterogeneous DLAs in data centers can be solved.

[0003] To solve this problem, source-to-source program translation (transcompilation) has become an attractive solution. An ideal transcompilation system can achieve the goal of "write once, run everywhere", that is, developers only need to write code once and it can run on different DLA platforms without manual adjustment and optimization for each hardware. However, despite the important practical significance of this goal, existing transcompilation technology still faces many challenges.

[0004] Existing DLA program translation methods can be mainly divided into three categories: rule-based methods, symbolic synthesis methods and data-driven methods.

[0005] (1) The rule-based approach requires experts to manually define a series of transformation rules for different programming languages ​​and platforms, such as the transformation rules of the abstract syntax tree (AST), and then use pattern matching technology to parse the input source program. However, the disadvantage of this approach is that it requires a large number of experts to manually define the rules and is difficult to adapt to the rapidly developing new hardware technologies.

[0006] (2) Symbolic synthesis methods, such as search-based program synthesis using SMT (Satisfiability Modulo Theories) solvers, can theoretically guarantee the semantic correctness of the generated code, but such methods are often computationally expensive and difficult to scale to large general programs. In addition, these methods usually require precise specification of input constraints, which is a major obstacle for non-expert users.

[0007] (3) Data-driven methods generate code by training neural networks, which have become increasingly popular in recent years due to the success of neural machine translation technology. It uses sequence-to-sequence models to automatically translate C++, Java, and Python programs. Although this method can reduce human involvement and generate code quickly in some cases, they often cannot match the functional correctness and performance of the generated program with the code manually optimized by experts.

[0008] In summary, although the current deep learning accelerator programming and translation technologies have made some progress in practical applications, they still have shortcomings in terms of program portability, correctness, and performance. These technical challenges are the core issues to be solved in this study and are also the key to promoting the further development of deep learning accelerator technology.

[0009] The above program translation strategies include three categories: rule-based methods, data-driven methods, and symbolic synthesis methods. Rule-based methods require expert-defined conversion rules, usually for specific language pairs. For example, FCUDA is a translator that automatically converts CUDA code into FPGA executable code. FCUDA defines a set of conversion rules for data communication, computational optimization, and parallel mapping. These rules are based on pattern matching based on the abstract syntax tree (AST) and are ultimately used for CUDA to FPGA translation. These techniques rely on manually created dedicated conversion rules. This method requires a deep understanding of both the source and target languages, and requires a lot of expert manual effort.

[0010] Data-driven approaches, by training neural networks from supervised or unsupervised corpora, have gained attention following the success of machine translation. Sequence-to-sequence models have been used to translate between C++, Java, and Python. Recent work has demonstrated excellent performance using large language models (such as CodeX, StarCoder, CodeGen, CodeT5, CodeGeeX, LLaMA, Gemini, and GPT-4) for program translation. However, although these methods have low labor costs, they often cannot ensure the functional correctness of the translated code, especially between different DLAs.

[0011] Symbolic synthesis is a third class of methods that generates semantically correct code from domain-specific languages ​​(DSLs) or input-output pairs. C2TACO leverages a guided enumeration synthesizer to translate C tensor code to TACO DSL through automatically generated input-output examples. While these methods are effective in small-scale projects, they heavily rely on costly search-based SMT solvers and face limitations in larger, real-world applications.

[0012] In summary, existing techniques require a large amount of expert human labor overhead, face challenges with incorrect functionality, or have limited scalability, thus failing to automatically translate programs between DLAs with different programming models. Summary of the invention

[0013] The present invention proposes a neural symbolic synthesis framework, in which translation is performed as a series of LLM-assisted transformations, and the functional equivalence of each transformation is ensured by small-scale symbolic synthesis, so as to achieve fully automatic translation without human intervention, translation that guarantees functional correctness, and translation methods that can solve general programs. The purpose of the present invention is to solve the automation, correctness and generality problems of the above-mentioned prior art, and propose a cross-platform program translation method and system based on neural symbolic synthesis.

[0014] In view of the shortcomings of existing technologies, such as Figure 8 As shown, the present invention proposes a cross-platform program translation method, which includes:

[0015] A parallelization step, obtaining a source program for running on a source platform, converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, and dividing the serial program into loops according to the hardware performance of the deep learning accelerator of the target platform and binding it to the target platform through the built-in parallel variables of the target platform to obtain input code;

[0016] A storage conversion step, checking the storage usage of the input code, and decorating the storage of the input code according to a preset storage hierarchy to obtain an input program;

[0017] In the tensor quantization step, the tensor operations in the input program are reduced to scalar operations, and the scalar operations are re-expressed using the tensor operation instructions of the target platform to obtain a target program for running on the target platform. The deep learning accelerator of the target platform runs the target program to obtain an operating result.

[0018] The cross-platform program translation method, wherein the parallelization step comprises:

[0019] After converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, the serial program is unit tested, and a symbolic synthesis-based repair process is performed on the serial program that fails the unit test;

[0020] According to the number of clusters and cores in the hardware performance, the serial program is loop-segmented, unit tests are performed on the segmented result programs, and symbol-synthesis-based repair processing is performed on the segmented result programs that fail the unit tests;

[0021] The segmentation result program is parallelized and bound to the target platform through the built-in parallel variables of the target platform to obtain input code, perform unit testing on the input code, and perform symbolic synthesis-based repair processing on the input code that fails the unit test.

[0022] The cross-platform program translation method, wherein the storage conversion step comprises:

[0023] The knowledge of the large language model is used to understand the computational semantics of the source program, the programming manual is retrieved to obtain the storage space requirements of the computational semantics, and the reading and writing of the input code are decorated with the help of the large language model to map the buffer to the storage space of the target platform MLU; the input program is unit tested, and symbolic synthesis-based repair processing is performed on the input program that fails the unit test.

[0024] The cross-platform program translation method, wherein the tensor quantization step comprises:

[0025] Identify the program area corresponding to the original scalar memory access or operation in the input program, retrieve the description and examples of the built-in tensor instructions from the programming manual to mark the program, and use a large language model to transform the marked program to re-express the scalar calculation; perform unit testing on the target program, and perform symbolic synthesis-based repair processing on the target program that fails the unit test.

[0026] The cross-platform program translation method, wherein the repair process includes:

[0027] Determine the local program fragment for program transformation according to the loop index or variable name, compare the program execution traces before and after the transformation by binary search method, and determine the wrong program fragment;

[0028] The constraints of each loop index variable, loop boundary, variable size or tensor semantics in the erroneous program fragment are enumerated, input into a symbolic synthesis solver, and the solved program fragment replaces the erroneous program fragment.

[0029] like Fig. 9 As shown, the present invention also proposes a cross-platform program translation device, which includes:

[0030] The parallelization module obtains the source program for running on the source platform, converts the parallel built-in variables in the source program into a serial program based on loop instructions through the large language model, and divides the serial program into loops according to the hardware performance of the deep learning accelerator of the target platform and binds it to the target platform through the built-in parallel variables of the target platform to obtain the input code;

[0031] A storage conversion module checks the storage usage of the input code and decorates the storage of the input code according to a preset storage hierarchy to obtain an input program;

[0032] The tensor quantization module converts the tensor operations in the input program into scalar operations, and re-expresses the scalar operations using the tensor operation instructions of the target platform to obtain a target program for running on the target platform. The deep learning accelerator of the target platform runs the target program to obtain an operating result.

[0033] The cross-platform program translation device, wherein the parallelization module comprises:

[0034] After converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, the serial program is unit tested, and a symbolic synthesis-based repair process is performed on the serial program that fails the unit test;

[0035] According to the number of clusters and cores in the hardware performance, the serial program is loop-segmented, unit tests are performed on the segmented result programs, and symbol-synthesis-based repair processing is performed on the segmented result programs that fail the unit tests;

[0036] The segmentation result program is parallelized, bound to the target platform through the built-in parallel variables of the target platform, input code is obtained, unit test is performed on the input code, and a repair process based on symbolic synthesis is performed on the input code that fails the unit test;

[0037] The storage conversion module includes:

[0038] Using the knowledge of the large language model to understand the computational semantics of the source program, searching the programming manual to obtain the storage space requirements of the computational semantics, and using the large language model to perform decoration on the reading and writing of the input code to map the buffer to the storage space of the target platform MLU; unit testing the input program, and performing symbolic synthesis-based repair processing on the input program that fails the unit test;

[0039] The tensor quantization module includes:

[0040] Identify the program area corresponding to the original scalar memory access or operation in the input program, retrieve the description and examples of the built-in tensor instructions from the programming manual to mark the program, and use the large language model to transform the marked program to re-express the scalar calculation; perform unit testing on the target program, and perform symbolic synthesis-based repair processing on the target program that fails the unit test;

[0041] The repair process includes:

[0042] Determine the local program fragment for program transformation according to the loop index or variable name, compare the program execution traces before and after the transformation by binary search method, and determine the wrong program fragment;

[0043] The constraints of each loop index variable, loop boundary, variable size or tensor semantics in the erroneous program fragment are enumerated, input into a symbolic synthesis solver, and the solved program fragment replaces the erroneous program fragment.

[0044] The present invention also proposes an electronic device, which includes the cross-platform program translation device described above, and the electronic device is connected to an information display device, which is used to display the operation results according to display parameters and attributes set by the user or through an artificial intelligence model.

[0045] The present invention also proposes a storage medium for storing a computer program for executing the cross-platform program translation method.

[0046] The present invention also provides a computer program product, comprising a computer program, wherein the computer program implements the steps of any of the cross-platform program translation methods when executed by a processor.

[0047] It can be seen from the above scheme that the advantages of the present invention are:

[0048] The cross-platform program translation method proposed in this invention is the first one that can complete fully automatic program translation on various deep learning accelerators. Specifically:

[0049] We conducted experiments on four different deep learning accelerators (DLA) and their programming interfaces, including Intel DL Boost using C language, NVIDIA GPU using CUDA C code, AMD MI using HIP code, and Cambricon MLU using BANG C code. The results show that the present invention correctly translates different DLA programs with an average functional accuracy of 95%, which is significantly better than existing technologies based on large language models and symbolic synthesis methods (their functional correctness is only 15% on average). In addition, the performance of the translated program even reaches 101% of its corresponding expert hand-optimized program. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is an overview diagram of the system of the present invention;

[0051] Figure 2 Locate the method graph for the faulty program fragment;

[0052] Figure 3 Diagram for a program repair method based on symbolic synthesis;

[0053] Figure 4 This is an example diagram of the present invention;

[0054] Figure 5 This is a schematic diagram of oop recovery.

[0055] Figure 6 This is a schematic diagram of Cache read;

[0056] Figure 7 This is an illustration of the automatic tuning prompt words based on the large language model;

[0057] Figure 8 is a flow chart of the method of the present invention;

[0058] Fig. 9 It is a module diagram of the device of the present invention;

[0059] Fig.10 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0060] Fig.11 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0061] Fig.12 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0062] Reference numerals:

[0063] A-First electronic device;

[0064] B-Cross-platform program translation device;

[0065] C-data acquisition equipment;

[0066] D-information display device;

[0067] 1000 - second electronic device;

[0068] Ⅰ-computational unit;

[0069] II-ROM;

[0070] III-RAM;

[0071] IV-bus;

[0072] V-interface;

[0073] VI - input unit;

[0074] VII-output unit;

[0075] VIII- Storage medium;

[0076] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0077] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0078] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0079] The processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0080] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0081] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0082] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0083] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0084] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0085] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0086] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0087] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0088] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0092] During the in-depth study of the programming and translation of the deep learning accelerator (DLA), the inventors found that the existing technology has significant defects in the automation, correctness and efficiency of program translation. These defects are mainly due to the inability of the translation tools to adapt to the high heterogeneity of the hardware, and the existing methods are poor in maintaining the functional correctness of the code and optimizing performance. Specifically, the rule-based translation method relies too much on manually defined rules, the data-driven method is difficult to ensure the functional correctness of the code, and the symbolic synthesis method is difficult to solve the translation problem of general programs. These technical challenges show that a new solution is needed that can automatically adapt to different hardware characteristics while ensuring the correctness and high performance of the translated code.

[0093] After an in-depth analysis of the existing DLA programming model and translation technology, the inventors realized that to solve these problems, they must start from two aspects: one is to improve the automation level and translation efficiency of the translation process, and the other is to ensure that the translated code can run correctly on different hardware. To achieve this goal, the inventors proposed a new translation method that uses a neural symbolic synthesis method, combined with a large language model (LLM) and search-based symbolic synthesis technology to improve the efficiency and correctness of translation.

[0094] In terms of technical implementation, the core technology of the method of the present invention is to decompose the entire translation process into multiple conversion stages assisted by LLM, and use small-scale symbolic synthesis to repair incorrect program fragments in each stage. First, the code conversion generated by LLM takes into account the abstract hardware characteristics of DLA, such as parallelism, memory hierarchy, and special instruction sets. After these codes are converted, their correctness is verified by unit testing. If the code fails the test, the system will use symbolic synthesis technology to accurately locate the error and repair it. The key to this method is that it can further optimize the code performance through auto-tuning while maintaining the correctness of the code function, and select the optimal functional equivalent program.

[0095] In addition, the method of the present invention also introduces a series of LLM prompts to guide the code conversion process. These prompts not only provide the specific details of the conversion, such as the correct conversion of parallel variables and the adaptation of memory access modes, but also help LLM better understand the requirements of the target hardware platform through specific examples. This method significantly improves the correctness and adaptability of the translation and can process codes from different programming languages ​​and hardware platforms.

[0096] The present invention successfully solves the problems of efficiency, correctness and versatility in DLA program translation by innovatively combining LLM and symbol synthesis technology. This invention provides a new and usable solution for program translation in heterogeneous computing environments. To achieve the above technical effects, the present invention includes the following key technologies:

[0097] Key point 1: Cross-platform program translation technology based on multi-step generation of a large language model; it breaks down the original one-step source-to-source program translation problem into a series of LLM-assisted program transformation processes, significantly improving the versatility and correctness of program translation.

[0098] Key point 2: Use symbolic synthesis technology to automatically complete program error correction; use unit testing to check the correctness of each transformation step, and use small-scale symbolic synthesis technology to correct incorrect program translation fragments, which significantly improves the functional correctness of the translated code.

[0099] Key point 3: Auto-tuning based on a large language model optimizes the performance of the translated program to reach or even exceed the performance of programs manually optimized for specific hardware.

[0100] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0101] The present invention adopts a neural symbol synthesis framework based on a large language model and a symbol synthesis method.

[0102] The present invention adopts a neural symbol synthesis method, that is, a large language model (LLM) is used to complete program transformation and errors are corrected through small-scale symbol synthesis. Figure 1 The system framework of the present invention is presented, which includes three main stages: serialization / parallelization, storage conversion, and (de)tensorization. These three stages are closely related to the key programming features of DLA, such as parallelism, storage hierarchy, and dedicated tensor instructions. Figure 1 The A between the modules represents the code comment pass, which is to mark the serial code with instruction information. For example, the matrix multiplication comment information is #pragma operation(matmul).

[0103] Specifically, the serialization / parallelization phase first automatically detects the parallel built-in variables in the input program and converts them into a serial program based on for loops (i.e., loop recovery), as shown in Error: Reference source not found. This serial program will further undergo loop segmentation and binding to the parallel built-in variables of the target platform (i.e., loop split, loop fusion, loop reorder). Figure 5 An example of loop recovery is given. Loop splitting and binding built-in variables refer to mapping the for loop to the parallel variables of the hardware, such as threadIdx on CUDA. Prompt words are given to the large model to tell it that loop splitting optimization or built-in variable binding optimization is required.

[0104] The storage conversion stage checks the storage usage of the input code and decorates it according to the specific storage hierarchy (such as __shared__ in GPU and NRAM, WRAM in Cambricon MLU). For example, an array originally declared as "float16A

[1024] ;" on the CPU needs to follow the requirements of the specific platform and become "__nram__float16A

[1024] " on the MLU or "__shared__float16A

[1024] " on the GPU.

[0105] Checking the storage usage of the input code specifically includes determining whether the program is performing gemm operations. Therefore, the relevant input and output data should be decorated through the storage hierarchy according to the storage hierarchy constraints on the target platform. The specific storage hierarchy is determined by the large language model, which automatically determines the storage hierarchy by analyzing the characteristics of the program and the storage hierarchy description recorded in the target platform programming manual. Each hardware platform has its own programming manual, such as the NVIDIA CUDA Programming Guide, the CAMBRICON BANG C Developer's Manual, and the CAMBRICON BANG C / C++ Programming Guide.

[0106] The (de)tensorization stage first converts the tensor operations in the input program (such as mma_sync in GPU Tensor Core) into scalar operations, and then re-expresses the operations using the dedicated tensor operation instructions on the target platform. In addition, the present invention also uses automatic tuning to generate multiple candidate programs through a large language model and search for the optimal program from them.

[0107] We use the translation of a 16×512×64 matrix multiplication program based on CUDA C language to its BANG C counterpart as an example to illustrate the entire workflow. The entire conversion process is shown in Error: Reference source not found.

[0108] In the serialization / parallelization process, in order to bridge the differences between different programming models, the original CUDA C code is first converted into a general for loop, and the loop is restored by removing the built-in parallel variables (such as blockIdx and blockDim). Then, according to the hardware characteristics of the MLU (i.e., the number of clusters and cores are both 4), the for loop is split and the split loop is parallelized and bound to the built-in parallel variables coreId and clusterId in the BANG C code accordingly. At each conversion stage, once the translated code fails the unit test, the present invention performs a symbolic synthesis SMT-based synthesis to fix incorrect indexes or loop boundaries.

[0109] The serialization process is that the large language model understands the semantics of tensor instructions, as well as the parameter introduction and examples in the corresponding user manual, and transforms tensor instructions into serial codes. For unit testing, we use Python code to call the corresponding kernel function for each kernel, and use numpy to implement operators with the same function, and compare the accuracy with the results of the operators on DLA. Currently, the test has two parts, one is whether the compilation is correct or not, and the other is the functional correctness (consistent with the results of numpy).

[0110] During the storage conversion process, the present invention first uses the knowledge of LLM to preliminarily understand the computational semantics of the source program for matrix multiplication operations. Then, it retrieves the programming manual to obtain the storage space requirements for the matrix multiplication instructions. According to the matrix multiplication instruction description in the programming manual, buffers A, B, and C are mapped to NRAM, WRAM, and NRAM spaces respectively. After that, with the help of LLM, we perform cache read and cache write transformations, modifying the program by reading the memory of buffers A and B, and writing the memory of buffer C to their respective storage spaces. In the case where LLM cannot correctly handle storage-related indexes, SMT-based code repair is also required.

[0111] In the (de)tensorization process, in order to automatically convert the scalar code into the built-in tensor instructions corresponding to the target platform, the present invention first identifies the program area corresponding to the original scalar memory access or operation, and then retrieves the description and examples of the built-in tensor instructions (such as __memcpy and __bang_mlp) from the programming manual to mark the program. Since the LLM-based transformation faces correctness challenges when processing the parameters of the built-in tensor instructions, the present invention uses the SMT-based symbolic synthesis technology to perform tensor operation repair to repair the erroneous code fragments (such as incorrect __bang_mlp input parameters).

[0112] 2. Use symbol synthesis technology to automatically complete program error correction

[0113] In each of the above stages, LLM is first used to perform preliminary program transformation (e.g., loop recovery in serialization / parallelization), and then the correctness of the transformed program is verified by unit testing. If the unit test fails, the program is repaired through a small-scale SMT-based symbolic synthesis. The key to making the above symbolic synthesis feasible is to limit the scale of the symbolic synthesis problem, which requires accurately locating the wrong program fragment during unit testing. We use Figure 2 The algorithm shown in the figure is used to accurately locate the faulty program fragment. First, we determine the local program fragment for program transformation based on the loop index or variable name, and then compare the execution trace of the faulty program fragment with that of the original program by binary search. Specifically, we insert print statements after the relevant variable positions and compare the print values ​​in the faulty program with those in the original program to accurately locate the error in the program fragment.

[0114] Subsequently, once the program transformation fails the unit test, it is necessary to correct the correctness of these programs through symbolic synthesis. We first enumerate each specific loop index variable, loop boundary, variable size or tensor semantic constraint in the program fragment, and generate input for the SMT solver. The correct transformed program fragment is solved by the SMT solver and replaced into the original program. The overall algorithm is shown in Error: Reference source not found.

[0115] 3. Auto-tuning based on large language models

[0116] Although the above-mentioned neural symbolic synthesis method can correctly complete the cross-platform program translation on different DLAs, we also hope to further improve the performance of the translated program to reduce the overhead of manual performance optimization. To this end, we implemented automatic tuning technology during the translation process, which is an effective method to automatically generate high-performance programs.

[0117] Generate search space

[0118] The basis of automatic tuning is to build a search space containing a large number of candidate programs, and our method generates such a space by interacting with LLM. As shown in Error: Reference source not found, taking the serialization / parallelization stage as an example, the automatic tuning module can instantiate the prompt words of the large language model and generate multiple programs with different tunable parameters, including the granularity of loop segmentation and different loop orders. This is achieved by using specially designed meta prompt words, which are then further transformed into prompt words for the loop split and loop reorder stages.

[0119] Exploring the Search Space

[0120] Once the search space is established, we propose to use methods such as XGBoost to explore the program with the best performance. Depending on the characteristics of the target program, for example, in the case of using BANG C language, due to the large instruction granularity, the search space is small, and we directly use the brute force search method. In fact, the search space size of different DLAs varies greatly. For example, for matrix multiplication operations, the search space size on GPU is 150, while on MLU it is only 10.

[0121] By introducing the automatic tuning method, the present invention can reduce the workload of developers in performance optimization and plays a key role in improving the performance of translated programs.

[0122] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0123] like Fig. 9 As shown, the present invention also proposes a cross-platform program translation device, which includes:

[0124] The parallelization module obtains the source program for running on the source platform, converts the parallel built-in variables in the source program into a serial program based on loop instructions through the large language model, and divides the serial program into loops according to the hardware performance of the deep learning accelerator of the target platform and binds it to the target platform through the built-in parallel variables of the target platform to obtain the input code;

[0125] A storage conversion module checks the storage usage of the input code and decorates the storage of the input code according to a preset storage hierarchy to obtain an input program;

[0126] The tensor quantization module converts the tensor operations in the input program into scalar operations, and re-expresses the scalar operations using the tensor operation instructions of the target platform to obtain a target program for running on the target platform. The deep learning accelerator of the target platform runs the target program to obtain an operating result.

[0127] The cross-platform program translation device, wherein the parallelization module comprises:

[0128] After converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, the serial program is unit tested, and a symbolic synthesis-based repair process is performed on the serial program that fails the unit test;

[0129] According to the number of clusters and cores in the hardware performance, the serial program is loop-segmented, unit tests are performed on the segmented result programs, and symbol-synthesis-based repair processing is performed on the segmented result programs that fail the unit tests;

[0130] The segmentation result program is parallelized, bound to the target platform through the built-in parallel variables of the target platform, input code is obtained, unit test is performed on the input code, and a repair process based on symbolic synthesis is performed on the input code that fails the unit test;

[0131] The storage conversion module includes:

[0132] Using the knowledge of the large language model to understand the computational semantics of the source program, searching the programming manual to obtain the storage space requirements of the computational semantics, and using the large language model to perform decoration on the reading and writing of the input code to map the buffer to the storage space of the target platform MLU; unit testing the input program, and performing symbolic synthesis-based repair processing on the input program that fails the unit test;

[0133] The tensor quantization module includes:

[0134] Identify the program area corresponding to the original scalar memory access or operation in the input program, retrieve the description and examples of the built-in tensor instructions from the programming manual to mark the program, and use the large language model to transform the marked program to re-express the scalar calculation; perform unit testing on the target program, and perform symbolic synthesis-based repair processing on the target program that fails the unit test;

[0135] The repair process includes:

[0136] Determine the local program fragment for program transformation according to the loop index or variable name, compare the program execution traces before and after the transformation by binary search method, and determine the wrong program fragment;

[0137] The constraints of each loop index variable, loop boundary, variable size or tensor semantics in the erroneous program fragment are enumerated, input into a symbolic synthesis solver, and the solved program fragment replaces the erroneous program fragment.

[0138] The present invention also provides a computer program product, comprising a computer program, wherein the computer program implements the steps of any of the cross-platform program translation methods when executed by a processor.

[0139] like Fig.10 As shown, the present invention further proposes a first electronic device A in another embodiment, which includes the cross-platform program translation device.

[0140] like Fig.11 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect and obtain the source program, and the information display device D is used to display the target program and operation results obtained by the analysis of the present invention.

[0141] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, such as the operation results, etc. The user can understand this information more promptly without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0142] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the cross-platform program translation methods provided by the above methods.

[0143] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the cross-platform program translation method. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0144] Fig.12 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0145] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0146] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0147] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S3. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0148] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A cross-platform program translation method, characterized in that: include: A parallelization step, obtaining a source program for running on a source platform, converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, and dividing the serial program into loops according to the hardware performance of the deep learning accelerator of the target platform and binding it to the target platform through the built-in parallel variables of the target platform to obtain input code; A storage conversion step, checking the storage usage of the input code, and decorating the storage of the input code according to a preset storage hierarchy to obtain an input program; In the tensor quantization step, the tensor operations in the input program are reduced to scalar operations, and the scalar operations are re-expressed using the tensor operation instructions of the target platform to obtain a target program for running on the target platform. The deep learning accelerator of the target platform runs the target program to obtain an operating result.

2. The cross-platform program translation method according to claim 1, characterized in that: The parallelization steps include: After converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, the serial program is unit tested, and a symbolic synthesis-based repair process is performed on the serial program that fails the unit test; According to the number of clusters and cores in the hardware performance, the serial program is loop-segmented, unit tests are performed on the segmented result programs, and symbol-synthesis-based repair processing is performed on the segmented result programs that fail the unit tests; The segmentation result program is parallelized and bound to the target platform through the built-in parallel variables of the target platform to obtain input code, perform unit testing on the input code, and perform symbolic synthesis-based repair processing on the input code that fails the unit test.

3. The cross-platform program translation method according to claim 1, characterized in that: The storage conversion steps include: The knowledge of the large language model is used to understand the computational semantics of the source program, the programming manual is retrieved to obtain the storage space requirements of the computational semantics, and the reading and writing of the input code are decorated with the help of the large language model to map the buffer to the storage space of the target platform MLU; the input program is unit tested, and symbolic synthesis-based repair processing is performed on the input program that fails the unit test.

4. The cross-platform program translation method according to claim 1, wherein: The tensor quantization step includes: Identify the program area corresponding to the original scalar memory access or operation in the input program, retrieve the description and examples of the built-in tensor instructions from the programming manual to mark the program, and use a large language model to transform the marked program to re-express the scalar calculation; perform unit testing on the target program, and perform symbolic synthesis-based repair processing on the target program that fails the unit test.

5. The cross-platform program translation method according to claim 2, 3 or 4, characterized in that: The repair process includes: Determine the local program fragment for program transformation according to the loop index or variable name, compare the program execution traces before and after the transformation by binary search method, and determine the wrong program fragment; The constraints of each loop index variable, loop boundary, variable size or tensor semantics in the erroneous program fragment are enumerated, input into a symbolic synthesis solver, and the solved program fragment replaces the erroneous program fragment.

6. A cross-platform program translation device, characterized in that: include: The parallelization module obtains the source program for running on the source platform, converts the parallel built-in variables in the source program into a serial program based on loop instructions through the large language model, and divides the serial program into loops according to the hardware performance of the deep learning accelerator of the target platform and binds it to the target platform through the built-in parallel variables of the target platform to obtain the input code; A storage conversion module checks the storage usage of the input code and decorates the storage of the input code according to a preset storage hierarchy to obtain an input program; The tensor quantization module converts the tensor operations in the input program into scalar operations, and re-expresses the scalar operations using the tensor operation instructions of the target platform to obtain a target program for running on the target platform. The deep learning accelerator of the target platform runs the target program to obtain an operating result.

7. The cross-platform program translation device according to claim 6, characterized in that: The parallelization module includes: After converting the parallel built-in variables in the source program into a serial program based on loop instructions through a large language model, the serial program is unit tested, and a symbolic synthesis-based repair process is performed on the serial program that fails the unit test; According to the number of clusters and cores in the hardware performance, the serial program is loop-segmented, unit tests are performed on the segmented result programs, and symbol-synthesis-based repair processing is performed on the segmented result programs that fail the unit tests; The segmentation result program is parallelized, bound to the target platform through the built-in parallel variables of the target platform, input code is obtained, unit test is performed on the input code, and a repair process based on symbolic synthesis is performed on the input code that fails the unit test; The storage conversion module includes: Using the knowledge of the large language model to understand the computational semantics of the source program, searching the programming manual to obtain the storage space requirements of the computational semantics, and using the large language model to perform decoration on the reading and writing of the input code to map the buffer to the storage space of the target platform MLU; unit testing the input program, and performing symbolic synthesis-based repair processing on the input program that fails the unit test; The tensor quantization module includes: Identify the program area corresponding to the original scalar memory access or operation in the input program, retrieve the description and examples of the built-in tensor instructions from the programming manual to mark the program, and use the large language model to transform the marked program to re-express the scalar calculation; perform unit testing on the target program, and perform symbolic synthesis-based repair processing on the target program that fails the unit test; The repair process includes: Determine the local program fragment for program transformation according to the loop index or variable name, compare the program execution traces before and after the transformation by binary search method, and determine the wrong program fragment; The constraints of each loop index variable, loop boundary, variable size or tensor semantics in the erroneous program fragment are enumerated, input into a symbolic synthesis solver, and the solved program fragment replaces the erroneous program fragment.

8. An electronic device, characterized in that: It includes a cross-platform program translation device as described in claim 6 or 7, and the electronic device is connected to an information display device, and the information display device is used to display the operation result according to the display parameters and attributes set by the user or through an artificial intelligence model.

9. A storage medium for storing a computer program for executing the cross-platform program translation method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the cross-platform program translation method described in any one of claims 1 to 5 are implemented.

Images