A method of compiler optimization

Abstract

The embodiment of the application provides a compilation optimization method, which comprises the following steps: obtaining an IR graph containing blocks with the same structure in a large language model; based on a preset loop operator semantic structure, folding multiple blocks in the IR graph into a single block to generate an IR graph containing the single block; and generating an optimized binary running file according to the IR graph containing the single block. By folding multiple blocks with the same structure into a single block, only the single block needs to be subjected to compilation optimization and code generation, so that the overall compilation time is greatly shortened, the size of the generated binary file is reduced, and the compilation efficiency of the large language model is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a compiler optimization method. Background Technology

[0002] Current artificial intelligence technology has entered a development stage centered on large language models, especially those based on the Transformer architecture. These models exhibit a significant increase in the number of parameters and network depth, leading to more complex model structures. Such models typically consist of numerous structurally similar, sequentially stacked Transformer blocks, necessitating compilation optimization to adapt to specific hardware before deployment. However, as the model size continues to expand, the time consumed by computational graph optimization and code generation during compilation increases dramatically, severely impacting model iteration and deployment efficiency.

[0003] Currently, compilation optimization methods for large language models typically employ a holistic compilation strategy, where the compiler directly traverses and optimizes the complete computation graph (including all block structures) one by one. In this process, each Transformer block is treated as an independent subgraph, undergoing optimization operations such as operator fusion, memory allocation, and instruction generation. This traditional approach suffers from the following main drawbacks: Long compilation time: Because all the repeated block structures in the model need to be optimized one by one, the compilation time increases almost linearly with the model depth (number of blocks), resulting in a compilation cycle of tens of days for large-scale models, which is difficult to meet the needs of rapid deployment and debugging.

[0004] The generated executable file is large: each block is independently compiled and generates a corresponding machine code segment during the code generation stage, resulting in a large amount of duplicate code in the final binary file, which significantly increases storage and loading overhead and is not conducive to model deployment in resource-constrained environments.

[0005] In summary, existing compilation methods suffer from significant problems such as low compilation efficiency and high code redundancy when dealing with large-scale AI models with highly repetitive structures. Therefore, there is an urgent need for a compilation optimization method that can identify and utilize model structural repetition during the compilation phase to achieve optimization and code reuse, thereby significantly shortening compilation time, reducing the size of generated files, and improving the overall efficiency of deploying large language models. Summary of the Invention

[0006] One object of this invention is to provide a compilation optimization method that, by folding multiple structurally identical blocks into a single block, only requires performing compilation optimization and code generation on that single block, thereby significantly reducing the overall compilation time and the size of the generated binary file, and significantly improving the compilation efficiency of large language models. Another object of this invention is to provide a computer-readable medium. A further object of this invention is to provide a computer device. A still other object of this invention is to provide a computer program product.

[0007] To achieve the above objectives, this invention discloses a compilation optimization method, comprising: Obtain the IR diagram of blocks with the same structure in a large language model; Based on the preset semantic structure of loop operators, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block; An optimized binary executable file is generated based on the IR map containing a single block.

[0008] Preferably, obtaining the IR diagram of blocks with the same structure in a large language model includes: Based on the preset model structure whitelist, determine the model structure description corresponding to the model name; Based on the model structure description, IR diagrams containing blocks with the same structure are extracted from the original IR diagrams of the large language model.

[0009] Preferably, the method further includes: Traverse the IR graph and identify input and output nodes; Assign a corresponding subgraph identifier to each input node; By identifying the structural boundaries of each block using subgraph identifiers and output nodes, multiple blocks are obtained.

[0010] Preferably, based on a preset cyclic operator semantic structure, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block, including: Based on the semantic structure of loop operators, multiple blocks are folded into a local loop of a single block according to the subgraph identifier; The defined data address operator is introduced into a local loop to generate an IR graph containing a single block.

[0011] Preferably, the semantic structure of the loop operator includes: input parameters, a self-attention module, a feedforward network, and an output result; Input parameters include: subgraph identifier, total number of blocks, and data address operator.

[0012] Preferably, the method further includes: Assign a corresponding input index to the global input of each block; Define the data address operator based on the input index and subgraph identifier.

[0013] Preferably, based on the IR map containing a single block, an optimized binary executable file is generated, including: Pass optimization is performed on the IR map containing a single block to generate an optimized IR map of the single block; The optimized IR map of a single block is converted into executable code, resulting in an optimized binary executable file.

[0014] The present invention also discloses a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0015] The present invention also discloses a computer device, including a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, wherein the processor executes the program to implement the method described above.

[0016] The present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described above.

[0017] This invention obtains an IR graph containing blocks with the same structure in a large language model; based on a preset loop operator semantic structure, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block; based on the IR graph containing a single block, an optimized binary executable file is generated. By folding multiple blocks with the same structure into a single block, only a single block needs to be optimized for compilation and generated for code, thereby significantly shortening the overall compilation time and reducing the size of the generated binary file, and significantly improving the compilation efficiency of large language models. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the basic network structure of a large language model provided in an embodiment of the present invention; Figure 2A flowchart of a compilation optimization method provided in an embodiment of the present invention; Figure 3 A flowchart illustrating yet another compilation optimization method provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To facilitate understanding of the technical solution provided in this application, the relevant content of the technical solution in this application will be explained below. Figure 1 This is a schematic diagram of the structure of the basic network of a large language model provided in an embodiment of the present invention, such as... Figure 1 As shown, this type of model is typically built on the Transformer architecture. Its core feature is composed of multiple structurally similar and functionally repetitive blocks stacked sequentially. This multi-layered stacking structure allows the model to extract and combine features layer by layer, thereby achieving powerful modeling capabilities for complex sequence data (such as text). The basic network of a large language model includes n blocks, namely block1, block2, ..., blockn; each block includes a self-attention module (Attention) and a feedforward network (FFN).

[0022] The self-attention module is used to calculate the association weights between elements at different positions in the input sequence, thereby capturing long-term dependencies and contextual information within the sequence.

[0023] A feedforward network (FFN) is typically a two-layer fully connected network that performs nonlinear transformations and feature mappings on the output of the self-attention module.

[0024] Each block includes m global inputs. The global inputs of block1 are: input 11, input 12, ..., input 1m; the global inputs of block2 are: input 21, input 22, ..., input 2m; and the global inputs of blockn are: input n1, input n2, ..., input nm.

[0025] It is worth noting that blocks with the same structure have the same number of global inputs, but the content of the inputs is different. These inputs usually come from the output of the previous layer, embedding vectors, positional encodings, or other network branches. As the network depth increases, the data is transformed and passed layer by layer.

[0026] In the traditional graph compilation process, the graph compiler treats the entire computation graph containing n identical blocks as a whole. That is, it needs to repeatedly perform the same optimization analysis on n identical substructures, causing the compilation time to increase linearly with the number of blocks. It also generates independent and highly similar code segments for each block, resulting in a large executable binary file containing a lot of redundant code.

[0027] This invention addresses the compilation efficiency and code size issues caused by the repetitive modular structure of the Transformer class model, and proposes a compilation optimization method.

[0028] The following uses a graph compiler as the execution subject to illustrate the implementation process of the compilation optimization method provided in this embodiment of the invention. It should be understood that the execution subject of the compilation optimization method provided in this embodiment of the invention includes, but is not limited to, a graph compiler.

[0029] Figure 2 A flowchart of a compilation optimization method provided in an embodiment of the present invention is shown below. Figure 2 As shown, the method includes: Step 101: Obtain the IR diagram of blocks with the same structure in the large language model.

[0030] In this embodiment of the invention, the network structure of a large AI language model (such as a large language model based on the Transformer architecture) is parsed by a graph compiler front-end and converted into a unified intermediate representation graph (IR graph). This IR graph describes the operator nodes and their data dependencies in the model in the form of a computation graph, which contains multiple subgraphs with identical structures and operators, i.e., blocks with the same structure.

[0031] Step 102: Based on the preset cyclic operator semantic structure, fold multiple blocks in the IR graph into a single block to generate an IR graph containing a single block.

[0032] In this embodiment of the invention, a semantic structure for a loop operator (BlockForOp) is pre-designed, representing the cyclic execution of a block. N independent blocks in the IR graph are replaced by a single BlockForOp operator, whose loop body consists of only a single block structure. Simultaneously, a data address operator (InputRefOp) is inserted within the loop body to correctly reference the global input data in each iteration based on the subgraph identifier (blockid) and the input index (inputid), ultimately outputting an IR graph containing only the semantics of a single block.

[0033] Step 103: Generate an optimized binary executable file based on the IR map containing a single block.

[0034] In this embodiment of the invention, the graph compiler performs optimization and code generation based on an IR graph containing a single block, specifically by performing Pass optimization and codegen serialization on the IR graph containing a single block.

[0035] During compilation, only the IR graph containing a single block is optimized using Pass, saving compilation time; the optimized IR graph is serialized using codegen to generate single-block code, and multiple blocks with the same structure reuse the single-block code, significantly reducing the size of the binary executable file of the large language model.

[0036] In the technical solution provided by this invention, an IR graph containing blocks with the same structure is obtained in a large language model; based on a preset loop operator semantic structure, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block; based on the IR graph containing a single block, an optimized binary executable file is generated. By folding multiple blocks with the same structure into a single block, only a single block needs to be optimized for compilation and generated for code, thereby significantly shortening the overall compilation time and reducing the size of the generated binary file, and significantly improving the compilation efficiency of the large language model.

[0037] Figure 3 A flowchart of another compilation optimization method provided in the embodiments of the present invention is shown below. Figure 3 As shown, the method includes: Step 201: Determine the model structure description corresponding to the model name based on the preset model structure whitelist.

[0038] In this embodiment of the invention, each step is executed by the graph compiler.

[0039] Specifically, the model name (Name) of the large language model to be optimized is matched with the model names in the model structure whitelist to retrieve the corresponding model structure description. The model structure description is the structure of all blocks in the model. For example, the deepseek model includes 61 blocks, and the structure of each block is recorded in the model structure description.

[0040] In this embodiment of the invention, the model structure whitelist includes the correspondence between model names and model structure descriptions. As an optional approach, the model structure description includes, but is not limited to, the total number of blocks and block structure templates. The block structure templates describe the standardized computation flow within a single block, facilitating subsequent determination of whether the structures are identical.

[0041] Step 202: Based on the model structure description, extract the IR diagram containing the same structural blocks from the original IR diagram of the large language model.

[0042] In this embodiment of the invention, based on the block structure template, the standardized computation flow inside a single block of each block is compared to extract the IR map containing blocks with the same structure.

[0043] It is worth noting that blocks with the same structure are grouped together and their IR maps are extracted. The IR maps of blocks with different structures need to be optimized separately.

[0044] For example, the Deepseek model consists of 61 blocks. The first to third blocks have the same structure, and the fourth to 61st blocks have the same structure. The first to third blocks are extracted as an IR map containing blocks with the same structure. The fourth to 61st blocks are extracted as another IR map containing blocks with the same structure. The two IR maps containing multiple blocks are then further optimized to finally obtain two IR maps containing a single block.

[0045] This invention utilizes a model structure whitelist to quickly match blocks with the same structure, achieving real-time and accurate identification of large-scale model topologies. Based on this identification result, the massive overall IR graph is deconstructed into a set of isomorphic subgraphs that can be processed repeatedly, reducing the compilation object of the graph compiler from a total of N blocks to a single block, fundamentally changing the working paradigm of compilation optimization when facing repetitive structures.

[0046] Step 203: Traverse the IR graph and identify input and output nodes.

[0047] In this embodiment of the invention, by traversing the nodes of the IR graph, key feature nodes that serve as block boundaries are identified, namely: input nodes (root) and output nodes (layerNorm). Each block has a unique input node (root) and a unique output node (layerNorm).

[0048] In this embodiment of the invention, the Transformer block includes an input node (root), intermediate nodes, and an output node (layerNorm). The input node identifies the starting boundary of the current block, the intermediate nodes are used to identify the boundaries of Attention and FFN, and the output node identifies the ending boundary of the current block.

[0049] As an alternative, the input node (root), intermediate node, and output node (layerNorm) can be identified through a front-end variable (flag).

[0050] Step 204: Assign a corresponding subgraph identifier to each input node.

[0051] In this embodiment of the invention, a corresponding subgraph identifier (blockid) is assigned to the input node of each block. The blockid is used to indicate the block. If there are N blocks with the same structure, the corresponding blockids are 1, 2, ..., N.

[0052] In this embodiment of the invention, each block has the same subgraph structure. By assigning a unique subgraph identifier (blockid) to the input node (root) of each identified block for identification and subsequent extraction, a set of all N blocks with the same structure in the model is obtained.

[0053] Step 205: Locate the structural boundary of each block using subgraph identifiers and output nodes to obtain multiple blocks.

[0054] In this embodiment of the invention, blockid is assigned to the input node of the block. The input node identifies the start boundary of the current block, and the output node identifies the end boundary of the current block. Based on the start and end boundaries of each block, the structural boundary of each block is located, thereby obtaining multiple blocks.

[0055] This invention identifies input / output nodes by traversing the IR graph, assigns unique subgraph identifiers (blockids), and uses these identifiers to locate the structural boundaries of each block. The graph compiler can clearly discretize the macroscopic, continuous computation graph into a series of independent, identifiable, and repetitive computation units, establishing a reliable structural foundation for optimization at the "single block granularity." By assigning a unique blockid to each block, the graph compiler can accurately manage the data flow and execution scheduling of a large number of repetitive modules, providing a key prerequisite for subsequent compilation optimization that achieves "single optimization, multiple reuses."

[0056] Step 206: Based on the semantic structure of the loop operator, according to the subgraph identifier, fold multiple blocks into a local loop of a single block.

[0057] In this embodiment of the invention, the semantic structure of the loop operator includes: input parameters, a self-attention module, a feedforward network, and an output result; the input parameters include: subgraph identifier blockid, total number of blocks N, and data address operator InputRefOp, where InputRefOp is defined based on the global input of the current block. Specifically, the semantic structure of the loop operator is as follows: %s = BlockForOp %blockid = %b1 to % bN{ %x1 = InputRefOp(%blockid,1) ………… %xm = InputRefOp(%blockid,m) %1 = attentionOp(%next1) / / use %x %1 = ffnOp(%1) } In this embodiment of the invention, the semantic structure of the loop operator is a for loop based on block execution. The entire model is regarded as being composed of multiple blocks with identical structures sequentially concatenated, and a high-level semantic reconstruction is performed. That is, the IR diagrams of multiple blocks are folded into the IR diagram of a single block with loop semantics, realizing local looping of the block. The BlockForOp operator encapsulates the computation logic of a single block and expresses the semantics of repeatedly executing the same block N times through blockid, N, and InputRefOp.

[0058] This invention introduces the BlockForOp operator, which achieves logical folding at the computation graph representation level, abstracting N independent blocks into a single computational unit with a loop variable. The BlockForOp operator internally defines the complete computation process of a single block, while the subgraph identifier blockid and the total number of blocks N indicate the number of loops.

[0059] Step 207: Introduce the defined data address operator into a local loop to generate an IR map containing a single block.

[0060] In this embodiment of the invention, a corresponding input index (inputid) is assigned to the global input of each block; a data address operator is defined based on the inputid and blockid. Wherein, if each block has m global inputs, the corresponding inputids are 0, 1, ..., m.

[0061] Specifically, the data address operator is: InputRefOp(%blockid, inputid). The InputRefOp operator is used for address pass-through, specifically locating the corresponding data address through blockid + inputid_index. Here, blockid is used to locate the input data of the IR of which block, and inputid_index is used to locate the nth input data point within the block of blockid.

[0062] It is worth noting that before introducing the InputRefOp operator, the input index needs to be registered, i.e.: %0 = InputOp(args0), where args0 is the inputid array (0, 1, ..., m).

[0063] In this embodiment of the invention, the InputRefOp operator is introduced into a local loop to generate an IR graph containing a single block, as follows: %0 = InputOp(args1) ………… %s = BlockForOp %blockid = %b1 to % bN{ %x1 = InputRefOp(%blockid,1) ………… %xm = InputRefOp(%blockid,m) %1 = attentionOp(%next1) / / use %x %1 = ffnOp(%1)} In this embodiment of the invention, during the merging of multiple structurally identical blocks into a single loop structure, the block loop is local, while the input data required for compilation comes from the global source. Each block has its own set of inputs within the large language model. These inputs are logically independent but continuously distributed in memory. Therefore, the InputRefOp operator is introduced to establish an address mapping relationship: during compilation, the global data address to be accessed in the current iteration is calculated based on blockid + inputid_index, and this address information is passed through to the block's local loop. This allows the computational logic of a single block to access different input data in different iterations, completing the transformation from global input data to local input data memory addresses. This prepares for the next step of pre-allocating device memory, i.e., determining the memory layout of each input data of each block in the device memory.

[0064] It's worth noting that the InputRefOp operator is only used for address pass-through during compilation; it doesn't participate in operator computation during runtime and doesn't generate actual computation instructions. In the final generated runtime code, these address mappings are translated into memory access operations, thus introducing no additional runtime overhead. This design ensures both the correctness of loop fusion and maintains runtime efficiency.

[0065] This invention solves the critical data access problem after logical folding by designing a dedicated InputRefOp operator, thus achieving data mapping. Since the input content of each block is logically independent, the InputRefOp operator statically calculates the exact memory location of the global input data that the block should access in each iteration during the compilation phase based on the blockid and inputid, and then passes this address information to the computation nodes within the block. This mechanism ingeniously separates computation logic from data layout: the computation logic only needs to be defined once, and the data is correctly accessed at runtime through the address mapping rules determined at compile time, thereby ensuring the correctness of loop fusion. Furthermore, because it is only parsed at compile time and does not generate runtime instructions, it achieves zero overhead optimization.

[0066] Taking an N-block structure, with each block having m global inputs as an example, the pseudocode of the network before optimization is: func.func @main() —>tuple{ %0 = InputOp(args1) ………… %100 = attentionOp(%1) %101 = ffnOp(%100) %102 = attentionOp(%101) %103 = ffnOp(%102) %104 = attentionOp(%103) %105 = ffnOp(%104) ... / / N blocks %n = attentionOp(%xxx) %n+1 = ffnOp(%n) %n+2 = outputOp(%n+1) } The optimized network pseudocode is as follows: func.func @main() —>tuple{ %0 = InputOp(args1) ………… %s = BlockForOp %blockid = %b1 to % bN{ %x1 = InputRefOp(%blockid,1) ………… %xm = InputRefOp(%blockid,m) %1 = attentionOp(%next1) / / use %x %1 = ffnOp(%1) } %s+1 = outputOp(%s) } Building upon the collaborative work in steps 206 and 207, the paradigm for handling repetitive structures in Transformer model compilation is revolutionized from traditional, inefficient space unrolling and repetitive optimization to efficient time loops and single-step optimization. This not only brings orders of magnitude savings in time and space for subsequent pass optimization and code generation, but also opens up new possibilities for deeper, loop-oriented optimizations (such as pipelined parallelism and memory access fusion) for graph compilers through clear semantic abstraction.

[0067] Step 208: Perform Pass optimization on the IR map containing a single block to generate an optimized IR map of the single block.

[0068] In this embodiment of the invention, a Pass is an independent module of the graph compiler that performs a "one-round processing" or "one-time optimization" of the computation graph. It is a modular and pluggable optimization step within the graph compiler. By sequentially running a series of Passes, the graph compiler gradually folds and optimizes the original computation graph into a highly streamlined and efficient version. Pass optimizations include, but are not limited to: constant folding Pass, dead code elimination Pass, operator fusion Pass, and layout transformation Pass.

[0069] Specifically, a series of Pass optimizations are performed on the IR map containing a single block to generate an optimized IR map of the single block.

[0070] This invention only requires optimizing a single block to achieve optimization of all blocks, making the compilation optimization process highly scalable. No matter how large the model size or how many blocks increase, the workload of the optimization stage remains constant, significantly saving compilation time. Ultimately, while ensuring that the quality of the generated code is not compromised, compilation efficiency is improved by orders of magnitude.

[0071] This invention breaks the traditional linear relationship between compilation optimization time and model size. Through previous steps, the computational logic of the entire large language model has been folded into a single block. Therefore, all high-level graph optimizations (such as operator fusion and constant folding) only need to be performed once on this template. This reduces the optimization time from the traditional O(N) (where N is the total number of blocks) to O(1), achieving constant time consumption in the compilation optimization stage. Regardless of whether the model has dozens or hundreds of blocks, the workload of the Pass optimization stage is exactly the same, fundamentally solving the key problem of excessively long compilation time for large language models.

[0072] Step 209: Convert the optimized IR map of a single block into executable code to obtain the optimized binary executable file.

[0073] In this embodiment of the invention, when the graph compiler processes the IR graph of an optimized single block, it generates only one executable code for that single block; through a lightweight loop scheduling logic, multiple blocks reuse the executable code of the generated single block to obtain an optimized binary runtime file.

[0074] Taking an N-block structure as an example, the pseudocode of codogen before optimization is: int attention 0 (params) {} int ffn 0 (params) {} int attention1 (params) {} int ffn1 (params) {} ... / / N block function definitions int attention n (params) {} int ffn n (params) {} void main(){ attention 0 (); ffn 0(); attention 1 (); ffn 1(); ... / / N block function definitions attention n (); ffn n(); } The optimized codogen pseudocode is as follows: int attention (params,block_info) {} int ffn (params,block_info) {} void main(){ for(auto i : block_nums){ attention (); ffn(); } } This invention significantly reduces the size of binary executable files (by nearly 1 / N) by reusing executable code, greatly reducing storage and transmission overhead and improving program loading speed. Hardware instruction cache can more efficiently reside in and reuse the same core computational code, avoiding cache turbulence caused by code bloat, thereby improving actual execution speed and achieving dual optimization of storage space and execution efficiency. Since the graph compiler can concentrate all optimization resources on extreme tuning of a single piece of code (such as more aggressive loop unrolling and vectorization), the quality of the generated code is usually higher and more optimized than when generating N copies in batches.

[0075] It is worth noting that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. The user information in the embodiments of this application was obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been authorized and agreed upon by the client.

[0076] It is worth noting that the information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0077] It is worth noting that the technical solution provided in this application provides users with a corresponding operation entry point, allowing users to choose to agree to or reject the automated decision-making result; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0078] In the technical solution of the compilation optimization method provided in this embodiment of the invention, an IR graph containing blocks with the same structure is obtained in a large language model; based on a preset loop operator semantic structure, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block; based on the IR graph containing a single block, an optimized binary executable file is generated. By folding multiple blocks with the same structure into a single block, compilation optimization and code generation only need to be performed on a single block, thereby significantly shortening the overall compilation time and reducing the size of the generated binary file, and significantly improving the compilation efficiency of the large language model.

[0079] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer device, specifically, a computer device can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0080] This invention provides a computer device including a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the steps of the above-described compiler optimization method. For a detailed description, please refer to the above-described compiler optimization method embodiments.

[0081] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer device 600 suitable for implementing the embodiments of this application.

[0082] like Figure 4As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer device 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0083] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal feedback (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed in storage section 608 as needed.

[0084] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611.

[0085] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0091] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0092] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0093] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0095] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0096] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A compiler optimization method, characterized in that, The method includes: Obtain the IR diagram of blocks with the same structure in a large language model; Traverse the IR graph to identify input nodes and output nodes. The input node identifies the start boundary of the current block, and the output node identifies the end boundary of the current block. Assign a corresponding subgraph identifier to each input node; By locating the structural boundary of each block using the subgraph identifier and output node, multiple blocks are obtained; Based on a preset cyclic operator semantic structure, multiple blocks in the IR graph are folded into a single block to generate an IR graph containing a single block, including: Based on the semantic structure of the loop operator, and according to the subgraph identifier, multiple blocks are folded into a local loop of a single block with loop semantics; a corresponding input index is assigned to the global input of each block; a data address operator is defined according to the input index and the subgraph identifier; the defined data address operator is introduced into the local loop to generate an IR graph containing a single block. The data address operator calculates the global data address accessed in the current iteration by locating the input data of which block's IR and the nth input data of the block identified by the subgraph identifier, and passes the global data address to the local loop, so that the single block's computation logic accesses different input data in different iterations, completing the transformation from global input data to local input data memory address; The semantic structure of the loop operator is a for loop based on block execution. The entire model is regarded as being composed of multiple blocks with identical structures sequentially concatenated and high-level semantic reconstruction is performed. The complete computation process of a single block is defined internally. The semantic structure of the loop operator includes: input parameters, self-attention module, feedforward network and output result. The input parameters include: subgraph identifier, total number of blocks, and data address operator, wherein the subgraph identifier and total number of blocks indicate the number of iterations; Based on the IR diagram containing a single block, an optimized binary executable file is generated, including: generating an executable code for a single block; and using lightweight loop scheduling logic to allow multiple blocks to reuse the executable code of the single block to obtain the optimized binary executable file.

2. The compilation optimization method according to claim 1, characterized in that, The process of obtaining the IR map of blocks with the same structure in a large language model includes: Based on the preset model structure whitelist, determine the model structure description corresponding to the model name; Based on the model structure description, IR diagrams containing the same structural blocks are extracted from the original IR diagrams of the large language model.

3. The compilation optimization method according to claim 1, characterized in that, The step of generating an optimized binary executable file based on the IR map containing a single block includes: Pass optimization is performed on the IR map containing a single block to generate an optimized IR map of the single block; The optimized IR map of a single block is converted into executable code, resulting in an optimized binary executable file.

4. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the compilation optimization method as described in any one of claims 1 to 3.

5. A computer device comprising a memory and a processor, the memory for storing information including program instructions, and the processor for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by the processor, they implement the compilation optimization method according to any one of claims 1 to 3.

6. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the compilation optimization method according to any one of claims 1 to 3.

Images