Code generation methods and apparatus, electronic devices, storage media, electronic equipment

Abstract

This disclosure provides a code generation method and apparatus, an electronic device, a storage medium, and an electronic device. The method includes: acquiring a plurality of execution modules, each execution module including a first code portion and a loop identifier; dividing the plurality of execution modules into a plurality of execution module groups according to the loop identifier and determining a first order of the plurality of execution module groups; concatenating a plurality of first code portions corresponding to each execution module group into a second code portion; and concatenating a plurality of second code portions corresponding to the plurality of execution module groups into a target code portion according to the first order of the plurality of execution module groups. This method can significantly improve code readability and maintainability, enhance code reusability, and reduce the skill requirements for developers without sacrificing code performance, thereby accelerating the iteration and update of code-based algorithms.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of artificial intelligence, and particularly to code generation methods, electronic devices, storage media, electronic devices, code generation apparatus, and computer program products. Background Technology

[0002] In the field of code writing and generation, different functionalities are often tightly coupled and encoded into a single code structure, resulting in chaotic code structure, extremely poor readability, and the formation of "spaghetti code" that is difficult to understand and maintain. This highly specialized and mixed coding style makes the code extremely unmaintainable and portable. Any minor adjustment to the algorithm or hardware can trigger a wide-ranging chain of modifications, leading to high testing and debugging costs. At the same time, this coding style greatly raises the technical threshold for developers and hinders rapid technological iteration.

[0003] Therefore, we desire a code writing scheme that combines high readability and high reusability. Summary of the Invention

[0004] According to at least one embodiment of the present disclosure, a code generation method is provided, comprising: acquiring a plurality of execution modules, each of the plurality of execution modules including a first code portion and a loop identifier; dividing the plurality of execution modules into a plurality of execution module groups according to the loop identifier and determining a first order of the plurality of execution module groups; concatenating a plurality of first code portions corresponding to each execution module group in the plurality of execution module groups into a second code portion; and concatenating a plurality of second code portions corresponding to the plurality of execution module groups into a target code portion according to the first order of the plurality of execution module groups.

[0005] For example, according to at least one embodiment of the code generation method of this disclosure, the step of dividing the plurality of execution modules into a plurality of execution module groups includes: grouping execution modules with the same loop identifier among the plurality of execution modules into the same execution module group.

[0006] For example, according to at least one embodiment of the code generation method of this disclosure, determining the first order of the plurality of execution module groups includes: sorting the loop identifiers corresponding to the plurality of execution module groups in lexicographical order; and determining the first order of the plurality of execution module groups based on the result of sorting the loop identifiers in lexicographical order.

[0007] For example, according to at least one embodiment of the code generation method of this disclosure, each of the plurality of execution modules further includes an iteration index, and the method further includes: determining a second order of each of the plurality of execution modules in a corresponding execution module group according to the iteration index, wherein concatenating a plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating the plurality of first code parts into a second code part according to the second order.

[0008] For example, according to at least one embodiment of the code generation method of this disclosure, determining the second order of each execution module in the corresponding execution module group includes: sorting the iteration indices corresponding to the execution modules included in each execution module group in the plurality of execution module groups in lexicographical order; and determining the second order of each execution module in the corresponding execution module group based on the result of the lexicographical sorting of the iteration indices.

[0009] For example, according to at least one embodiment of the code generation method of this disclosure, the method further includes: obtaining the dependency relationship of at least a portion of the execution modules among the plurality of execution modules; determining a third order of the at least a portion of the execution modules in the corresponding execution module group according to the dependency relationship, wherein concatenating a plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating a plurality of first code parts, including at least the first code parts corresponding to the at least a portion of the execution modules, into the second code part according to the third order.

[0010] For example, in a code generation method according to at least one embodiment of the present disclosure, the loop identifier and the iteration index of the at least part of the execution modules are the same.

[0011] For example, according to at least one embodiment of the code generation method of this disclosure, obtaining the dependencies of at least a portion of the execution modules among the plurality of execution modules includes at least one of the following: obtaining the dependencies included in the at least a portion of the execution modules; and analyzing a first code portion included in the at least a portion of the execution modules to obtain the dependencies.

[0012] For example, according to at least one embodiment of the code generation method of this disclosure, the method further includes: obtaining a startup function for the target code portion; and encapsulating the startup function and the target code portion into a target code file.

[0013] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the plurality of execution modules include one or more of the following: a first matrix loading module configured to perform a first loading operation, the first loading operation including loading data blocks included in a first matrix from global memory; a second matrix loading module configured to perform a second loading operation, the second loading operation including loading data blocks included in a second matrix from global memory; a synchronization module configured to synchronize the first loading operation of the first matrix loading module and the second loading operation of the second matrix loading module; a calculation module configured to perform matrix multiplication based on the loaded data blocks included in the first matrix and the loaded data blocks included in the second matrix, and store the result of the matrix multiplication in a register; and a write-back module configured to write the result of the matrix multiplication back to global memory.

[0014] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the loop identifier is used to identify the outer loop of the execution module in the matrix multiplication operation.

[0015] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the iteration index is used to identify at least one of the loop unrolling order and pipeline order in the outer loop of the execution module in the matrix multiplication operation.

[0016] An electronic device is provided according to at least one embodiment of the present disclosure, comprising: at least one processing unit; at least one memory including one or more computer program modules, wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processing unit, the one or more computer program modules being used to implement the method described above.

[0017] According to at least one embodiment of the present disclosure, a computer-readable storage medium is provided having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described above.

[0018] According to at least one embodiment of the present disclosure, a code generation apparatus is provided, comprising: an acquisition module configured to acquire a plurality of execution modules, each of the plurality of execution modules including a first code portion and a loop identifier; a division module configured to divide the plurality of execution modules into a plurality of execution module groups according to the loop identifier and determine a first order of the plurality of execution module groups; a first splicing module configured to splice a plurality of first code portions corresponding to each of the plurality of execution module groups into a second code portion; and a second splicing module configured to splice a plurality of second code portions corresponding to the plurality of execution module groups into a target code portion according to the first order of the plurality of execution module groups.

[0019] An electronic device is provided according to at least one embodiment of the present disclosure, including an electronic device of the embodiment described above or a code generation device of the embodiment described above.

[0020] The method according to at least one embodiment of this disclosure can significantly improve the readability and maintainability of code. For example, code generated or written according to the method of at least one embodiment of this disclosure can be organized according to corresponding functional modules, thus resulting in a clear code structure. In this way, developers can independently understand, modify, and test each code module, greatly reducing maintenance costs and the risk of errors.

[0021] The method according to at least one embodiment of this disclosure can enhance code reusability. For example, a predetermined code module can be reused by other algorithms that need to execute the function corresponding to the predetermined code module. Furthermore, the code module can be easily modified to adapt to different application scenarios. For example, the predetermined code module can be easily modified to support different data types (e.g., 16-bit floating-point numbers (FP16), 16-bit floating-point numbers (BF16), and 8-bit integer numbers (INT8, but this disclosure is not limited thereto) or different hardware (e.g., different computing cores, but this disclosure is not limited thereto).

[0022] Furthermore, the method according to at least one embodiment of this disclosure can reduce the skill requirements for developers. For example, developers do not need to be proficient in all the details of all functions of the algorithm, such as memory hierarchy, instruction pipelining, and synchronization mechanisms. Instead, developers can focus on their areas of expertise (e.g., the specific implementation logic of the algorithm) and use pre-built or standardized code modules. In this way, more developers can be involved in the development of algorithms such as those for high-performance computing.

[0023] The method according to at least one embodiment of this disclosure can accelerate the iteration and updating of algorithms. When new hardware architectures (e.g., the introduction of new processing units such as matrix computation units) or new algorithmic ideas emerge, developers only need to replace or modify the affected code modules, without rewriting the entire code. In this way, the development and adaptation cycle of new algorithms (e.g., new operators) can be greatly shortened.

[0024] According to at least one embodiment of the method disclosed herein, while significantly improving code readability and maintainability, reducing skill requirements for developers, and accelerating algorithm iteration and updates as described above, the algorithm's performance can be guaranteed without loss. For example, since the final generated code is constructed using logical equivalence, the execution details of the final generated code (such as, but not limited to, instruction flow, memory access patterns, and synchronization behavior) are completely consistent with the code manually optimized by developers, thus achieving performance comparable to hand-written code. Logical equivalence can refer to the fact that the computational results, execution timing, and performance metrics (e.g., throughput, latency) produced by the code generation method provided by this invention are functionally and performance-wise equivalent to the code obtained through traditional hand-written optimization methods. In this way, the performance loss problem that may be caused by modular programming can be solved. Attached Figure Description

[0025] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flowchart of a code generation method according to at least one embodiment of the present disclosure is shown.

[0027] Figure 2 A schematic diagram of a code generation method according to at least one embodiment of the present disclosure is shown.

[0028] Figure 3 A schematic diagram of a code generation method according to at least one embodiment of the present disclosure is shown.

[0029] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0030] Figure 5 A schematic diagram of an electronic device according to at least one embodiment of the present disclosure is shown.

[0031] Figure 6 A schematic diagram of a code generation apparatus according to at least one embodiment of the present disclosure is shown.

[0032] Figure 7A schematic block diagram of an electronic device provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0033] Before proceeding with the detailed description below, it may be advantageous to define certain words and phrases used throughout this disclosure. The terms “comprising” and “including” and their derivatives mean including but not limited to. The term “or” is inclusive, meaning and / or. The phrase “associated with” and its derivatives mean including, comprising, interconnecting, containing, contained within, connected or connected to, coupled or coupled to, communicating with, cooperating, intertwining, juxtaposing, proximate, binding or bound to, having, possessing attributes, having a relationship or being related to, etc. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware, or a combination of hardware and software and / or firmware. The functionality associated with any particular controller may be centralized or distributed, local or remote. The phrase “at least one,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and that only one item from the list may be required. For example, "at least one of A, B, and C" includes any one of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C.

[0034] Definitions of other specific words and phrases are provided throughout this disclosure. Those skilled in the art will understand that, in many, if not most, cases, such definitions apply to the prior and future use of the words and phrases thus defined.

[0035] The various embodiments of the principles of this disclosure described below with reference to the accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or device. In some cases, the actions described in the specification may be performed in a different order and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require a specific order or sequential sequence to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0036] In the field of code writing and generation, different functionalities are often tightly coupled and encoded into a single code structure, resulting in chaotic code structure, extremely poor readability, and the formation of "spaghetti code," which is difficult to understand and maintain. "Spaghetti code" refers to code with a chaotic structure, tangled logic, and difficulty in maintenance. This highly specialized and hybrid coding style makes code maintainability and portability extremely low. Any minor adjustment to the algorithm or hardware can trigger widespread cascading modifications, leading to high testing and debugging costs. Furthermore, this coding style significantly raises the technical barrier for developers, hindering rapid technological iteration.

[0037] For example, in the field of high-performance computing using general-purpose graphics processing units (GPGPUs), especially for applications such as deep learning and scientific computing, developers pursue the ultimate computing performance.

[0038] For example, in some programming scenarios, a warp consisting of multiple threads is the basic unit of scheduling and execution. To maximize hardware utilization, developers typically need to write highly optimized kernel code for warps. In programming such as Graphics Processing Unit (GPU) programming, kernel code can refer to functions executed on the GPU and can be the basic unit of task in the GPU computation process. Taking the General Matrix to Matrix Multiplication (GEMM) operator as an example, a high-performance GEMM kernel can include various different types of operations in its core computation loop. Different types of operations can have different functionalities, and these types of operations are tightly coupled within the function or loop body.

[0039] To reduce instruction overhead and control flow divergence, functional code indicating different types of operations is often manually inlined, loop-expanded, and mixed with inline assembly instructions, resulting in large, flat, and extremely complex code blocks. This programming paradigm is currently the common way to achieve high-performance code in processing systems without native coroutine support.

[0040] However, the aforementioned code writing or generation methods suffer from at least one of the following drawbacks: poor readability and maintainability, low code reusability, high skill requirements for developers, and slow algorithm iteration and update speed. For example, multiple code logics can be mixed together in the same loop, making it extremely difficult for developers to understand the true intent and data flow of the code. Furthermore, even minor modifications to any part of the logic (e.g., adjusting the size of the data to be computed (e.g., matrix partitioning) or changing the data type) may require reviewing and modifying the entire code. This approach easily introduces new bugs, resulting in high debugging and testing costs. For instance, because computational logic is encoded together and cannot be reused independently, code highly optimized for specific computational tasks (e.g., operators) and hardware architectures is difficult to port to other computational tasks (e.g., operators) or new hardware architectures. Moreover, very few developers are capable of writing or modifying such complex code, leading to long development cycles and high human resource costs for programming tasks (e.g., high-performance operators), hindering rapid algorithm iteration and innovation.

[0041] Figure 1 A flowchart of a code generation method according to at least one embodiment of the present disclosure is shown. Figure 1 As shown, the code generation method 1000 includes steps S1100-S1400. Figure 2 A schematic diagram of a code generation method according to at least one embodiment of the present disclosure is shown.

[0042] In step S1100, multiple execution modules can be obtained, each of which includes a first code portion and a loop identifier. According to at least one embodiment of this disclosure, such as... Figure 2 As shown, multiple execution modules 2110, 2120, 2130, ..., 21N0 can be provided, but this disclosure is not limited to this, and more or fewer execution modules can be provided. Each of the multiple execution modules 2110, 2120, 2130, ..., 21N0 may include a corresponding first code portion and a tag. For example, execution module 2110 may include a first code portion 2111 and a tag 2112, execution module 2120 may include a first code portion 2121 and a tag 2122, execution module 2130 may include a first code portion 2131 and a tag 2132, ..., and execution module 21N0 may include a first code portion 21N1 and a tag 21N2.

[0043] According to at least one embodiment of this disclosure, the first code portions 2111, 2121, 2131, ..., 21N1 included in each of the plurality of execution modules 2110, 2120, 2130, ..., 21N0 can correspond to different functions in the algorithm. For example, the first code portions 2111, 2121, 2131, ..., 21N1 included in each of the plurality of execution modules 2110, 2120, 2130, ..., 21N0 can be a series of independent, single-function code segments that decompose the code of an algorithm for a complex computational task according to its semantics. Additionally or alternatively, the first code portions 2111, 2121, 2131, ..., 21N1 included in each of the plurality of execution modules 2110, 2120, 2130, ..., 21N0 can be a series of independent, single-function code segments written separately for the semantics of an algorithm for a complex computational task.

[0044] According to at least one embodiment of this disclosure, tags 2112, 2122, 2132, ..., 21N2 can be metadata attached to execution modules 2110, 2120, 2130, ..., 21N0 to describe the position and context information of the execution module in the original execution flow or the execution flow of the concatenated code. The tags 2112, 2122, 2132, ..., 21N0 of multiple execution modules 2110, 2120, 2130, ..., 21N0 can each include a corresponding loop identifier. Tags 2112, 2122, 2132, ..., 21N2 can provide a basis for subsequent sorting and merging. For example, the code of an algorithm for a complex computational task may have multiple loops, and the multiple execution modules 2110, 2120, 2130, ..., 21N0 may correspond to the corresponding loops in the multiple loops. For example, execution modules 2110, 2120, 2130, ..., 21N0 may have a first loop identifier, which indicates a first loop; and execution modules 2130, ..., 21N0 may have a second loop identifier, which indicates a second loop. However, this disclosure is not limited thereto, and the multiple execution modules 2110, 2120, 2130, ..., 21N0 may include loop identifiers indicating more or fewer loops.

[0045] In step S1200, multiple execution modules can be divided into multiple execution module groups based on the loop identifier, and a first order of the multiple execution module groups can be determined. According to at least one embodiment of this disclosure, execution modules having the same or at least partially identical loop identifiers can be grouped into the same execution module group. For example, as... Figure 2As shown, execution modules 2110 and 2120 with first loop identifiers can be assigned to the first execution module group, and execution modules 2130, ..., 21N0 with second loop identifiers can be assigned to the second execution module group. However, this disclosure is not limited to this, and more or fewer execution module groups can be formed depending on the number of loops indicated by the loop identifiers of the multiple execution modules 2110, 2120, 2130, ..., 21N0.

[0046] According to at least one embodiment of this disclosure, a code refactoring tool can determine a first order of multiple execution module groups based on loop identifiers. For example, the first order may indicate that the concatenation order of a first execution module group including execution modules 2110, 2120 with first loop identifiers (described in detail with reference to step S1400) precedes the concatenation order of a second execution module group including execution modules 2130, ..., 21N0 with second loop identifiers. For example, when determining the first order, the loop identifiers corresponding to the multiple execution module groups can be sorted lexicographically, but this disclosure is not limited to this, and other sorting methods are also possible. The first order of multiple execution module groups can be determined based on the result of the lexicographical sorting of the loop identifiers. For example, for execution modules of the same type, the concatenation order of execution modules with first loop identifiers may precede the concatenation order of execution modules with second loop identifiers.

[0047] In step S1300, multiple first code portions corresponding to each execution module group in the multiple execution module groups can be concatenated into a second code portion. According to at least one embodiment of this disclosure, such as... Figure 2 As shown, the code refactoring tool can concatenate the first code portions 2111 and 2121 of execution modules 2110 and 2120 in the first execution module group into a second code portion 2210, and can concatenate the first code portions 2131, ..., and 21N1 of execution modules 2130, ..., and 21N0 in the second execution module group into a second code portion 2220. According to at least one embodiment of this disclosure, the code refactoring tool can parse execution modules with tags and concatenate the execution modules into a complete, compilable code file based on the tag information.

[0048] In step S1400, multiple second code portions corresponding to multiple execution module groups can be concatenated into a target code portion according to a first order of the multiple execution module groups. According to at least one embodiment of this disclosure, such as... Figure 2As shown, following the first execution module group determined in step S1300, the code refactoring unit concatenates the second code portion 2210 and the second code portion 2220 into the target code portion 2300, with the second code portion 2210 preceding the second code portion 2220. The target code portion 2300 can be code executed by a thread bundle of the artificial intelligence processor.

[0049] Figure 3 A schematic diagram of a code generation method according to at least one embodiment of the present disclosure is shown. Figure 3 The process of concatenating the first code portions corresponding to multiple execution modules into a second code portion is shown in each execution module group. Figure 3 Zhongyu Figure 2 Identical or similar parts will not be described again to avoid redundancy.

[0050] According to at least one embodiment of this disclosure, a plurality of execution modules 3110, 3120, 3130, ..., 3200 may be provided, but this disclosure is not limited thereto, and more or fewer execution modules may be provided. Each of the plurality of execution modules 3110, 3120, 3130, ..., 3200 may include a corresponding first code portion and a tag. For example, execution module 3110 may include a first code portion 3111 and a label 3112; execution module 3120 may include a first code portion 3121 and a label 3122; execution module 3130 may include a first code portion 3131 and a label 3132; execution module 3140 may include a first code portion 3141 and a label 3142; execution module 3150 may include a first code portion 3151 and a label 3152; execution module 3160 may include a first code portion 3161 and a label 3162; execution module 3170 may include a first code portion 3171 and a label 3172; execution module 3180 may include a first code portion 3181 and a label 3182; execution module 3190 may include a first code portion 3191 and a label 3192; and execution module 3200 may include a first code portion 3201 and a label 3202.

[0051] According to at least one embodiment of this disclosure, the first code portions 3111, 3121, 3131, ..., 3201 of each of the plurality of execution modules 3110, 3120, 3130, ..., 3200 can correspond to different functions in the algorithm. The labels 3112, 3122, 3132, ..., 3202 of the plurality of execution modules 3110, 3120, 3130, ..., 3200 can each include a corresponding loop identifier. Taking the general GEMM operator as an example, the target code portion obtained by concatenation can include the general matrix multiplication operation kernel code, but this disclosure is not limited to this; execution modules can be written for other operators (e.g., convolution operators, pooling operators, etc.) and the corresponding target code portions can be obtained. In the embodiment of the GEMM operator, the loop identifier can be used to identify the outer loop of the execution module in the matrix multiplication operation. The outer loop can refer to the process by which thread bundles, etc., traverse each block of the output matrix and cooperate to schedule the computation tasks sequentially. According to at least one embodiment of this disclosure, multiple execution modules 3110, 3120, 3130, ..., 3200 may have the same cycle identifier.

[0052] According to at least one embodiment of this disclosure, the labels 2112, 2122, 2132, ..., 21N2 of the plurality of execution modules 2110, 2120, 2130, ..., 21N0 may each include a corresponding iteration index. The iteration index can be used to identify at least one of the loop unrolling order and pipelined order of the corresponding execution module in the outer loop of the matrix multiplication operation. As described above, the plurality of execution modules 3110, 3120, 3130, ..., 3200 may have the same loop identifier, therefore the plurality of execution modules 3110, 3120, 3130, ..., 3200 may correspond to the same outer loop. The iteration index can be used to identify at least one of the loop unrolling order and pipelined order of the corresponding execution module in that outer loop. Loop unrolling can refer to repeating the loop body multiple times in the code, thereby reducing the overhead of loop control instructions (e.g., conditional judgments, increments, etc.). Loop unrolling can improve instruction-level parallelism and register reuse by increasing the workload within a single loop. Pipelines can break down a single task into multiple consecutive stages and allow different stages of different tasks to be executed in overlapping time.

[0053] According to at least one embodiment of this disclosure, taking loop unrolling as an example, multiple execution modules belonging to a single loop unrolling can have the same iteration index. For example, execution modules 3110, ..., 3150 can have a first iteration index indicating a first iteration process, and execution modules 3160, ..., 3200 can have a second iteration index indicating a second iteration process. A second order of each execution module in the corresponding execution module group can be determined based on the iteration index. According to at least one embodiment of this disclosure, a code refactoring tool can determine the second order of multiple execution modules in the execution module group based on the iteration index. For example, the second order can indicate that the concatenation order of execution modules with the first iteration index precedes the concatenation order of execution modules with the second iteration index. For example, when determining the second order, the iteration indices corresponding to the multiple execution module groups can be sorted lexicographically, but this disclosure is not limited to this, and other sorting methods are also possible. The second order of multiple execution modules can be determined based on the result of the lexicographical sorting of the iteration indices. Further, multiple first code parts can be concatenated into a second code part according to the second order. For example, the assembly order of execution modules 3110, ..., 3150 corresponding to the first iteration process can precede the assembly order of execution modules 3160, ..., 3200 corresponding to the second iteration process.

[0054] According to at least one embodiment of this disclosure, taking the GEMM operator as an example, multiple execution modules include one or more of a first matrix loading module, a second matrix loading module, a synchronization module, a calculation module, and a write-back module. For example, execution modules 3110 and 3160 may correspond to the first matrix loading module, execution modules 3120 and 3170 may correspond to the second matrix loading module, execution modules 3130 and 3180 may correspond to the synchronization module, execution modules 3140 and 3190 may correspond to the calculation module, and execution modules 3150 and 3200 may correspond to the write-back module. The first matrix loading module can be configured to perform a first loading operation. The first loading operation includes loading data blocks included in a first matrix to be multiplied in matrix multiplication from global memory. The first loading operation may load the data blocks included in the first matrix into shared memory or registers, and this disclosure is not limited thereto. The second matrix loading module can be configured to perform a second loading operation. The second loading operation includes loading data blocks included in a second matrix to be multiplied in matrix multiplication from global memory. The second loading operation may load the data blocks included in the second matrix into shared memory or registers, and this disclosure is not limited thereto. The synchronization module can be configured to synchronize the first loading operation of the first matrix loading module with the second loading operation of the second matrix loading module. In this way, it can be ensured that the data blocks of the first and second matrices have been loaded respectively before matrix multiplication calculations are performed. The calculation module can be configured to perform matrix multiplication based on the data blocks included in the loaded first and second matrices, and store the result of the matrix multiplication in a register. The write-back module can be configured to write the result of the matrix multiplication back to global memory. For example, the write-back module can be configured to write the result of the matrix multiplication from a register back to global memory.

[0055] like Figure 3As shown, the loop identifiers and iteration indices of execution modules 3110, ..., 3150 can be the same (e.g., having a first loop identifier and a first iteration index), and the loop identifiers and iteration indices of execution modules 3160, ..., 3200 can be the same (e.g., having a second loop identifier and a second iteration index). According to at least one embodiment of this disclosure, a code refactoring tool can obtain the dependencies of at least a portion of the execution modules among a plurality of execution modules. Dependencies can refer to the dependencies formed between different execution modules due to the association of data to be processed. For example, a code refactoring tool can obtain the dependencies of execution modules with the same loop identifier and iteration index, thereby obtaining a third order. The code refactoring tool can determine the third order of at least a portion of the execution modules in the corresponding execution module group based on the dependencies. In one embodiment, the label of the execution module can further include the dependency. In another embodiment, the first code portion included in execution modules having the same loop representation and iteration index can be analyzed to obtain the dependency. For example, data flow analysis or control flow analysis can be performed on the first code portion to obtain the dependency, but this disclosure is not limited thereto.

[0056] According to at least one embodiment of this disclosure, multiple first code portions, including at least the first code portions corresponding to execution modules having the same loop identifier and iteration index, can be concatenated into a second code portion according to a third order. For example, for execution modules 3110, ..., 3150 having a first loop identifier and a first iteration index, it can be determined based on dependencies that the first matrix loading module and the second matrix loading module need to precede the synchronization module, the synchronization module needs to precede the calculation module, and the calculation module needs to precede the write-back module. However, this disclosure is not limited to this, and other third orders are also possible. Similarly, for execution modules 3160, ..., 3200 having a first loop identifier and a first iteration index, it can be determined based on dependencies that the first matrix loading module and the second matrix loading module need to precede the synchronization module, the synchronization module needs to precede the calculation module, and the calculation module needs to precede the write-back module. However, this disclosure is not limited to this, and other third orders are also possible. In this way, the third order of execution modules 3110, ..., 3150 and the third order of execution modules 3160, ..., 3200 can be determined respectively. Furthermore, based on the third and second orders, the execution modules 3110, ..., 3200 are concatenated into the second code section 3300.

[0057] For reference Figure 3The second code portion, including the second code portion 3300, can be concatenated into a target code portion in a first order. Further, a startup function for the target code portion can be obtained. According to at least one embodiment of this disclosure, the startup function may be the gemm_kernel function, but this disclosure is not limited thereto. The startup function and the target code portion can be encapsulated into a target code file for subsequent execution of computational tasks.

[0058] The method according to at least one embodiment of this disclosure can significantly improve code readability and maintainability, enhance code reusability, and reduce the skill requirements for developers without compromising code performance, thereby accelerating the iteration and update of code-based algorithms.

[0059] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0060] Artificial intelligence processors may include graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), tensor processing units (TPUs), deep learning processing units (DPUs), accelerated processing units (APUs), neural network processing units (NPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs), but this disclosure is not limited thereto.

[0061] like Figure 4 As shown, an example of an artificial intelligence processor is an array of programmable multiprocessors, which can be used in the code generation method of this disclosure embodiment. For example, the programmable multiprocessors of the artificial intelligence processor can be a Streaming Processor Cluster (SPC), such as including streaming processor cluster 1, ..., streaming processor cluster M as shown in the figure, where M is a positive integer greater than 1. In the artificial intelligence processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0062] like Figure 4 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 4 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores (also called computational cores), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These Cores are used to execute specific computational tasks. Furthermore, a Computation Unit may also include registers (e.g., ...). Figure 4 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0063] like Figure 4 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0064] like Figure 4 As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0065] In parallel computing, computational tasks are typically executed through multiple threads. These threads are divided into multiple thread blocks before execution in the artificial intelligence processor (or general-purpose graphics processor or parallel computing processor), and then dispatched via a thread block dispatch module. Figure 4 (Not shown in the diagram) Multiple thread blocks are distributed to various computing units. All threads in a thread block must be assigned to the same computing unit for execution. Simultaneously, thread blocks are divided into minimum execution thread bundles (or simply thread bundles, warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple threads from multiple thread blocks can execute on multiple computing cores within the same computing unit, or on multiple computing cores in different computing units, sharing registers, shared memory, etc., and transferring data. The target code portion in the code generation method according to at least one embodiment of this disclosure can be executed by a single thread bundle.

[0066] In each computing unit, the thread beam scheduling / distribution module ( Figure 4(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 4 It is used for reading and writing operations in high-bandwidth memory (HBM).

[0067] Figure 4 The AI ​​processor shown can execute the above code generation method, which includes: acquiring multiple execution modules, each of the multiple execution modules including a first code portion and a loop identifier; dividing the multiple execution modules into multiple execution module groups according to the loop identifier and determining a first order of the multiple execution module groups; concatenating multiple first code portions corresponding to each execution module group in the multiple execution module groups into a second code portion; and concatenating multiple second code portions corresponding to the multiple execution module groups into a target code portion according to the first order of the multiple execution module groups.

[0068] For example, according to at least one embodiment of the code generation method of this disclosure, dividing a plurality of execution modules into a plurality of execution module groups includes: grouping execution modules with the same loop identifier into the same execution module group.

[0069] For example, according to at least one embodiment of the code generation method of this disclosure, determining the first order of a plurality of execution module groups includes: sorting the loop identifiers corresponding to the plurality of execution module groups in lexicographical order; and determining the first order of the plurality of execution module groups based on the result of the lexicographical sorting of the loop identifiers.

[0070] For example, according to at least one embodiment of the code generation method of this disclosure, each of the plurality of execution modules further includes an iteration index, and the method further includes: determining a second order of each of the plurality of execution modules in a corresponding execution module group according to the iteration index, wherein concatenating a plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating a plurality of first code parts into a second code part according to the second order.

[0071] For example, according to at least one embodiment of the code generation method of this disclosure, determining the second order of each execution module in a corresponding execution module group in a plurality of execution modules includes: sorting the iteration indices corresponding to the execution modules included in each execution module group in a lexicographical order; and determining the second order of each execution module in a corresponding execution module group in a plurality of execution modules based on the result of the lexicographical sorting of the iteration indices.

[0072] For example, according to at least one embodiment of the code generation method of this disclosure, the method further includes: obtaining the dependency relationship of at least a portion of the execution modules in a plurality of execution modules; determining the third order of the at least a portion of the execution modules in the corresponding execution module group according to the dependency relationship, wherein concatenating a plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating a plurality of first code parts, including at least a portion of the first code parts corresponding to the execution modules, into a second code part according to the third order.

[0073] For example, in a code generation method according to at least one embodiment of the present disclosure, at least a portion of the execution modules have the same loop identifier and iteration index.

[0074] For example, according to at least one embodiment of the code generation method of this disclosure, obtaining the dependencies of at least a portion of the execution modules among a plurality of execution modules includes at least one of the following: obtaining the dependencies included in at least a portion of the execution modules; and analyzing a first code portion included in at least a portion of the execution modules to obtain the dependencies.

[0075] For example, according to at least one embodiment of the code generation method of this disclosure, the method further includes: obtaining a startup function for a target code portion; and encapsulating the startup function and the target code portion into a target code file.

[0076] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the plurality of execution modules include one or more of the following: a first matrix loading module configured to perform a first loading operation, the first loading operation including loading data blocks included in a first matrix from global memory; a second matrix loading module configured to perform a second loading operation, the second loading operation including loading data blocks included in a second matrix from global memory; a synchronization module configured to synchronize the first loading operation of the first matrix loading module and the second loading operation of the second matrix loading module; a calculation module configured to perform matrix multiplication based on the loaded data blocks included in the first matrix and the loaded data blocks included in the second matrix, and store the result of the matrix multiplication in a register; and a write-back module configured to write the result of the matrix multiplication back to global memory.

[0077] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the loop identifier is used to identify the outer loop of the execution module in the matrix multiplication operation.

[0078] For example, according to at least one embodiment of the code generation method of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and wherein the iteration index is used to identify at least one of the loop unrolling order and pipeline order in the outer loop of the execution module in the matrix multiplication operation.

[0079] Figure 5 A schematic diagram of an electronic device according to at least one embodiment of the present disclosure is shown.

[0080] like Figure 5 As shown, the electronic device 5000 may include at least one processing unit 5100 and at least one memory 5200. The at least one memory 5200 includes one or more computer program modules 5210. The one or more computer program modules 5210 are stored in the at least one memory 5200 and can be configured to be read and executed by the at least one processing unit 5100. The one or more computer program modules 5210 include instructions for performing the various methods described above according to at least one embodiment of the present disclosure. When executed by the at least one processing unit 5100, they can perform one or more steps of the various methods described above according to at least one embodiment of the present disclosure and their additional aspects.

[0081] At least one memory 5200 and at least one processing unit 5100 may be interconnected via a bus system and / or other forms of connection mechanism (not shown). For example, the bus may be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0082] Exemplarily, at least one processing unit 5100 may be a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), or other processing elements with data processing capabilities and / or program execution capabilities, such as a field-programmable gate array (FPGA). At least one processing unit 5100 may be a general-purpose processor or a dedicated processor, capable of controlling other components in the electronic device 5000 to perform desired functions. According to at least one embodiment of this disclosure, the electronic device 5000 may be such as... Figure 4 The artificial intelligence processor shown.

[0083] Exemplarily, at least one memory 5200 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules 5210 may be stored on the computer-readable storage medium, and at least one processing unit 5100 may run one or more computer program modules 5210 to implement various functions of the electronic device 5000. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0084] For example, electronic device 5000 may also include input devices such as cameras, touchscreens, touchpads, keyboards, mice, webcams, microphones, accelerometers, and gyroscopes; output devices such as liquid crystal displays, speakers, and vibrators; storage devices such as magnetic tapes and hard disks (HDDs or SDDs); and communication devices such as network interface cards like LAN cards and modems. The communication devices allow electronic device 5000 to communicate wirelessly or wiredly with other devices to exchange data and perform communication processing via networks such as the Internet. A drive is connected to the I / O interface as needed. Removable storage media, such as disks, optical disks, magneto-optical disks, and semiconductor memories, are installed on the drive as needed so that computer programs read from them can be installed into the storage device as required.

[0085] For example, the electronic device 5000 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device can communicate wirelessly with networks and other devices, such as the Internet, intranets and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0086] The electronic device 5000 may be, for example, a system-on-a-chip (SOC) or a device including the SOC. For instance, it can be any device such as a mobile phone, tablet computer, laptop computer, e-reader, game console, television, digital photo frame, navigator, home appliance, communication base station, industrial controller, server, etc., or any combination of data processing devices and hardware. The embodiments of this disclosure do not limit this. The specific functions and technical effects of the electronic device 5000 can be found in the foregoing description of the method and its additional aspects according to at least one embodiment of this disclosure, and will not be repeated here.

[0087] Figure 6 A schematic diagram of a code generation apparatus according to at least one embodiment of the present disclosure is shown. Figure 6 As shown, the code generation device 6000 may include an acquisition module 6100, a division module 6200, a first splicing module 6300, and a second splicing module 6400.

[0088] The acquisition module 6100 can be configured to acquire multiple execution modules, each of which includes a first code portion and a loop identifier.

[0089] The partitioning module 6200 can be configured to divide multiple execution modules into multiple execution module groups based on a loop identifier and determine the first order of the multiple execution module groups.

[0090] The first concatenation module 6300 can be configured to concatenate multiple first code portions corresponding to each of the multiple execution module groups into a second code portion.

[0091] The second splicing module 6400 can be configured to splice multiple second code parts corresponding to multiple execution module groups into a target code part according to the first order of multiple execution module groups.

[0092] According to at least one embodiment of the code generation apparatus of the present disclosure, the partitioning module 6200 is further configured to partition execution modules with the same cycle identifier among a plurality of execution modules into the same execution module group.

[0093] According to at least one embodiment of the code generation apparatus of the present disclosure, the acquisition module 6100 may further be configured to sort the loop identifiers corresponding to the plurality of execution module groups in lexicographical order; and to determine a first order of the plurality of execution module groups based on the result of sorting the loop identifiers in lexicographical order.

[0094] According to at least one embodiment of the code generation apparatus of the present disclosure, the acquisition module 6100 may be further configured to determine the second order of each of the plurality of execution modules in the corresponding execution module group according to the iteration index, wherein concatenating the plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating the plurality of first code parts into a second code part according to the second order.

[0095] According to at least one embodiment of the code generation apparatus of the present disclosure, the acquisition module 6100 may further be configured to sort the iteration indices corresponding to the execution modules included in each of the plurality of execution module groups in lexicographical order; and determine the second order of each execution module in the corresponding execution module group based on the result of the lexicographical sorting of the iteration indices.

[0096] According to at least one embodiment of the code generation apparatus of this disclosure, the acquisition module 6100 may further be configured to acquire the dependency relationship of at least a portion of the execution modules among a plurality of execution modules; and determine a third order of at least a portion of the execution modules in the corresponding execution module group according to the dependency relationship, wherein concatenating a plurality of first code parts corresponding to each execution module group in the plurality of execution module groups into a second code part includes: concatenating a plurality of first code parts, including at least a portion of the first code parts corresponding to the execution modules, into a second code part according to the third order.

[0097] According to at least one embodiment of the code generation apparatus of the present disclosure, at least a portion of the execution modules have the same loop identifier and iteration index.

[0098] According to at least one embodiment of the code generation apparatus of the present disclosure, the acquisition module 6100 may further be configured to acquire dependencies included in at least a portion of the execution modules; and to analyze a first code portion included in at least a portion of the execution modules to acquire dependencies.

[0099] According to at least one embodiment of the code generation apparatus of the present disclosure, the acquisition module 6100 may further be configured to acquire a startup function for a target code portion; and encapsulate the startup function and the target code portion into a target code file.

[0100] According to at least one embodiment of the code generation apparatus of this disclosure, the target code portion includes general matrix multiplication operation kernel code; and the plurality of execution modules include one or more of the following: a first matrix loading module configured to perform a first loading operation, the first loading operation including loading data blocks included in a first matrix from global memory; a second matrix loading module configured to perform a second loading operation, the second loading operation including loading data blocks included in a second matrix from global memory; a synchronization module configured to synchronize the first loading operation of the first matrix loading module and the second loading operation of the second matrix loading module; a calculation module configured to perform matrix multiplication based on the loaded data blocks included in the first matrix and the loaded data blocks included in the second matrix, and store the result of the matrix multiplication in a register; and a write-back module configured to write the result of the matrix multiplication back to global memory.

[0101] According to at least one embodiment of the code generation apparatus of the present disclosure, the target code portion includes general matrix multiplication operation kernel code; and the loop identifier is used to identify the outer loop of the execution module in the matrix multiplication operation.

[0102] According to at least one embodiment of the code generation apparatus of the present disclosure, the target code portion includes general matrix multiplication operation kernel code; and wherein the iteration index is used to identify at least one of the loop unrolling order and pipeline order in the outer loop of the execution module in the matrix multiplication operation.

[0103] Figure 7 A schematic block diagram of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0104] like Figure 7 As shown, the electronic device 300 is, for example, suitable for implementing the data transmission method for an artificial intelligence processor provided in embodiments of this disclosure. It should be noted that... Figure 7 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0105] For example, such as Figure 7 As shown, in some examples, electronic device 300 includes a processing device (e.g., including as referenced). Figure 4 The AI ​​processor 301 shown is a parallel processor such as a graphics processor, a general-purpose graphics processor, etc. It can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303 to achieve various functions, such as executing the code generation method provided in at least one embodiment of this disclosure.

[0106] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the code generation method according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the code generation method can be found in the relevant descriptions in the embodiments of the above code generation method; repeated details will not be repeated here.

[0107] RAM 303 also stores various programs and data required for the operation of the computer system. Processing device 301, ROM 302, and RAM 303 are connected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0108] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0109] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage device 308 as needed. Although Figure 7 An electronic device 300 including various devices is shown; however, it should be understood that implementation or inclusion of all shown devices is not required. More or fewer devices may be implemented or included alternatively. For example, a processing device 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing and / or program execution capabilities, such as a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a parallel processor, an artificial intelligence processor, a tensor processor (TPU), or a graphics processing unit (GPU). The CPU may be based on an x86 or ARM architecture, etc. The GPU may be integrated directly onto the motherboard or built into the motherboard's northbridge chip. Alternatively, the GPU may be built into the CPU.

[0110] For example, the electronic device 300 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device 309 can communicate wirelessly with a network and other devices, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0111] For example, electronic device 300 can be any device such as mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, navigator, etc., or any combination of electronic devices and hardware. The embodiments disclosed herein do not limit this.

[0112] For example, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, the code generation method described above, as defined in the methods of embodiments of this disclosure, is performed.

[0113] According to at least one embodiment of this disclosure, electronic device 300 may include the electronic device or code generation device described in the above embodiments.

[0114] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0115] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. For example, a computer-readable storage medium provided according to at least one embodiment of the present disclosure may store instructions thereon that, when executed by one or more processors, cause the processors to perform the methods described above.

[0116] The code generation method, electronic device, storage medium, electronic device, code generation apparatus, and computer program product provided according to at least one embodiment of the present disclosure can significantly improve code readability and maintainability, enhance code reusability, and reduce the skill requirements for developers without compromising code performance, thereby accelerating the iteration and update of code-based algorithms.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0119] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0120] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0121] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0122] The following points should be noted regarding this disclosure:

[0123] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0124] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0125] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A code generation method, characterized in that, include: Obtain multiple execution modules, each of which includes a first code portion, a loop identifier, and an iteration index; Based on the cycle identifier, the plurality of execution modules are divided into a plurality of execution module groups and the first order of the plurality of execution module groups is determined; The multiple first code portions corresponding to each of the multiple execution module groups are concatenated into a second code portion; According to the first order of the plurality of execution module groups, the plurality of second code parts corresponding to the plurality of execution module groups are concatenated into the target code part; as well as The second order of each execution module in the corresponding execution module group is determined based on the iterative index. The step of concatenating the multiple first code portions corresponding to each of the multiple execution module groups into a second code portion includes: The plurality of first code portions are concatenated into the second code portion according to the second order.

2. The code generation method according to claim 1, characterized in that, The step of dividing the plurality of execution modules into a plurality of execution module groups includes: Execution modules with the same loop identifier are grouped into the same execution module group.

3. The code generation method according to claim 2, characterized in that, The first order of determining the plurality of execution module groups includes: Sort the loop identifiers corresponding to the plurality of execution module groups lexicographically; and The first order of the plurality of execution module groups is determined based on the lexicographical order of the loop identifiers.

4. The code generation method according to claim 1, characterized in that, The step of determining the second order of each of the plurality of execution modules in the corresponding execution module group includes: The iteration indices corresponding to the execution modules included in each of the plurality of execution module groups are sorted lexicographically; and Based on the lexicographical sorting of the iterative index, the second order of each of the plurality of execution modules in the corresponding execution module group is determined.

5. The code generation method according to claim 1, characterized in that, The method further includes: Obtain the dependencies of at least a portion of the execution modules among the plurality of execution modules; Based on the dependencies, determine the third order of at least a subset of execution modules within the corresponding execution module group. The step of concatenating the multiple first code portions corresponding to each of the multiple execution module groups into a second code portion includes: According to the third order, multiple first code parts, including at least a portion of the first code parts corresponding to the execution modules, are concatenated into the second code part.

6. The code generation method according to claim 5, characterized in that, The loop identifier and the iteration index are the same for at least a portion of the execution modules.

7. The code generation method according to claim 5, characterized in that, The step of obtaining the dependencies of at least a portion of the execution modules among the plurality of execution modules includes at least one of the following: Obtain the dependencies included in at least a portion of the execution modules; as well as The first code portion included in at least a portion of the execution module is analyzed to obtain the dependencies.

8. The code generation method according to claim 1, characterized in that, The method further includes: Obtain the startup function for the target code segment; and The startup function and the target code portion are encapsulated into a target code file.

9. The code generation method according to any one of claims 1-8, characterized in that, The target code portion includes kernel code for general matrix multiplication operations; and The plurality of execution modules include one or more of the following: The first matrix loading module is configured to perform a first loading operation, which includes loading data blocks included in the first matrix from global memory. The second matrix loading module is configured to perform a second loading operation, which includes loading data blocks included in the second matrix from global memory. The synchronization module is configured to synchronize the first loading operation of the first matrix loading module and the second loading operation of the second matrix loading module. The calculation module is configured to perform matrix multiplication based on data blocks included in a first loaded matrix and data blocks included in a second loaded matrix, and store the results of the matrix multiplication in a register. The write-back module is configured to write the result of the matrix multiplication back to global memory.

10. The code generation method according to any one of claims 1-8, characterized in that, The target code portion includes kernel code for general matrix multiplication operations; and The loop identifier is used to identify the outer loop of the execution module in the matrix multiplication operation. The outer loop includes the process of traversing each block of the output matrix of the general matrix multiplication operation kernel code.

11. The code generation method according to claim 10, characterized in that, The target code portion includes kernel code for general matrix multiplication operations; and The iteration index is used to identify at least one of the loop unrolling order and pipeline order in the outer loop of the execution module in the matrix multiplication operation.

12. An electronic device, characterized in that, include: At least one processing unit; At least one memory, including one or more computer program modules, The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processing unit, and the one or more computer program modules are used to implement the method of any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-11.

14. A code generation apparatus, characterized in that, include: The acquisition module is configured to acquire multiple execution modules, each of which includes a first code portion, a loop identifier, and an iteration index. The partitioning module is configured to divide multiple execution modules into multiple execution module groups based on the loop identifier and determine a first order of the multiple execution module groups; The first concatenation module is configured to concatenate multiple first code parts corresponding to each of the multiple execution module groups into a second code part; as well as The second concatenation module is configured to concatenate multiple second code portions corresponding to the multiple execution module groups into a target code portion according to a first order of the multiple execution module groups. The acquisition module is further configured to determine the second order of each of the plurality of execution modules in the corresponding execution module group based on the iteration index. The step of concatenating the multiple first code portions corresponding to each of the multiple execution module groups into a second code portion includes: The plurality of first code portions are concatenated into the second code portion according to the second order.

15. An electronic device, characterized in that, This includes the electronic device as described in claim 12 or the code generation device as described in claim 14.

Images