Fusion computing kernel compiling method and device, electronic equipment and storage medium

Abstract

The disclosure provides a fusion computing kernel compilation method and device, electronic equipment and storage medium. The method comprises the following steps: determining a first thread bundle configuration for a master thread part and a second thread bundle configuration for a work function part, the first thread bundle configuration being a single thread bundle configuration, and the second thread bundle configuration being a multi-thread bundle configuration; compiling the master thread part based on the first thread bundle configuration to obtain master code; compiling the work function part based on the second thread bundle configuration to obtain work code; and generating a compilation result comprising the master code and the work code. The master code performs control and scheduling according to the first thread bundle configuration, and the work code performs vector calculation according to the second thread bundle configuration under the scheduling of the master code. Thus, the fusion computing kernel can simultaneously meet the single thread bundle execution requirement of the master thread part and the multi-thread bundle parallel demand of the work function part, release the multi-thread bundle parallel capability of the work function part, and reduce the idle of the computing resources.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence compilation technology, and in particular to a method, apparatus, electronic device and storage medium for compiling a fusion computing kernel. Background Technology

[0002] Artificial intelligence compilers are used to convert AI computing tasks into target code that can be executed by the target chip. AI computing tasks typically include various types of operations such as control flow, matrix operations, data handling, and vector computation. By merging multiple operations into a single computing kernel, the compiler can produce a fused computing kernel.

[0003] In related technologies, the same fused computing kernel typically uses a globally uniform number of thread bundles. Different computation stages, different instruction types, and different execution units within the fused computing kernel share the same thread bundle configuration.

[0004] However, when the fused computing kernel adopts a unified thread bundle configuration, the compiler needs to set the total number of thread bundles in the entire fused computing kernel to 1 to meet the single-threaded execution requirement of the main control thread. At this time, the worker functions also run in a single-threaded manner, which limits the multi-threaded parallelism capability of the worker functions. Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, and storage medium for compiling a converged computing kernel. It can reduce the limitation on the parallelism of the working function part by the single-threaded bundle operation requirements of the main control thread part, release the multi-threaded bundle parallelism capability of the working function part, and reduce idle computing resources.

[0006] According to one aspect of this disclosure, a method for compiling a fusion computing kernel is provided. The fusion computing kernel includes a master control thread portion for performing control and scheduling, and a working function portion for performing vector computation under the scheduling of the master control thread portion. The method includes: determining a first thread bundle configuration for the master control thread portion, wherein the first thread bundle configuration is a single-thread bundle configuration; determining a second thread bundle configuration for the working function portion based on the number of compile-time thread bundles, wherein the second thread bundle configuration is a multi-thread bundle configuration; compiling the master control thread portion based on the first thread bundle configuration to obtain master control code; compiling the working function portion based on the second thread bundle configuration to obtain working code; and generating a compilation result including the master control code and the working code; wherein the compilation result is used to enable the master control code to perform control and scheduling according to the first thread bundle configuration, and to enable the working code to perform vector computation under the scheduling of the master control code according to the second thread bundle configuration, during the runtime of the fusion computing kernel.

[0007] Optionally, determining the second thread bundle configuration for the working function portion based on the compile-time thread bundle count includes: writing the compile-time thread bundle count into the module-level attribute corresponding to the fused computing kernel; determining the thread bundle count corresponding to the second thread bundle configuration based on the compile-time thread bundle count recorded in the module-level attribute; wherein the first thread bundle configuration is independent of the compile-time thread bundle count recorded in the module-level attribute.

[0008] Optionally, the working function portion includes multiple working functions, which correspond to the same module-level attribute; the step of compiling the working function portion based on the second thread bundle configuration to obtain working code includes: having the multiple working functions read the number of the same compile-time thread bundle recorded in the module-level attribute; using the number of compile-time thread bundles, generating target code for the multiple working functions respectively to obtain working function code corresponding to each working function; and generating the working code based on the working function code corresponding to each working function.

[0009] Optionally, the working function part includes vector processing operations for tensor data; compiling the working function part based on the second thread bundle configuration to obtain working code includes: obtaining tensor encoding information corresponding to the vector processing operation, the tensor encoding information being used to characterize the distribution of tensor elements between threads and thread bundles; determining the mapping relationship between the tensor elements and multiple thread bundles based on the tensor encoding information and the number of thread bundles corresponding to the second thread bundle configuration; generating vectorized memory access instructions and vectorized computation instructions based on the mapping relationship between the tensor elements and multiple thread bundles; and generating the target code corresponding to the vector processing operation based on the vectorized memory access instructions and the vectorized computation instructions.

[0010] Optionally, the vector processing operations include at least one of layout transformation operations, vector arithmetic operations, vector logic operations, reduction operations, and special function operations.

[0011] Optionally, compiling the main control thread portion based on the first thread bundle configuration to obtain the main control code includes: compiling at least one of the scalar control flow operations, hardware scheduling operations, and worker function call operations in the main control thread portion according to the first thread bundle configuration to obtain code corresponding to the at least one operation; and compiling the code corresponding to the at least one operation into the main control code according to the control relationship of the fused computing kernel; wherein the scalar control code corresponding to the scalar control flow operation, the device scheduling code corresponding to the hardware scheduling operation, and the worker function call code corresponding to the worker function call operation are all generated according to the single thread bundle execution semantics.

[0012] Optionally, when the at least one operation includes the hardware scheduling operation, the hardware scheduling operation is compiled, including: reducing the hardware scheduling operation to device scheduling code executed according to the first thread bundle configuration; determining the scheduling interface information of the device scheduling code, wherein the scheduling interface information includes source operand address, target address, and device description information; wherein the device scheduling code is used to enable the main control code to schedule a target hardware execution unit based on the scheduling interface information, and the target hardware execution unit includes at least one of a matrix operation execution unit and a data transport execution unit.

[0013] Optionally, when the at least one operation includes the working function call operation, the working function call operation is compiled, including: determining the target working function corresponding to the working function call operation; generating a working function call instruction for calling the target working function; wherein the working function call instruction is used to cause the main control code running according to the first thread bundle configuration to schedule the working function code corresponding to the target working function generated according to the second thread bundle configuration.

[0014] Optionally, the master control code is configured to be executed by a master control execution unit, which is used to perform scalar control and hardware scheduling according to the first thread bundle configuration; the working code is configured to be executed by a working execution unit, which supports the parallel execution of multiple thread bundles and is used to perform the vector calculation according to the second thread bundle configuration; the master control execution unit is also used to schedule the working execution unit to execute the working code based on the master control code.

[0015] According to one aspect of this disclosure, a fusion computing kernel compilation apparatus is provided. The fusion computing kernel includes a master control thread portion for execution control and scheduling, and a working function portion for performing vector computation under the scheduling of the master control thread portion. The apparatus includes: a first thread bundle configuration determination module, configured to determine a first thread bundle configuration for the master control thread portion; the first thread bundle configuration is a single-thread bundle configuration; a second thread bundle configuration determination module, configured to determine a second thread bundle configuration for the working function portion based on the number of compile-time thread bundles; the second thread bundle configuration is a multi-thread bundle configuration; a master control code compilation module, configured to compile the master control thread portion based on the first thread bundle configuration to obtain master control code; a working code compilation module, configured to compile the working function portion based on the second thread bundle configuration to obtain working code; and a compilation result generation module, configured to generate a compilation result including the master control code and the working code; the compilation result is used to: during the runtime of the fusion computing kernel, cause the master control code to perform control and scheduling according to the first thread bundle configuration, and cause the working code to perform vector computation under the scheduling of the master control code according to the second thread bundle configuration.

[0016] According to one aspect of this disclosure, an electronic device is proposed, the electronic device including a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, wherein the program is executed by the processor to implement the fusion computing kernel compilation method as described above.

[0017] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores one or more programs, which can be executed by one or more processors to implement the fused computing kernel compilation method described above.

[0018] This disclosure proposes a method, apparatus, electronic device, and storage medium for compiling a converged computing kernel. The converged computing kernel includes a main control thread for execution control and scheduling, and a worker function part for performing vector computations under the scheduling of the main control thread. The main control thread adopts a single-threaded configuration, while the worker function part adopts a multi-threaded configuration. The two parts are compiled into main control code and worker code, respectively. When the converged computing kernel runs, the main control code performs control and scheduling, and the worker code performs vector computations under the scheduling of the main control code. Therefore, while ensuring stable scheduling on the main control side, the multi-threaded parallelism capability of the worker function part can be released, reducing idle computing resources.

[0019] Furthermore, the number of compile-time thread bundles is written into the module-level attribute corresponding to the fused computing kernel, and the number of thread bundles corresponding to the second thread bundle configuration is determined based on this module-level attribute; the first thread bundle configuration is independent of the compile-time thread bundle number recorded in this module-level attribute. Thus, the single-thread bundle configuration on the master side and the module-level multi-thread bundle configuration on the worker side are independent of each other, and a unified configuration source is provided for the worker function part.

[0020] Furthermore, when the working function section includes multiple working functions, each working function reads the compile-time thread bundle count recorded in the same module-level attribute and uses that count to generate the corresponding working function code. Thus, multiple working functions share the same module-level multi-thread bundle configuration, avoiding configuration dispersion or inconsistencies caused by repeated settings for each function.

[0021] Furthermore, when the working function includes vector processing operations for tensor data, tensor encoding information characterizing the distribution of tensor elements among threads and thread bundles is obtained. This information, combined with the second thread bundle configuration, determines the mapping relationship between tensor elements and multiple thread bundles, and accordingly generates vectorized memory access instructions and vectorized computation instructions. Vector processing operations may include at least one of layout transformation operations, vector arithmetic operations, vector logic operations, reduction operations, and special function operations, thereby adapting the corresponding operations to the parallel execution of multiple thread bundles.

[0022] Furthermore, at least one of the scalar control flow operations, hardware scheduling operations, and worker function call operations is compiled according to the first thread bundle configuration, and the corresponding code is compiled into the main control code according to the control relationship of the fused computing kernel; among them, the scalar control code, device scheduling code, and worker function call code are all generated according to the single thread bundle execution semantics, thereby ensuring that the main control code stably executes control and scheduling.

[0023] Furthermore, when the main control thread includes hardware scheduling operations, the hardware scheduling operations are reduced to device scheduling code executed according to the configuration of the first thread bundle. The source operand address, target address, and device description information are used as scheduling interface information, so that the main control code can schedule at least one of the matrix operation execution unit and the data transport execution unit, thereby decoupling single-thread bundle scheduling from multi-thread bundle vector calculation.

[0024] Furthermore, when the main control thread includes a worker function call operation, the target worker function corresponding to the worker function call operation is determined, and a worker function call instruction for calling the target worker function is generated, so that the main control code running according to the first thread bundle configuration can schedule the target worker function code generated according to the second thread bundle configuration, thereby establishing the execution connection between single-thread bundle control scheduling and multi-thread bundle vector calculation.

[0025] Furthermore, the main control code is configured to be executed by the main control execution unit, and the working code is configured to be executed by the working execution unit that supports the parallel execution of multiple thread bundles. The main control execution unit schedules the working execution units to execute the working code based on the main control code, thereby implementing the differentiated thread bundle configuration to the heterogeneous target hardware execution unit.

[0026] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0027] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0028] Figure 1 This is a schematic diagram of the hardware architecture of a heterogeneous chip provided in an embodiment of this disclosure;

[0029] Figure 2 This is a flowchart of a fusion computing kernel compilation method provided in an embodiment of this disclosure;

[0030] Figure 3 This is a flowchart illustrating the process of determining the number of thread bundles corresponding to the second thread bundle configuration in a fusion computing kernel compilation method provided in this embodiment of the present disclosure;

[0031] Figure 4 This is a flowchart illustrating the code generation process of multiple working functions in a fusion computing kernel compilation method provided in this embodiment of the disclosure;

[0032] Figure 5 This is a flowchart illustrating the target code generation process corresponding to vector processing operations in a fusion computing kernel compilation method provided in this disclosure embodiment;

[0033] Figure 6 This is a flowchart of the compilation process of the main control thread in a fusion computing kernel compilation method provided in this embodiment of the disclosure;

[0034] Figure 7 This is a flowchart of the compilation process of hardware scheduling operations in a fusion computing kernel compilation method provided in this disclosure embodiment;

[0035] Figure 8 This is a flowchart of the compilation process of working function call operations in a fusion computing kernel compilation method provided in this embodiment of the disclosure;

[0036] Figure 9 This is a schematic diagram of the overall compilation process of a fusion computing kernel provided in an embodiment of this disclosure;

[0037] Figure 10 This is a schematic diagram of the operation flow of a fusion computing kernel provided in an embodiment of this disclosure;

[0038] Figure 11 This is a schematic diagram of the structure of a fusion computing kernel compilation device proposed in an embodiment of this disclosure;

[0039] Figure 12 This is a schematic diagram of the structure of the electronic device proposed in the embodiments of this disclosure. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0041] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0042] An AI compiler is software that analyzes, transforms, and generates code for AI computational tasks. The process of an AI compiler includes stages such as front-end processing, intermediate representation transformation, compilation and downscaling, and target code generation.

[0043] A fused computation kernel is a computational kernel that includes multiple computational operations. These operations are processed through operator fusion, function fusion, and computational graph fusion to form a fused computational kernel. A fused computational kernel is also called a fused kernel or a hybrid kernel.

[0044] Module-level attributes: These are attributes associated with the compilation module corresponding to the fused computing kernel and applied to multiple functions within that module. Module-level attributes can record the number of compile-time thread bundles and provide a unified source of thread bundle configurations for multiple worker functions.

[0045] The main control thread refers to the code portion of the fused computing kernel used for control and scheduling. Operations in the main control thread are executed according to scalar semantics or scheduling semantics.

[0046] The worker function section refers to the code portion of the fused computing kernel used to perform vector computations under the scheduling of the main control thread. The worker function section includes at least one worker function, which executes on a worker execution unit.

[0047] Vector processing operations refer to operations that perform vectorization processing on tensor data, including at least one of the following: layout transformation operations, vector arithmetic operations, vector logic operations, reduction operations, and special function operations.

[0048] A warp is the basic unit for parallel thread scheduling in a target chip. A warp contains multiple execution threads. The execution threads in a warp perform data processing according to a unified instruction scheduling process.

[0049] Thread bundle configuration refers to the compiler configuration that specifies the number of thread bundles used in a section of code. Thread bundle configuration can be recorded in a thread bundle count field, compiler parameters, intermediate representation properties, or compiler internal configuration objects.

[0050] Single-threaded beam configuration: refers to a beam configuration where the number of corresponding beams is 1.

[0051] Multi-threaded bundle configuration: refers to a thread bundle configuration with a corresponding number of thread bundles greater than 1.

[0052] In related technologies, the compiler sets the global thread count per fused computing kernel. Functions within the same fused computing kernel are compiled based on this global thread count. Taking a fused computing kernel for a heterogeneous chip as an example, the main control thread executes in a single-threaded manner. In one related implementation, the global thread count of the fused computing kernel is set to 1. Worker functions are compiled based on this global thread count and executed in a single-threaded manner. Therefore, the multi-threaded parallelism supported by the worker execution unit is not utilized, resulting in low hardware utilization of the worker execution unit.

[0053] Based on this, this disclosure proposes a method, apparatus, electronic device, and storage medium for compiling a converged computing kernel. The converged computing kernel includes a main control thread for execution control and scheduling, and a working function part for performing vector computations under the scheduling of the main control thread. The main control thread and the working function part correspond to single-threaded beam configuration and multi-threaded beam configuration, respectively, and are compiled into main control code and working code, respectively, thereby simultaneously satisfying the single-threaded beam execution requirements on the main control side and the multi-threaded beam parallelism requirements on the working side.

[0054] Description of heterogeneous chip architecture applied in the embodiments of this disclosure

[0055] Figure 1 This is a schematic diagram of a hardware architecture for a heterogeneous chip provided in an embodiment of this disclosure. Figure 1 As shown, the heterogeneous chip 100 includes a main control execution unit 110, a working execution unit 120, a matrix operation unit 130, and a data transport unit 140.

[0056] The main control execution unit 110 is used to perform control operations and scheduling operations. The control operations are used to control the execution order in the fused computing kernel. The scheduling operations are used to trigger the worker execution unit 120, matrix operation unit 130, and data transport unit 140 to execute corresponding tasks.

[0057] The main control execution unit 110 operates in a single-threaded manner. The thread running on the main control execution unit 110 assumes the role of the main control thread. In some embodiments, this main control thread role is a meta-thread role.

[0058] The worker execution unit 120 is used to execute operations corresponding to the worker code, including scalar operations and vector operations. The worker execution unit 120 supports the parallel execution of multiple thread bundles. Threads running on the worker execution unit 120 assume the role of worker threads. In some embodiments, this worker thread role is a working thread role.

[0059] The matrix operation unit 130 is used to perform matrix operations. Matrix operations include matrix multiplication. The main control execution unit 110 provides the matrix operation unit 130 with source operand addresses, destination addresses, and device description information. The matrix operation unit 130 performs matrix operations based on the above information.

[0060] The data transfer unit 140 performs data transfer. Data transfer includes data transfer between different storage areas. The master control execution unit 110 provides the data transfer unit 140 with source operand addresses, destination addresses, and device description information. The data transfer unit 140 performs data transfer based on the above information.

[0061] In some embodiments, the heterogeneous chip 100 is a computing chip employing a target instruction set. The main control execution unit 110 can be implemented as an instruction scheduling unit. This instruction scheduling unit executes the main control instructions in the target instruction set and runs in a single-threaded manner. The worker execution unit 120 can be implemented as an execution unit (EU). The EU performs arithmetic and logical operations and vector calculations, and supports parallel execution of multiple threads. The matrix operation unit 130 can be implemented as a tensor processing engine (TPE). The TPE performs matrix multiplication operations. The instruction scheduling unit schedules the TPE using matrix operation scheduling instructions. The data transport unit 140 can be implemented as a copy engine (CE). The CE performs data transport. The instruction scheduling unit schedules the CE using data transport scheduling instructions.

[0062] exist Figure 1In the illustrated architecture, the main control code is configured to be executed by the main control execution unit 110, which performs scalar control and hardware scheduling according to the first thread bundle configuration. The working code is configured to be executed by the working execution unit 120, which supports parallel execution of multiple thread bundles and is used to execute the operations corresponding to the working code according to the second thread bundle configuration. The main control execution unit 110 is also used to schedule the working execution unit 120 to execute the working code based on the main control code. The matrix operation unit 130 and the data transport unit 140 execute their respective tasks under the scheduling of the main control code. Thus, the first and second thread bundle configurations can be applied to the main control execution unit 110 and the working execution unit 120, respectively, enabling the main control code and the working code to execute according to their respective thread bundle configurations.

[0063] Overall Implementation Method of the Fusion Computing Kernel Compilation Method in this Disclosure

[0064] This disclosure proposes a method for compiling a fusion computing kernel, wherein the fusion computing kernel includes a main control thread for execution control and scheduling, and a working function part for performing vector computation under the scheduling of the main control thread. Figure 2 This is a flowchart of a fusion computing kernel compilation method provided in an embodiment of this disclosure, such as... Figure 2 As shown, taking the method executed by an artificial intelligence compiler as an example, the method includes the following steps S201 to S205.

[0065] Step S201: Determine the first thread bundle configuration for the main control thread section; the first thread bundle configuration is a single thread bundle configuration.

[0066] The first thread bundle configuration specifies the number of thread bundles corresponding to the main control thread. The number of thread bundles corresponding to the first thread bundle configuration is 1. Determining the first thread bundle configuration establishes a correspondence between the main control thread and the single-thread bundle configuration. This correspondence is used for the subsequent compilation of the main control thread. The main control thread executes according to single-thread bundle semantics. During runtime, the main control thread is controlled and scheduled by a single thread bundle.

[0067] The representation of the first thread bundle configuration is not limited to the code form of the main control thread. When the main control thread takes the form of a function, code region, basic block set, or intermediate representation subgraph, the first thread bundle configuration corresponds to that main control thread.

[0068] Step S202: Based on the number of compile-time thread bundles, determine the configuration of the second thread bundle for the working function part; the second thread bundle is configured as a multi-thread bundle configuration.

[0069] The compile-time thread bundle count refers to the number of thread bundles determined before the target code generation is complete. The compile-time thread bundle count specifies the number of thread bundles corresponding to the worker function section. This count is greater than 1. Determining the second thread bundle configuration establishes the correspondence between the worker function section and the compile-time thread bundle count. The second thread bundle configuration records or references this compile-time thread bundle count. The worker function section executes according to multi-threaded bundle semantics. Multiple thread bundles jointly execute vector calculations within the worker function section.

[0070] The second thread beam configuration applies to the worker function part, while the first thread beam configuration applies to the main control thread part. The two thread beam configurations participate in the code generation of their respective parts.

[0071] Step S203: Compile the main control thread part based on the configuration of the first thread bundle to obtain the main control code.

[0072] The compilation of the main control thread involves converting its intermediate representation into code executable by the target device. During this compilation process, the AI ​​compiler generates main control code that executes in a single-threaded manner based on the first thread bundle configuration. This main control code is used for control and scheduling within the fused computing kernel. Control includes managing the execution order within the fused computing kernel. Scheduling includes triggering worker code or other execution units within the target device.

[0073] Step S204: Compile the working function part based on the second thread bundle configuration to obtain the working code.

[0074] The compilation of the working function part refers to converting the intermediate representation of the working function part into code that can be executed by the target device. During this compilation process, the AI ​​compiler generates working code that executes in a multi-threaded manner based on the number of thread bundles configured in the second thread bundle. This working code is used to perform vector computations. Vector computation refers to the calculation of vector elements or tensor elements by multiple execution threads.

[0075] Step S205: Generate a compilation result including main control code and working code; wherein, the compilation result is used to enable the main control code to perform control and scheduling according to the first thread bundle configuration when the fused computing kernel is running, and to enable the working code to perform vector computation according to the second thread bundle configuration under the scheduling of the main control code.

[0076] The AI ​​compiler organizes the main control code and worker code into a compiled result of the fused computing kernel. The compilation result records both the main control code and the worker code. It also records the scheduling relationship between the main control code and the worker code.

[0077] When the target device runs the converged computing kernel, the main control code performs control and scheduling according to the first thread bundle configuration. Under the scheduling of the main control code, the worker code performs vector computation according to the second thread bundle configuration.

[0078] In steps S201 to S205 above, the main control thread and the working function correspond to the first thread bundle configuration and the second thread bundle configuration, respectively. The main control code performs control and scheduling according to the single-thread bundle configuration. The working code performs vector computation according to the multi-thread bundle configuration. Thus, the fused computing kernel can simultaneously meet the single-thread bundle execution requirements of the main control thread and the multi-thread bundle parallel requirements of the working function, releasing the multi-thread bundle parallel capability of the working function and reducing the idle computing resources of the working execution unit.

[0079] It should be noted that, Figure 2 The following description uses the example of step S201 being executed before step S202. In some other embodiments, step S202 is executed before step S201. In still other embodiments, steps S201 and S202 are executed in parallel. Figure 2 The following explanation uses the example of step S203 being executed before step S204. In some other embodiments, step S204 is executed before step S203. In still other embodiments, steps S203 and S204 are executed in parallel. Step S205 is executed after both the main control code and the working code have been generated. When there is a data dependency between steps S203 and S204, the compiler determines their execution order according to this data dependency.

[0080] Specifically, in some implementations, see Figure 3 The above step S202 includes the following steps S301 and S302.

[0081] In step S301, the number of compile-time thread bundles is written into the module-level attribute corresponding to the fused computing kernel.

[0082] Module-level attributes refer to attributes associated with the compiled module corresponding to the fused computing kernel. The scope of module-level attributes covers multiple functions within the compiled module. The first thread bundle configuration is independent of the number of compiler thread bundles recorded in the module-level attributes.

[0083] In some embodiments, the intermediate representation of the fused computing kernel includes a kernel module. Module-level attributes are stored in this kernel module.

[0084] The source of the compile-time thread warp count can be implemented in several ways. In some embodiments, the compilation device receives the compile-time thread warp count through a compilation interface. The compilation interface then passes this count to the AI ​​compiler. Exemplarily, the compile-time thread warp count is represented by the thread warp count (“num_warps”) parameter. The compilation user inputs the “num_warps” parameter when initiating compilation. The AI ​​compiler writes this parameter into the kernel module corresponding to the fused computing kernel.

[0085] In other embodiments, the compiler interface does not receive the number of thread bundles. The compiler backend uses a preset number of thread bundles. For example, the default number of thread bundles used by the compiler backend is 2. The compiler writes the value 2 to the "num_warps" module-level attribute.

[0086] If a module-level property has not yet been created, the compiler first creates the property to record the number of compile-time thread bundles, and then performs the write operation. If the module-level property has already been created, the compiler writes the number of compile-time thread bundles into that property.

[0087] In step S302, the number of thread bundles corresponding to the second thread bundle configuration is determined based on the number of compile-time thread bundles recorded in the module-level attributes.

[0088] The compiler accesses the module-level attribute corresponding to the fused computing kernel and obtains the compile-time thread bundle count recorded in that attribute. The compiler then associates this compile-time thread bundle count with the second thread bundle configuration. This count becomes the thread bundle count corresponding to the second thread bundle configuration. For example, the value recorded in the "num_warps" module-level attribute is 2. The thread bundle count corresponding to the second thread bundle configuration is 2.

[0089] It should be noted that the first thread bundle configuration is independent of the compile-time thread bundle number recorded in the module-level properties. The code generation for the main control thread does not require changing the first thread bundle configuration based on this module-level property, thus making the single-thread bundle configuration on the main control side independent of the module-level multi-thread bundle configuration on the worker side.

[0090] When multiple worker functions reside within the same kernel module corresponding to the same fused computing kernel, module-level attributes override these multiple worker functions. Therefore, module-level attributes provide a unified source of the number of compile-time thread bundles for the worker function portion.

[0091] In steps S301 and S302 above, the number of compile-time thread bundles is written into the module-level attribute corresponding to the fused computing kernel, and the number of thread bundles corresponding to the second thread bundle configuration is determined based on the module-level attribute. At the same time, the first thread bundle configuration is made independent of the compile-time thread bundle number recorded in the module-level attribute, so that the master control side and the working side have stable and clear sources of thread bundle configurations, and provide a consistent multi-thread bundle configuration basis for the compilation reduction of multiple working functions and the continuous generation of target code.

[0092] In some embodiments, see Figure 4 The working function part includes multiple working functions, which correspond to the same module-level attribute. Step S204 above includes steps S401 to S403.

[0093] In step S401, multiple working functions are made to read the number of the same compile-time thread bundles recorded in the module-level attributes.

[0094] In this context, multiple working functions refer to two or more functions within a working function section, each including vector processing operations to be compiled. Multiple working functions corresponding to the same module-level attribute means that the scope of that module-level attribute covers all multiple working functions, and the compilation degrade process for each working function obtains the same compile-time thread count from that module-level attribute.

[0095] In step S402, the number of compile-time thread bundles is used to generate target code for multiple working functions, thereby obtaining the working function code corresponding to each working function.

[0096] For example, multiple working functions (EU functions) reside in the same kernel module, which includes a "num_warps" module-level attribute. The compilation lowering process of each EU function reads the same "num_warps" attribute and uses the number of thread bundles recorded in the attribute to generate the corresponding EU working function code.

[0097] In some embodiments, the first EU function includes a layout transformation operation (ConvertLayoutOp), and the compiler generates the working function code of the first EU function based on the "num_warps" attribute and the distributed encoding corresponding to ConvertLayoutOp; the second EU function includes other vector processing operations, and the compiler generates the working function code of the second EU function based on the same "num_warps" attribute.

[0098] It should be noted that, Figure 4Taking the sequential generation of target code for multiple working functions as an example. In other embodiments, when there are no compilation data dependencies between multiple working functions, the compiler generates target code for multiple working functions in parallel.

[0099] In step S403, working code is generated based on the working function code corresponding to each working function.

[0100] The compiler organizes the code corresponding to each working function into working code. In some embodiments, the working function code forms different function code segments, and the different working function call instructions in the main control code point to the corresponding function code segments. When running the fused computing kernel, the main control code schedules each working function code according to the execution flow, and each working function is executed according to the number of compile-time thread bundles recorded in the same module-level attributes.

[0101] In steps S401 to S403 above, multiple working functions read the number of compile-time thread bundles recorded in the same module-level attributes, use this number to generate the code for each working function, and generate working code based on the code for each working function. This allows multiple working functions in the same fused computing kernel to be compiled and executed with a consistent multi-thread bundle scale, reducing redundant configuration per function and configuration differences between different working functions.

[0102] In some embodiments, see Figure 5 The working function part includes vector processing operations for tensor data, and the above step S204 includes the following steps S501 to S504.

[0103] In step S501, tensor encoding information corresponding to the vector processing operation is obtained. This tensor encoding information is used to characterize the distribution of tensor elements between threads and thread bundles.

[0104] Vector processing operations refer to operations that perform vectorized processing on tensor data. A tensor consists of multiple tensor elements organized along one or more dimensions. Tensor encoding information is used to characterize the distribution of tensor elements among threads and thread bundles, including the correspondence between tensor elements and thread bundles, and the correspondence between tensor elements and execution positions within a thread bundle; execution positions within a thread bundle are also called lanes, and multiple lanes together form a thread bundle.

[0105] In some embodiments, the tensor encoding information is distributed encoding. Distributed encoding describes how the elements in the tensor are distributed across multiple lanes and multiple warps. The warp-splitting layout of the tensor elements is not exposed to the device instruction interface layer.

[0106] Distributed encoding takes several specific forms. In some embodiments, distributed encoding is block-based encoding. Blocked encoding divides tensor elements into data blocks and maps different data blocks to different thread bundles and different lanes. In other embodiments, distributed encoding is tiled layout. Tiled layout determines the mapping relationship between tensor elements and multiple thread bundles according to the tensor block layout.

[0107] In some embodiments, vector processing operations include at least one of layout transformation operations, vector arithmetic operations, vector logic operations, reduction operations, and special function operations.

[0108] The layout transformation operation is used to transform a tensor from a first layout to a second layout; vector arithmetic operations are used to perform arithmetic operations such as addition, subtraction, multiplication, and division on tensor elements; vector logic operations are used to perform logical operations such as comparison, AND, OR, and NOT on tensor elements; reduction operations are used to merge multiple tensor elements along one or more dimensions; and special function operations are used to perform exponential, logarithmic, trigonometric, and other function operations on tensor elements. For example, the compiler represents the layout transformation operation as an intermediate representation operation (ConvertLayoutOp), and converts this intermediate representation operation into target instructions executed by the working execution unit during the compilation downscaling phase.

[0109] In step S502, based on the tensor encoding information and the number of thread bundles corresponding to the second thread bundle configuration, the mapping relationship between tensor elements and multiple thread bundles is determined.

[0110] The compiler determines the number of thread bundles participating in the vector processing operation based on the number of thread bundles corresponding to the second thread bundle configuration, and determines the tensor elements processed by each thread bundle and the lane in each thread bundle based on the tensor encoding information, thereby obtaining the mapping relationship between tensor elements and multiple thread bundles.

[0111] For example, the second thread bundle configuration corresponds to a thread bundle number of 2. The compiler determines the tensor element set processed by the first thread bundle and the tensor element set processed by the second thread bundle according to DistributedEncoding. The compiler generates vectorized memory access instructions and vectorized computation instructions corresponding to the two thread bundles respectively.

[0112] When the tensor encoding information changes, the mapping relationship between tensor elements and thread bundles changes accordingly. When the number of thread bundles corresponding to the second thread bundle configuration changes, the number of thread bundles participating in vector processing changes accordingly. The compiler re-determines the tensor element mapping based on the changed tensor encoding information and the number of thread bundles.

[0113] In step S503, vectorized memory access instructions and vectorized computation instructions are generated based on the mapping relationship between tensor elements and multiple thread bundles.

[0114] The compiler generates vectorized memory access instructions for each thread bundle to read or write the corresponding tensor elements based on the mapping relationship obtained in step S502, and vectorized computation instructions for each thread bundle to perform vector processing on the corresponding tensor elements.

[0115] In step S504, target code corresponding to vector processing operations is generated based on vectorized memory access instructions and vectorized computation instructions.

[0116] The target code constitutes at least a part of the working code.

[0117] For example, the compiler reduces ConvertLayoutOp to the vectorized memory access instructions and vectorized computation instructions corresponding to the EU function, and the EU function executes the above instructions according to the number of thread bundles recorded in the module-level "num_warps" attribute.

[0118] In steps S501 to S504 above, tensor encoding information is obtained to characterize the distribution of tensor elements between threads and thread bundles. The mapping relationship between tensor elements and multiple thread bundles is determined by combining the number of thread bundles corresponding to the second thread bundle configuration. Vectorized memory access instructions and vectorized computation instructions are generated accordingly, so that vector processing operations such as layout transformation, vector arithmetic, vector logic, reduction or special functions can be executed in parallel on multiple thread bundles according to the distribution of the corresponding tensor elements.

[0119] It should be noted that in embodiments where the working function does not include the aforementioned vector processing operations, the above-mentioned vector processing operations are not executed. Figure 5 As shown in the process, the compiler generates target code based on other operations in the working function section.

[0120] Specifically, in some embodiments, see [link to relevant documentation]. Figure 6 The above step S203 includes steps S601 and S602.

[0121] In step S601, according to the first thread bundle configuration, at least one of the scalar control flow operations, hardware scheduling operations, and worker function call operations in the main control thread part is compiled to obtain the code corresponding to the at least one operation.

[0122] Among them, the scalar control code corresponding to the scalar control flow operation, the device scheduling code corresponding to the hardware scheduling operation, and the working function call code corresponding to the working function call operation are all generated according to the single-threaded bundle execution semantics.

[0123] Scalar control flow operations refer to operations that perform calculations on scalar data or control the code execution path. Scalar control flow operations include at least one of scalar arithmetic operations, scalar comparison operations, branch control operations, and loop control operations. For example, scalar arithmetic operations are represented in the Arithmetic Dialect (arith). Loop control operations are used to control the number of loops and loop jumps in the fused computing kernel.

[0124] Hardware scheduling operations refer to operations used to trigger dedicated hardware execution units in the target device to execute tasks. The hardware scheduling operation itself handles the scheduling, and the corresponding computational tasks are executed by the dedicated hardware execution units. Worker function call operations refer to operations used to cause the main control thread to call the worker functions. Worker function call operations establish the calling relationship between the main control thread and the worker functions.

[0125] In step S602, the code corresponding to the at least one operation is programmed into the main control code according to the control relationship of the fusion computing kernel.

[0126] In some embodiments, the master control code includes scalar control code. The scalar control code is compiled from scalar control flow operations. In some embodiments, the master control code includes device scheduling code. The device scheduling code is compiled from hardware scheduling operations. In some embodiments, the master control code includes worker function call instructions. The worker function call instructions are compiled from worker function call operations.

[0127] When the main control thread includes multiple operations, the main control code includes code corresponding to each operation. These codes are arranged according to the control order of the fused computing kernel or connected via jump relationships. The above process unifies the different operations in the main control thread into the main control code, which is executed according to the first thread bundle configuration. This allows the main control code to perform scalar control, hardware scheduling, and worker function calls in a single-threaded manner.

[0128] It should be noted that scalar control flow operations, hardware scheduling operations, and worker function call operations are determined based on the content of the fused computing kernel. If the main control thread does not include one of these types of operations, the compiler will not perform the corresponding compilation processing.

[0129] In steps S601 and S602 above, at least one operation in the main control thread is compiled according to the first thread bundle configuration, and the corresponding code is compiled into the main control code according to the control relationship of the fused computing kernel. This ensures that the scalar control code, device scheduling code, and working function call code are all generated according to the single thread bundle execution semantics, thereby ensuring that the main control code stably executes the control and scheduling of the fused computing kernel.

[0130] In some embodiments, see Figure 7 If at least one operation processed in step S601 includes a hardware scheduling operation, the compilation process for the hardware scheduling operation includes the following steps S701 and S702.

[0131] In step S701, the hardware scheduling operation is reduced to device scheduling code executed according to the first thread bundle configuration.

[0132] The hardware scheduling operation includes at least one of matrix operation scheduling and data transfer scheduling. The device scheduling code enables the main control code to schedule the target hardware execution unit based on scheduling interface information. The target hardware execution unit includes at least one of matrix operation execution unit and data transfer execution unit.

[0133] Compilation downgrade refers to converting higher-level operations into operations or code that are closer to the instruction form of the target device.

[0134] Device scheduling code refers to the code executed by the main control execution unit and used to schedule dedicated hardware execution units. Matrix operation scheduling operations are used to schedule matrix operation execution units. Matrix operation execution units perform matrix multiplication and other matrix operations. Data transfer scheduling operations are used to schedule data transfer execution units. Data transfer execution units perform data transfer between storage spaces.

[0135] For example, the target device is a computing chip including a Tensor Processing Engine (TPE) and a Copy Engine (CE). The TPE is a matrix operation unit responsible for performing matrix multiplication operations; the hardware scheduling operation corresponding to the matrix multiplication operation is a matrix operation scheduling operation (DotOp), which the compiler reduces to TPE scheduling code executed by the main control thread. The CE is a data transfer execution unit responsible for performing data transfer operations; the hardware scheduling operation corresponding to the data transfer operation is a data transfer scheduling operation (AsyncCopy), which the compiler reduces to CE scheduling code executed by the main control thread. In this specific implementation, the main control thread runs instructions from the target instruction set, and the device scheduling code is device scheduling code adapted to the target instruction set.

[0136] In step S702, the scheduling interface information of the device scheduling code is determined, wherein the scheduling interface information includes the source operand address, the target address, and the device description information.

[0137] The scheduling interface information refers to the information used by the device scheduling code when initiating scheduling to the dedicated hardware execution unit. The source operand address indicates the location where the dedicated hardware execution unit reads the input data. The destination address indicates the location where the dedicated hardware execution unit writes the processing result. The device description information describes the device task corresponding to this scheduling. The device description information includes at least one of the following: operation type, data specification, and execution parameters.

[0138] Before generating device scheduling code, the compiler prepares the source operand addresses, target addresses, and device description information. The device scheduling code schedules either the TPE or CE based on this information. The TPE reads the input data for matrix operations according to the source operand address and writes the matrix operation result according to the target address. The CE reads the data to be transferred according to the source operand address and writes the data to be transferred according to the target address. The device scheduling code uses the source operand address, target address, and device description information as the scheduling interface information. Therefore, the scheduling interface for TPE and CE remains in the form of address and description information, keeping the implementation of the device instruction layer concise.

[0139] In some embodiments, the main control execution unit also prepares tensor description information and memory addresses. The tensor description information and memory addresses are used to generate scheduling interface information for the device scheduling code.

[0140] After generating the device scheduling code, it is compiled into the main control code. Specifically, the compiler compiles the device scheduling code generated in step S701 into the main control code. The device scheduling code executes according to the first thread bundle configuration.

[0141] For example, the main control thread runs in a single-warp manner. The main control thread executes the device scheduling code corresponding to matrix operation scheduling operations to schedule the tensor processing engine; the main control thread executes the device scheduling code corresponding to data transfer scheduling operations to schedule the copy engine. The tensor processing engine and the copy engine execute their respective device tasks. The tensor processing engine and the copy engine are scheduled by the main control thread. The device tasks of the tensor processing engine and the copy engine are not divided according to the number of thread bundles in the worker function section.

[0142] In steps S701 and S702 above, the hardware scheduling operation is reduced to device scheduling code executed according to the first thread bundle configuration. The scheduling interface information, including the source operand address, the target address, and the device description information, is determined, and the device scheduling code is compiled into the main control code. This enables the main control code to schedule at least one of the matrix operation execution unit and the data transport execution unit based on the scheduling interface information. At the same time, the amount of processing of tensor thread bundle distribution information by the device scheduling code is reduced, and the complexity of the device instruction compilation process is reduced.

[0143] In some embodiments, see Figure 8If at least one operation processed in step S601 includes a working function call operation, the compilation processing of the working function call operation includes the following steps S801 and S802.

[0144] In step S801, the target working function corresponding to the working function call operation is determined.

[0145] The compiler identifies the worker function call operations in the main control thread section and determines the target worker function in the worker function section based on the function identifier, call relationship, or target address carried by the worker function call operation.

[0146] For example, the target device is a computing chip. The main control thread runs on the instruction scheduling unit, and the working functions run on the execution unit (EU). The EU is the working execution unit in the computing chip used to perform arithmetic and logical operations and vector calculations, and supports the parallel execution of multiple thread bundles. The compiler determines the target working function to be called based on the working function call operation. In some embodiments, the EU is located inside the Compute Unit (CU).

[0147] In step S802, a working function call instruction for calling the target working function is generated.

[0148] Specifically, the compiler generates a working function call instruction that the target device can execute based on the target working function determined in step S801. This working function call instruction is used to enable the main control code running according to the first thread bundle configuration to schedule the working function code corresponding to the target working function generated according to the second thread bundle configuration.

[0149] For example, the worker function call instruction is a worker function call instruction (“call_eu” instruction) in the target instruction set. This worker function call instruction is used to cause the main thread to call the target worker function executed by the worker execution unit. The compiler reduces the worker function call operation to a worker function call instruction pointing to the target worker function.

[0150] After the working function call instructions are generated, they are programmed into the main control code according to the control relationship of the fused computing kernel.

[0151] When running the fused computing kernel, the main control code running according to the first thread bundle configuration executes the working function call instruction to schedule the working function code corresponding to the target working function generated according to the second thread bundle configuration.

[0152] In steps S801 and S802 above, the target working function corresponding to the working function call operation is determined, a working function call instruction for calling the target working function is generated, and the instruction is compiled into the main control code, so that the main control code running according to the first thread bundle configuration can schedule the target working function code generated according to the second thread bundle configuration, thereby establishing the execution connection between single-thread bundle control scheduling and multi-thread bundle vector calculation in the same fused computing kernel.

[0153] In some embodiments, during the compilation downscaling phase of the working function section, target code is generated for the working function section based on the number of thread bundles corresponding to the second thread bundle configuration, resulting in working code.

[0154] The compilation downscaling phase of the working function part refers to the stage of converting the intermediate representation operations in the working function part into low-level operations or target instructions corresponding to the target device. The second thread bundle configuration has been determined in step S202. The compilation downscaling phase obtains the number of thread bundles corresponding to the second thread bundle configuration and provides this number to the target code generation process.

[0155] For example, the working function section includes at least one EU function. The compiler obtains the number of thread bundles corresponding to the second thread bundle configuration during the compilation downscaling phase of the working function. In a specific implementation that uses the "num_warps" module-level attribute to carry the second thread bundle configuration, the compilation downscaling process of the EU function reads the "num_warps" attribute from the kernel module. This read occurs after the second thread bundle configuration is determined. This read provides the determined number of thread bundles to the target code generation process. The compilation downscaling process of all EU functions accesses the same module-level "num_warps" attribute.

[0156] Target code generation refers to generating code that the target device can load and execute. Based on the number of thread bundles obtained, the compiler performs instruction selection, thread mapping, and code generation for the operations in the working function section. When there are multiple thread bundles, the compiler generates working code adapted for execution by multiple thread bundles.

[0157] In some embodiments, the master control thread and worker functions separately perform target code generation. The master control thread generates master target code. The worker functions generate worker target code. Exemplarily, the master control target code is executed in a single-warp manner. The worker target code is executed according to the number of warps recorded in the "num_warps" attribute. The master control thread and worker functions respectively form target code. The compiler organizes both types of target code into the compilation result of the same fused computing kernel.

[0158] In the above embodiments, during the compilation reduction stage of the working function part, target code is generated for the working function part based on the number of thread bundles corresponding to the second thread bundle configuration, so that the number of multi-thread bundles represented by the second thread bundle configuration is realized in the instruction organization and parallel execution scale of the working code, thereby adapting the generated working code to the parallel execution vector calculation of multiple thread bundles.

[0159] In some embodiments, the master control code is configured to be executed by the master control execution unit, which is used to perform scalar control and hardware scheduling according to a first thread bundle configuration; the worker code is configured to be executed by the worker execution unit, which supports the parallel execution of multiple thread bundles and is used to perform vector computation according to a second thread bundle configuration; the master control execution unit is also used to schedule the worker execution units to execute the worker code based on the master control code.

[0160] The following is through Figure 9 and Figure 10 The process of compiling the above-mentioned fused computing kernel is illustrated with an example.

[0161] Figure 9 This is a schematic diagram of the overall compilation process of a fusion computing kernel provided in an embodiment of this disclosure. Figure 9 As shown, the compiler identifies the master thread portion in the fused computing kernel used for control and scheduling, and the worker function portion used for computation. The first thread bundle configuration of the master thread portion is fixed at 1; the second thread bundle configuration of the worker function portion is determined by the module-level "num_warps" attribute, which has a default value of 2 in some embodiments. The compiler compiles the master thread portion and the worker function portion separately, generating a compilation result that includes both master code and worker code.

[0162] Figure 10 This is a schematic diagram illustrating the operation flow of a fusion computing kernel provided in an embodiment of this disclosure. For example... Figure 10 As shown, the main control thread acts as the meta-thread and runs in a single-threaded bundle manner. The main control thread schedules matrix operation units through the matrix operation scheduling operation (DotOp), data transport units through the data transport scheduling operation (AsyncCopy), and worker functions (EU functions) through the worker function call instruction ("call_eu"). Each worker function acts as a worker thread and runs according to the number of thread bundles specified by the thread bundle count parameter (num_warps).

[0163] It should be noted that the above Figure 9 and Figure 10 The examples shown are merely illustrative and do not limit the scope of the embodiments disclosed herein.

[0164] In a test of a converged computing kernel, when the relevant implementations use a uniform number of thread bundles, the total number of thread bundles in the converged computing kernel is 1. After adopting the compilation method of the embodiment of this application, the main control thread runs in a single-thread bundle manner, and the worker functions run in multiple thread bundles corresponding to the module-level attributes of the number of thread bundles, thereby improving the hardware utilization of the worker execution unit.

[0165] Description of apparatus and devices according to embodiments of this disclosure

[0166] See Figure 11 This disclosure also provides a fusion computing kernel compilation apparatus 1100. The fusion computing kernel includes a main control thread for execution control and scheduling, and a working function part for performing vector computations under the scheduling of the main control thread. The fusion computing kernel compilation apparatus 1100 includes a first thread bundle configuration determination module 1101, a second thread bundle configuration determination module 1102, a main control code compilation module 1103, a working code compilation module 1104, and a compilation result generation module 1105.

[0167] The first thread bundle configuration determination module 1101 is used to determine the first thread bundle configuration for the main control thread part, and the first thread bundle configuration is a single thread bundle configuration.

[0168] The second thread bundle configuration determination module 1102 is used to determine the second thread bundle configuration for the working function part based on the number of compile-time thread bundles, and the second thread bundle configuration is a multi-thread bundle configuration.

[0169] The main control code compilation module 1103 is used to compile the main control thread part based on the first thread bundle configuration to obtain the main control code;

[0170] The working code compilation module 1104 is used to compile the working function part based on the second thread bundle configuration to obtain the working code;

[0171] The compilation result generation module 1105 is used to generate compilation results including main control code and working code; wherein, the compilation results are used to enable the main control code to perform control and scheduling according to the first thread bundle configuration when the fused computing kernel is running, and enable the working code to perform vector computation according to the second thread bundle configuration under the scheduling of the main control code.

[0172] The specific processing procedure of the fusion computing kernel compilation device 1100 disclosed herein is the same as the fusion computing kernel compilation method of the above embodiments, and will not be repeated here.

[0173] This disclosure also provides an electronic device 1200, comprising:

[0174] At least one processor, and,

[0175] A memory that is communicatively connected to at least one processor; wherein,

[0176] The memory stores instructions that are executed by at least one processor to cause the at least one processor to perform the method as described in any of the above embodiments of the present disclosure when executing the instructions.

[0177] The following is combined with Figure 12 The hardware structure of the electronic device is described in detail. The electronic device includes: a processor 1210, a memory 1220, an input / output interface 1230, a communication interface 1240, and a bus 1250.

[0178] The processor 1210 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0179] The memory 1220 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1220 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1220 and is called and executed by the processor 1210 using the fusion computing kernel compilation method of the embodiments of this disclosure.

[0180] The input / output interface 1230 is used to implement information input and output.

[0181] The communication interface 1240 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0182] Bus 1250 transmits information between various components of the device (e.g., processor 1210, memory 1220, input / output interface 1230, and communication interface 1240);

[0183] The processor 1210, memory 1220, input / output interface 1230 and communication interface 1240 are connected to each other within the device via bus 1250.

[0184] This disclosure also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the fusion computing kernel compilation method of the above embodiments, which will not be described in detail here.

[0185] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0186] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0187] It should be understood that in the description of the embodiments of this disclosure, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0188] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0189] The units described as separate units may or may not be physically separated. The units shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0192] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for compiling a fusion computing kernel, characterized in that, The fusion computing kernel includes a main control thread for execution control and scheduling, and a working function part for performing vector computations under the scheduling of the main control thread. The method includes: Determine a first thread bundle configuration for the main control thread portion, wherein the first thread bundle configuration is a single thread bundle configuration; Based on the number of compile-time thread bundles, a second thread bundle configuration for the working function portion is determined, wherein the second thread bundle configuration is a multi-thread bundle configuration. The main control thread portion is compiled based on the first thread bundle configuration to obtain the main control code; The working function part is compiled based on the second thread bundle configuration to obtain the working code; Generate a compilation result that includes the main control code and the working code; The compilation result is used to enable the main control code to perform control and scheduling according to the first thread bundle configuration when the fused computing kernel is running, and to enable the working code to perform vector computation according to the second thread bundle configuration under the scheduling of the main control code.

2. The fusion computing kernel compilation method according to claim 1, characterized in that, The determination of the second thread bundle configuration for the working function portion based on the number of compile-time thread bundles includes: Write the number of compile-time thread bundles into the module-level attribute corresponding to the fusion computing kernel; Based on the number of compile-time thread bundles recorded in the module-level attributes, determine the number of thread bundles corresponding to the second thread bundle configuration; The first thread bundle configuration is independent of the number of compile-time thread bundles recorded in the module-level attributes.

3. The fusion computing kernel compilation method according to claim 2, characterized in that, The working function section includes multiple working functions, and the multiple working functions correspond to the same module-level attribute; The process of compiling the working function part based on the second thread bundle configuration to obtain working code includes: The multiple working functions are each instructed to read the number of the same compile-time thread bundle recorded in the module-level attributes. Using the specified number of compile-time thread bundles, target code is generated for each of the multiple working functions to obtain the working function code corresponding to each working function. The working code is generated based on the working function code corresponding to each of the aforementioned working functions.

4. The fusion computing kernel compilation method according to claim 1, characterized in that, The working function part includes vector processing operations for tensor data; The process of compiling the working function part based on the second thread bundle configuration to obtain working code includes: Obtain the tensor encoding information corresponding to the vector processing operation, wherein the tensor encoding information is used to characterize the distribution of tensor elements between threads and thread bundles; Based on the tensor encoding information and the number of thread bundles corresponding to the second thread bundle configuration, the mapping relationship between the tensor elements and multiple thread bundles is determined; Vectorized memory access instructions and vectorized computation instructions are generated based on the mapping relationship between the tensor elements and multiple thread bundles; Based on the vectorized memory access instructions and the vectorized computation instructions, the target code corresponding to the vector processing operation is generated.

5. The fusion computing kernel compilation method according to claim 4, characterized in that, The vector processing operations include at least one of the following: layout transformation operations, vector arithmetic operations, vector logic operations, reduction operations, and special function operations.

6. The fusion computing kernel compilation method according to claim 1, characterized in that, The process of compiling the main control thread based on the first thread bundle configuration to obtain the main control code includes: According to the first thread bundle configuration, at least one of the scalar control flow operations, hardware scheduling operations, and worker function call operations in the main control thread part is compiled to obtain code corresponding to the at least one operation; According to the control relationship of the fusion computing kernel, the code corresponding to the at least one operation is compiled into the main control code; The scalar control code corresponding to the scalar control flow operation, the device scheduling code corresponding to the hardware scheduling operation, and the working function call code corresponding to the working function call operation are all generated according to single-threaded bundle execution semantics.

7. The fusion computing kernel compilation method according to claim 6, characterized in that, When the at least one operation includes the hardware scheduling operation, compiling the hardware scheduling operation includes: The hardware scheduling operation is reduced to device scheduling code executed according to the first thread bundle configuration; Determine the scheduling interface information of the device scheduling code, wherein the scheduling interface information includes the source operand address, the target address, and the device description information; The device scheduling code is used to enable the main control code to schedule the target hardware execution unit based on the scheduling interface information. The target hardware execution unit includes at least one of a matrix operation execution unit and a data transport execution unit.

8. The fusion computing kernel compilation method according to claim 6, characterized in that, When at least one of the operations includes the worker function call operation, the worker function call operation is compiled, including: Determine the target working function corresponding to the working function call operation; Generate a working function call instruction for invoking the target working function; The working function call instruction is used to cause the main control code running according to the first thread bundle configuration to schedule the working function code corresponding to the target working function generated according to the second thread bundle configuration.

9. The fusion computing kernel compilation method according to claim 1, characterized in that, The main control code is configured to be executed by the main control execution unit, which is used to perform scalar control and hardware scheduling according to the first thread bundle configuration; The working code is configured to be executed by a working execution unit, which supports the parallel execution of multiple thread bundles and is used to perform the vector calculation according to the second thread bundle configuration; The master control execution unit is also used to schedule the work execution unit to execute the work code based on the master control code.

10. A fusion computing kernel compilation device, characterized in that, The fusion computing kernel includes a main control thread for performing control and scheduling, and a working function part for performing vector computations under the scheduling of the main control thread. The device includes: The first thread bundle configuration determination module is used to determine the first thread bundle configuration for the main control thread portion; the first thread bundle configuration is a single thread bundle configuration. The second thread bundle configuration determination module is used to determine the second thread bundle configuration for the working function part based on the number of compile-time thread bundles, wherein the second thread bundle configuration is a multi-thread bundle configuration; The main control code compilation module is used to compile the main control thread part based on the first thread bundle configuration to obtain the main control code; The working code compilation module is used to compile the working function part based on the second thread bundle configuration to obtain the working code; The compilation result generation module is used to generate compilation results including the main control code and the working code; The compilation result is used to enable the main control code to perform control and scheduling according to the first thread bundle configuration when the fused computing kernel is running, and to enable the working code to perform vector computation according to the second thread bundle configuration under the scheduling of the main control code.

11. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. The program is executed by the processor to implement the fusion computing kernel compilation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the fusion computing kernel compilation method as described in any one of claims 1 to 9.

Images