Machine-readable medium and computer system for generating uniform intermediary code

Abstract

A machine-readable medium and a computer system for generating uniform intermediate code are provided. The program stored in the machine-readable medium, when executed, causes the computer system to: receive a source code file; and compile the source code file into an intermediate code file containing a plurality of intermediate code primitives. The intermediate code primitives include a slicing primitive. The slicing primitive has at least one parameter to express a slicing of a tensor into a plurality of sub-tensors along a specified axis of the tensor in time or space selectively.

Description

Technical Field

[0001] This invention relates to an electronic device, and more particularly to a machine-readable medium and computer system for generating a uniform intermediate code. Background Technology

[0002] Intermediate representation (IR) is a type of instruction, also known as a primitive. A compiler can compile a source code file into an intermediate representation file. The intermediate representation file can then be further converted into machine code (binary code) files for different system architectures. Summary of the Invention

[0003] The present invention provides a non-transitory machine-readable medium and a computer system storing a program to generate a unified intermediate code (IR).

[0004] In an embodiment of the invention, when the program stored on the machine-readable medium is executed, the program causes a computer system to: receive a source code file; and compile the source code file into a medium code file containing a plurality of medium code primitives. These medium code primitives include a slice primitive. The slice primitive has at least one parameter to describe selectively sliceing the tensor into a plurality of subtensors along a specified axis in time or space.

[0005] In an embodiment of the invention, the computer system includes a machine-readable medium and a computing processor. The machine-readable medium stores a program. The computing processor is coupled to the machine-readable medium to execute the program. When the program is executed, the program causes the computing processor to receive a source code file and compile the source code file into an intermediate code file containing a plurality of intermediate code primitives. These intermediate code primitives include a segmentation primitive. The segmentation primitive has at least one parameter to indicate selectively segmenting the tensor into a plurality of subtensors along a specified axis in time or space.

[0006] Based on the above, the program described in the embodiments of the present invention can compile source code files into intermediate code files. In some embodiments, the unified intermediate code file can describe the spatial and temporal parallelism of a neural network computing graph. In the intermediate code file, the partitioning primitive has at least one parameter. Based on the at least one parameter, the partitioning primitive can express selectively partitioning the tensor into multiple subtensors along a specified axis in time or space. The use of a partitioning primitive can replace other general primitives (e.g., "broadcast", "replicate", "partial_sum", "shard", etc.). According to the technical content disclosed in the present invention, programming efficiency is improved. Attached Figure Description

[0007] Figure 1 This is a schematic diagram illustrating the input and output matrices for matrix multiplication, as shown in an example of the present invention.

[0008] Figure 2 This is a schematic diagram illustrating the input and output matrices for matrix multiplication, as shown in another example of the present invention.

[0009] Figure 3 This is a schematic diagram of an operation phase for performing batch-to-space operations, illustrated according to another example of the present invention.

[0010] Explanation of reference numerals in the attached figures

[0011] h: Height axis

[0012] Matrix_A, Matrix_B, Matrix_C: Matrix

[0013] SPC0, SPC1, SPC2, SPC3: Computational cores

[0014] w: Width axis Detailed Implementation

[0015] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts.

[0016] The term "coupled (or connected)" as used throughout this specification (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some means of connection. The terms "first," "second," etc., used throughout this specification (including the claims) are used to name components, and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. Furthermore, wherever possible, components / components / steps using the same reference numerals in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same reference numerals or the same terms in different embodiments may be referred to mutually in the relevant descriptions.

[0017] A machine-readable medium storing a program will be described in the following embodiments. In some embodiments, the machine-readable medium includes, for example, a tape, disk, card, semiconductor memory, programmable logic circuitry, and / or a storage device. The storage device includes a hard disk drive (HDD), a solid-state drive (SSD), or other storage devices. A computing processor of a computer system is coupled to the machine-readable medium to execute the program. In some practical applications, the computing processor may include a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. In some practical applications, the program may be implemented as compiler software, electronic design automation (EDA) software, computer-aided design (CAD) software, or other design software.

[0018] When executed, the program causes the computer system's computing processor to receive a source code file. In some practical applications, the source code file may include any computational software that processes tensors. In some practical applications, the source code file may include Convolutional Neural Network (CNN) computational software or other application software. The computing processor executing the program can compile the source code file into an intermediate code file containing multiple intermediate representation (IR) primitives.

[0019] These intermediate code primitives in the intermediate code file include slice primitives. Each slice primitive has at least one parameter. Based on this parameter, the slice primitive can express selectively slicing the tensor into multiple subtensors along a specified axis in time or space. The at least one parameter (parameter field) of the slice primitive can be set according to the actual design. For example (but not limited to), Table 1 shows an example of the parameter field of the slice primitive.

[0020] Table 1: Parameter fields of the slice primitive

[0021]

[0022]

[0023] In the embodiment shown in Table 1, the tensor parameter "tensor" specifies the tensor to be split, while the axis parameter "axis" specifies the splitting axis (split axis) of the tensor "tensor". For example, the specified axis can be the sampling axis "n", channel axis "c", height axis "h", or width axis "w" of the tensor "tensor_A". The length parameter "length" specifies the split length along the specified axis. The mapping parameter "map_to" specifies the multiple hardware cores (or hardware modules) corresponding to the multiple sub-tensors after splitting, so as to split the tensor into the multiple sub-tensors in space along the specified axis. When the splitting primitive includes the mapping parameter "map_to", the splitting primitive can indicate that the multiple sub-tensors split in space are used to be split one-to-one to the multiple hardware cores. That is, the splitting primitive can replace the general primitive "split".

[0024] Based on practical applications, the segmentation primitive can be called recursively. For example, assuming slice() is the segmentation primitive, the recursive call to the segmentation primitive can be represented as slice(slice()). This recursive calling method can be used to segment multiple subtensors on different axes.

[0025] When the partitioning primitive does not include the mapping parameter "map_to", it can be used to partition a tensor into multiple subtensors in time according to a specified axis. When the partitioning primitive does not include the mapping parameter "map_to", it can indicate that each partitioned subtensor is executed sequentially by a hardware core in time. For example, in some embodiments, the partitioning primitive can indicate that each partitioned subtensor in time is used to be broadcast to multiple hardware cores. In other embodiments, the partitioning primitive can indicate that these partitioned subtensors in time are sequentially provided to the same hardware core.

[0026] These intermediate code primitives in the intermediate code file can also contain tensor primitives. The tensor primitives can declare the memory architecture mode of the tensor. The parameter fields of the tensor primitives can be set according to the actual design. For example (but not limited to), Table 2 shows an example of the parameter fields of the tensor primitives.

[0027] Table 2: Parameter fields of tensor primitives

[0028] Parameter fields illustrate Parameter Examples shape Axis and Dimensions of a Tensor (2,128,56,56) layout tensor form Matrix mem_arch_mode Memory architecture mode uma,uma8,numa

[0029] In the embodiments shown in Table 2, the parameter "shape" is used to declare the axes and dimensions of the tensor. For example, the parameter "shape = (2, 128, 56, 56)" indicates that the sampling axis "n" of this tensor has a dimension of 2, the channel axis "c" has a dimension of 128, the height axis "h" has a dimension of 56, and the width axis "w" has a dimension of 56. The parameter "layout" is used to declare the form of the tensor. The parameter "mem_arch_mode" is used to declare the memory architecture mode of the tensor, such as non-uniform memory access (NUMA), uniform memory access (UMA), or other architecture modes. When the source code file indicates "dividing the tensor into multiple sub-tensors in space," the tensor primitive can declare the memory architecture mode of the tensor as "numa," that is, NUMA architecture. In the case where the source code file indicates "splitting the tensor into multiple sub-tensors in time", the tensor primitive can declare the tensor's memory architecture mode as "uma", that is, UMA architecture.

[0030] These intermediate code primitives in the intermediate code file may also include at least one operational primitive. The parameter fields of the operational primitives can be set according to the actual design. For example (but not limited to), Table 3 shows an example of the parameter fields of the operational primitives. In the embodiment shown in Table 3, the parameter "tensors" is used to declare the tensors to be operated on. For example, the parameter "tensors" may include a first tensor parameter and a second tensor parameter. The first tensor parameter indicates a first tensor, and the second tensor parameter indicates a second tensor. The operational primitives can perform operations on the first tensor and the second tensor. The operations can be arbitrary operations, such as matrix multiplication or other operations.

[0031] Table 3: Parameter fields of operation primitives

[0032]

[0033] Assuming op() is the computational primitive, then tensor_result = op(tensor1, tensor2) can represent "the computational primitive performs operations on tensors tensor1 and tensor tensor2 to produce the computational result tensor_result". Assuming tensor tensor1 is partitioned, the rules for deriving partial_sum / all_reduce / broadcast from the partitioning method are shown in Table 4.

[0034] Table 4: Segmentation Derivation Rules

[0035]

[0036] Figure 1 This is a schematic diagram illustrating the input and output matrices for matrix multiplication, drawn according to an example of the present invention. Figure 1 In the illustrated embodiment, the intermediary code file may contain the operation primitive "MatrixMultiply()", that is, the matrix multiplication operation primitive. Figure 1 The matrices Matrix_A and Matrix_B shown are the input matrices for the matrix multiplication operation primitives, while matrix Matrix_C is the output matrix for the matrix multiplication operation primitives. For ease of explanation, as an exemplary embodiment, the following examples of intermediate code files will use "tensor()" as a specific example of the tensor primitive and "slice()" as a specific example of the segmentation primitive. An example of the content of the intermediate code file is as follows.

[0037]

[0038]

[0039] After transformation, the format of the intermediary code is as follows.

[0040]

[0041] Figure 2 This is a schematic diagram illustrating the input and output matrices for matrix multiplication, according to another operational example of the present invention. Figure 2 In the illustrated embodiment, the intermediary code file may contain the matrix multiplication operation primitive "MatrixMultiply()". Figure 2 The matrices Matrix_A and Matrix_B shown are the input matrices for the matrix multiplication operation primitives, while matrix Matrix_C is the output matrix for the matrix multiplication operation primitives. For ease of explanation, as an exemplary embodiment, the following examples of intermediate code files will use "tensor()" as a specific example of the tensor primitive and "slice()" as a specific example of the segmentation primitive. An example of the content of the intermediate code file is as follows.

[0042]

[0043]

[0044] After transformation, the format of the intermediary code is as follows.

[0045]

[0046]

[0047] Figure 3 This diagram illustrates the operation phase of a batch-to-space operation, according to another example of the present invention. The batch-to-space operation can also be called axis expansion (or matrix concatenation). Figure 3 The following is an example of a medcode file for the axis extension operation.

[0048]

[0049]

[0050] In summary, the programs described in the embodiments can compile source code files into intermediate code files. In some applications, a unified intermediate code file can describe the spatial and temporal parallelism of a neural networking computing graph. Based on at least one parameter of the partitioning primitive in the intermediate code file, the partitioning primitive can represent selectively partitioning the tensor into multiple subtensors along a specified axis in time or space. For example, when the partitioning primitive includes the mapping parameter "map_to", the partitioning primitive can indicate that the partitioned subtensors are spatially distributed one-to-one to multiple hardware cores. Conversely, when the partitioning primitive does not include the mapping parameter "map_to", the partitioning primitive can indicate that each partitioned subtensor is executed sequentially by the hardware cores in time. For example, in some embodiments, the partitioning primitive can indicate that each partitioned subtensor in time is used to be broadcast to multiple hardware cores. In other embodiments, these partitioned subtensors in time can be sequentially provided to the same hardware core. Therefore, the same segmentation primitive can replace multiple general primitives (such as "broadcast", "replicate", "partial_sum", "shard", etc.). This results in higher programming efficiency.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine-readable medium storing a program, characterized in that, When the program is executed, it causes the computer system to: Receive source code files; and The source code file is compiled into a median code file containing multiple median code primitives, wherein the multiple median code primitives include a segmentation primitive, and the segmentation primitive has at least one parameter to indicate whether the tensor is segmented into multiple subtensors along a specified axis of the tensor in time or space. Wherein, if the partitioning primitive does not contain a mapping parameter, the partitioning primitive is used to partition the tensor into the plurality of sub-tensors in time according to the specified axis, and the partitioning primitive indicates that each of the plurality of sub-tensors partitioned in time is used to be broadcast to a plurality of hardware cores.

2. The machine-readable medium according to claim 1, characterized in that, The at least one parameter includes a specified axis parameter, used to specify the specified axis of the tensor.

3. The machine-readable medium according to claim 1, characterized in that, The at least one parameter includes a length parameter, used to specify the cutting length along the specified axis.

4. The machine-readable medium according to claim 1, characterized in that, The at least one parameter includes a mapping parameter, which specifies the multiple hardware cores corresponding to the multiple sub-tensors, so as to divide the tensor into the multiple sub-tensors in space according to the specified axis.

5. The machine-readable medium according to claim 4, characterized in that, When the partitioning primitive includes the mapping parameter, the partitioning primitive represents the plurality of sub-tensors partitioned in space for being split one-to-one to the plurality of hardware cores.

6. The machine-readable medium according to claim 1, characterized in that, The plurality of escrow primitives also include tensor primitives, and the tensor primitives are used to declare the memory architecture pattern of the tensor.

7. The machine-readable medium according to claim 6, characterized in that, In the case where the source code file indicates that the tensor is divided into the multiple sub-tensors in space, the tensor primitive declares that the memory architecture mode of the tensor is a non-uniform memory access architecture.

8. The machine-readable medium according to claim 6, characterized in that, In the case where the source code file indicates that the tensor is divided into the multiple sub-tensors in time, the tensor primitive declares that the memory architecture pattern of the tensor is a unified memory access architecture.

9. The machine-readable medium according to claim 1, characterized in that, The plurality of intermediate code primitives further include operational primitives, each operational primitive including a first tensor parameter and a second tensor parameter, the first tensor parameter indicating a first tensor, the second tensor parameter indicating a second tensor, and the operational primitive being used to perform operations on the first tensor and the second tensor.

10. A computer system, comprising: A machine-readable medium containing a program; as well as A computing processor, coupled to the machine-readable medium, executes the program, wherein the program, when executed, causes the computing processor to receive a source code file and compile the source code file into a medium code file containing a plurality of medium code primitives. The plurality of intermediate code primitives include a segmentation primitive, and the segmentation primitive has at least one parameter to indicate whether the tensor is segmented into a plurality of subtensors along a specified axis of the tensor in time or space. Wherein, if the partitioning primitive does not contain a mapping parameter, the partitioning primitive is used to partition the tensor into the plurality of sub-tensors in time according to the specified axis, and the partitioning primitive indicates that each of the plurality of sub-tensors partitioned in time is used to be broadcast to a plurality of hardware cores.

11. The computer system according to claim 10, characterized in that, The at least one parameter includes a specified axis parameter, used to specify the specified axis of the tensor.

12. The computer system according to claim 10, characterized in that, The at least one parameter includes a length parameter, used to specify the cutting length along the specified axis.

13. The computer system according to claim 10, characterized in that, The at least one parameter includes a mapping parameter, which specifies the multiple hardware cores corresponding to the multiple sub-tensors, so as to divide the tensor into the multiple sub-tensors in space according to the specified axis.

14. The computer system according to claim 13, characterized in that, When the partitioning primitive includes the mapping parameter, the partitioning primitive represents the plurality of sub-tensors partitioned in space for being split one-to-one to the plurality of hardware cores.

15. The computer system according to claim 10, characterized in that, The plurality of escrow primitives also include tensor primitives, and the tensor primitives are used to declare the memory architecture pattern of the tensor.

16. The computer system according to claim 15, characterized in that, In the case where the source code file indicates that the tensor is divided into the multiple sub-tensors in space, the tensor primitive declares that the memory architecture mode of the tensor is a non-uniform memory access architecture.

17. The computer system according to claim 15, characterized in that, In the case where the source code file indicates that the tensor is divided into the multiple sub-tensors in time, the tensor primitive declares that the memory architecture pattern of the tensor is a unified memory access architecture.

18. The computer system according to claim 10, characterized in that, The plurality of intermediate code primitives further include operational primitives, each operational primitive including a first tensor parameter and a second tensor parameter, the first tensor parameter indicating a first tensor, the second tensor parameter indicating a second tensor, and the operational primitive being used to perform operations on the first tensor and the second tensor.

Images