Accelerator, data processing method, and compiler apparatus

Abstract

To improve processing performance of an arithmetic processing apparatus.SOLUTION: An accelerator includes an interface for receiving an instruction sequence including a plurality of instructions, a plurality of 0-th memories, a plurality of first memories, and one or more second memories. Each of the one or more second memories is connected to a corresponding first memory of the first memories. Each of the first memories includes a hierarchical memory connected to a corresponding 0-th memory of the 0-th memories, and a plurality of computing units that run in parallel using the hierarchical memory. The hierarchical memory executes data transfer between the 0-th memories and the first memories, according to a data transfer instruction dedicated to data transfer between the 0-th memories and the first memories, the instruction being included in the instruction sequence received by the interface.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an accelerator, a data processing method, and a compiler device. [Background technology]

[0002] Deep learning is generally performed using a processor with many built-in cores, such as a Graphics Processing Unit (GPU). Recently, processors (accelerators) specialized for deep learning have been developed to improve the calculation speed of deep learning.

[0003] The architecture of a processor specialized for deep learning (such as the number of computing units, the number of blocks including computing units, the number of block hierarchies, and instructions) may differ from the architecture of general-purpose products such as GPUs. For this reason, in order to efficiently operate a processor specialized for deep learning, a compiler device that appropriately generates instructions to be executed by the processor is important. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure improves the processing performance of a processing unit. [Means for solving the problem]

[0005] An accelerator according to an embodiment of the present invention includes an interface that receives an instruction sequence including a plurality of instructions, a plurality of zeroth memories, a plurality of first memories, and one or more second memories, each of which is connected to a corresponding first memory among the plurality of first memories and each of which is connected to a corresponding zeroth memory among the plurality of zeroth memories; and a hierarchical memory that operates in parallel using the hierarchical memory, wherein the hierarchical memory performs data transfer between the plurality of zeroth memories and the plurality of first memories in accordance with a data transfer instruction that is included in the instruction sequence received by the interface and is dedicated to data transfer between the plurality of zeroth memories and the plurality of first memories. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a functional block diagram illustrating an example of a compiler device according to an embodiment of the present invention. [Figure 2] 2 is a block diagram showing an example of a processing unit that executes a program generated by the compiler device of FIG. 1. FIG. [Figure 3] 3 is a block diagram showing an example of a system including a processor in which the arithmetic processing device of FIG. 2 is installed and a host. [Figure 4] FIG. 3 is an explanatory diagram illustrating an example of the operation of the arithmetic processing device of FIG. 2. [Figure 5] 3 is an explanatory diagram showing another example of the operation of the arithmetic processing device of FIG. 2. [Figure 6] 1. FIG. 4 is an explanatory diagram showing an example of processing using instruction codes generated by a compiler device when the instruction merge unit in FIG. 1 is not operated and when it is operated. [Figure 7] 2 is an explanatory diagram showing an example of merging instructions by the instruction merging unit of FIG. 1; FIG. [Figure 8] FIG. 10 is an explanatory diagram showing an example of merging two instructions. [Figure 9] FIG. 10 is an explanatory diagram showing another example of merging two instructions. [Figure 10]FIG. 10 is an explanatory diagram showing an example in which instructions cannot be merged due to data dependency; [Figure 11] FIG. 10 is an explanatory diagram showing an example in which instructions cannot be merged due to a memory shortage; [Figure 12] FIG. 10 is an explanatory diagram illustrating an example of merging two instructions in a compiler device according to another embodiment. [Figure 13] 2 is a block diagram showing an example of a hardware configuration of the compiler device shown in FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0008] Fig. 1 is a functional block diagram showing an example of a compiler device according to an embodiment of the present invention. For example, the compiler device 10 shown in Fig. 1 is realized by a processor such as a CPU (Central Processing Unit) built into an information processing device (computer) such as a server executing a compiler program. By executing the compiler program, the compiler device 10 generates instruction codes to be executed by a processing device 100 shown in Fig. 2. The compiler device 10 includes a computation graph generation unit 11, a first instruction code generation unit 12, a computation graph generation unit 13, a second instruction code generation unit 14, an instruction merging unit 15, and a finalization unit 16.

[0009] The computation graph generation unit 11 generates a computation graph based on a learning model (e.g., source code) generated using, for example, a general-purpose library (framework) for deep learning. A computation graph represents the dependencies between computations (e.g., which computation output or which model's overall input is given to the input of a certain computation) when expressing a learning model as a combination of computational processes commonly used in deep learning, such as convolution operations, batch normalization, and activation function operations. The first instruction code generation unit 12 generates abstraction-level instruction codes using the computation graph generated by the computation graph generation unit 11. Here, the abstraction-level instruction codes may be written in, for example, a high-level language that cannot be executed by a processor installed in a computer.

[0010] The first instruction code generation unit 12 does not convert a computation graph based on a learning model into a single instruction code sequence including instruction codes at an abstraction level, but determines the execution order of processes at the level of basic operations and generates an instruction code for each basic operation. Here, the basic operations include, for example, processing of a layer such as a convolution layer or a pooling layer, processing such as a ReLU (Rectified Linear Unit) function, or processing such as data transfer.

[0011] The operation graph generation unit 13 generates an operation graph based on the instruction codes generated by the first instruction code generation unit 12. The operation graph indicates the dependency relationships between operations when the calculation process of the calculation graph is expressed as a combination of operations (e.g., matrix multiplication operation, addition operation) that can be directly executed by the arithmetic processing device 100. The second instruction code generation unit 14 generates instruction codes at the machine language level based on the operation graph generated by the operation graph generation unit 13. For example, the second instruction code generation unit 14 generates an instruction code sequence including multiple instruction codes for each layer of the deep neural network. When generating the instruction codes, the second instruction code generation unit 14 may perform instruction scheduling to determine the execution order of the instruction codes and memory allocating to determine the allocation of memory areas to be used for executing the instruction codes.

[0012] The instruction merging unit 15 searches for instructions that can be executed in duplicate among the instructions included in the multiple instruction code sequences generated by the second instruction code generating unit 14, and performs a merging process to merge the instructions that can be executed in duplicate. The instruction merging unit 15 merges some of the instructions included in the multiple instruction code sequences generated by the second instruction code generating unit 14, and generates one instruction code sequence at the machine language level.

[0013] The finalization unit 16 finalizes the instruction code sequence generated by the instruction merge unit 15 and outputs an instruction code sequence in machine language that can be directly executed by the processor. The finalized instruction code sequence is, for example, a deep learning program executed by the processor. In the following description, the instruction code is also referred to as an instruction, and the instruction code sequence is also referred to as an instruction sequence.

[0014] Fig. 2 is a block diagram showing an example of a processing device that executes a program generated by the compiler device 10 of Fig. 1. For example, the processing device 100 shown in Fig. 1 may function as an accelerator that executes deep learning. The processing device 100 may be in the form of a chip or a package such as a CSP (Chip Size Package).

[0015] The arithmetic processing device 100 has a plurality of hierarchical arithmetic units and a plurality of memories. For example, the arithmetic processing device 100 has a predetermined number of second-level blocks BLK2. Each second-level block BLK2 has a plurality of first-level blocks BLK1 and a memory MEM2 shared by the plurality of first-level blocks BLK1. Each first-level block BLK1 has a plurality of arithmetic units OPU and a memory MEM1 shared by the plurality of arithmetic units OPU. Each of the hierarchical arithmetic units OPU, first-level blocks BLK1, and second-level blocks BLK2 is an example of a processing unit that executes processing based on an issued instruction. The arithmetic processing device 100 can efficiently transfer data between hierarchical blocks by executing various data transfer instructions.

[0016] In the example shown in FIG. 2, the arithmetic processing device 100 has four second-hierarchical blocks BLK2. Each second-hierarchical block BLK2 has eight first-hierarchical blocks BLK1. Each first-hierarchical block BLK1 has 16 arithmetic units OPU. However, the number of second-hierarchical blocks BLK2 in the arithmetic processing device 100, the number of first-hierarchical blocks BLK1 implemented in each second-hierarchical block BLK2, and the number of arithmetic units OPU implemented in each first-hierarchical block BLK1 are not limited to those shown in FIG. 1. However, it is preferable that the number of second-hierarchical blocks BLK2 implemented in the arithmetic processing device 100, the number of first-hierarchical blocks BLK1 implemented in each second-hierarchical block BLK2, and the number of arithmetic units OPU implemented in each first-hierarchical block BLK1 are each 2 n (n is an integer greater than or equal to 1).

[0017] The arithmetic unit OPU has a memory MEM0 as a memory in the zeroth tier. The first-tier block BLK1 has a memory MEM1 as a memory in the first tier, and the second-tier block BLK2 has a memory MEM2 as a memory in the second tier. The types of memories MEM1 and MEM2 are not particularly limited and may be selected appropriately depending on the performance required of the arithmetic processing device 100. For example, in this embodiment, a first data transfer instruction for transferring data between memories MEM0 and MEM1 in the arithmetic unit OPU and a second data transfer instruction for transferring data between memories MEM1 and MEM2 are provided. That is, a data transfer instruction is provided between the memory MEM2 in the two adjacent tiers. In addition, a third data transfer instruction is provided for transferring data between memory MEM2 and memory MEM3 (a memory outside the arithmetic processing device 100) shown as a memory in the third tier in FIG. 3. That is, a data transfer instruction is provided between the arithmetic processing device 100 and memory MEM3 outside the arithmetic processing device 100. In this way, the arithmetic processing device 100 can execute data transfer instructions between each of two adjacent memory tiers.

[0018] For example, the arithmetic processing device 100 executes a first data transfer instruction to transfer data between memory MEM0 and memory MEM1 in the operation unit OPU in each of all first-level block BLK1 in the arithmetic processing device 100. The arithmetic processing device 100 executes a second data transfer instruction to transfer data between memories MEM1 and MEM2 in each of all second-level block BLK2 in the arithmetic processing device 100. The arithmetic processing device 100 executes a third data transfer instruction to transfer data between memory MEM2 and memory MEM3 (FIG. 3) in each of all second-level block BLK2 in the arithmetic processing device 100.

[0019] Each data transfer instruction is a SIMD (Single Instruction Multiple Data) instruction that executes multiple data transfers with a single instruction. Note that, in order to prevent the loss of data that you do not want to overwrite, the arithmetic processing device 100 may have a function to mask data transfers between some elements.

[0020] Each arithmetic unit OPU has an arithmetic unit EX (FIG. 3) that performs various operations. The arithmetic unit EX may include a plurality of arithmetic elements that perform different types of operations (for example, matrix multiplication and addition).

[0021] The various arithmetic instructions executed by the arithmetic processing device 100 are SIMD instructions that execute multiple operations with one instruction. In this embodiment, the arithmetic processing device 100 can cause all of the arithmetic units EX in the arithmetic processing device 100 to execute operations in parallel (including simultaneously) based on one arithmetic instruction.

[0022] Here, the degree of parallelism of SIMD operations performed by a normal SIMD processor depends on the number of registers allocated to the SIMD registers. On the other hand, the degree of parallelism of SIMD operations performed by the arithmetic processing device 100 depends on the number of processing units EX within the arithmetic processing device 100. Therefore, the arithmetic processing device 100 can operate as a huge SIMD processor.

[0023] Data transfer instructions and operation instructions at different hierarchical levels can be executed in parallel if there is no resource contention. Therefore, the compiler device 10 in Fig. 1 can generate an instruction set including multiple instructions that do not conflict with resources. The instruction set generated by the compiler device 10 will be described with reference to Figs. 6 to 11.

[0024] 3 is a block diagram showing an example of a system including a processor and a host on which the arithmetic processing device 100 of FIG. 2 is mounted. The processor 200 shown in FIG. 3 has a plurality of arithmetic processing devices 100 and a memory MEM3 connected to each other. For example, the processor 200 may have the form of an electronic circuit board on which a plurality of arithmetic processing devices 100 and a memory MEM3 are mounted. The plurality of arithmetic processing devices 100 may be mounted in a multi-chip package. The type of memory MEM3 is not particularly limited and may be selected appropriately depending on the performance required of the arithmetic processing device 100.

[0025] 3, the processor 200 has four arithmetic processing units 100, but the number of arithmetic processing units 100 is not limited to four and may be one or more. A memory MEM3 may be provided corresponding to each arithmetic processing unit 100. A processor 200 having multiple arithmetic processing units 100 operates as a single arithmetic processing unit.

[0026] In each arithmetic processing device 100, the memory MEM2 of each second-level block BLK2 is connected to the memory MEM3 and the memory MEM1 of each first-level block BLK1 in the second-level block BLK2, and data can be transferred between them.

[0027] The memory MEM1 of each first-level block BLK1 is connected to the memory MEM0 in each operation unit OPU in the first-level block BLK1, and data can be transferred between them. Data transfer instructions and operation instructions may be transferred from each memory MEM1 to the memory MEM0. The operation unit EX in the operation unit OPU performs various operations using the memory MEM0 in that operation unit OPU.

[0028] The host memory HOSTM and the memory MEM3 may transmit and receive information such as data and commands via a PCIe (Peripheral Component Interconnect express) interface. Furthermore, the transfer of information between the host memory HOSTM and the memory MEM3 may be performed by DMA (Direct Memory Access).

[0029] Each arithmetic processing unit 100 executes arithmetic processing based on an arithmetic instruction received from the host 300, and executes data transfer processing based on a data transfer instruction received from the host 300. A program including various instruction codes transmitted from the host 300 to the arithmetic processing unit 100 is generated by the compiler device 10 in FIG. 1 and stored in the host memory HOSTM in the host 300.

[0030] 4 and 5 are explanatory diagrams showing an example of the operation of the arithmetic processing device 100 of FIG. 2. The operations shown in FIGS. 4 and 5 are realized by the arithmetic processing device 100 executing a plurality of data transfer instructions and arithmetic instructions in parallel. In FIGS. 4 and 5, elements shown in bold frames indicate resources used for data transfer or arithmetic execution. Arrows connecting elements shown in bold frames indicate the direction of data transfer.

[0031] The number of cycles required for data transfer between memory MEM2 and memory MEM3 is greater than the number of cycles required for data transfer between memory MEM1 and memory MEM2, and the number of cycles required for data transfer between memory MEM1 and memory MEM0 in the arithmetic unit OPU. Furthermore, the execution cycles of various operations by the arithmetic unit EX may differ. However, in the following description, when at least some of the execution cycles of the data transfer process and the arithmetic process overlap, the processes are assumed to be executed in parallel (including simultaneously).

[0032] 4A, the arithmetic unit EX executes an arithmetic instruction for performing an operation in parallel using data stored in each of the memories MEM0. The arithmetic processing device 100 also executes a data transfer instruction for transferring data from the memory MEM3 to the memory MEM2 in parallel with the arithmetic instruction.

[0033] In Fig. 4(A), an arithmetic instruction and a data transfer instruction that use different resources can be executed in parallel. If the compiler device 10 determines that there is no resource conflict as shown in Fig. 4(A), it generates a single instruction group that allows the arithmetic instruction and the data transfer instruction to be executed in parallel.

[0034] In FIG. 4B, the arithmetic unit EX executes an arithmetic instruction that executes an arithmetic operation in parallel using data stored in each of the memories MEM0. The arithmetic processing device 100 also executes a data transfer instruction that transfers (writes) data in parallel from memory MEM1 to each of the memories MEM0 in the arithmetic unit OPU that includes the arithmetic unit EX that executes the arithmetic operation, in parallel with the arithmetic instruction. It is assumed here that the units (resources) involved in the above arithmetic operation and data transfer in each memory MEM0 do not compete with each other.

[0035] In Figure 4(B), there is no conflict between the units involved in the calculation and the units involved in the data transfer, so the calculation instructions and the data transfer instructions can be executed in parallel. On the other hand, if there is conflict between the units involved in the calculation and the units involved in the data transfer, the calculation instructions and the data transfer instructions cannot be executed in parallel and are executed sequentially.

[0036] 4(B), if the compiler device 10 determines that there is no resource conflict, it generates one instruction set that allows the operation instruction and the data transfer instruction to be executed in parallel. On the other hand, if the compiler device 10 determines that there is a resource conflict, it generates two instruction sets that execute the operation instruction and the data transfer instruction sequentially.

[0037] In FIG. 4(C), the arithmetic unit EX executes an arithmetic instruction that executes an arithmetic operation in parallel using data stored in each of the memories MEM0. The arithmetic processing device 100 also executes a data transfer instruction that transfers data in parallel from memory MEM0 to different areas in memory MEM1 in the arithmetic unit OPU that includes the arithmetic unit EX that executes the arithmetic operation, in parallel with the arithmetic instruction. It is assumed here that the units (resources) involved in the above arithmetic operation and data transfer in each memory MEM0 do not compete with each other. Furthermore, the arithmetic processing device 100 also executes a data transfer instruction that transfers data from memory MEM2 to memory MEM3, in parallel with the arithmetic instruction.

[0038] 4(C), there is no conflict between the resources used for the operation in the arithmetic unit EX and the two data transfers, so the compiler device 10 generates a single instruction set that can execute one operation instruction and two data transfer instructions in parallel. On the other hand, when there is conflict between the resources related to the operation and the resources related to the data transfer to memory MEM1, the compiler device 10 generates a single operation instruction and a single instruction set that can execute two data transfer instructions in parallel.

[0039] 5(D), the arithmetic processing device 100 executes a data transfer instruction to transfer data from a plurality of memories MEM0 to memory MEM1 in the arithmetic unit OPU. The arithmetic processing device 100 also executes a data transfer instruction to transfer data from memory MEM2 to memory MEM3 in parallel with a data transfer instruction from memory MEM0 to memory MEM1.

[0040] In FIG. 5(D), the two data transfers do not conflict and can therefore be executed in parallel. Therefore, the compiler device 10 generates a single instruction group that allows the two data transfer instructions to be executed in parallel. Whether the two data transfers conflict depends on the architecture of the arithmetic processing device 100, and the compiler device 10 may determine whether the two data transfers conflict by taking this architecture into consideration. The compiler device 10 of this embodiment determines that the data transfers conflict when the data transfer destinations of multiple data transfer instructions are the same.

[0041] 5(E), the first and second processing elements of the processing unit EX execute a processing instruction to execute the first and second processing in parallel using data stored in each of the memories MEM0. Note that, in this case, it is assumed that the units (resources) involved in the first and second processing in each memory MEM0 and the processing unit EX do not compete with each other.

[0042] In FIG. 5(E), there is no contention between resources related to the first and second operations. Therefore, the compiler device 10 generates one set of instructions that allows the first and second operations to be executed in parallel by the first and second operation elements. Note that the compiler device 10 can include data transfer between the memory MEM1 and the memory MEM2 or data transfer between the memory MEM2 and the memory MEM3 in the set of instructions. In FIG. 5(E), if the compiler device 10 determines that there is contention for the memory MEM0, it does not generate a set of instructions, but instead sequentially generates two operation instructions that execute the first and second operations in the first and second operation elements, respectively.

[0043] In FIG. 5(F), the processor EX executes an operation instruction that performs an operation using data stored in memory MEM0. Furthermore, the processor 100 executes a data transfer instruction that transfers data from memory MEM1 to memory MEM0 in parallel with the operation instruction in the operation unit including the processor EX that executes the operation. It is assumed here that the units (resources) involved in the above operation and data transfer in each memory MEM0 do not conflict with each other. Furthermore, the processor 100 executes a data transfer instruction that transfers data from memory MEM3 to memory MEM2 in parallel with the above operation instruction and data transfer instruction.

[0044] In Fig. 5(F), there is no conflict between the resources used for the operation by the arithmetic unit EX and the two data transfers, so the compiler device 10 generates one instruction set that can execute one operation instruction and two data transfer instructions in parallel. The method for generating an instruction set when the compiler device 10 determines that there is a resource conflict is the same as that described in Fig. 4(C).

[0045] 6 to 11 will be used to explain an example of processing for merging instructions by the instruction merging unit 15. For example, the processing shown in Figures 6 to 11 shows an example of processing by a compiler program executed by the compiler device 10 and an example of processing by a compiling method executed by the compiler device 10. The method by which the compiler device 10 generates instructions by the processing shown in Figures 6 to 11 shows an example of an instruction generation method.

[0046] 6 is an explanatory diagram showing an example of processing using instruction codes generated by the compiler device 10 when the instruction merge unit 15 in FIG. 1 is not operating and when it is operating. The method in which the compiler device 10 operates the instruction merge unit 15 to generate instruction codes is an example of an instruction generation method. Download DL (DL1, DL2), upload UL, convolution operation Conv (Conv1, Conv2, Conv3), batch normalization BN, and ReLU function are each examples of basic operations. The rectangles showing the frequencies of calculation instructions and data transfer instructions have a darker shaded area, indicating a higher frequency of instruction execution.

[0047] In the example shown in FIG. 6, the arithmetic processing device 100 downloads data DL1 from memory MEM3 to memory MEM0, and executes convolution operation Conv1 using the downloaded data (FIGS. 6(a) and 6(b)). The process of download DL1 is executed using a data transfer instruction. The process of convolution operation Conv1 is executed mainly using an arithmetic instruction.

[0048] Next, the arithmetic processing device 100 uploads (UL) the data obtained by the convolution operation Conv1 from the memory MEM0 to the memory MEM3 (FIG. 6(c)). The upload (UL) process is executed using a data transfer instruction.

[0049] Furthermore, the arithmetic processing device 100 sequentially executes batch normalization BN processing and ReLU function processing on the data obtained by the convolution operation Conv1 (FIGS. 6(d) and 6(e)). The batch normalization BN processing and ReLU function processing are executed mainly using calculation instructions. Next, the arithmetic processing device 100 executes convolution operation Conv2 (FIG. 6(f)).

[0050] Furthermore, the arithmetic processing device 100 downloads data DL2 from memory MEM3 to memory MEM0 (FIG. 6(g)). Then, the arithmetic processing device 100 executes the convolution operation Conv3 using the downloaded data (FIG. 6(h)). In this way, when the instruction merge unit 15 is not operated, the arithmetic processing device 100 sequentially executes the processing of the deep neural network.

[0051] On the other hand, when the instruction merge unit 15 is operated, the instruction merge unit 15 generates a single instruction group including multiple instructions that do not compete for resources, thereby enabling multiple basic operations to be executed in parallel. This makes it possible to reduce the calculation time required for training a deep neural network, for example, and improve the processing performance of the arithmetic processing device 100.

[0052] For example, the upload UL1 process is an example of a first process, and the batch normalization BN process and the ReLU function process are examples of a second process executed after the first process. In this case, an instruction sequence including an instruction to execute the upload UL1 process is an example of a first instruction sequence including a first instruction. An instruction sequence including an instruction to execute the batch normalization BN process and an instruction sequence including an instruction to execute the ReLU function process are examples of a second instruction sequence including a second instruction.

[0053] Furthermore, the ReLU function processing and the convolution operation Conv2 processing are an example of a first processing, and the download DL2 processing is an example of a second processing. In this case, the instruction sequence including instructions for executing the ReLU function processing and the convolution operation Conv2 processing is an example of a first instruction sequence including a first instruction. The instruction sequence including instructions for executing the download DL2 processing is an example of a second instruction sequence including a second instruction.

[0054] For example, the instruction merging unit 15 assembles into at least one instruction group the arithmetic instructions used in the batch normalization BN and ReLU function processing and the data transfer instructions used in the upload UL processing that do not conflict with resources (FIG. 6(i)). Also, the instruction merging unit 15 assembles into at least one instruction group the arithmetic instructions used in the ReLU function and convolution operation Conv2 processing and the data transfer instructions used in the download DL2 processing that do not conflict with resources (FIG. 6(j)).

[0055] An instruction group formed by grouping multiple instructions is an example of a third instruction. The instruction merge unit 15 combines the group of instructions formed by grouping multiple instructions with instructions other than the instructions grouped in the instruction group to generate an instruction sequence (third instruction sequence) indicated by the dashed line frame in FIG. 6. By grouping instructions, for example, it is possible to increase the execution frequency of data transfer instructions and improve data transfer efficiency. As a result, it is possible to improve the processing performance of the arithmetic processing device 100.

[0056] The resource that conflicts between the arithmetic instructions and the data transfer instructions is the memory MEM0. Therefore, the instruction merge unit 15 can generate an instruction group by combining arithmetic instructions and data transfer instructions as long as there is no resource conflict in the memory MEM0. Resource conflicts between arithmetic instructions and data transfer instructions often occur less frequently than resource conflicts between arithmetic instructions. Therefore, by combining arithmetic instructions and data transfer instructions, the efficiency of compressing the number of instructions can be improved compared to combining arithmetic instructions, and the processing performance of the arithmetic processing device 100 can be improved.

[0057] Note that the instruction merge unit 15 does not merge instructions if merging the instructions would change the processing result of the basic operation. In other words, the instruction merge unit 15 determines whether the processing result of the instruction sequence after merging the target instruction is equivalent to the processing result of the instruction sequence before merging. If the processing result is equivalent, the instruction merge unit 15 merges the instructions to generate an instruction set. This can prevent malfunctions that would cause the arithmetic processing device 100 executing the merged instruction sequence to not operate normally.

[0058] For example, examples of when the processing result of a basic operation changes include when data dependency is broken or when the memory size used is insufficient. An example of when data dependency is broken is shown in Figure 10, and an example of when the memory size used is insufficient is shown in Figure 11.

[0059] FIG. 7 is an explanatory diagram showing an example of merging instructions by the instruction merging unit 15 of FIG. 1. For example, uppercase letters indicate an instruction sequence IS1 including an instruction IS used in the batch normalization BN process of FIG. 6. Lowercase letters indicate an instruction sequence IS2 including an instruction IS used in the upload UL process of FIG. 6. The instruction sequence IS1 is an example of a first instruction sequence, and the instruction IS included in the instruction sequence IS1 is an example of a first instruction. The instruction sequence IS2 is an example of a second instruction sequence, and the instruction IS included in the instruction sequence IS2 is an example of a second instruction.

[0060] As will be described later with reference to Figures 8 and 9, one instruction IS includes multiple different instructions for multiple processing units capable of executing different types of processing in parallel. That is, one instruction IS includes an instruction for a first processing unit, an instruction for a second processing unit, ..., an instruction for an Nth processing unit. The instructions provided to the processing units may include NOP (NO OPERATION: do nothing). The multiple processing units execute different types of processing (including NOP) in parallel according to the multiple different instructions provided simultaneously.

[0061] The instruction merge unit 15 extracts instructions IS that can be overlapped from the instructions IS included in the instruction sequences IS1 and IS2, and generates an instruction sequence IGr by merging the extracted instructions IS. The instruction sequence IGr is an example of a third instruction. The instruction merge unit 15 then generates an instruction sequence IS3 that includes an instruction sequence IGr in which instructions IS are merged and a single instruction IS that is not merged. The instruction sequence IS3 is an example of a third instruction sequence. For example, the instructions IS and the instruction sequence IGr included in the instruction sequences IS1, IS2, and IS3 are written at the machine language level.

[0062] As shown in Fig. 7, the number of instructions IS that can be merged in two instruction sequences IS1 and IS2 is limited. However, by merging some instructions IS as an instruction group IGr and making them executable in parallel, the instruction execution speed can be increased compared to the case where instructions IS are executed sequentially, as shown in Fig. 6.

[0063] The brackets in Figure 7 indicate the state during the merge process. The bold lines in the instruction sequences IS1, IS2, and IS3 indicate that the merge process of the mergeable instruction IS has already been performed, while the lines after the bold lines indicate that the merge process has not yet been performed. The instructions G and e shown in dashed lines indicate the next instruction IS to be merged.

[0064] As shown by the bold line, the instruction merge unit 15 recognizes how far the merging process of the instruction sequences IS1 and IS2 has been performed, determines whether the next instruction (e.g., G, e) can be merged, and if so, repeats the process of generating the instruction group IGr.

[0065] 8 and 9 are explanatory diagrams showing an example of merging two instructions. For example, instruction 1 corresponds to instruction IS in instruction sequence IS1 in FIG. 7, and instruction 2 corresponds to instruction IS in instruction sequence IS2 in FIG. 7. Instruction 1+2 indicates a group of instructions obtained by merging instruction 1 and instruction 2. In instruction 1 and instruction 2, the symbol "-" indicates that no instruction is executed (NOP). That is, instruction 1, instruction 2, and instruction 1+2 obtained by merging instruction 1 and instruction 2 each contain multiple different instructions for multiple processing units capable of executing different types of processing in parallel, similar to instruction IS in FIG. 7. FIGS. 8(A)-8(C) and 9(D)-9(F) correspond to the operations shown in FIGS. 4(A)-4(C) and 5(D)-5(F), respectively.

[0066] 8A, the instruction merge unit 15 determines that merging is possible because there is no conflict in resources used between instruction 1, which includes an arithmetic instruction for executing a first arithmetic operation by the arithmetic unit EX, and instruction 2, which includes a data transfer instruction for transferring data from memory MEM3 to memory MEM2. Therefore, the instruction merge unit 15 generates instruction 1+2 by merging the arithmetic instruction and the data transfer instruction.

[0067] 8(B), the instruction merge unit 15 determines that there is no conflict in resources used by instruction 1, which includes an arithmetic instruction for executing a first arithmetic operation, and instruction 2, which includes a data transfer instruction for transferring data from memory MEM1 to processor element PE, under certain conditions. A condition for no resource conflict is, for example, that in each memory MEM0, the unit to which the result of the arithmetic instruction is written is different from the unit to which the data is transferred. If the condition is satisfied, the instruction merge unit 15 generates instruction 1+2 by merging the arithmetic instruction and the data transfer instruction.

[0068] 8(C), the instruction merge unit 15 determines that there is no conflict, under a condition, between the resources used by instruction 1 including an arithmetic instruction for executing a first arithmetic operation, instruction 2 including a data transfer instruction for transferring data from memory MEM0 to memory MEM1, and a data transfer instruction for transferring data from memory MEM2 to memory MEM3. The condition for there to be no conflict between resources is that, in each memory MEM0, the unit used for the arithmetic instruction and the unit used for data transfer are different.

[0069] If the condition is satisfied, the instruction merge unit 15 generates instruction 1+2 by merging the arithmetic instruction and two data transfer instructions. If the condition is not satisfied, the instruction merge unit 15 does not merge the single arithmetic instruction and the two data transfer instructions, and as a result, instruction 1 and instruction 2 are maintained as separate instructions. However, instruction 1 may be merged with another instruction, and instruction 2 may be merged with another instruction.

[0070] 9(D), the instruction merge unit 15 determines that instruction 1, which includes a data transfer instruction for transferring data from memory MEM0 to memory MEM1, and instruction 2, which includes a data transfer instruction for transferring data from memory MEM2 to memory MEM3, can be merged because there is no conflict in the resources used between them. Therefore, the instruction merge unit 15 generates instruction 1+2 by merging the two data transfer instructions. The data transfer instruction for transferring data from memory MEM0 to memory MEM1 is an example of a first data transfer instruction, and the data transfer instruction for transferring data from memory MEM2 to memory MEM3 is an example of a second data transfer instruction.

[0071] In this embodiment, a data transfer instruction is prepared for each memory hierarchy. Therefore, as shown in FIG. 9D, the instruction merge unit 15 can determine that there is no conflict between data transfer instructions for which memory hierarchies do not overlap. As a result, the compiler device 10 can efficiently generate instructions.

[0072] 9(E), the instruction merge unit 15 determines that there is no conditional conflict between the resources used by instruction 1, which includes a first operation instruction that causes the arithmetic unit EX to execute a first operation, and instruction 2, which includes a second operation instruction that causes the arithmetic unit EX to execute a second operation. The condition for there to be no resource conflict is that the operation elements that execute the operations are different from each other, and the units in memory MEM0 used in the first and second operations are different from each other. If the condition is satisfied, the instruction merge unit 15 generates instruction 1+2 by merging the first operation instruction and the second operation instruction.

[0073] 9(F), the instruction merge unit 15 determines that there is no conditional conflict between the resources used by instruction 1, which includes an arithmetic instruction that causes arithmetic unit EX to execute a first arithmetic operation, and instruction 2, which includes a data transfer instruction that transfers data from memory MEM1 to memory MEM0 and a data transfer instruction that transfers data from memory MEM3 to memory MEM2. The condition for there to be no resource conflict is that, in each memory MEM0, the unit used for the arithmetic instruction and the unit used for data transfer are different.

[0074] If the condition is satisfied, the instruction merge unit 15 generates instruction 1+2 by merging the arithmetic instruction and the two data transfer instructions. If the condition is not satisfied, the instruction merge unit 15 does not merge the single arithmetic instruction and the two data transfer instructions, and as a result, instruction 1 and instruction 2 are maintained as separate instructions.

[0075] 10 is an explanatory diagram showing an example in which instructions cannot be merged due to data dependency. In FIG. 10, it is assumed that a process in layer A that writes a value x to memory MEM0 and a process in layer B that reads the value x written to memory MEM0 by the process in layer A are executed sequentially.

[0076] When merging an instruction sequence of layer A with an instruction sequence of layer B, the instruction merge unit 15 determines whether there is a dependency relationship between the data read from and written to the memory MEM0. In the example of FIG. 10, there is a dependency relationship between the data read from and written to the memory MEM0. When merging an instruction sequence of layer A with an instruction sequence of layer B, a process of reading the value x from the memory MEM0 is executed before the value x is written to the memory MEM0. For this reason, the instruction merge unit 15 determines that merging the instruction sequences will not result in correct processing, and does not merge the instruction sequences.

[0077] In this way, the instruction merge unit 15 temporarily restricts the merging of instruction sequences when there is a data dependency between the memory MEM0 or the like commonly used by the layers A and B to be merged, and the read and write of the merged data are not executed in the correct order. That is, as can be understood from the description of FIG. 7 , for a data read instruction and a data write instruction to the same memory MEM0, the instruction merge unit 15 waits to merge the read instruction of the instruction sequence in layer B until the merging of at least the write instruction of the instruction sequence in layer A is completed. This makes it possible to prevent a malfunction in which the arithmetic processing device 100 does not operate normally due to a breakdown in the data dependency. Note that instead of temporarily restricting the merging of instruction sequences, it is also possible to abandon the merging of the instruction sequences between the layers and not merge the instruction sequences between the layers at all.

[0078] FIG. 11 is an explanatory diagram showing an example in which instructions cannot be merged due to insufficient memory. Note that the numerical values ​​of memory capacity described below are examples. For example, assume that a sequence of instructions in layer A and a sequence of instructions in layer B share a memory with a storage capacity of 800 to execute calculation processes (calculations 1-1, 1-2, 2-1, 2-2). Note that the memory used for the calculation processes may be distributed among multiple locations among memories MEM0, MEM1, and MEM2.

[0079] The memory capacity required for calculations 1-1 and 2-1 is 300 each, and the memory capacity required for calculations 1-2 and 2-2 is 400 each. After performing calculation 1-2, layer A releases the memory capacity 700 reserved for calculations 1-1 and 1-2. After performing calculation 2-2, layer B releases the memory capacity 700 reserved for calculations 2-1 and 2-2. If merging is not performed in this way, no memory shortage will occur.

[0080] When merging an instruction sequence of layer A with an instruction sequence of layer B, the instruction merge unit 15 determines whether the calculation processes of layer A and layer B can be executed without the memory capacity used exceeding the memory storage capacity. In the example shown in Fig. 11, when the instruction sequences are merged, after executing calculations 1-1 and 2-1, the memory capacity 400 required for calculations 1-2 and 2-2 cannot be secured, and the calculation process fails.

[0081] If merging of instruction sequences would result in a shortage of memory capacity, the instruction merge unit 15 suppresses merging of instruction sequences, thereby preventing the arithmetic processing device 100 from operating normally due to a shortage of memory capacity.

[0082] As described above, in this embodiment, the instruction merge unit 15 generates a single instruction group including multiple instructions that do not conflict with each other for resources, thereby enabling multiple basic operations to be executed in parallel. This makes it possible to reduce the calculation time required for training a deep neural network, for example, and improve the processing performance of the arithmetic processing device 100.

[0083] The instruction merge unit 15 determines whether the processing result of the instruction sequence after merging the target instruction is equivalent to the processing result of the instruction sequence before merging. If the processing result is equivalent, the instruction merge unit 15 merges the instructions to generate a new instruction sequence. This makes it possible to prevent malfunctions that would cause the arithmetic processing device 100 executing the merged instruction sequence to not operate normally.

[0084] The instruction merge unit 15 temporarily restricts the merging of instruction sequences when there is a data dependency in a memory MEM0 or the like that is shared by layers A and B to be merged and the order of reading and writing data after merging is incorrect. This makes it possible to prevent malfunctions that would cause the arithmetic processing device 100 to malfunction due to a breakdown in the data dependency.

[0085] If merging of instruction sequences would result in a shortage of memory capacity, the instruction merge unit 15 suppresses merging of instruction sequences, thereby preventing the arithmetic processing device 100 from operating normally due to a shortage of memory capacity.

[0086] As described in FIG. 6, by combining an arithmetic instruction and a data transfer instruction into a single instruction, the efficiency of compressing the number of instructions can be improved compared to combining arithmetic instructions, thereby improving the processing performance of the arithmetic processing device 100. Data transfer instructions are prepared for each different memory hierarchies. Therefore, as shown in FIG. 9(D), the instruction merge unit 15 can determine that there is no conflict between data transfer instructions in non-overlapping memory hierarchies. Therefore, the instruction merge unit 15 can easily determine conflicts between data transfer instructions in different memory hierarchies, allowing the compiler device 10 to efficiently perform instruction generation processing.

[0087] 12 is an explanatory diagram showing an example of merging two instructions in a compiler device of another embodiment. Fig. 12 shows an example of an instruction generation method by the compiler device. The compiler device of this embodiment has the same functions as the second instruction code generation unit 14, the instruction merging unit 15, and the finalization unit 16 of the compiler device 10 of Fig. 1. That is, the compiler device of this embodiment has the function of generating a new instruction by merging an instruction included in a first instruction sequence with an instruction included in a second instruction sequence.

[0088] In this embodiment, the instructions included in the first instruction sequence, the instructions included in the second instruction sequence, and the new instructions generated by merging these two instructions each include multiple different instructions for multiple processing units capable of executing different types of processing in parallel. That is, the instructions included in the instruction sequence and the new instructions generated by merging include instructions for the first processing unit, instructions for the second processing unit,..., instructions for the Nth processing unit. The instructions provided to the processing units may include NOP (NO OPERATION: do nothing). The multiple processing units execute different types of processing (including NOP) in parallel according to the multiple different instructions provided simultaneously.

[0089] For example, the arithmetic processing device that executes Instruction 1 and Instruction 2 shown in Fig. 12 is a VLIW (Very Long Instruction Word) processor, and has multiple processing units that can execute different types of processing, such as an ALU (Arithmetic and Logic Unit) and two FPUs (Floating Point Units) 0 and 1. Note that, for ease of explanation, Fig. 12 shows an example in which Instruction 1 includes two instructions for the ALU and FPU0 and a NOP instruction for FPU1, and Instruction 2 includes a NOP instruction for the ALU and FPU0 and one instruction for FPU1, but each of Instruction 1 and Instruction 2 may include four or more instructions.

[0090] The instruction merge unit 15 determines whether there is a resource conflict between instruction 1, which includes an integer addition instruction (c<-a+b) and a floating-point addition instruction (f<-d+.e), and instruction 2, which includes a floating-point multiplication instruction (z<-x*.y). Since there is no resource conflict between instruction 1 and instruction 2, the instruction merge unit 15 determines that instruction 1 and instruction 2 can be merged, and generates an instruction group (instruction 1+2).

[0091] This integer addition instruction adds the data held in integer registers a and b and stores the result in integer register c. The floating-point addition instruction adds the data held in floating-point registers d and e and stores the result in floating-point register f. The addition sign "+." indicates floating-point addition. The floating-point multiplication instruction multiplies the data held in floating-point registers x and y and stores the result in floating-point register z. The multiplication sign "*." indicates floating-point multiplication.

[0092] As described above, this embodiment can also achieve the same effects as the above-described embodiments. For example, in this embodiment, instruction 1 and instruction 2, each of which contains at least one instruction executed by the VLIW processor, are merged to generate an instruction group. This improves the execution efficiency of the VLIW processor and reduces the calculation time compared to when instruction 1 and instruction 2 are not merged.

[0093] In the above-described embodiments, the process of merging (combining) a first instruction in a first instruction sequence and a second instruction in a second instruction sequence into a single instruction has been described. However, the process of combining a first instruction in a first instruction sequence and a second instruction in a second instruction sequence can also be applied to an instruction sequence supplied to a superscalar processor. A superscalar processor can automatically detect two adjacent instructions in an instruction sequence that can be executed in parallel and execute them in parallel. For such a superscalar processor, the compiler device 10 may perform a process of combining multiple instructions as follows. If it is expected that the processor will automatically execute the first instruction in the first instruction sequence and the second instruction in the second instruction sequence in parallel by arranging the first instruction and the second instruction adjacent to each other in a single instruction sequence, the compiler device 10 generates an instruction in which the first instruction and the second instruction are merged (combined) by arranging the first instruction and the second instruction adjacent to each other. In this case, the first and second instructions arranged in a row correspond to the third instruction. This process is expected to result in a sequence of instructions that can be executed by a processor with a higher degree of parallelism.

[0094] In the above-described embodiments, the compiler device 10 or the host 300 may be partially or entirely configured with hardware, or may be configured with software (program) information processing executed by a CPU, a GPU, or the like. When configured with software information processing, software that realizes at least some of the functions of each device in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or a USB (Universal Serial Bus) memory, and the software information processing may be executed by loading the software into a computer. Alternatively, the software may be downloaded via a communication network. Furthermore, the software information processing may be executed by hardware by implementing all or part of the software processing in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0095] The storage medium that stores the software may be a removable medium such as an optical disk, or a fixed medium such as a hard disk, memory, etc. The storage medium may be provided inside the computer (main storage device, auxiliary storage device, etc.) or outside the computer.

[0096] Fig. 13 is a block diagram showing an example of the hardware configuration of the compiler device 10 shown in Fig. 1. The hardware configuration of the host 300 shown in Fig. 3 is also the same as the hardware configuration of the compiler device 10 shown in Fig. 13. As an example, the compiler device 10 may be realized as a computer including a processor 20, a main storage device 30 (memory), an auxiliary storage device 40 (memory), a network interface 50, and a device interface 60, which are connected via a bus 70. For example, the processor 20 executes a compiler program, thereby performing the operations described in Figs. 6 to 12.

[0097] Although the compiler device 10 includes one of each component, it may also include multiple of the same component. Although FIG. 13 shows one compiler device 10, the software may be installed in multiple compiler devices 10, and each of the multiple compiler devices 10 may execute the same or different parts of the software. In this case, a distributed computing configuration may be adopted in which each compiler device 10 communicates with the other compiler devices 10 via a network interface 50 or the like to execute processing. That is, the compiler device 10 in the above-described embodiment may be configured as a system in which one or more compiler devices 10 execute instructions stored in one or more storage devices to realize its functions. Furthermore, the compiler device 10 may be configured such that information transmitted from a terminal is processed by one or more compiler devices 10 provided on a cloud, and the processing results are transmitted to the terminal.

[0098] 6 to 11 may be executed in parallel using one or more processors 20, or using multiple computers via a network. Furthermore, the various calculations may be distributed to multiple processing cores within the processor 20 and executed in parallel. Furthermore, some or all of the processes, means, etc. disclosed herein may be implemented by at least one of a processor and a storage device provided on a cloud that can communicate with the compiler device 10 via a network. Thus, the compiler device 10 in the above-described embodiment may be implemented in the form of parallel computing using one or more computers.

[0099] The processor 20 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that performs at least one of computer control and calculation. The processor 20 may also be a general-purpose processor, a dedicated processing circuit designed to perform a specific calculation, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. The processor 20 may also include an optical circuit or a calculation function based on quantum computing.

[0100] The processor 20 may perform arithmetic processing based on data or software input from each device, etc., configured internally of the compiler device 10, and may output arithmetic results or control signals to each device, etc. The processor 20 may control each component constituting the compiler device 10 by executing the OS (Operating System) of the compiler device 10, applications, etc.

[0101] The compiler apparatus 10 in the above-described embodiment may be realized by one or more processors 20. Here, the processor 20 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the respective electronic circuits may communicate with each other via wire or wirelessly.

[0102] The main memory device 30 may store instructions to be executed by the processor 20, various data, etc., and information stored in the main memory device 30 may be read by the processor 20. The auxiliary memory device 40 is a memory device other than the main memory device 30. Note that these memory devices refer to any electronic component capable of storing electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The memory device for saving various data, etc. in the compiler device 10 in the above-described embodiment may be realized by the main memory device 30 or the auxiliary memory device 40.

[0103] When the compiler device 10 in the above-described embodiment is configured with at least one storage device (memory) and at least one processor 20 connected (coupled) to this at least one storage device, at least one processor 20 may be connected to one storage device. Also, at least one storage device may be connected to one of the processors 20. Also, a configuration may be included in which at least one processor 20 of the multiple processors 20 is connected to at least one storage device of the multiple storage devices. Also, this configuration may be realized by storage devices and processors 20 included in multiple compiler devices 10. Furthermore, a configuration in which a storage device is integrated with a processor 20 (for example, a cache memory including an L1 cache and an L2 cache) may be included.

[0104] The network interface 50 is an interface for connecting to the communication network 600 wirelessly or via a wire. The network interface 50 may be an appropriate interface, such as one conforming to an existing communication standard. Information may be exchanged with an external device 710 connected via the communication network 600 through the network interface 50. The communication network 600 may be any one of a wide area network (WAN), a local area network (LAN), a personal area network (PAN), etc., or a combination thereof, as long as information is exchanged between the compiler device 10 and the external device 710. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0105] The device interface 60 is an interface such as a USB that directly connects to an external device 720 .

[0106] The external device 710 is connected to the compiler device 10 via the communication network 600. The external device 720 is connected directly to the compiler device 10.

[0107] For example, the external device 710 or the external device 720 may be an input device. The input device is, for example, a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, a touch panel, or the like, and provides acquired information to the compiler apparatus 10. Alternatively, the external device 710 or the external device 720 may be a device including an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0108] Furthermore, the external device 710 or the external device 720 may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or may be a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0109] Furthermore, the external device 710 or the external device 720 may be a storage device (memory). For example, the external device 710 may be a network storage or the like, and the external device 720 may be a storage such as an HDD.

[0110] Furthermore, the external device 710 or the external device 720 may be a device having some of the functions of the components of the compiler device 10 in the above-described embodiment. That is, the compiler device 10 may transmit some or all of the processing results of the external device 710 or the external device 720, or may receive some or all of the processing results from the external device 710 or the external device 720.

[0111] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0112] In this specification (including claims), when expressions such as "using data as input / based on / according to / in response to data" (including similar expressions) are used, unless otherwise specified, this includes cases where the data itself is used, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of data, etc.) is used. Furthermore, when a statement is made that a result is obtained "using data as input / based on / according to / in response to data" (including similar expressions), this includes cases where the result is obtained based solely on the data, or where the result is influenced by other data, factors, conditions, and / or states other than the data itself, unless otherwise specified. Furthermore, when a statement is made that "data is output" (including similar expressions), this includes cases where the data itself is used as output, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of various data, etc.) is used as output, unless otherwise specified.

[0113] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that encompass any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0114] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure, etc. of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0115] Whenever words implying containing or possessing (e.g., "comprising / including," "having," etc.) are used in this specification (including the claims), they are intended to be open-ended terms that include the inclusion or possession of things other than the object designated by the object of the term. When the object of such words implying containing or possessing does not specify a quantity or suggests a singular number (e.g., expressions using the articles "a" or "an"), the expression should be construed as not being limited to a specific number.

[0116] In this specification (including the claims), even if expressions such as "one or more" and "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0117] In this specification, when a particular advantage / result is described as being obtained with respect to a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained with one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or circumstances, and that the effect is not necessarily obtained with the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or circumstances are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same or a similar configuration.

[0118] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" (including similar expressions) is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that hardware may include electronic circuits, devices including electronic circuits, etc.

[0119] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices may store only a portion of the data, or may store the entire data. Also, a configuration in which only some of the multiple storage devices store data may be included.

[0120] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention, which is derived from the content defined in the claims and their equivalents. For example, when numerical values ​​or formulas are used in the above-described embodiments, they are shown for illustrative purposes and do not limit the scope of the present disclosure. Furthermore, the order of each operation shown in the embodiments is also illustrative and does not limit the scope of the present disclosure.

Claims

1. an interface for receiving a command sequence including a plurality of commands; a hierarchical memory including a plurality of 0th memories, a plurality of first memories, and one or more second memories, wherein each of the one or more second memories is connected to a corresponding first memory among the plurality of first memories, and each of the plurality of first memories is connected to a corresponding 0th memory among the plurality of 0th memories; a plurality of computing units that operate in parallel using the hierarchical memory; and the hierarchical memory executes data transfer between the plurality of zeroth memories and the plurality of first memories in accordance with a data transfer instruction dedicated to data transfer between the plurality of zeroth memories and the plurality of first memories, the data transfer instruction being included in an instruction sequence received by the interface; Accelerator.

2. the data transfer instruction is a SIMD instruction; The accelerator of claim 1 .

3. 3. The accelerator according to claim 1, wherein the hierarchical memory executes the data transfer in parallel between the plurality of 0th memories and the plurality of first memories in accordance with the data transfer command.

4. a plurality of computing units; the plurality of arithmetic units include a corresponding arithmetic unit among the plurality of arithmetic units and a corresponding 0th memory among the plurality of 0th memories, a first arithmetic unit included in a first arithmetic unit of the plurality of arithmetic units executes an arithmetic operation using a 0th memory included in the first arithmetic unit; each of the plurality of first memories is shared by a corresponding one of the plurality of arithmetic units; 4. The accelerator according to claim 1.

5. the sequence of instructions includes one instruction that is a collection of multiple instructions that do not conflict with each other for resources; 5. The accelerator according to claim 1.

6. Whether or not multiple instructions compete for resources is determined based on the accelerator architecture. The accelerator of claim 5 .

7. the hierarchical memory and the plurality of arithmetic units execute the data transfer instruction and the arithmetic operation instruction in parallel according to one instruction included in the instruction sequence, the one instruction being a combination of the data transfer instruction and the arithmetic operation instruction that do not conflict with each other for resources; 7. The accelerator according to claim 1.

8. The instruction sequence is generated based on a learning model generated using a deep learning framework.

8. The accelerator according to claim 1.

9. Perform deep learning, 9. The accelerator according to claim 1.

10. the interface receives the sequence of instructions including the data transfer instruction from a host external to the accelerator; 10. The accelerator according to any one of claims 1 to 9.

11. each of the plurality of computing units includes a plurality of computing elements that perform different types of computations; 11. An accelerator according to any one of claims 1 to 10.

12. each of the plurality of arithmetic units includes an arithmetic element that performs a matrix multiplication operation and an arithmetic element that performs an addition operation; 11. An accelerator according to any one of claims 1 to 10.

13. 13. The accelerator according to claim 1, wherein, in accordance with the sequence of instructions, at least two data transfers in the hierarchical memory are executed in parallel, or at least two operations in the plurality of arithmetic units are executed in parallel, or one or more data transfers in the hierarchical memory and one or more operations in the plurality of arithmetic units are executed in parallel.

14. the at least two operations are executed in parallel using data stored in the plurality of 0th memories in accordance with the sequence of instructions; The accelerator of claim 13 .

15. the one or more data transfers between the plurality of zeroth memories and the plurality of first memories and the one or more operations in the plurality of arithmetic units are executed in parallel according to the instruction sequence; 15. The accelerator according to claim 13 or 14.

16. One instruction included in the instruction sequence includes a plurality of individual instructions corresponding to different types of processes that can be executed in parallel, the plurality of individual instructions includes the data transfer instruction as one of the plurality of individual instructions; 16. An accelerator according to any one of claims 1 to 15.

17. The plurality of individual instructions includes a NOP instruction as one of the plurality of individual instructions. The accelerator of claim 16.

18. One instruction included in the instruction sequence includes a plurality of different individual instructions for a plurality of processing units each capable of executing different types of processing in parallel, the plurality of individual instructions include the data transfer instruction and a NOP instruction that causes a corresponding processing unit not to execute processing; the plurality of processing units execute different types of processing or NOPs in parallel according to the plurality of individual instructions included in the one instruction; 16. An accelerator according to any one of claims 1 to 15.

19. the one or more second memories are a plurality of second memories; 19. An accelerator according to any one of claims 1 to 18.

20. the hierarchical memory includes a third memory connected to the plurality of second memories; 20. The accelerator of claim 19.

21. the number of cycles required for data transfer between any one of the plurality of second memories and the third memory is greater than the number of cycles required for data transfer between any one of the plurality of first memories and any one of the plurality of 0th memories; 21. The accelerator of claim 20.

22. A memory device including a means for masking the data transfer between some elements of the plurality of zeroth memories and the plurality of first memories.

22. An accelerator according to any one of claims 1 to 21.

23. the plurality of instructions and the data transfer instruction are written in machine language level; 23. An accelerator according to any one of claims 1 to 22.

24. the data transfer instruction dedicated to data transfer between the plurality of zeroth memories and the plurality of first memories does not cause data transfer between the plurality of first memories and the one or more second memories to be executed; 24. An accelerator according to any one of claims 1 to 23.

25. the data transfer instruction dedicated to data transfer between the plurality of zeroth memories and the plurality of first memories causes the hierarchical memory to execute data transfer only between the plurality of zeroth memories and the plurality of first memories; 25. An accelerator according to any one of claims 1 to 24.

26. the hierarchical memory executes data transfer between the plurality of first memories and the one or more second memories in accordance with a data transfer instruction that is independent of the data transfer instruction dedicated to data transfer between the plurality of zeroth memories and the plurality of first memories and is dedicated to data transfer between the plurality of first memories and the one or more second memories; 26. An accelerator according to any one of claims 1 to 25.

27. Provided in chip form.

27. An accelerator according to any one of claims 1 to 26.

28. Mounted in a multi-chip package.

28. An accelerator according to any one of claims 1 to 27.

29. A data processing method using an accelerator including a plurality of arithmetic units and a hierarchical memory having a 0th hierarchical level, a 1st hierarchical level, and a 2nd hierarchical level in order of proximity to the plurality of arithmetic units, receiving a data transfer command dedicated to transferring data between the 0th hierarchical layer and the 1st hierarchical layer of the hierarchical memory; executes a data transfer between the 0th hierarchical layer and the 1st hierarchical layer of the hierarchical memory in accordance with the data transfer command; receiving an operation instruction for an operation using data stored in the 0th layer of the hierarchical memory; executing, in parallel by the plurality of arithmetic units, an operation using data transferred from the first hierarchical layer of the hierarchical memory in accordance with the data transfer instruction and stored in the zeroth hierarchical layer, in accordance with the operation instruction; Data processing methods.

30. Each of the 0th layer, the 1st layer, and the 2nd layer has a plurality of memories.

30. A data processing method according to claim 29.

31. The hierarchical memory includes a third tier that is farther from the plurality of arithmetic units than the second tier.

31. A data processing method according to claim 29 or 30.

32. 1. A compiler device for generating an instruction sequence to be transferred to an accelerator including a plurality of arithmetic units and a hierarchical memory having a 0th hierarchical layer, a 1st hierarchical layer, and a 2nd hierarchical layer in order of proximity to the plurality of arithmetic units, one or more memories; one or more processors; The one or more processors: generating a data transfer command dedicated to data transfer between the 0th hierarchical level and the 1st hierarchical level of the hierarchical memory; generating an operation instruction that causes the plurality of operation units to execute in parallel an operation using data stored in the 0th layer of the hierarchical memory; generating the instruction sequence including the data transfer instruction and the operation instruction; Compiler device.

33. The one or more processors generate the instruction sequence based on a learning model generated using a deep learning framework.

33. A compiler apparatus according to claim 32.

34. The one or more processors analyze the dependency of instructions based on the learning model, and generate the instruction sequence by ordering the data transfer instructions and the operation instructions that do not conflict so that the data transfer between the 0th layer and the 1st layer and the operation using the data stored in the 0th layer are executed in parallel in the accelerator.

34. A compiler apparatus according to claim 33.

35. The one or more processors: generating a first graph having a relatively high level of abstraction based on the learning model; generating a second graph having a relatively lower level of abstraction based on the first graph; generating the instruction sequence based on the second graph; 35. A compiler apparatus according to claim 33 or 34.

36. The second graph represents dependencies between operations when the computational process represented by the first graph is represented by a combination of operations that can be executed by the accelerator; The operation includes a matrix multiplication operation or an addition operation.

36. A compiler apparatus according to claim 35.

37. The generation of the instruction sequence based on the second graph schedules the execution order of each instruction of the instruction sequence, including the data transfer instruction and the calculation instruction, based on dependencies between operations when the calculation process represented by the first graph represents a combination of operations that the accelerator can execute.

37. A compiler apparatus according to claim 35 or 36.

38. The first graph represents dependencies between computational processes in a combination of computational processes that represent the learning model; The calculation process is performed in units larger than the unit of operations that the accelerator can execute.

38. A compiler apparatus according to any one of claims 35 to 37.

39. Each of the 0th layer, the 1st layer, and the 2nd layer has a plurality of memories.

39. A compiler apparatus according to any one of claims 32 to 38.

40. The hierarchical memory includes a third tier that is farther from the plurality of arithmetic units than the second tier.

40. A compiler apparatus according to any one of claims 32 to 39.

41. An interface for receiving the instruction sequence generated by the compiler device according to any one of claims 32 to 40; the hierarchical memory that transfers data between the 0th hierarchical layer and the 1st hierarchical layer in accordance with the data transfer command included in the command sequence; the plurality of arithmetic units that execute the arithmetic operations in parallel using data stored in the 0th layer of the hierarchical memory in accordance with the arithmetic instructions included in the instruction sequence; An accelerator having:

Images