Processing unit and system

Abstract

This reduces the computation time of the processing unit or system by performing data transfer at a low cost in accordance with the processor architecture. [Solution] An arithmetic processing unit having a plurality of second blocks, each of which includes a plurality of first blocks and at least one second memory, and each of which includes at least one arithmetic unit and at least one first memory, executes an instruction sequence to perform at least one of the following: a first data transfer between first blocks, a second data transfer from a first block to a second block, a third data transfer from a second block to a first block, and a fourth data transfer between second blocks, the instruction sequence including information on the combination of at least one multicast instruction selected from a plurality of multicast instructions and the execution order.

Description

Technical Field

[0001] The present disclosure relates to an arithmetic processing unit and a system.

Background Art

[0002] Generally, deep learning is executed using a processor having a large number of cores such as a GPU (Graphics Processing Unit). Recently, in order to improve the calculation speed in deep learning, a processor (accelerator) specialized for deep learning has been developed. The architecture of a processor specialized for deep learning (the number of arithmetic units, the number of blocks including arithmetic units, the number of block hierarchies, instructions, etc.) may be different from the architecture of general-purpose products such as GPUs. Therefore, in order to efficiently operate a processor specialized for deep learning, an instruction generation device such as a compiler that appropriately generates instructions to be executed by the processor is important.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In the present disclosure, data transfer is executed at a low cost according to the architecture of a processor, and the calculation time of an arithmetic processing unit or a system is shortened.

Means for Solving the Problems

[0004] An instruction generation method according to an embodiment of the present invention is an arithmetic processing device having a plurality of second blocks, each of the plurality of second blocks including a plurality of first blocks and at least one second memory, and each of the plurality of first blocks including at least one arithmetic unit and at least one first memory, wherein by executing an instruction sequence generated by the information processing device, at least one of a parallel first data transfer, a parallel second data transfer, a parallel third data transfer, or a parallel fourth data transfer is performed, the source of the first data transfer is one or more first blocks from the plurality of first blocks, the destination of the first data transfer is one or more first blocks from the plurality of first blocks, and the transfer of the second data transfer The source is one or more first blocks from the plurality of first blocks, the destination of the second data transfer is one or more second blocks from the plurality of second blocks, the source of the third data transfer is one or more second blocks from the plurality of second blocks, the destination of the third data transfer is one or more first blocks from the plurality of first blocks, the source of the fourth data transfer is one or more second blocks from the plurality of second blocks, the destination of the fourth data transfer is one or more second blocks from the plurality of second blocks, and the instruction sequence includes information on the combination and execution order of at least one multicast instruction selected from a plurality of multicast instructions. [Brief explanation of the drawing]

[0005] [Figure 1] A block diagram showing an example of a processing unit in one embodiment of the present invention. [Figure 2] Figure 1 is a block diagram showing an example of a system including a board and host on which the arithmetic processing unit is mounted, and an information processing unit that generates instructions to be executed by the arithmetic processing unit. [Figure 3] This is an explanatory diagram illustrating an example of data transfer classification based on the data transfer path. [Figure 4] Figure 2 is a flowchart illustrating an example of the compiler's operation as implemented by the information processing unit. [Figure 5]This block diagram shows an example of a system including a board and a host in another embodiment, and an information processing device that generates instructions to be executed by a processing unit. [Figure 6] Figure 5 is an explanatory diagram showing an example of a data transfer instruction that can be executed by the arithmetic processing unit. [Figure 7] Figure 6 is an explanatory diagram showing an example of data transfer using multicast instructions. [Figure 8] This is an explanatory diagram illustrating an example of data transfer between eight second-tier blocks BLK2 within a third-tier block BLK3. [Figure 9] Figure 5 is a flowchart illustrating an example of the compiler's operation as implemented by the information processing unit. [Figure 10] Figures 2 and 5 are block diagrams showing examples of the hardware configuration of the host and information processing device. [Modes for carrying out the invention]

[0006] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0007] Figure 1 is a block diagram showing an example of a processing unit in one embodiment of the present invention. For example, the processing unit 100 shown in Figure 1 may function as an accelerator for performing deep learning. The present invention may be applied to a processor such as an accelerator specialized for deep learning, or to other processors that are not specialized for deep learning.

[0008] An example of a processor, the arithmetic processing unit 100, has a plurality of first-level blocks BLK1, a plurality of second-level blocks BLK2 containing a plurality of first-level blocks BLK1, and a plurality of third-level blocks BLK3 containing a plurality of second-level blocks BLK2. That is, the arithmetic processing unit 100 has second-level blocks BLK2 and third-level blocks BLK3 that are layered and contain a predetermined number of first-level blocks BLK1. In the following description, when the first-level blocks BLK1, second-level blocks BLK2, and third-level blocks BLK3 are described without distinction, they will simply be referred to as blocks BLK. The arithmetic processing unit 100 can efficiently perform data transfer such as scatter, gather, broadcast, and reduction between the layered blocks BLK by executing various data transfer instructions. Note that the number of layers is just an example, and the arithmetic processing unit 100 may be composed of four or more layers. In addition, each block BLK in each layer may contain at least one of either memory or an arithmetic unit, and the arithmetic unit may perform matrix operations.

[0009] The arithmetic processing unit 100 may be in the form of a chip or a package such as a CSP (Chip Size Package). The second-tier block BLK2 has memory MEM2, and the third-tier block BLK3 has memory MEM3. The first-tier block BLK1 is an example of the first block, and the second-tier block BLK2 and the third-tier block BLK3 are examples of the second block.

[0010] In the example shown in Figure 1, the arithmetic processing unit 100 has four third-tier blocks BLK3. Each third-tier block BLK3 has eight second-tier blocks BLK2. Each second-tier block BLK2 has sixteen first-tier blocks BLK1. However, the number of third-tier blocks BLK3 mounted in the arithmetic processing unit 100, the number of second-tier blocks BLK2 mounted in each third-tier block BLK3, and the number of first-tier blocks BLK1 mounted in each second-tier block BLK2 are not limited to those shown in Figure 1.

[0011] Each first-level block BLK1 has an arithmetic unit EX2 and multiple arithmetic units OPU. The arithmetic unit OPU has a memory MEM1 (Figure 2) that holds data to be executed by arithmetic units EX2 and EX1 (Figure 2), and supplies data to arithmetic units EX1 and EX2 according to the instructions. For example, arithmetic unit EX1 (Figure 2) may be an integer arithmetic unit. The arithmetic unit OPU is an example of an arithmetic execution unit. For example, each arithmetic unit EX1 and EX2 can execute SIMD (Single Instruction Multiple Data) instructions. Since multiple arithmetic units EX1 and multiple arithmetic units EX2 can each execute instructions in parallel, each arithmetic processing unit 100 or board 200 (Figure 2) can operate as a large SIMD execution machine.

[0012] Furthermore, when the memories MEM1, MEM2, and MEM3 within each block BLK are described without distinction, they are simply referred to as memory MEM. Memory MEM1 within each arithmetic unit OPU mounted on the first-tier block BLK1 (Figure 2) may be described as the memory of the first-tier block BLK1.

[0013] Figure 2 is a block diagram showing an example of a system including a board 200 on which the arithmetic processing unit 100 shown in Figure 1 is mounted, a host 300, and an information processing unit 400 that generates instructions for the arithmetic processing unit 100 to execute. The board 200 shown in Figure 2 has a plurality of interconnected arithmetic processing units 100 and memory MEM4. For example, the board 200 may have the form of a board on which a plurality of arithmetic processing units 100 and memory MEM4 are mounted. Alternatively, the plurality of arithmetic processing units 100 may be mounted in a multi-chip package. In this case, it is preferable that the plurality of arithmetic processing units 100 be arranged side by side on the substrate to improve heat dissipation.

[0014] In Figure 2, the board 200 has four arithmetic processing units 100, but the number of arithmetic processing units 100 is not limited to four; it can be one or more. The memory MEM4 is provided in common to the four arithmetic processing units 100, but it may also be provided corresponding to each arithmetic processing unit 100. A board 200 having multiple arithmetic processing units 100 operates as a single arithmetic processing unit. When the board 200 has multiple arithmetic processing units 100, each arithmetic processing unit 100 or the board 200 may function as the highest-level second block.

[0015] In each arithmetic processing unit 100, the memory MEM3 of each third-tier block BLK3 is connected to memory MEM4 and to the memory MEM2 of each second-tier block BLK2 within the third-tier block BLK3, enabling data transfer between them. Data transfer instructions and arithmetic instructions may also be transferred from memory MEM4 to each memory MEM3, and from each memory MEM3 to each memory MEM2, respectively.

[0016] Each memory MEM2 is connected to a memory MEM1 mounted on each arithmetic unit (OPU) within the second-tier block BLK2, and data can be transferred between them. Data transfer instructions and arithmetic instructions may be transferred from each memory MEM2 to memory MEM1. Each first-tier block BLK1 and each arithmetic unit (OPU) may have registers.

[0017] Memory MEM4 can transfer data to and from host memory HOSTM installed in host 300. Data transfer instructions and arithmetic instructions may be transferred from host memory HOSTM to memory MEM4. In addition to the data transfer path shown in FIG. 2, a transfer path (not shown) for transferring data transfer instructions and arithmetic instructions from memory MEM4 to each memory MEM3, from each memory MEM3 to each memory MEM2, and from each memory MEM2 to each memory MEM1 may be provided.

[0018] Host memory HOSTM and memory MEM4 may transmit and receive information such as data and instructions via, for example, a PCIe (Peripheral Component Interconnect express) interface. Also, the transfer of information between host memory HOSTM and memory MEM4 may be executed by DMA (Direct Memory Access).

[0019] Each arithmetic processing unit 100 on board 200 executes arithmetic processing using the data received from host 300 based on the instructions (data transfer instructions and arithmetic instructions) received from host 300. Various instructions transmitted from host 300 to arithmetic processing unit 100 are generated by information processing device 400, transferred from information processing device 400 to host 300, and stored in host memory HOSTM. Information processing device 400 may be, for example, a server.

[0020] The information processing apparatus 400 functions as a compiler 500 (code generator) that generates an instruction sequence to be executed by the arithmetic processing apparatus 100 through a program executed by a processor such as a built-in CPU (Central Processing Unit). For example, the processor of the information processing apparatus 400 executes an instruction generation method by executing an instruction generation program stored in a memory mounted on the information processing apparatus 400, and generates an instruction sequence. The information processing apparatus 400 is an example of an instruction generation apparatus. The dashed arrow between the information processing apparatus 400 and the host 300 indicates that the instruction sequence generated by the compiler 500 is transferred to the host 300. Note that the transfer of the instruction sequence may be performed via a network.

[0021] For example, the compiler 500 generates an instruction (instruction code) to cause the board 200 to execute deep learning. At this time, for example, the compiler 500 generates an instruction sequence to efficiently execute deep learning on the board 200 based on a learning model generated using a general-purpose library (framework) for deep learning. For example, the compiler 500 combines a sequence of queries instructing the movement of data from a transfer source to a transfer destination into a plurality of queries that can be processed simultaneously, and generates an instruction code or the like indicating one or more data transfer instructions for each of the plurality of combined queries. Thereby, it becomes possible to improve the calculation speed of deep learning by the board 200, and it becomes possible to shorten the calculation time required for deep learning. Although not particularly limited, for example, the instruction code may be a machine language obtained by assembling a description in an assembly language.

[0022] Figure 3 is an explanatory diagram illustrating an example of data transfer classification according to the data transfer path. The data transfer instructions generated by the compiler 500 in Figure 2 include instructions for moving data between adjacent hierarchical blocks BLK. The arrows shown in Figure 3 indicate the data transfer path by the data transfer instructions. For example, a data transfer indicated by a single arrow is realized by one or more data transfer instructions. The black circles shown in Figure 3 indicate that data passes through memory MEM during data transfer. In the plan view of the arithmetic processing unit 100 shown in Figure 3, the numbers shown for one of the 16 first-level blocks BLK1 in the second-level block BLK2 indicate the identifiers of the first-level block BLK1. The same identifiers are assigned to the other 16 first-level blocks BLK1 in the other second-level blocks BLK2 that do not show numbers. Note that the identifiers of the first-level blocks BLK1 may be assigned in mirror symmetry.

[0023] Although not shown in the diagram, identifiers from 0 to 3 are sequentially assigned to the four arithmetic units (OPUs) within each first-tier block BLK1, as shown in Figure 1. Identifiers from 0 to 7 are also sequentially assigned to the eight second-tier blocks BLK2 within each third-tier block BLK3. Identifiers from 0 to 3 are also sequentially assigned to the four third-tier blocks BLK3 within each arithmetic processing unit 100. Furthermore, identifiers from 0 to 3 are also sequentially assigned to the four arithmetic processing units 100 within the board 200.

[0024] The identifiers assigned to each element are merely examples and are not limited to numbers; any identifier that can identify each element is acceptable. For example, an address that can identify the location of each element may be used as the identifier. The address may be a memory address. Alternatively, a register number may be used as the identifier.

[0025] Data transfers executed by data transfer instructions include data transfers between arithmetic units (OPUs) within the first-level block BLK1. Additionally, data transfers executed by data transfer instructions include data transfers between first-level blocks BLK1 within the second-level block BLK2, and data transfers between second-level blocks BLK2 within the third-level block BLK3. Furthermore, data transfers executed by data transfer instructions include data transfers between third-level blocks BLK3 within the arithmetic processing unit 100.

[0026] By combining these data transfers, the data transfers shown in classifications 1 to 4 are realized. Note that the data transfers in classifications 1 to 4 are just examples. For example, if the number of block layers increases, the number of classifications will also increase. Also, in Figure 3, for the sake of clarity, one data transfer is shown for each of classifications 1 to 4, but in reality, each arithmetic processing unit 100 can execute multiple data transfers for each classification. Each of the data transfers in classifications 1 to 4 corresponds to a single query that moves data from memory MEM, to which an arbitrary address is assigned, to memory MEM, to which a different address is assigned, between arithmetic units OPUs or between first-level blocks BLK1. Classification 1 is a data transfer between arithmetic units OPUs (memory MEM1 or registers) within the first-level block BLK1, and the data does not pass through memories MEM2, MEM3, or MEM4.

[0027] Classification 2 is data transfer between first-tier block BLK1s within second-tier block BLK2, with one block BLK tier through which the data passes (second-tier block BLK2). Classification 3 is data transfer between first-tier block BLK1s belonging to different second-tier block BLK2s within third-tier block BLK3, with two block BLK tiers through which the data passes (second-tier block BLK2 and third-tier block BLK3).

[0028] Classification 4 is data transfer between first-tier blocks BLK1 belonging to different third-tier blocks BLK3 within the arithmetic processing unit 100. The number of block BLK layers through which the data passes is three (second-tier block BLK2, third-tier block BLK3, and the arithmetic processing unit 100 (memory MEM4)). Each of the four arithmetic processing units 100 on the board 200 can perform data transfer from classification 1 to classification 4 independently of the other three arithmetic processing units 100.

[0029] For example, the compiler 500 can generate at least one data transfer instruction that commonly executes multiple data transfers where the identifiers of the data source and the identifiers of the data destination are the same. For example, a data transfer instruction may be generated for each of the data transfers from classification 1 to classification 4. By generating a data transfer instruction for each classification, data transfers that follow similar paths can be easily grouped together, and at least one data transfer instruction that commonly executes multiple data transfers can be easily generated. Note that the data transfer instruction may also be generated for data transfers in some of the data transfer paths included in classifications 2 to 4.

[0030] Furthermore, the compiler 500 may add mask information (optional information) to each data transfer instruction to disable the storage of certain data to the destination (memory MEM1, MEM2, MEM3, MEM4, or storage units such as registers). This prevents the writing of data to the destination specified by the mask information. In other words, it is possible to write any data to the destination of any of the multiple data transfers that can be executed with a single data transfer instruction. Note that masking (disabling) data transfers using mask information may also be achieved by masking (disabling) reading from the data source.

[0031] For example, in classification 1, data transfer between pairs of arithmetic units (OPUs) in each of the 512 first-level blocks BLK1 of the arithmetic processing unit 100 can be executed by at least one data transfer instruction. Multiple data transfers included in classification 1 can be executed by at least one common data transfer instruction if the identifiers of the source arithmetic units (OPUs) and the destination arithmetic units (OPUs) are the same. Here, the identity of identifiers in classification 1 may be determined not only by the identifier assigned to the arithmetic unit (OPU), but also by the identifier of a register within the arithmetic unit (OPU) or the identifier of memory MEM1.

[0032] Furthermore, a data transfer instruction that performs a data transfer of category 1 can mask the storage of data in the destination arithmetic unit (OPU) by masking the data transfer instruction. This allows data transfer between any arithmetic units (OPUs) even when the address indicating the data source and the address indicating the data destination are specified in the data transfer instruction. For example, each data transfer of category 1 may be performed by a unicast instruction that transfers data from one source to one destination.

[0033] In Classification 2, data transfer from one First-Level Block BLK1 to another in each of the 32 Second-Level Blocks BLK2 of the arithmetic processing unit 100 can be executed by at least one data transfer instruction. For example, in Classification 2, data transfers where the identifiers of the data sources are the same and the identifiers of the data destinations are the same can be executed by at least one common data transfer instruction (e.g., a multicast instruction). For example, each data transfer in Classification 2 may be executed by a unicast instruction, or by a combination of a unicast instruction and a multicast instruction.

[0034] In classification 2, the identity of the source identifier may be determined if the identifiers of the source arithmetic unit OPU and the first-level block BLK1 are the same. Similarly, in classification 2, the identity of the destination identifier may be determined if the identifiers of the source arithmetic unit OPU and the first-level block BLK1 are the same. Here, in addition to the above, the identity of the identifier in classification 2 may be determined using the identifier of a register in the arithmetic unit OPU, the identifier of memory MEM1, or the identifier of memory MEM2.

[0035] In Classification 3, data transfer between the first-level block BLK1 and the second-level block BLK2 can be performed by at least one data transfer instruction (e.g., a unicast instruction). Also in Classification 3, data transfer between the second-level blocks BLK2 in each of the four third-level blocks BLK3 of the arithmetic processing unit 100 can be performed by at least one data transfer instruction (e.g., a multicast instruction). For example, there are multiple types of multicast instructions depending on the number of source and destination instructions. Unicast instructions and multicast instructions are also explained in Figures 6 and 7. For example, in Classification 3, data transfers where the identifiers of the data source and destination are the same can be performed by at least one common data transfer instruction.

[0036] In classification 3, the identity of the source identifier for data transfer between the first-level block BLK1 and the second-level block BLK2 may be determined if the identifiers of the source arithmetic unit OPU, the first-level block BLK1, and the second-level block BLK2 are the same. Similarly, in classification 3, the identity of the destination identifier for data transfer between the first-level block BLK1 and the second-level block BLK2 may be determined if the identifiers of the destination arithmetic unit OPU, the first-level block BLK1, and the second-level block BLK2 are the same.

[0037] Furthermore, in classification 3, the identity of the source identifier for data transfer between second-level blocks BLK2 may be determined if the identifiers of the second-level blocks BLK2 that are the data source are the same. Similarly, in classification 3, the identity of the destination identifier for data transfer between second-level blocks BLK2 may be determined if the identifiers of the second-level blocks BLK2 that are the data destination are the same. Here, the identity of identifiers in classification 3 may be determined using the identifier of a register in the arithmetic unit OPU, the identifier of memory MEM1, the identifier of memory MEM2, or the identifier of memory MEM3.

[0038] In Classification 4, data transfer between the first-level block BLK1 and the second-level block BLK2 can be performed by at least one data transfer instruction (e.g., a unicast instruction), similar to Classification 3. In Classification 4, data transfer between the second-level block BLK2 and the third-level block BLK3 can be performed by at least one data transfer instruction (e.g., a unicast instruction). Furthermore, in Classification 4, data transfer between different third-level blocks BLK3 within the arithmetic processing unit 100 can be performed by at least one data transfer instruction (e.g., a multicast instruction).

[0039] In classification 4, the identity of the source identifier for data transfer between the first-level block BLK1 and the second-level block BLK2 may be determined, as in classification 3, by whether the identifiers of the source arithmetic unit OPU, the first-level block BLK1, and the second-level block BLK2 are the same. Similarly, in classification 4, the identity of the destination identifier for data transfer between the first-level block BLK1 and the second-level block BLK2 may be determined by whether the identifiers of the destination arithmetic unit OPU, the first-level block BLK1, and the second-level block BLK2 are the same.

[0040] In classification 4, the identity of the source identifier for data transfer between the second-tier block BLK2 and the third-tier block BLK3 may be determined if the identifiers of the second-tier block BLK2 and the third-tier block BLK3 are the same. Similarly, in classification 4, the identity of the destination identifier for data transfer between the second-tier block BLK2 and the third-tier block BLK3 may be determined if the identifiers of the second-tier block BLK2 and the third-tier block BLK3 are the same.

[0041] Furthermore, in classification 4, the identity of the source identifier for data transfer between third-tier blocks BLK3 may be determined if the identifiers of the source third-tier blocks BLK3 are the same. Similarly, in classification 4, the identity of the destination identifier for data transfer between third-tier blocks BLK3 may be determined if the identifiers of the destination third-tier blocks BLK3 are the same. Here, the identity of identifiers in classification 4 may be determined using the identifier of a register in the arithmetic unit OPU, the identifier of memory MEM1, the identifier of memory MEM2, or the identifier of memory MEM3.

[0042] The data transferred in the data transfer from classification 2 to classification 4 is output from the arithmetic processing unit (OPU) and input to other arithmetic processing units (OPUs). Therefore, as explained in classification 1, by masking the storage of data in the destination arithmetic processing unit (OPU), it is possible to disable a portion of the data transfer executed by one data transfer command per classification.

[0043] Note that data transfers may be performed without classification. For example, in the data transfers for classifications 2, 3, and 4 shown in Figure 3, the identifier of the source first-level block BLK1 is "4". Note that the identifier may include the register number within the first-level block BLK1. In this case, the data transfer from the first-level block BLK1 to the second-level block BLK2 (memory MEM2) may be performed with a single data transfer instruction. Also, in the data transfers for classifications 3 and 4 shown in Figure 3, the identifier of the destination first-level block BLK1 is "11". Note that the identifier may include the register number within the first-level block BLK1. In this case, the data transfer from the second-level block BLK2 (memory MEM2) to the first-level block BLK1 may be performed with a single data transfer instruction. Thus, regardless of classification, multiple data transfer instructions with the same source and destination identifiers may be executed with at least one common data transfer instruction.

[0044] Figure 4 is a flowchart showing an example of the operation of the compiler 500 in Figure 2. In other words, the flow shown in Figure 4 shows an example of an instruction generation method by the compiler 500, which is realized when the CPU of the information processing device 400 executes an instruction generation program.

[0045] First, in step S10, the compiler 500 sequentially inputs multiple queries from an external source to the board 200 to perform deep learning. Next, in step S20, the compiler 500 classifies each of the input queries into one of four categories, from category 1 to category 4. Then, in step S30, if the compiler 500 has classified all the input queries, it executes step S40; if there are still unclassified queries remaining, it returns to step S20.

[0046] In step S40, the compiler 500 divides the data transfers that can be executed with one data transfer instruction for each classification into groups. That is, each group corresponds to one data transfer instruction. This makes it possible to generate at least one common data transfer instruction for each group that can transfer multiple data in parallel. In this case, the compiler 500 divides the data transfers between adjacent blocks BLK in the hierarchy shown in Figure 3 into at least one group for each classification. For example, in classification 3, the compiler 500 divides the data transfers between the first-level block BLK1 and the second-level block BLK2, and between the second-level block BLK2 and the third-level block BLK3, into at least one group each.

[0047] Next, in step S50, the compiler 500 generates data transfer instructions for each data transfer group separated in step S40. For example, the data transfer instructions generated by the compiler 500 for each group include either one of several types of unicast instructions that transfer data to a single destination, or one of several types of multicast instructions that transfer data to multiple destinations. By combining the unicast instructions or multicast instructions generated for each group, the compiler 500 can, for example, execute data transfer between adjacent blocks BLKs with the minimum number of data transfer instructions.

[0048] For example, if data transfer between two adjacent block blocks (BLKs) can be performed using multiple types of multicast instructions, the compiler 500 may use dynamic programming to determine the combination and execution order of multicast instructions with a small number of instructions for at least a portion of the data transfer between the block blocks. Here, dynamic programming includes a method of recursively dividing the problem into multiple subproblems and solving them while reusing the computation results of the divided subproblems. Also, if data transfer between two adjacent block blocks (BLKs) can be performed using at least one of multiple types of unicast instructions, the compiler 500 generates a unicast instruction that will be executed after the multiple types of multicast instructions. The arithmetic processing unit 100 executes the data transfer instructions generated by the compiler 500 and sent from the host 300 in the order they were generated by the compiler 500.

[0049] Next, in step S60, if the compiler 500 has generated instructions from all queries, it executes step S70. If there are any queries that have not generated instructions, it returns to step S50. In step S70, the compiler 500 outputs the instructions generated in step S50 in the order they were generated, and then terminates the operation shown in Figure 4.

[0050] In this embodiment, the compiler 500 can generate at least one data transfer instruction for each classification corresponding to the number of layers of block blocks (BLKs) through which the data passes, which executes multiple data transfers in parallel from among the data transfers included in multiple queries. As a result, in the arithmetic processing unit 100, in which block blocks including the arithmetic unit (OPU) are arranged in layers, a large amount of data can be moved between block blocks with fewer data transfer instructions than before. In other words, the compiler 500 can generate data transfer instructions that enable data transfer to be executed at a lower cost than before, in accordance with the architecture of the board 200 and the arithmetic processing unit 100. As a result, the computation time required for deep learning by the arithmetic processing unit 100 or the board 200 can be reduced.

[0051] For example, in the data transfer of classification 3 shown in Figure 3, the arithmetic processing unit 100 can execute multiple data transfers from the first-level block BLK1 to the second-level block BLK2 and multiple data transfers from the second-level block BLK2 to the first-level block BLK1 with the minimum number of instructions. Furthermore, in the data transfer of classification 3, the arithmetic processing unit 100 can execute multiple data transfers between second-level blocks BLK2 via the third-level block BLK3 with the minimum number of instructions.

[0052] On the other hand, for example, in the data transfer of classification 3, if data transfer is performed between two first-tier blocks BLK1 within the third-tier block BLK3 without consolidating queries, each data transfer is executed using four data transfer instructions. These four data transfer instructions transfer data from the first-tier block BLK1 to the second-tier block BLK2, from the second-tier block BLK2 to the third-tier block BLK3, from the third-tier block BLK3 to the second-tier block BLK2, and from the second-tier block BLK2 to the first-tier block BLK1. In this case, four instructions are required for the data transfer of classification 3 in each third-tier block BLK3.

[0053] Each arithmetic processing unit 100 has four third-tier blocks BLK3, and the board 200 has sixteen third-tier blocks BLK3. Therefore, when performing data transfer for classification 3 without grouping queries, each arithmetic processing unit 100 requires 16 instructions, and the board 200, which has four arithmetic processing units 100, requires 64 instructions. In this embodiment, each arithmetic processing unit 100 and the board 200 can execute multiple data transfers in parallel for each data transfer instruction by using at least one common data transfer instruction per group. In this case, each arithmetic processing unit 100 may issue SIMD type data transfer instructions to each of the first-tier blocks BLK1 and second-tier blocks BLK2. In this case, a large number of data transfers can be executed in parallel with fewer instructions compared to issuing individual data transfer instructions to each of the first-tier blocks BLK1 and second-tier blocks BLK2. For example, data transfer for classification 3 can be executed with 4 instructions across the entire board 200. This represents approximately 6% of the total data transfers of category 3 performed by 64 instructions across the entire board 200. Note that some of the multiple data transfers performed by a single data transfer instruction may be masked (invalidated) using mask information.

[0054] Figure 5 is a block diagram showing an example of a system including a board 200 and a host 300 in another embodiment, and an information processing device 400 that generates instructions to be executed by the arithmetic processing device 100. The configuration of the system including the board 200 and the host 300 is the same as in Figure 2. The information processing device 400 functions as a compiler 500A (code generator) that generates a sequence of instructions to be executed by the arithmetic processing device 100 using a program executed by a processor such as a built-in CPU.

[0055] Similar to Figure 2, the compiler 500A generates instructions (instruction code) that cause the board 200 to execute deep learning. In this process, the compiler 500A uses dynamic programming to determine some of the multiple data transfer instructions that the board 200 should execute. For example, the compiler 500A uses dynamic programming to determine the combination and execution order of multicast instructions that minimize the number of instructions for at least some of the data transfer between two adjacent blocks BLK. By using dynamic programming, unnecessary combinations can be eliminated from the combinations of data transfer instructions, which increase exponentially with the increase in the scale of data transfer, making it possible to determine the combination that minimizes the number of data transfer instructions within the given time. Furthermore, by using dynamic programming, it is possible to find combinations of data transfer instructions with fewer instructions, which improves the computation speed of deep learning by the board 200 and reduces the computation time required for deep learning.

[0056] Figure 6 is an explanatory diagram showing an example of a data transfer instruction that can be executed by the arithmetic processing unit 100 in Figure 5. In Figure 6, examples of data transfer instructions are given for a data transfer instruction between a second-level block BLK2 and a first-level block BLK1, a data transfer instruction between two second-level blocks BLK2 via a third-level block BLK3, and a data transfer instruction between a second-level block BLK2 and a third-level block BLK3. Note that the data transfer instructions that can be executed by the arithmetic processing unit 100 are not limited to those shown in Figure 6. For example, a data transfer instruction between a second-level block BLK2 and a first-level block BLK1 is supplied to a second-level block BLK2, and a data transfer instruction between a second-level block BLK2 and a third-level block BLK3 is supplied to a third-level block BLK3.

[0057] Data transfer instructions between the second-tier block BLK2 and the first-tier block BLK1 include unicast instructions. In a unicast instruction, data in memory MEM2 or memory MEM1 is moved within each second-tier block BLK2. In a broadcast instruction, data in memory MEM2 is moved within each second-tier block BLK2 to memory MEM1 in the 16 first-tier blocks BLK1 within the second-tier block BLK2. Note that data movement represents a copy of the data, and the original data remains unless it is overwritten.

[0058] Data transfer instructions between second-tier block BLK2 include three types of multicast instructions. In data transfer instructions between second-tier block BLK2, data is moved via third-tier block BLK3. Data movement by multicast instructions is explained in Figure 4. Data transfer instructions between second-tier block BLK2 and third-tier block BLK3 include unicast instructions. In a unicast instruction moving data from third-tier block BLK3 to second-tier block BLK2, data is moved from memory MEM3 of the third-tier block BLK3 to memory MEM2 of the second-tier block BLK2 in each third-tier block BLK3. In a unicast instruction moving data from second-tier block BLK2 to third-tier block BLK3, data is moved from memory MEM2 of the second-tier block BLK2 to memory MEM3 of the third-tier block BLK3 in each third-tier block BLK3.

[0059] Similar to the embodiments described above, each instruction shown in Figure 6 can move data in parallel across multiple first-level blocks BLK1, multiple second-level blocks BLK2, or multiple third-level blocks BLK3 using a source address, destination address, or relative address. Therefore, the arithmetic processing unit 100 can move a large amount of data in parallel by executing one of the instructions shown in Figure 3.

[0060] Figure 7 is an explanatory diagram illustrating an example of data transfer using the multicast instruction in Figure 6. In Figure 7, an example is shown where each third-tier block BLK3 has eight second-tier blocks BLK2, and each second-tier block BLK2 has sixteen first-tier blocks BLK1. "r" indicates the identifier of the eight second-tier blocks BLK2 within each third-tier block BLK3. "p" indicates the source address of the data transfer in memory MEM2 within the second-tier block BLK2. "q" indicates the destination address of the data transfer in memory MEM2 within the second-tier block BLK2. In Figure 7, for the sake of simplicity, the values ​​of "p" and "q" are assumed to be fixed. Also, Figure 7 shows an example of operation when "r" is "1".

[0061] The multicast instruction "1 to 7" moves the data at address p in memory MEM2 of the r-th second-tier block BLK2 to address q in memory MEM2 of the seven other second-tier blocks BLK2. Because there are eight possible data sources ("r") in the eight second-tier blocks BLK2, the multicast instruction "1 to 7" allows for eight different data transfers.

[0062] In the multicast instruction "2 to 6", data at address p in memory MEM2 of the r-th tier 2 block BLK2 is moved to address q in memory MEM2 of tier 2 blocks BLK2 from 0th to 3rd (excluding rth). Also, data at address p in memory MEM2 of the 4+r-th tier 2 block BLK2 is moved to address q in memory MEM2 of tier 2 blocks BLK2 from 4th to 7th (excluding 4+r). Because there are four possible data sources ("r") for each of the four tier 2 blocks BLK2, the multicast instruction "2 to 6" allows for four different types of data transfer.

[0063] In the multicast instruction "4 to 4", data located at address p in memory MEM2 of the r-th, 2+r-th, 4+r-th, and 6+r-th second-tier block BLK2 is moved to address q in memory MEM2 of the 1-r-th, 3-r-th, 5-r-th, and 7-r-th second-tier block BLK2, respectively. Since there are two possible data sources ("r") for each pair of second-tier block BLK2, the multicast instruction "4 to 4" allows for two types of data transfer. Therefore, with the three types of multicast instructions shown in Figure 7, 14 types (8+4+2) of data transfer are possible. In other words, in this embodiment, the arithmetic processing unit 100 can use 14 types of multicast instructions for data transfer between second-tier block BLK2 and third-tier block BLK3.

[0064] Figure 8 is an explanatory diagram showing an example of data transfer between eight second-tier blocks BLK2 within the third-tier block BLK3. Below, we will explain an example of changing state (A) to state (D), assuming that the number of instructions is minimized by executing multicast instructions "2 to 6" and "4 to 4" in this order, as determined by the single-source shortest path problem, a type of dynamic programming. State (D) is the state in which the data S1-S7 in memory MEM2 of second-tier blocks BLK2(1)-BLK2(7) in state (A) have been moved to memory MEM2 of second-tier blocks BLK2(0)-BLK(6), respectively.

[0065] First, the arithmetic processing unit 100 executes the multicast instruction "2 to 6" in state (A). As a result, the data S2 in the second-tier block BLK2(2) (k=2) is moved to the second-tier blocks BLK2(0), BLK2(1), and BLK2(3), and the data S6 in the second-tier block BLK2(6) (4+k=6) is moved to the second-tier blocks BLK2(4), BLK2(5), and BLK2(7), resulting in state (B). Note that with each multicast instruction, the data in the memory MEM2 of the second-tier block BLK2 is transferred to the memory MEM2 of the other second-tier block BLK2 via memory MEM3.

[0066] Next, the arithmetic processing unit 100 executes the multicast instruction "4 to 4" in state (B). As a result, data S1 in the second-level block BLK2(1) (k=1) is moved to the second-level block BLK2(0), data S3 in the second-level block BLK2(3) is moved to the second-level block BLK2(2), data S5 in the second-level block BLK2(5) is moved to the second-level block BLK2(4), and data S7 in the second-level block BLK2(7) is moved to the second-level block BLK2(6), resulting in state (C).

[0067] Next, the arithmetic processing unit 100 executes a unicast instruction in state (C) after all multicast instructions have been executed. As a result, the data S4 in the second-tier block BLK2(4) is moved to the second-tier block BLK2(3), resulting in state (D), and the desired data transfer is completed. Note that between state (C) and state (D), a unicast instruction is executed to transfer data S4 to memory MEM3 in the third-tier block BLK3, and another unicast instruction is executed to transfer data S4 from memory MEM3 to the second-tier block BLK2(3). Thus, the data transfer shown in Figure 7 can be executed by two multicast instructions and two unicast instructions.

[0068] The following describes an example that generalizes the state change shown in Figure 8. For example, let S0, S1, ..., S7 be the data located at address src in the second-level blocks BLK2(0)-BLK2(7), respectively. Address src is the source address. As described above, by combining multicast and unicast instructions, data D0, D1, ..., D7 are placed at address dst (an address different from src) in the second-level blocks BLK2(0)-BLK2(7) with the minimum number of instructions. Address dst is the destination address. Here, "data D0, ..., D7 ∈ {data S0, ..., S7, Wild}". Wild can be any data and represents data with no specific purpose. For example, "D0, ..., D7 = S1, ..., S7, Wild".

[0069] Next, scheduling using the single-source shortest path problem, a type of dynamic programming, is explained. Unicast instructions are used in the final data transfer because they are used for the final adjustment of data transfers. That is, a predetermined sequence of multiple multicast instructions is executed, followed by a predetermined sequence of unicast instructions. Below, the general idea is explained first, and then the reduction to the single-source shortest path problem is explained.

[0070] The state that changes due to a sequence of multicast instructions is represented by a set (x0, x1, ..., x7), where the state of each set is x0, ..., x7 ∈ {o, x, -}. "xk = o" (where k is any of 0-7) means that the address dst of the k-th second-level block BLK2(k) is updated by the sequence of multicast instructions and Dk is placed there. "xk = x" means that the address dst of the k-th second-level block BLK2(k) is updated by the sequence of multicast instructions and Dk is not placed there. "xk = -" means that the address dst of the k-th second-level block BLK2(k) is not updated by the sequence of multicast instructions.

[0071] In the single-source shortest path problem, the optimal sequence of multicast instructions is basically explored by searching all possible patterns. However, here, compiler 500A determines the instructions in order of how they will be used later. In the actual execution order of instructions, for example, as shown in Figure 7, multicast instructions "2 to 6@2" (k=2) and multicast instructions "4 to 4@1" (k=1) are used in this order. However, when compiler 500A uses dynamic programming, it first considers what state will change due to multicast instruction "4 to 4@1". For example, the state changes from "s0=(-,-,...,-)" to "S1,-,S3,-,S5,-,S7,-", so the state becomes "s1=(o,-,o,-,o,-,o,-)".

[0072] Next, compiler 500A considers inserting the multicast instruction "2 to 6@2" before the multicast instruction "4 to 4@1". This results in "S1,S2,S3,S2,S5,S6,S7,S6", and the state becomes "s2=(o,o,o,x,o,o,o,x)". Thus, regardless of which of the multiple multicast instructions is inserted at the beginning, "o" and "x" remain unchanged, and only "-" changes.

[0073] Next, we consider reducing the problem to a single-source shortest path problem. First, a vertex corresponding to each state is prepared. V(s) represents the vertex corresponding to state s. Let the initial state be "s0=(-,-,...,-)" and consider reducing the problem to a single-source shortest path problem from the initial state. Transitions are made by inserting a multicast instruction at the beginning.

[0074] More precisely, for each vertex V(s) and each multicast instruction m (where m is one of 14 different multicast instructions), an edge with cost 1 is drawn from V(s) to V(s'). Here, "s'" represents the state resulting from inserting multicast instruction m at the beginning of the sequence of multicast instructions that change "s0" to "s", and then executing the multicast instructions in order from m onwards, relative to the initial state. For example, in the above example, when "s=s1" and "m=Multicast 2 to 6@2", then "s'=s2". Also, any sequence of multicast instructions that change "s0" to "s" should result in the same "s'".

[0075] Compiler 500A can obtain the optimal sequence of multicast instructions to bring a given state s to all possible states by solving the single-source shortest path problem from "s0" of the weighted directed graph constructed in this way. After this, for each "s", Compiler 500A obtains the one that minimizes "(number of multicast instructions to get s from s0) + (number of unicast instructions to get s from (o,...,o))". The "number of unicast instructions to get s from (o,...,o)" is equal to "(number of non-o states in s) × 2".

[0076] The computational complexity and optimization are explained below. "o" and "x" can be considered identical. Since the cost of "x" is added at the end, which is twice the number of "x" elements, the edge cost should be increased by +2. Although the number of states appears large, the identification of "o" and "x" shows that there are only 15 states due to the nature of multicast instructions. The number of states is proportional to the number of second-level blocks BLK2.

[0077] Furthermore, excluding self-loops, there are only transitions where "-" decreases, and there are no cycles, so the calculation can be performed linearly. As a result, the computational complexity can be further reduced. If the number of multicast instruction types is proportional to the number of second-tier blocks BLK2, then the time complexity is proportional to "(number of second-tier blocks BLK2)^2".

[0078] Figure 9 is a flowchart showing an example of the operation of the compiler 500A implemented by the information processing device 400 in Figure 5. Specifically, the flow shown in Figure 9 illustrates an example of the instruction generation method by the compiler 500A, which is implemented when the CPU of the information processing device 400 executes an instruction generation program. The same reference numerals are used for operations similar to those in Figure 4, and detailed explanations are omitted.

[0079] The processing in steps S10, S20, S30, S40, S60, and S70 is the same as in Figure 4. After step S40, in step S42, the compiler 500A determines whether the data transfer instruction that performs data transfer between two adjacent block BLKs among the grouped data transfers includes a multicast instruction. If it includes a multicast instruction, the compiler 500A executes step S44; otherwise, it executes step S46.

[0080] In step S44, as described above, compiler 500A uses dynamic programming to determine the combination and execution order of multiple multicast instructions for at least a portion of the data transfer between blocks BLK, and further determines the unicast instructions to be executed after the multicast instructions. After step S44, compiler 500A executes step S46.

[0081] Then, in step S46, for data transfers that do not include multicast instructions, the compiler 500A generates data transfer instructions for each data transfer group separated in step S40. Also in step S46, for data transfers that include multicast instructions, the compiler 500A generates the multicast instructions and unicast instructions determined in step S44. After processing in step 46, the compiler 500A executes steps S60 and S70, and completes the operation shown in Figure 9.

[0082] As described above, in this embodiment as well, the compiler 500A can generate at least one data transfer instruction for each classification according to the number of layers of blocks through which the data passes, which executes multiple data transfers in parallel from among the data transfers included in multiple queries. As a result, in the arithmetic processing unit 100 in which block BLKs including the arithmetic unit OPU are arranged in layers, a large amount of data can be moved between block BLKs with fewer data transfer instructions than in the conventional method. As a result, the computation time required for deep learning by the arithmetic processing unit 100 or the board 200 can be reduced.

[0083] Furthermore, in this embodiment, the compiler 500A uses dynamic programming to determine the combination and execution order of data transfer instructions that perform data transfer between two adjacent blocks BLK in the hierarchy with the fewest number of instructions, and generates data transfer instructions according to the determination. This allows the compiler 500A to terminate the search for instruction sequences that have a large number of instructions, thereby gradually reducing the search space. As a result, the compiler 500A can find an appropriate combination of data transfer instructions with a small number of instructions while minimizing the computational cost.

[0084] Furthermore, in dynamic programming, by searching for multicast instructions to be used in descending order of execution, it is possible to suppress the overwriting of data transferred by earlier multicast instructions by later multicast instructions. This reduces unnecessary data transfer by multicast instructions, and the compiler 500A can generate appropriate combinations of multicast instructions with fewer instructions.

[0085] Furthermore, compiler 500A determines which multicast instructions to use, after it has been determined which unicast instructions to use, as unicast instructions offer greater flexibility in data transfer but lower data transfer efficiency compared to multicast instructions. This allows compiler 500A to minimize the number of unicast instructions used, which have lower data transfer efficiency, and thus minimize the number of instructions used for data transfer between adjacent blocks (BLKs) in the hierarchy.

[0086] In this embodiment, an example was described in which dynamic programming is used to minimize the number of instructions executed for data transfer (classification 3) between first-tier blocks BLK1 belonging to different second-tier blocks BLK2 within the third-tier block BLK3. However, the appropriate instruction sequence to be searched by dynamic programming is not limited to data transfer of classification 3, but may also be an instruction sequence used for data transfer of classification 1, classification 2, or classification 4. Furthermore, the instruction sequence determined by the search is not limited to multicast instructions, but may also be other special instructions other than unicast instructions. Here, a special instruction is, for example, an instruction that transfers data in parallel to multiple locations.

[0087] In the embodiments described above, part or all of the host 300 or information processing device 400 may be composed of hardware, or it may be composed of information processing by software (programs) executed by a CPU or GPU, etc. If it is composed of information processing by software, the software that realizes at least some of the functions of each device in the embodiments described above may be stored on a non-temporary storage medium (non-temporary computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or USB (Universal Serial Bus) memory, and the information processing by the software may be executed by loading it into a computer. Alternatively, the software may be downloaded via a communication network. Furthermore, all or part of the processing by the software may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), so that the information processing by the software is executed by hardware.

[0088] The storage medium for the software may be a removable medium such as an optical disc, or a fixed storage medium such as a hard disk or memory. Furthermore, the storage medium may be located inside the computer (main memory, auxiliary storage, etc.) or outside the computer.

[0089] Figure 10 is a block diagram showing an example of the hardware configuration of the host 300 and information processing device 400 shown in Figures 2 and 5. The information processing device 400 may be implemented as a computer comprising, for example, a processor 20, a main memory 30 (e.g., memory such as DRAM), an auxiliary memory 40 (memory), a network interface 50, and a device interface 60, all connected via a bus 70. For example, the processor 20 executes an instruction generation program to perform the operation described in Figure 4 or Figure 9.

[0090] The information processing device 400 has one of each component, but it may have multiple identical components. Also, although Figure 10 shows one information processing device 400, the software may be installed on multiple information processing devices 400, and each of the multiple information processing devices 400 may execute the same or different parts of the software. In this case, it may be a distributed computing configuration in which each information processing device 400 communicates via a network interface 50 or the like to execute processing. In other words, the information processing device 400 in the above-described embodiment may be configured as a system that realizes its function by having one or more information processing devices 400 execute instructions stored in one or more storage devices. Alternatively, it may be configured so that information transmitted from a terminal is processed by one or more information processing devices 400 located on the cloud, and the processing results are transmitted to the terminal.

[0091] The operations described in the flowchart of Figure 4 and the operations described in the flowchart of Figure 9 may be executed in parallel using one or more processors 20, or using multiple computers via a network. Furthermore, various calculations may be distributed to multiple arithmetic cores within the processor 20 and executed in parallel. Also, some or all of the processing and means of this disclosure may be implemented by at least one of a processor and a storage device located on a cloud that can communicate with the information processing device 400 via a network. Thus, the information processing device 400 in the above-described embodiments may be in the form of parallel computing using one or more computers.

[0092] The processor 20 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that performs either computer control or calculations. The processor 20 may also be a general-purpose processor, a dedicated processing circuit designed to perform specific calculations, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. Furthermore, the processor 20 may include optical circuits or quantum computing-based calculation functions.

[0093] The processor 20 may perform calculations based on data and software input from each of the internal components of the information processing device 400, and may output calculation results and control signals to each of the internal components. The processor 20 may also control each component of the information processing device 400 by executing the OS (Operating System) or applications of the information processing device 400.

[0094] The information processing device 400 in the above-described embodiment may be implemented by one or more processors 20. Here, the processor 20 may refer to one or more electronic circuits arranged on one chip, or one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, each electronic circuit may communicate by wire or wireless.

[0095] The main memory 30 may store instructions executed by the processor 20 and various data, and the information stored in the main memory 30 may be read by the processor 20. The auxiliary memory 40 is a memory other than the main memory 30. These memory devices refer to any electronic component capable of storing electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. In the information processing device 400 in the above-described embodiment, the memory for storing various data may be implemented by the main memory 30 or the auxiliary memory 40, or it may be implemented by the memory built into the processor 20.

[0096] In the above-described embodiment, if the information processing device 400 consists of at least one storage device (memory) and at least one processor connected to (coupled with) this at least one storage device, at least one processor 20 may be connected to one storage device. Also, at least one storage device may be connected to one processor. Furthermore, the configuration may include at least one processor among a plurality of processors being connected to at least one storage device among a plurality of storage devices. This configuration may also be realized by the storage devices and processors included in a plurality of information processing devices 400. Moreover, the configuration may include a storage device integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache).

[0097] The network interface 50 is an interface for connecting to the communication network 600 wirelessly or via a wired connection. The communication network 600 may use an appropriate interface, such as one that conforms to an existing communication standard. Information may be exchanged between the network interface 50 and an external device 710 connected via the communication network 600. The communication network 600 may be a WAN (Wide Area Network), LAN (Local Area Network), PAN (Personal Area Network), or a combination thereof, as long as information is exchanged between the information processing device 400 and the external device 710. An example of a WAN is the Internet, an example of a LAN is IEEE 802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication).

[0098] The device interface 60 is an interface such as USB that connects directly to the external device 720.

[0099] External device 710 is a device connected to the information processing device 400 via a network. External device 720 is a device directly connected to the information processing device 400.

[0100] External device 710 or external device 720 may, for example, be an input device. The input device may be a camera, microphone, motion capture device, various sensors, keyboard, mouse, touch panel, etc., and will provide the acquired information to the information processing device 400. Alternatively, external device 710 or external device 720 may be a device equipped with an input unit, memory, and processor, such as a personal computer, tablet terminal, or smartphone.

[0101] Furthermore, the external device 710 or external device 720 may, for example, be an output device. The output device may be a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or it may be a speaker that outputs sound, etc. It may also be a device equipped with an output unit, memory, and a processor, such as a personal computer, tablet terminal, or smartphone.

[0102] Furthermore, the external device 710 or external device 720 may be a storage device (memory). For example, the external device 710 may be network storage, and the external device 720 may be storage such as an HDD.

[0103] Furthermore, the external device 710 or external device 720 may be a device having some of the functions of the components of the information processing device 400 in the embodiment described above. In other words, the information processing device 400 may transmit some or all of the processing results to the external device 710 or external device 720, or may receive some or all of the processing results from the external device 710 or external device 720.

[0104] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. Furthermore, any element may have multiple instances, such as aa, abb, aabbcc, etc. In addition, it is also possible to add other elements other than the enumerated elements (a, b, and c), such as abcd which has d.

[0105] In this specification (including the claims), when expressions such as "using data as input / based on data / according to / in accordance with data" (including similar expressions) are used, unless otherwise specified, this includes using the data itself or using data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of the data, etc.). Furthermore, when it is stated that some result is obtained "using data as input / based on data / according to / in accordance with data" (including similar expressions), unless otherwise specified, this includes cases where the result is obtained based solely on the data in question or where the result is influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes cases where the data itself is used as output or where data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of various types of data, etc.) is used as output.

[0106] In this specification (including the claims), the terms “connected” and “coupled” are intended to be non-restrictive terms that include any direct connection / coupling, indirect connection / coupling, electrical connection / coupling, communicative connection / coupling, operational connection / coupling, physical connection / coupling, etc. The terms should be interpreted as appropriate in the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted non-restrictively as being included in the terms.

[0107] In this specification (including the claims), when the expression "A configured to B" is used, it may include that the physical structure of element A has a configuration capable of performing operation B, and that the permanent or temporary setting / configuration of element A is configured to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and that it is configured to actually perform operation B by the setting of a permanent or temporary program (instruction). Furthermore, if element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0108] Wherever terms meaning "comprising" or "possessing" (e.g., "comprising / including," "having," etc.) are used herein, they are intended to be open-ended terms, including cases where the subject matter of such terms is not the object of the term. Where the object of such terms meaning "comprising" or "possessing" is an expression that does not specify a quantity or suggests a singular number (an expression with the article "a" or "an"), such expression should be interpreted as not being limited to a specific number.

[0109] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in some places, and expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) should not necessarily be interpreted as not being limited to a specific number.

[0110] In this specification, if a particular configuration of an embodiment is described as having a specific advantage or result, it should be understood, unless otherwise stated, that the same advantage or result can also be obtained from one or more other embodiments having that configuration. However, it should be understood that the presence or absence of such advantage or result generally depends on various factors, conditions, and / or states, and that the configuration does not necessarily guarantee that the advantage or result can be obtained. The advantage or result can only be obtained from the configuration described in the embodiment when various factors, conditions, and / or states are met, and the advantage or result cannot necessarily be obtained in the invention claimed to define that configuration or a similar configuration.

[0111] In this specification (including the claims), when terms such as "maximize" are used, they include finding the global maximum value, finding an approximation of the global maximum value, finding the local maximum value, and finding an approximation of the local maximum value, and should be interpreted appropriately depending on the context in which the term is used. They also include finding these approximations of maximum values ​​probabilistically or heuristically. Similarly, when terms such as "minimize" are used, they include finding the global minimum value, finding an approximation of the global minimum value, finding the local minimum value, and finding an approximation of the local minimum value, and should be interpreted appropriately depending on the context in which the term is used. They also include finding these approximations of minimum values ​​probabilistically or heuristically. Similarly, when terms such as "optimize" are used, they include finding the global optimal value, finding an approximation of the global optimal value, finding the local optimal value, and finding an approximation of the local optimal value, and should be interpreted appropriately depending on the context in which the term is used. This also includes finding approximate values ​​of these optimal values ​​probabilistically or heuristically.

[0112] In this specification (including the claims), when multiple hardware components perform a predetermined process, each component may cooperate to perform the predetermined process, or some components may perform all of the predetermined process. Alternatively, some components may perform part of the predetermined process, while other components perform the remainder. In this specification (including the claims), when expressions such as "one or more hardware components perform a first process, and the one or more hardware components perform a second process" (including similar expressions) are used, the hardware component performing the first process and the hardware component performing the second process may be the same or different. In other words, it is sufficient that the hardware component performing the first process and the hardware component performing the second process are included in the one or more hardware components. Hardware may include electronic circuits, devices containing electronic circuits, etc.

[0113] In this specification (including the claims), when multiple memory devices store data, each of the multiple memory devices may store only a portion of the data or the entire data. Furthermore, a configuration in which some of the multiple memory devices store data is also included.

[0114] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, and partial deletions are possible, provided that they do not depart from the conceptual idea and spirit of the present invention derived from the claims and their equivalents. For example, where numerical values ​​or mathematical formulas are used in the description of the embodiments described above, these are provided for illustrative purposes only and do not limit the scope of this disclosure. Similarly, the sequence of operations shown in the embodiments is also illustrative and does not limit the scope of this disclosure. [Explanation of Symbols]

[0115] 20 processors 30 Main memory 40 Auxiliary storage 50 Network Interfaces 60 device interfaces 70 bus 100 Arithmetic Processing Unit 200 boards 300 hosts 400 Information Processing Devices 500, 500A compilers 600 Communication Networks 710, 720 External device BLK1 1st Layer Block BLK2 Second Layer Block BLK3 Third Layer Block EX1, EX2 computing unit HOSTM Host Memory MEM1, MEM2, MEM3, MEM4 memory OPU (Operating Unit)

Claims

1. An arithmetic processing device having a plurality of second blocks, each of the plurality of second blocks including a plurality of first blocks and at least one second memory, and each of the plurality of first blocks including at least one arithmetic unit and at least one first memory, By executing the instruction sequence generated by the information processing device, at least one of the following is performed: a parallel first data transfer, a parallel second data transfer, a parallel third data transfer, or a parallel fourth data transfer. The source of the first data transfer is one or more first blocks from the plurality of first blocks, and the destination of the first data transfer is one or more first blocks from the plurality of first blocks. The source of the second data transfer is one or more first blocks from the plurality of first blocks, and the destination of the second data transfer is one or more second blocks from the plurality of second blocks. The source of the third data transfer is one or more second blocks from the plurality of second blocks, and the destination of the third data transfer is one or more first blocks from the plurality of first blocks. The source of the fourth data transfer is one or more second blocks from the plurality of second blocks, and the destination of the fourth data transfer is one or more second blocks from the plurality of second blocks. The instruction sequence includes information about the combination of at least one multicast instruction selected from multiple types of multicast instructions and the execution order. Processing unit.

2. The source and destination are identified by at least one of the following: the address of the first memory, the address of the second memory, the register identifier, the identifier of the arithmetic unit contained in each of the plurality of first blocks, the identifier of the first memory, the identifier of the second memory, the identifier of each of the plurality of first blocks, or the identifier of each of the plurality of second blocks. The arithmetic processing device according to claim 1.

3. The source identifiers of the data transfers executed in parallel among the first, second, third, and fourth data transfers are the same, and the destination identifiers of the data transfers executed in parallel among the first, second, third, and fourth data transfers are the same. The arithmetic processing device according to claim 1.

4. A first identifier is assigned to each of the plurality of first blocks contained within each of the plurality of second blocks, and the first identifier is different in each of the plurality of first blocks and common among the plurality of second blocks. Each of the plurality of second blocks is assigned a second identifier, and each of the plurality of second blocks is different from the others. The arithmetic processing device according to claim 1.

5. The aforementioned at least one multicast instruction is a SIMD type data transfer instruction. The arithmetic processing device according to claim 1.

6. SIMD execution device, The arithmetic processing device according to claim 1.

7. It further comprises a plurality of third blocks, each of which includes the plurality of second blocks and at least one third memory. By executing the instruction sequence generated by the information processing device, at least one of the following is performed: a parallel fifth data transfer, a parallel sixth data transfer, or a parallel seventh data transfer. The source of the fifth data transfer is one or more of the plurality of third blocks, and the destination of the fifth data transfer is one or more of the plurality of third blocks. The source of the sixth data transfer is one or more second blocks from the plurality of second blocks, and the destination of the sixth data transfer is one or more third blocks from the plurality of third blocks. The source of the seventh data transfer is one or more third blocks from the plurality of third blocks, and the destination of the seventh data transfer is one or more second blocks from the plurality of second blocks. The arithmetic processing device according to claim 1.

8. The at least one multicast instruction is an instruction that causes the arithmetic processing unit to execute the same type of data transfer in parallel in each of the plurality of third blocks. The arithmetic processing unit according to claim 7.

9. The third block further comprises the plurality of second blocks and at least one third memory, The instruction sequence includes at least one of the first multicast instruction or the second multicast instruction, By executing the first multicast instruction, data transfer is performed in parallel from one of the plurality of second blocks in the third block to the other plurality of second blocks in the third block. By executing the second multicast instruction, data transfers are performed in parallel from at least two second blocks included in the plurality of second blocks of the third block to other plurality of second blocks included in the plurality of second blocks of the third block. The arithmetic processing device according to claim 1.

10. The first data transfer includes data transfer from one or more of the plurality of first blocks to one or more of the plurality of first blocks via the second memory. The arithmetic processing device according to claim 1.

11. A processing unit according to claim 1 and an information processing unit according to claim 1, system.

12. The aforementioned information processing device Based on dynamic programming, select at least one multicast instruction, The instruction sequence is generated using the selected at least one multicast instruction. The system according to claim 11.

13. The aforementioned information processing device Based on dynamic programming, the above combinations and execution order are determined, Based on the determined combination and execution order, the instruction sequence is generated. The system according to claim 11.

14. The aforementioned information processing device After determining the aforementioned combination and execution order, another combination and execution order of data transfer instructions is determined. Based on the determined combination and execution order of the other data transfer instructions, the instruction sequence is generated. The system according to claim 11.

15. The aforementioned information processing device Classify data transfers based on the data transfer path. Based on the results of the above classification, the instruction sequence is generated. The system according to claim 11.

16. The aforementioned information processing device Information is generated to invalidate at least a portion of multiple data transfers included in at least one of the first data transfer, the second data transfer, the third data transfer, or the fourth data transfer. A sequence of instructions is generated that includes the generated information. The system according to claim 11.

17. An arithmetic processing unit having a plurality of second blocks, each of which includes a plurality of first blocks, By executing the instruction sequence generated by the information processing device, at least one of the following is performed: data transfer between two blocks in adjacent hierarchies or data transfer between two blocks in the same hierarchical level. The instruction sequence includes information on the combination and execution order of data transfer instructions that utilize at least one multicast instruction, determined based on dynamic programming. Processing unit.

18. By executing the aforementioned sequence of instructions, at least one of the following is performed: parallel data transfer between the two blocks in adjacent hierarchies or parallel data transfer between the two blocks in the same hierarchical level. The arithmetic processing unit according to claim 17.

19. The aforementioned one or more multicast instructions are SIMD type data transfer instructions. The arithmetic processing unit according to claim 17.

20. SIMD execution device, The arithmetic processing unit according to claim 17.

21. A processing unit according to claim 17 and an information processing unit according to claim 17, system.

22. The aforementioned information processing device Based on dynamic programming, the combination and execution order of the data transfer instructions are determined. Based on the determined combination and execution order, the instruction sequence is generated. The system according to claim 21.

23. The aforementioned information processing device Determine the combination and execution order of the data transfer instructions that utilize at least one type of unicast instruction, Based on the determined combination and execution order, the instruction sequence is generated. The system according to claim 21.

24. The aforementioned information processing device Based on dynamic programming, the multicast instruction to be used is searched for from the last of the execution order among the one or more multicast instructions, Based on the search results, the instruction sequence is generated. The system according to claim 21.

Images