Arithmetic device, information processing device, and arithmetic device control method

Abstract

To provide an arithmetic device, an information processing device, and an arithmetic device control method that improve reliability while maintaining processing performance.SOLUTION: A memory sequentially receives and stores data to be used for a matrix operation including multiple individual operations. A systolic array has multiple operators arranged in a matrix, and causes each operator to execute a different individual operation in sequence, thereby executing the matrix operation. A control unit calculates, upon receiving a matrix operation processing request, the number of excessive operators for each cycle which is the number of operators that exceeds an upper limit number to be newly activated in the increased number of operators that are caused to execute the respective individual operations for each cycle, pre-activates the operators based on the number of excessive operators on the basis of the start of data reception by the memory, and causes the systolic array to execute the matrix operation based on the data held by the memory when the memory completes the data reception.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a computing device, an information processing device, and a method for controlling a computing device. [Background technology]

[0002] In recent years, AI (Artificial Intelligence) has been attracting attention in various fields. In this environment, AI is required to perform complex processing at high speed, and the performance requirements for hardware that runs AI are increasing significantly. In particular, the current trend in the amount of computational demands of AI is far exceeding previous performance improvements, and is increasing exponentially to the point where the number of computing units is expanding to the limits of heat and power consumption.

[0003] In response to this sudden increase in the amount of required computation, various hardware solutions have been proposed for running AI. One example of dedicated hardware is a computing device that uses a configuration called a systolic array (SA), in which computing units are arranged in tiles. Here, we refer to the complex of computing units configured in a systolic array as simply a "systolic array."

[0004] Systolic arrays take advantage of the fact that the processing performed by each computing unit in matrix operations, etc., is regular, and data flows cyclically, performing the operation when two pieces of data are aligned in the flow. Systolic arrays allow for the concentration of resources, improving the density of operations. However, there are also aspects where the chips that make up systolic arrays have not been sufficiently considered in terms of quality assurance and continuous operation from a product perspective.

[0005] For example, as a technology related to systolic arrays, a technology has been proposed that improves the speed of machine learning by selectively controlling the output destination at each pipeline stage in a pipeline that performs systolic processing. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-177366 Summary of the Invention [Problem to be solved by the invention]

[0007] However, because systolic arrays incorporate an extremely large number of arithmetic units, controlling the operation timing of each arithmetic unit is crucial, and depending on the operation timing, there is a risk of malfunctions occurring. Specifically, when many arithmetic units in a circuit start operating simultaneously, power consumption increases rapidly, resulting in a temporary drop in power supply voltage known as IR drop. If a sudden change in the number of operating arithmetic units causes IR drop, destabilizing the supply current and voltage, noise will be generated. If the specified voltage cannot be guaranteed for all arithmetic units in this way, memory results and calculation results may be corrupted, resulting in errors.

[0008] One way to avoid voltage shortages when the number of active processors suddenly increases is to limit the number of newly activated processors to suppress voltage increases. However, while limiting the number of newly activated processors can avoid voltage shortages when the number of active processors suddenly increases, it is difficult to operate a sufficient number of processors for the operations to be executed, which may result in delays in the execution of the operations. Delays in the execution of the operations can lead to a decrease in the performance of the systolic array as a whole, resulting in problems such as not being able to meet the required performance.

[0009] Furthermore, even with the technology for selecting and controlling the output destination at each pipeline stage of a pipeline that performs systolic processing, the voltage increase caused by the new operation of an arithmetic unit is not taken into consideration, making it difficult to improve the reliability of the systolic array.

[0010] The disclosed technology has been made in view of the above, and aims to provide a computing device, an information processing device, and a method for controlling a computing device that improve reliability while maintaining processing performance. [Means for solving the problem]

[0011] In one aspect of the arithmetic device, information processing device, and control method for the arithmetic device disclosed herein, a memory sequentially receives and stores data used for a matrix operation including a number of individual operations. The systolic array has a plurality of arithmetic units arranged in a matrix, and executes the matrix operation by having each arithmetic unit perform a different individual operation in turn. In response to a processing request for the matrix operation, a control unit calculates, for each cycle, an excess number of arithmetic units, which is the number of arithmetic units that exceed an upper limit number of arithmetic units to be newly started in operation among the increased number of arithmetic units performing individual operations per cycle, pre-operates arithmetic units based on the excess number of arithmetic units based on the start of data reception from the memory, and, when the memory completes receiving data, causes the systolic array to execute the matrix operation based on the data held in the memory. [Effects of the Invention]

[0012] In one aspect, the present invention can improve reliability while maintaining processing performance. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram of a processor equipped with a systolic array. [Figure 2] FIG. 2 is a diagram illustrating the behavior of a systolic array. [Figure 3] FIG. 3 is a diagram for explaining the idling control by the control unit. [Figure 4] FIG. 4 is a diagram showing an example of an outline of the sequence of idling. [Figure 5] FIG. 5 is a diagram showing an example of a specific sequence of idling. [Figure 6] FIG. 6 is a sequence diagram of an operation performed by the processor according to the embodiment. [Figure 7] FIG. 7 is a flowchart of the idle rotation control. [Figure 8]FIG. 8 is a diagram showing a comparison of the time required for the processor according to the embodiment to execute an operation with a processor that does not idle. [Figure 9] FIG. 9 is a diagram illustrating an example of hardware of an information processing device equipped with a processor according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the arithmetic device, information processing device, and arithmetic device control method disclosed herein will be described in detail with reference to the accompanying drawings. Note that the arithmetic device, information processing device, and arithmetic device control method disclosed herein are not limited to the following embodiments. [Example]

[0015] 1 is a block diagram of a processor equipped with a systolic array. As shown in FIG. 1, the processor 1, which is an arithmetic device, has a systolic array 10, a plurality of memories 11, an instruction issuing unit 12, and a control unit 13.

[0016] Figure 2 shows the behavior of a systolic array. A systolic array 10 is equipped with multiple computing units 100. The computing units 100 are arranged in a tiled pattern, with multiple units arranged in a straight line, vertically and horizontally, to form a matrix. The matrix formed by the computing units 100 is called an "operation matrix."

[0017] Data is input from the end of the same side of each column of the operation matrix formed by the calculator 100. Data is also input from the end of the same side of each row of the operation matrix formed by the calculator 100. For example, in FIG. 2, data is input to each column from the top as you face the paper, and data is input to each row from the left as you face the paper.

[0018] Here, in accordance with Figure 2, the side in the column direction of the operation matrix formed by the operator 100 to which data is first input is called the top, and the opposite side is called the bottom, and the side in the row direction of the operation matrix to which data is first input is called the left, and the opposite side is called the right.

[0019] The arithmetic units 100 receive data input from above and from the left. For example, the leftmost arithmetic unit 100 receives data input from the left from a buffer memory. The uppermost arithmetic unit 100 receives data input from above from a buffer memory. In other cases, each arithmetic unit 100 receives data input from the left from the adjacent arithmetic unit 100 on the left. Also, each arithmetic unit 100 receives data input from above from the adjacent arithmetic unit 100 above.

[0020] When the computing unit 100 acquires both data from the left and data from above, it executes a calculation using each data in one cycle. After the calculation is completed, the computing unit 100 outputs the calculation result to the adjacent computing unit 100 on the right and the computing unit 100 below. The computing unit 100 propagates the result to the adjacent computing unit 100 in each cycle. However, the last computing unit 100 in each row and column of the calculation matrix outputs the calculation result of the respective row or column to the outside.

[0021] For example, let us consider the case of a 4×4 operation matrix shown in Fig. 2. The operation units 100 marked with diagonal lines in Fig. 2 are operation units 100 that perform calculations. The operation units 100 without patterns are operation units 100 that do not perform calculations.

[0022] When an operation starts, in the first cycle, the leftmost and uppermost arithmetic unit 100 performs an operation using data input from the left and above. Here, the leftmost and uppermost arithmetic unit 100 is referred to as the first arithmetic unit. Then, as shown in state 101 in FIG. 2, the first arithmetic unit outputs the operation result to the arithmetic unit 100 adjacent to the right and the arithmetic unit 100 adjacent below. Then, in the next cycle, the first arithmetic unit receives input of the next data from above and the next data from the left. Then, the first arithmetic unit performs the next operation using the data input in one cycle. In this cycle, the arithmetic unit 100 adjacent below the first arithmetic unit performs an operation using data input from the buffer memory on the left and data input from the first arithmetic unit. Also, the arithmetic unit 100 adjacent to the right of the first arithmetic unit performs an operation using data input from the first arithmetic unit and data input from the buffer memory above.

[0023] Thereafter, with each cycle, the front of the computing unit 100 that executes the operation propagates diagonally downward to the right, and the behavior of the systolic array 10 transitions to state 102. As the cycle progresses further, the front of the computing unit 100 that executes the operation moves diagonally downward to the right, and the behavior of the systolic array 10 transitions to state 103. If further operation data follows the data that executes the operation indicated by the diagonal pattern, the computing unit 100 indicated by the dotted pattern in state 103 executes the next operation. If no operation data follows the data that executes the operation indicated by the diagonal pattern, the computing unit 100 indicated by the dotted pattern in state 103 does not execute the operation. Here, the operation by the entire operation matrix, which is performed by propagating the operation results of the computing units 100, is an example of a "matrix operation," and the operation performed by each computing unit 100 is an example of an "individual operation."

[0024] Here, power supply to the arithmetic unit 100 and start of operation when performing calculations in a conventional manner, unlike the systolic array 10 according to this embodiment, will be described. In this case, a problem occurs because an error occurs due to the influence of IR drop.

[0025] 2 receives power supply and starts operation at the timing when the state changes from one in which no pattern is added to one in which a diagonal line pattern is added. In this case, the timing when the IR drop due to the start of operation of each computing unit 100 is the largest and most dangerous is the timing of state 102, when the computing units 100 on the diagonal line connecting the right end and top end with the left end and bottom end all start operation at the same time.

[0026] This is also true for other numbers of computing units 100 included in the computation matrix, and the most dangerous timing is when all the computing units 100 on the diagonal of the computation matrix formed by the computing units 100 start operating at the same time. In other words, the number of computing units 100 on the diagonal of the computation matrix is ​​the maximum number of newly operating units.

[0027] For example, if the size of a matrix is ​​expressed as the product of the respective numbers, then for a 32 x 32 operation matrix, a maximum of 32 new operations will be activated. Also, for a 128 x 128 operation matrix, a maximum of 128 new operations will be activated. For a 512 x 512 operation matrix, a maximum of 512 new operations will be activated. And when the maximum number of new operations is activated, there is the greatest risk of an error occurring due to the effects of IR drop.

[0028] Therefore, the systolic array 10 according to this embodiment receives an instruction to start idling the arithmetic unit 100 from the control unit 13. Idle operation is a preliminary operation of the arithmetic unit 200, in which power is supplied to the arithmetic unit 100 in advance to start operation and keep it in an operating state. For example, the processor 1 is provided with a signal line connecting the control unit 13 to the systolic array 10 for transmitting a VLD (Valid) signal, which is a circuit operation valid signal. When the value of the VLD signal sent from the control unit 13 via the signal line is 0, this indicates that the idling instruction has not been received, and when the value of the VLD signal changes to 1, the systolic array 10 determines that the idling instruction has been received.

[0029] When an instruction to idle the arithmetic units 100 is received, the systolic array 10 supplies power to the number of arithmetic units 100 that can be newly issued in each cycle, causing them to start operating sequentially. The number of arithmetic units that can be newly issued is the upper limit number of arithmetic units 100 that can be newly started operating in order to avoid the occurrence of IR drop.

[0030] When the computing unit 100 is idle, it executes a dummy operation after power is supplied. In this case, the computing unit 100 discards the calculated data. By idling the computing unit 100 and operating it in advance, the systolic array 10 does not need to supply power to the computing unit 100 to start it up when it is to be used for a calculation, and a stall can be avoided.

[0031] The systolic array 10 executes an operation when data is input from the memory 11 after the arithmetic unit 100 has been idled. When an operation is to be performed, the arithmetic unit 100 is used to execute the operation in the normal order shown in FIG. 2. In this case, if the arithmetic unit 100 to be used is not idled, the systolic array 10 supplies power to that arithmetic unit 100, starts operation, and then executes the operation. If the arithmetic unit 100 to be used is idled, the systolic array 10 simply causes the arithmetic unit 100 that was previously put into operation to execute the operation.

[0032] In this way, the systolic array 10 has a plurality of computing units 100 arranged in a matrix, and performs a matrix operation by having each computing unit 100 perform a different individual operation in turn. More specifically, the systolic array 10 operates up to an upper limit number of computing units 100 in each cycle, and inputs data stored in the memory 11 or data propagated from an adjacent computing unit 100 to one or more of the operating computing units 100 in each cycle to perform the individual operation. The systolic array 10 then repeatedly propagates the operation results to adjacent computing units 100 to perform the matrix operation. The systolic array 10 also pre-activates the computing units 100 arranged in a matrix, starting with the computing units 100 closest to one diagonal of the matrix. The systolic array 10 also assigns the computing units 100 that will perform the individual operation in each cycle from among the computing units 100 that have been started in each cycle and the pre-activated computing units 100.

[0033] When the calculation using the calculation matrix is ​​completed, the calculator 100 outputs the calculation result to the memory 11. Next, the calculator 100 notifies the control unit 13 of the completion of the calculation.

[0034] The instruction issuing unit 12 receives an execution command for an operation using the systolic array 10, such as a matrix operation, from an external processor, etc. Next, the instruction issuing unit 12 notifies the control unit 13 of the acquired execution command, causing the control unit 13 to start managing each process.

[0035] The control unit 13 receives an input of an execution command for an operation from the command issuing unit 12. Next, the control unit 13 instructs the external memory 2 to transmit data to be used in the specified operation, and causes each memory 11 to transmit the data to be used in the operation.

[0036] Thereafter, the control unit 13 receives from each memory 11 a notification of the reception completion timing calculated when the memory 11 starts receiving data. Here, the reception completion timing is the time until reception of all data predicted by the memory 11 at the start of reception is completed, for example, the number of cycles until reception is complete. If the reception completion timing is N cycles, the control unit 13 receives logN (bits) of information. More specifically, if the packet data length is 8 / 16 / 32 / 64 cycles, for example, the control unit 13 receives the reception completion timing notification by a 3-bit signal including a 1-bit VLD and a 2-bit compressed packet. The reception completion timing information is stored in the packet header or the like.

[0037] Then, the control unit 13 determines the latest reception completion timing of all the reception completion timings of all memories 11 as the reception completion timing for the entire memory 11, at which all the data to be used for executing the calculation is available. The control unit 13 calculates the reception completion time from the reception completion timings for the entire memory 11. Because the memory 11 receives large amounts of data, it takes many cycles from the start of reception to the completion of reception, so the control unit 13 can obtain information about the reception completion timing with ample time to allow for the completion of reception.

[0038] Next, the control unit 13 calculates the elapsed time from when the data to be used for executing the specified operation is collected in each memory 11 until the operation is started by the systolic array 10. Here, the control unit 13 may store information on the elapsed time for each type of operation in advance, or may calculate the elapsed time from the content of the operation.

[0039] FIG. 3 is a diagram for explaining the idle control by the control unit. In FIG. 3, the horizontal axis represents the passage of time in units of one cycle, and the vertical axis represents the number of operating calculators 100. In FIG. 3, the elapsed time is α. The control unit 13 calculates the calculation start time by adding the elapsed time to the reception completion time when all the data used to execute the calculation is available. In FIG. 3, the calculation start time is time T0.

[0040] The control unit 13 calculates the required number of computing units, which is the total number of computing units 100 to be used for executing the specified computation, from the specified computation. Here, the control unit 13 may store information on the required number of computing units 100 to be used for each type of computation in advance, or may calculate the required number of computing units 100 to be used from the content of the computation.

[0041] For example, when the computing unit 100 forming an operation matrix of N rows and N columns is used for the operation, the control unit 13 sets the required number of computing units to N 2 In the systolic array 10, the number of arithmetic units 100 used increases proportionally with each cycle from the start of the operation, and decreases proportionally with each cycle once the diagonal of the matrix is ​​crossed. In FIG. 3, the operation starts at time T0, so the required number of arithmetic units is represented by the area of ​​triangle 201 enclosed by a dashed line in FIG. 3.

[0042] Furthermore, the control unit 13 holds in advance the number of computing units that can be newly issued. In FIG. 3, the number of computing units that can be newly issued is k. The control unit 13 determines whether the maximum number of computing units 100 that have started operation in the computation to be executed is equal to or less than the number of computing units that can be newly issued. Here, the maximum number of computing units 100 that have started operation in the computation to be executed is the number of rows or columns of the computation matrix, which is N in FIG. 3. If the maximum number of computing units 100 that have started operation is equal to or less than the number of computing units that can be newly issued, the number of computing units 100 that have started operation per cycle falls within the number of computing units that can be newly issued, so the control unit 13 determines that idle running is unnecessary and waits until all memories 11 have completed receiving data.

[0043] On the other hand, if the number of operation start times of the largest computing units 100 is greater than the number of computing units that can be newly issued, the control unit 13 executes the following idle control process. Here, when the computing units 100 to be used are operated sequentially, the part of the triangle 201 in FIG. 3 that exceeds the number k of computing units that can be newly issued is the part whose operation is delayed. The timing when the delayed computing units 100 accumulate the most is time T1 in FIG. 3. In this case, if all of the computing units 100 used in the calculation are in time to start operation by time T1, it is possible to avoid the occurrence of computing units 100 whose operation is delayed, in other words, to avoid stalling of the calculations executed by the computing units 100.

[0044] To achieve this, the control unit 13 only needs to start operating the number of computing units 100 that are expected to stall by time T2. That is, it is only necessary that the number of computing units 100 included in the region 202 of the triangle 201 that exceeds the number k of computing units that can be newly issued start operating by time T2. Hereinafter, the number of computing units 100 whose operation will be delayed, that is, the number of computing units 100 that are required to be operated in advance, is referred to as the "total number of requested computing units." The total number of requested computing units is the sum of the number of excess computing units, which is the number of computing units 100 that exceed the number of computing units that can be newly issued among the computing units 100 that are newly used in computing in each cycle from the start to the end of the computation.

[0045] The control unit 13 calculates the total number of requests using the number of requested arithmetic units and the number of newly issuable arithmetic units k. For example, as shown in FIG. 3, the total number of requests is the number of arithmetic units 100 included in the area 202. If the total number of requests is A, the control unit 13 calculates A=N 2 -0.5×k 2 The total number of requests is calculated as follows.

[0046] Then, in order to activate the same number of arithmetic units 100 as the total number requested by time T2, the control unit 13 uses the number of newly issuable arithmetic units k to find an area 203 containing the same number of arithmetic units 100 as the number of arithmetic units 100 included in area 202. Hereinafter, the number of arithmetic units 100 included in area 203, which is the number of arithmetic units 100 that are to be started up in operation in advance, is referred to as the total number of newly issuable arithmetic units. In the case of FIG. 3, if the total number of newly issuable arithmetic units is B, the control unit 13 calculates the total number of newly issuable arithmetic units as B=k(k+x). x is the limit time during which all of the total number of requested arithmetic units 100 can be operated.

[0047] Then, the control unit 13 calculates x when the number of arithmetic units 100 included in the area 202 and the area 203 matches, that is, x when the total number of requests A matches the number of new issuances B. If x cannot be calculated as an integer, the control unit 13 calculates the smallest x when the total number of requests A is exceeded by the number of new issuances B. In this case, the control unit 13 subtracts the time x from the time T0 when the calculation starts to calculate the limit time T4 at which all of the total number of requested arithmetic units 100 can be operated.

[0048] Next, the control unit 13 compares the limit time T4 with the current time. If the current time has passed the limit time T4, the control unit 13 determines that it will not be possible to operate all of the requested total number of arithmetic units 100 in time, and decides to immediately start operating the number k of newly issuable arithmetic units 100. The control unit 13 then instructs the systolic array 10 to start idling the number k of newly issuable arithmetic units 100.

[0049] Furthermore, if the limit time T4 coincides with the current time, the control unit 13 determines to immediately start operation of the k number of newly issuable arithmetic units, and then instructs the systolic array 10 to start idling the arithmetic units 100 of the k number of newly issuable arithmetic units.

[0050] Furthermore, if the limit time T4 is later than the current time, the control unit 13 waits until the limit time T4 at which all of the requested number of arithmetic units 100 can be operated. When the limit time T4 arrives, the control unit 13 instructs the systolic array 10 to start idling the arithmetic units 100, which are the number k of arithmetic units that can be newly issued.

[0051] As described above, the control unit 13 can use a 1-bit VLD signal as a method of instructing the systolic array 10 to start idle operation. For example, when idle operation is not to be performed, the control unit 13 continues to input a VLD signal with a value of 0 to the systolic array 10 using the signal line for idle operation. When instructing the start of idle operation, the control unit 13 inputs a VLD signal with a value of 1 to the systolic array 10 using the signal line for idle operation until the operation is completed.

[0052] Thereafter, the control unit 13 receives a notification of completion of reception from each memory 11. When the control unit 13 receives a notification of completion of reception from all memories 11, it instructs each memory 11 to transfer data. Thereafter, the control unit 13 notifies the systolic array 10 of the start of calculation. When the calculation is completed, the control unit 13 receives a notification of the calculation completion from the systolic array 10. Then, the control unit 13 ends control of the calculation.

[0053] In this way, upon receiving a processing request for a matrix operation, the control unit 13 calculates, for each cycle, the number of excess arithmetic units, which is the number of arithmetic units 100 that exceed the upper limit number of arithmetic units 100 to be newly started in operation among the increased number of arithmetic units 100 that perform individual operations per cycle. Next, the control unit 13 pre-operates the arithmetic units 100 based on the number of excess arithmetic units, based on the start of data reception from the memory 11. Then, when the memory 11 completes receiving data, the control unit 13 causes the systolic array 10 to perform the matrix operation based on the data stored in the memory 11. The control unit 13 also calculates the time required for pre-operation of the excess number of arithmetic units 100 based on the upper limit number of newly issuable arithmetic units, and calculates the timing to start the pre-operation based on the timing when the reception is completed. Furthermore, if the timing to start the pre-operation has already arrived when calculating the timing to start the pre-operation, the control unit 13 starts the pre-operation at that time. Furthermore, if the timing to start the pre-operation has not yet arrived, the control unit 13 waits until the timing for starting the pre-operation is reached before starting the pre-operation.

[0054] A RAM (Random Access Memory) can be used as the memory 11. Each memory 11 receives data used for calculation from the external memory 2. In this way, the memory 11 sequentially receives and stores data used for matrix calculations including a plurality of individual calculations.

[0055] Next, each memory 11 calculates the reception completion timing when it starts receiving data. Then, each memory 11 notifies the calculated reception completion timing to the control unit 13. In this way, the memory 11 predicts the data reception completion timing when it starts receiving data and notifies the control unit 13.

[0056] Thereafter, when each memory 111 has completed receiving all data addressed to itself to be used in the calculation, it notifies the control unit 13 of the completion of reception. Thereafter, when each memory 111 receives a data transfer instruction from the control unit 13, it outputs the data to be used in the calculation to the systolic array 10.

[0057] Next, a specific operation of idle execution of the computing units 100 in the systolic array 10 will be described. If the limit time is before the current time, it will not be possible to operate all of the requested number of computing units 100 in time. Therefore, the systolic array 10 starts operation of the computing units 100 at the timing when the number of operation starts is large. The timing when the number of operation starts is largest is when the computing units 100 on the diagonal line start operation. In other words, the systolic array 10 starts operation from the computing units 100 as close to the diagonal line as possible. Furthermore, even if the current time is after the current time, it is preferable for the systolic array 10 to operate the computing units 100 in the same procedure to ensure safety.

[0058] 4 is a diagram showing an example of the outline of the sequence of idle operations, in which the operation matrix is ​​4×4 and the number of newly issuable operation units is four.

[0059] Here, the arithmetic units 100 start operating when dummy data is input. Therefore, the arithmetic units 100 that can start operating are the arithmetic units 100 connected to the buffer memory or the arithmetic units 100 connected to the arithmetic units 100 that are already operating. For example, as shown in state 211, the systolic array 10 can initially start operating two arithmetic units 100 connected to the diagonal buffer and two arithmetic units 100 connected to the adjacent buffer memory, for a total of four arithmetic units. That is, in state 211, the systolic array 10 starts operating the arithmetic unit 100 numbered #1.

[0060] Furthermore, the systolic array 10 moves the arithmetic unit 100 to be started to the output side in each cycle. That is, the systolic array 10 moves the position of the arithmetic unit 100 to be started to be operated in each cycle in the order of numbers #1 to #4 as shown in state 212. If there are not enough operable arithmetic units 100, the systolic array 10 starts the arithmetic unit 100 that is closest to the diagonal line and connected to the buffer among the arithmetic units 100 that have not started to be started.

[0061] The systolic array 10 operates as many of the arithmetic units 100 as the number of arithmetic units that can be newly issued as possible. However, if the number of operable arithmetic units 100 is less than the number of arithmetic units that can be newly issued due to the propagation status of dummy data or the like, the systolic array 10 operates as many arithmetic units 100 as possible that are less than the number of arithmetic units that can be newly issued.

[0062] 5 is a diagram showing an example of a specific sequence of idle operations. When the systolic array 10 receives an instruction to idle from the control unit 13, it starts the operation of the arithmetic unit 100 numbered #1, as shown in state 221. In the next cycle, the systolic array 10 starts the operation of the arithmetic unit 100 numbered #2, as shown in state 222. In the next cycle, the systolic array 10 starts the operation of the arithmetic unit 100 numbered #3, as shown in state 223.

[0063] By idling the arithmetic units 100 in this way, the systolic array 10 can reduce the occurrence of stalls as much as possible even when it is not possible to operate all of the total number of arithmetic units 100 in time. However, in the systolic array 10 according to this embodiment, when it is not possible to operate all of the total number of arithmetic units 100 in time, stalls occur in the operations performed by the arithmetic units 100 that were not able to be operated in time.

[0064] Also, for safety reasons, the arithmetic units 100 are started in the same order even if the current time is after the current time, but the start-up order may be different as long as it is possible to start them in time.

[0065] Fig. 6 is a sequence diagram of an operation executed by a processor according to an embodiment. Next, the overall flow of an operation executed by the processor 1 according to an embodiment will be described with reference to Fig. 6. Here, the memory 11 is considered as one entity, and an operation to calculate C, C=A×B, is performed. A and B are two pieces of data. Here, the operation of the instruction issuing unit 12 is omitted.

[0066] The control unit 13 acquires an execution command for the operation C=A×B (step S1), and then requests the external memory 2 for the data A and B (step S2).

[0067] Upon receiving the request from the control unit 13, the external memory 2 returns responses for data A and B in order. Specifically, the external memory 2 outputs data A and sends it to the memory 11 (step S3). Upon receiving data A, the memory 11 calculates the reception completion timing and notifies the control unit 13 (step S4).

[0068] When the control unit 13 receives the notification of the reception completion timing, it determines whether or not to execute idle running. If executing idle running, the control unit 13 transmits an idle running execution command to the systolic array 10 to instruct the execution of idle running (step S5). The idle running execution command is sent, for example, as a 1-bit VLD signal via a dedicated signal line. When the systolic array 10 receives the idle running execution command, it executes idle running of the arithmetic units 100 of the number of arithmetic units that can be newly issued per cycle (step S6).

[0069] During this time, the external memory 2 outputs and sends data B to the memory 11 (step S7). When the memory 11 receives both data A and B, it notifies the control unit 13 that reception is complete (step S8).

[0070] When the control unit 13 receives the notification of the completion of reception, it instructs the memory 11 to transfer data (step S9) and also instructs the systolic array 10 to start calculation (step S10).

[0071] The memory 11 transmits the data to the systolic array 10 (step S11). The systolic array 10 executes an operation using the received data (step S12).

[0072] After that, when the operation is completed, the systolic array 10 notifies the control unit 13 of the completion of the operation (step S13).

[0073] 7 is a flowchart of the idling control. Next, the detailed flow of the idling control by the control unit 13 will be described with reference to FIG.

[0074] The control unit 13 receives notification of the reception completion timing calculated at the start of data reception from each memory 11. The control unit 13 determines the latest completion timing of the reception completion timings of all memories 11 as the reception completion timing of all memories 11 (step S11).

[0075] Next, the control unit 13 calculates the elapsed time from when the data to be used for executing the specified operation is collected in each memory 11 until the systolic array 10 starts the operation (step S12).

[0076] Next, the control unit 13 adds the elapsed time to the reception completion timing of the entire memory 11 to calculate the calculation start time (step S13).

[0077] Next, the control unit 13 determines whether the number of operation start times of the largest computing units 100 in the computation to be executed is equal to or less than the number of newly issuable computing units (step S14). If the number of operation start times of the largest computing units 100 is equal to or less than the number of newly issuable computing units (step S14: No), the control unit 13 determines that idle running is unnecessary and ends the idle running control.

[0078] On the other hand, if the number of operation start times of the largest computing units 100 is greater than the number of newly issuable computing units (step S14: Yes), the control unit 13 calculates the required number of computing units, which is the number of all computing units 100 to be used to execute the specified computation (step S15).

[0079] Next, the control unit 13 calculates the total number of requests using the number of requested arithmetic units and the number of newly issuable arithmetic units (step S16).

[0080] Next, the control unit 13 calculates the time until the calculation start time when the total number of newly issuable calculation units, expressed using the number of newly issuable calculation units, matches the total requested number. Then, the control unit 13 subtracts the calculated time from the calculation start time to calculate the limit time until all of the total requested number of calculation units 100 can be operated (step S17).

[0081] Next, the control unit 13 determines whether the limit time is after the current time (step S18). If the limit time is before the current time (step S18: No), the control unit 13 proceeds to step S20.

[0082] On the other hand, if the limit time is later than the current time (step S18: Yes), the control unit 13 waits until the limit time (step S19).

[0083] Thereafter, the control unit 13 instructs the systolic array 10 to start idling the arithmetic units 100 of the number k of newly issuable arithmetic units (step S20).

[0084] Here, the systolic array 10 receives an instruction from the control unit 13 and executes an idle run (step S21).

[0085] 8 is a diagram showing a comparison of the time required to execute an operation between a processor according to the embodiment and a processor that does not idle. Next, the effect of the processor 1 according to the embodiment will be described with reference to FIG. 8. Here, the case where an 8×8 matrix operation is executed using an 8×8 operation matrix will be described. In this case, the number of newly issuable operation units is four.

[0086] Table 301 shows the execution results of operations when there is no limit on the number of newly issueable arithmetic units. In Table 301, the number of newly issueable arithmetic units is listed as a comparison with the number of newly operating units, but there is no limit on the number of newly issueable arithmetic units. In this case, the operation starts from the first cycle and ends in the 15th cycle. However, in reality, the number of newly issueable arithmetic units is exceeded in the 5th to 11th cycles, and an IR drop occurs during this period.

[0087] Therefore, if the same calculation as in Table 301 is performed without idle operation and a limit is placed on the number of newly issueable calculation units, the execution result will be as shown in Table 302. In this case, a calculation stall occurs due to calculation unit 100 not starting operation for cycles 5 to 11. As a result, it takes 22 cycles to complete the calculation.

[0088] In contrast, when the processor 1 according to this embodiment is used, the results are as shown in Table 303. In this case, the systolic array 10 idles the arithmetic units 100 by a portion of the arithmetic units 100 that exceeds the number of newly issueable arithmetic units in the range 312 corresponding to the 9th to 15th cycles, and executes them in the range 311 corresponding to the 1st to 4th cycles. Therefore, in the processor 1 according to this embodiment, the time required to operate the arithmetic units 100 is 19 cycles, but the 4 cycles in the range 311 are hidden, so the time required to execute the operation is 15 cycles. Therefore, by using the processor 1 according to this embodiment, it is possible to complete the operation in the same time as when no limit is placed on the number of newly issueable arithmetic units. In this way, by using the processor 1 according to this embodiment, it is possible to reduce the speed reduction that occurs when a limit is placed on the number of newly issueable arithmetic units without idle running.

[0089] As described above, the processor according to this embodiment executes operations by idling in advance the number of arithmetic units in the systolic array equal to the number of arithmetic units predicted to stall due to the limit on the number of newly issueable arithmetic units established to avoid IR drop. This reduces the occurrence of stalls and suppresses delays in operation time. Therefore, it is possible to suppress the occurrence of stalls while reducing the occurrence of IR drop, thereby maintaining processing performance and improving reliability. In particular, the processor according to this embodiment is effective when targeting floating-point operations that consume a lot of power. Furthermore, the processor according to this embodiment can reduce power consumption by delaying the timing of starting pre-operation until the latest timing at which the operation of the number of arithmetic units predicted to stall can be completed.

[0090] Specifically, the number of arithmetic units in a systolic array continues to increase, from 32 x 32 = 256, to 64 x 64 = 512, to 128 x 128 = 16,384. Meanwhile, the number of arithmetic units required for the latest applications reaches as high as 2,048 x 2,048. This significantly increases the risk of IR drop, and simply limiting the number of units that can be activated at the same time can result in numerous stalls and significant processing delays. The processor according to this embodiment can demonstrate greater effectiveness in executing the latest applications using systolic arrays with such a large number of arithmetic units, thereby maintaining processing performance and improving reliability.

[0091] 9 is a diagram illustrating an example of hardware of an information processing device equipped with a processor according to an embodiment. For example, an information processing device 90 includes a processor 1, a CPU (Central Processing Unit) 91, a memory 92, a hard disk 93, and a network interface 94. The CPU 91 is connected to the processor 1, the memory 92, the hard disk 93, and the network interface 94 via a bus.

[0092] The network interface 94 relays communication between the CPU 91 and external devices. The hard disk 93 is an auxiliary storage device, and stores various programs such as an OS (Operating System) and applications.

[0093] The memory 92 is a type of storage device, such as a RAM (Random Access Memory). The memory 92 realizes the function of the external memory 2 in Fig. 1. This memory 92 is an example of a "main memory."

[0094] The CPU 91 reads out various programs stored on the hard disk 93, expands them in the memory 92, and executes them. When executing the programs, the CPU 91 instructs the processor 1 to perform matrix operations, etc. Then, the CPU 91 acquires the results of operations performed by the systolic array 10 of the processor 1 using data, etc. stored in the memory 92, and executes the programs. This CPU 19 is an example of a "main processing unit."

[0095] Here, the processor 1 may be connected to the CPU 91 using a host interface or the like. [Explanation of symbols]

[0096] 1 processor 2. External memory 10 Systolic Array 11. Memory 12 Command Issuance Department 13 Control Unit

Claims

1. a memory for sequentially receiving and storing data used in a matrix calculation including a plurality of individual calculations; a systolic array in which a plurality of computing units are arranged in a matrix, and the matrix computation is performed by causing each computing unit to perform a different individual computation in turn; a control unit that receives a processing request for the matrix operation, calculates for each cycle an excess number of arithmetic units which is the number of arithmetic units that exceed an upper limit number of arithmetic units to be newly started to operate among an increase in the number of arithmetic units that perform individual arithmetic operations for each cycle, pre-operates arithmetic units based on the excess number of arithmetic units based on a start of data reception from the memory, and causes the systolic array to execute the matrix operation based on the data held by the memory when the memory has completed receiving the data; An arithmetic device comprising:

2. 2. The arithmetic device according to claim 1, wherein the systolic array operates up to the upper limit number of arithmetic units in each cycle, inputs data held in the memory or data propagated from an adjacent arithmetic unit to one or more arithmetic units that have already been operated in each cycle, causes the data to be input to one or more arithmetic units that have already been operated in each cycle, performs individual arithmetic operations, and propagates the arithmetic results to adjacent arithmetic units, thereby executing the matrix arithmetic operations.

3. 2. The arithmetic device according to claim 1, wherein the systolic array pre-activates the arithmetic units arranged in the matrix in order from the arithmetic units closest to one diagonal of the matrix.

4. 2. The arithmetic device according to claim 1, wherein the systolic array assigns an arithmetic unit that executes an individual arithmetic operation for each cycle from among the arithmetic units that have started operation in each cycle and the arithmetic units that have been pre-operated.

5. the memory predicts a timing at which data reception will be completed when reception of the data starts, and notifies the control unit of the predicted timing; The control unit calculates a time required for preliminary operation of the excess number of arithmetic units based on the upper limit number, and calculates a timing to start the preliminary operation based on the timing of completion of reception.

2. The computing device according to claim 1 .

6. 6. The computing device according to claim 5, wherein, when the timing to start the preliminary operation is calculated, if the timing to start the preliminary operation has already arrived, the control unit starts the preliminary operation at that time; and, if the timing to start the preliminary operation has not yet arrived, the control unit waits until a timing is reached via the preliminary operation before starting the preliminary operation.

7. An information processing device having a main processing unit, a main memory, and an arithmetic unit, The computing device a memory for sequentially receiving and storing data used in a matrix operation including a plurality of individual operations from the main memory; a systolic array in which a plurality of computing units are arranged in a matrix, and the matrix computation is performed by causing each computing unit to perform a different individual computation in turn; a control unit that receives a processing request for the matrix operation from the main processing unit, calculates a number of excess processing units for each cycle, which is the number of processing units that exceeds an upper limit number for starting operation among an increase in processing units that perform individual processing for each cycle, pre-operates processing units based on the number of excess processing units based on a start of data reception from the memory, and causes the systolic array to execute the matrix operation based on the data held by the memory when the memory completes receiving the data; An information processing device comprising:

8. A control method for a computing device having a memory that sequentially receives and stores data used for a matrix computation including a plurality of individual computations, and a systolic array in which a plurality of computing units are arranged in a matrix and each computing unit is caused to perform a different individual computation in turn, thereby executing the matrix computation. In response to a request for processing the matrix operation, Calculating the number of excess computing units per cycle, which is the number of computing units that exceed the upper limit number of units that start operation among the increase in the number of computing units that perform individual computations per cycle, Based on the start of receiving data from the memory, the arithmetic units are pre-activated based on the excess number of arithmetic units; When the memory has completed receiving the data, the systolic array is caused to execute the matrix operation based on the data held in the memory. A control method for a computing device, comprising causing the computing device to perform processing.

Images