Method and electronic apparatus for executing deep learning calculations

Abstract

To provide a method and electronic device for performing deep learning operations.SOLUTION: An electronic device for performing deep learning operations, according to an embodiment, comprises a processor configured to simultaneously perform multiple deep learning operations. The processor comprises a systolic array having a plurality of processing elements, and a first on-chip network for data propagation between the processing elements.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The following embodiments relate to a method for executing deep learning operations and an electronic device.

Background Art

[0002] An apparatus for processing an Artificial Neural Network requires a large amount of operations on complex input data. Therefore, in order to analyze a large amount of input data in real time using an artificial neural network and extract desired information, a technology capable of efficiently processing operations related to the artificial neural network is required.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present invention is to provide a method and an apparatus for executing deep learning operations.

Means for Solving the Problems

[0004] An electronic device according to an embodiment includes a processor that simultaneously performs a plurality of tasks using a systolic array, the processor including the systolic array including a plurality of Processing Elements (PEs), and a first on-chip network that performs data propagation between the processing elements, and the plurality of tasks include one or more deep learning operations.

[0005] The processor can distribute the processing elements so as to simultaneously perform deep learning operations of the plurality of artificial neural networks based on characteristics of the plurality of artificial neural networks, and the distribution can be performed based on the characteristics of the plurality of artificial neural networks.

[0006] The processor can distribute the plurality of processing elements included in the systolic array.

[0007] The processor can set the propagation direction of each input data and the propagation direction of the partial sum based on the characteristics of multiple artificial neural networks.

[0008] The processor can separate one artificial neural network into multiple sub-artificial neural networks and distribute the processing elements so that deep learning calculations can be performed simultaneously on the sub-artificial neural networks.

[0009] The processor can set the propagation direction of the input data and the partial sum based on the characteristics of the sub-artificial neural network.

[0010] The processor may further include an input data transmission module for inputting data to other sides of the systolic array.

[0011] Other sides of the systolic array face the left and right sides of the systolic array, and the input data transmission module may include a first systolic data setup module that adjusts the timing of inputting the first input data to the left side of the systolic array and transmits the first input data to the left side of the systolic array, a second systolic data setup module that adjusts the timing of inputting the second input data to the right side of the systolic array, and a second on-chip network that transmits the second input data to the right side of the systolic array.

[0012] Other sides of the systolic array face the left and right sides of the systolic array, the first input data is input via a first on-chip network, the second input data is input via a second on-chip network, and the processor further includes an input data transmission module for inputting weight input data to the upper and lower stages of the systolic array, the input data transmission module may include a weight buffer that coordinates the timing of inputting the first and second weight input data to the systolic array and transmits the first weight input data to each first processing element via the upper stage of the systolic array, and a third on-chip network that transmits the second weight input data to each second processing element of the processing elements via the lower stage of the systolic array.

[0013] The processor may further include a second input data transmission module for inputting input data to the upper and lower stages of the processing element.

[0014] The input data transmission module may include a weight buffer that adjusts the timing of inputting the first weight input data to the first processing element among the processing elements and transmits the first weight input data to the upper stage of the first processing element, and another on-chip network that transmits the second weight input data to the lower stage of the second processing element among the processing elements.

[0015] The weight buffer can adjust the timing of inputting the second weight input data to the second processing element.

[0016] The processor may further include an output data receiving module for receiving output data, which is the result of a calculation between first input data and second input data, from the upper and lower edges of the systolic array.

[0017] The output data receiving module includes an output result accumulation register and another on-chip network that transmits a partial sum propagated to the upper edge of the systolic array to the lower stage of the output result accumulation register, the partial sum propagated to the lower edge of the systolic array can be transmitted to the upper stage of the output result accumulation register.

[0018] A deep learning operation execution method according to one embodiment includes the steps of: determining whether or not there is a first artificial neural network currently running; if there is a first artificial neural network currently running, distributing processing elements based on the characteristics of the first artificial neural network and the characteristics of a newly executed second artificial neural network so that the deep learning operations of the first artificial neural network and the deep learning operations of the second artificial neural network can be performed simultaneously; setting the propagation direction of input data and partial sums based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network; and simultaneously performing the deep learning operations of the first artificial neural network and the deep learning operations of the second artificial neural network using the distributed processing elements.

[0019] The step of distributing the processing elements may include determining the distribution method and distribution ratio of the processing elements based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network.

[0020] The step of distributing the processing elements may include, based on the distribution method and the distribution ratio, the step of preempting the deep learning operations of the first artificial neural network; the step of allocating the processing elements reserved via the preemption to perform the deep learning operations of the second artificial neural network; and the step of allocating further processing elements from among the processing elements reserved via the preemption to perform the deep learning operations of the first artificial neural network.

[0021] The processing element may be a processing element of the systolic array.

[0022] A deep learning calculation execution method according to one embodiment may further include the steps of: determining whether the second artificial neural network has multiple batches if there is no first artificial neural network currently being executed; separating the second artificial neural network into multiple sub-artificial neural networks if there are multiple batches; distributing processing elements based on the characteristics of the sub-artificial neural networks so that deep learning calculations can be performed simultaneously on the sub-artificial neural networks; setting the propagation direction of input data and partial sums based on the characteristics of the sub-artificial neural networks; and simultaneously performing deep learning calculations on the sub-artificial neural networks using the distributed processing elements.

[0023] The step of distributing the processing elements may include determining the distribution method and distribution ratio of the processing elements based on the characteristics of the sub-artificial neural network.

[0024] A deep learning calculation execution method according to one embodiment may include the steps of: dividing the second artificial neural network into a plurality of sub-artificial neural networks according to each batch of the second artificial neural network; distributing the processing elements to simultaneously perform deep learning calculations on the sub-artificial neural networks according to the characteristics of the sub-artificial neural networks; setting the propagation direction of input data and partial sums based on the characteristics of the sub-artificial neural networks; and simultaneously performing deep learning calculations on the first artificial neural network and the sub-artificial neural networks using the distributed processing elements.

[0025] An electronic device for performing deep learning operations according to an embodiment includes a processor, the processor includes a systolic array including a plurality of processing elements and a first on-chip network for data propagation between the plurality of processing elements, and the processor separates an artificial neural network into a plurality of sub-artificial neural networks and distributes the processing elements so as to be able to simultaneously perform deep learning operations of the sub-artificial neural networks.

[0026] The separation of the artificial neural network into the sub-artificial neural networks can be performed according to tasks of respective different layers of the artificial neural network.

[0027] The separation of the artificial neural network into the sub-artificial neural networks can be performed according to different batches of the artificial neural network.

[0028] The processor can set propagation directions of input data and partial sums with respect to the processing elements based on characteristics of the sub-artificial neural networks.

[0029] The processor can determine a distribution method and a distribution ratio of the processing elements based on characteristics of the sub-artificial neural networks and distribute the processing elements.

[0030] The processor can perform deep learning operations of other artificial neural networks using other processing elements among the plurality of processing elements simultaneously with deep learning operations of the sub-artificial neural networks executed using the processing elements.

Advantages of the Invention

[0031] [[ID= 26]] According to the present invention, a method and an apparatus for executing deep learning operations can be provided.

Brief Description of the Drawings

[0032] [Figure 1A]This is a diagram illustrating a deep learning computation method using an artificial neural network. [Figure 1B] This diagram illustrates the data and filters of the input feature maps provided as input in deep learning operations. [Figure 1C] This diagram illustrates the process of performing convolution operations on a deep learning platform. [Figure 1D] This is a diagram illustrating how to perform convolution operations using a systolic array. [Figure 2A] This diagram illustrates how to implement temporal multitasking on a systolic array, taking into account the prioritization of multiple artificial neural networks. [Figure 2B] This figure illustrates the operation of a deep learning computing device that supports spatial multitasking according to one embodiment. [Figure 3A] This is a diagram illustrating a spatial multitasking operation method according to one embodiment. [Figure 3B] This is a diagram illustrating a spatial multitasking operation method according to another embodiment. [Figure 4] This figure shows an example of hardware implementation for a deep learning computing device that performs multiple deep learning operations simultaneously according to one embodiment. [Figure 5A] This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 5B] This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 5C] This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 5D] This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 5E] This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 5F]This figure illustrates a specific calculation execution process of a deep learning computing device according to one embodiment. [Figure 6] This is a flowchart illustrating how to perform deep learning operations via spatial multitasking. [Figure 7] This figure illustrates a method for utilizing a neural processing unit for spatial multitasking according to one embodiment. [Modes for carrying out the invention]

[0033] The specific structural or functional descriptions disclosed herein are illustrative for the purpose of illustrating embodiments, and embodiments can be carried out in various different forms. The present invention is not limited to the embodiments described herein, and the scope of the present invention includes modifications, equivalents, or substitutions that are included in the technical ideas described in the embodiments.

[0034] Terms such as "first" or "second" may be used to describe multiple components, but such terms should be interpreted solely for the purpose of distinguishing one component from others. For example, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0035] When it is mentioned that one component is “linked” or “connected” to another component, it should be understood that it is directly linked to or connected to the other component, but that other components may be present in between.

[0036] A singular expression includes plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “has” indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood not to preemptively exclude the possibility of the presence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which this embodiment belongs. Commonly used, predefined terms should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as ideal or overly formal unless expressly defined herein.

[0038] The embodiments will be described in detail below with reference to the attached drawings. In describing with reference to the attached drawings, the same reference numerals will be used for the same components regardless of the reference numerals in the drawings, and redundant explanations will be omitted.

[0039] Figure 1A is a diagram illustrating a deep learning computation method using an artificial neural network.

[0040] An artificial intelligence model performing deep learning operations may be characterized by input data 10 being input to the model and output data 30 being an exemplary output of the model. For example, a model performing deep learning operations may be embodied as a neural network trained via deep learning and generate output data 30 that depends on convolutional operations. Such convolutional operations are referred to as inference operations. The trained neural network may be trained via deep learning for a specific purpose, such as face recognition, based on feature extraction from the neural network.

[0041] Artificial intelligence (AI) algorithms, including deep learning, are characterized by inputting input data 10 into an artificial neural network (ANN) and learning output data 30 through operations such as convolution. Within the ANN, nodes corresponding to neurons in the brain are connected to each other and work collectively to process the input data. Various types of neural networks are examples, such as convolutional neural networks (CNN), recurrent neural networks (RNN), deep belief networks (DBN), and restricted Boltzmann machines (RBM), but AI is not limited to these. In a feed-forward neural network, neurons in the neural network have connections (links) with other neurons. Such connections may be extended in one direction, for example, forward, through the neural network.

[0042] For example, referring to Figure 1A, it shows a structure in which input data 10 is input to an artificial neural network, and output data 30 is output via an artificial neural network (e.g., a convolutional neural network) 20 containing one or more layers. The artificial neural network may also be a deep neural network with two or more layers. Furthermore, the reference to CNN20 shown in Figure 1A is a reference to one or more processors and / or deep learning computing units configured with CNN20.

[0043] The convolutional neural network 20 may be used to extract "features" such as frames, lines, and colors from the input data 10. The convolutional neural network 20 may include multiple layers (for example, multiple convolutional layers). Each layer receives data, processes the data input to the layer, and generates data output from that layer. The data output from a layer may be the image input to the convolutional neural network 20, or a feature map generated by convolving the input feature map with the weight values ​​of one or more filters. The initial layer of the convolutional neural network 20 operates to extract low-level features such as edges or gradients from the input. Subsequent layers of the convolutional neural network 20 can extract increasingly complex features in the image, such as eyes and noses.

[0044] Figure 1B is a diagram illustrating the data and filters of the input feature map provided as input in a deep learning operation.

[0045] Referring to Figure 1B, the input feature map 100 is, but is not limited to, a set of pixel values ​​or numerical data of an image input to the artificial neural network. In Figure 1B, the input feature map 100 is defined as the pixel values ​​of an image to be learned via the artificial neural network. For example, the input feature map 100 may have 256 × 256 pixels and a depth of K (e.g., K channels of the input image or output feature maps of previous hierarchies). However, the specific values ​​are illustrative, and the size of the pixels in the input feature map 100 is not limited to these examples.

[0046] Filters 110-1 to 110-n may be formed in N quantities. Each of the filters 110-1 to 110-n may contain n by n (n × n) weight values. For example, each of the filters 110-1 to 110-n may have 3 × 3 pixels and a depth value of K (e.g., K channels). However, the size of the filters is illustrative, and the size of each of the filters 110-1 to 110-n is not limited to the examples. However, as mentioned in the examples above, the depth K of each of the filters 110-1 to 110-n may be the same as the depth K of the input feature map 100.

[0047] Figure 1C is a diagram illustrating the process of performing convolution operations on a deep learning platform.

[0048] Referring to Figure 1C, the process of performing a convolution operation in an artificial neural network involves generating output values ​​by multiplying and adding the input feature map 100 and filter 110 at each layer, and then accumulating and summing these output values ​​to generate the output feature map 120.

[0049] The convolution operation is performed by applying a filter 110 of a fixed size, i.e., n × n size, from the upper left to the lower right of the input feature map 100 in the current layer, and performing multiplication and addition operations. Below, we will explain the process of performing the convolution operation when the size of the filter 110 is 3 × 3.

[0050] For example, first, in the first region 101 in the upper left of the input feature map 100, a 3x3 operation is performed to multiply a total of 9 data points x11 to x33, which include 3 data points in the first direction and 3 data points in the second direction, by the weight values ​​w11 to w33 of the filter 110. Then, by accumulating and summing all the output values ​​of the multiplication operation, i.e., x11×w11, x12×w12, x13×w13, x21×w21, x22×w22, x23×w23, x31×w31, x32×w32, and x33×w33, the 1-1 output data y11 of the output feature map 120 is generated.

[0051] Subsequently, the operation is performed by moving data units from the first region 101 to the second region 102 in the upper left of the input feature map 100. During this convolution operation, the number of data units moved within the input feature map 100 is called the stride, and the size of the generated output feature map 120 is determined according to the size of the stride. For example, if the stride is 1, the total of 9 input data x12 to x34 contained in the second region 102 are multiplied by the weight values ​​w11 to w33 of the filter 110. The output values ​​of the multiplication operation, x12×w11, x13×w12, x14×w13, x22×w21, x23×w22, x24×w23, x32×w31, x33×w32, x34×w33, are all accumulated and summed up to generate the first-to-second output data y12 of the output feature map 120.

[0052] Similarly, the following operations are performed: multiply the nine input data x13~x35 contained in the next region by the weight values ​​w11~w33 of filter 110, accumulate the results to generate Y13, and then multiply the following nine input data x14~x36. The next region is then multiplied by the weights w11~w33 of filter 110, and the results are accumulated to generate Y14. In the illustrated example, the stride is 1, so the application of filter 110 is moved down one row to generate output Y21, and the stride allows the remaining multiplications and accumulations to be performed until all outputs Y11~Y44 are generated.

[0053] When there is one or more additional filters 110, each additional filter 110 applied similarly to the input data may generate corresponding additional output depths or channels in the output feature map 120 for the input data.

[0054] Figure 1D is a diagram illustrating how to perform a convolution operation using a systolic array.

[0055] Referring to Figure 1D, each data point in the input feature map 130 is mapped to a systolic array that is sequentially input to processing elements (PEs) 141-149 by a clock with a constant latency. The processing elements may also be arithmetic units that perform multiplication and addition.

[0056] In the first clock cycle, the first-first data x11 of the first row (circled number (1)) of the systolic array may be input to the first processing element 141. The first-first data x11 is multiplied by the weight value of w11 in the first clock cycle. Then, in the second clock cycle, the first-first data x11 may be input to the second processing element 142, the second-first data x21 may be input to the first processing element 141, and the first-second data x12 may be input to the fourth processing element 144. Similarly, in the third clock cycle, the first-first data x11 may be input to the third processing element 143, the second-first data x21 may be input to the second processing element 142, and the first-second data x12 may be input to the fifth processing element 145. In the third clock cycle, the third-first data x31 may be input to the first processing element 141, the second-second data x22 may be input to the fourth processing element 144, and the first-third data x13 may be input to the seventh processing element 147.

[0057] As described above, the input feature map 130 is input to each processing element within processing elements 141 to 149 in accordance with sequential clocks, and multiplication and addition operations are performed with the input weight values ​​by each clock. An output feature map can be generated by accumulating and summing the values ​​output through the multiplication and addition operations of each data in the sequentially input input feature map 130 and the weight values.

[0058] Figure 2A illustrates how to implement temporal multitasking on a systolic array, taking into account the prioritization of multiple artificial neural networks.

[0059] Referring to Figure 2A, the deep learning computing unit can execute multiple artificial neural networks on a single systolic array using temporal multitasking.

[0060] The deep learning computing unit may execute artificial neural network A 210 in the first time interval t0-t1, perform context switching at time t1 to execute artificial neural network B 220 in the second time interval t1-t2, perform context switching at time t2 to execute artificial neural network A 210 again in the third time interval t2-t3. Executing an artificial neural network means performing deep learning calculations on the artificial neural network.

[0061] However, even with the use of temporal multitasking, deep learning computing systems still cannot execute multiple artificial neural networks simultaneously on a single systolic array. Due to the nature of temporal multitasking, it is not possible to distribute the processing elements of the same systolic array to multiple artificial neural networks. Therefore, deep learning computing systems that support temporal multitasking have the disadvantage of not being able to achieve higher throughput and top-of-the-line processing speeds (TOPS / Watt) compared to executing only one artificial neural network. Furthermore, deep learning computing systems that support temporal multitasking cannot guarantee high real-time performance due to the long context switching time between artificial neural networks.

[0062] Figure 2B is a diagram illustrating the operation of a deep learning computing device that supports spatial multitasking according to one embodiment.

[0063] Referring to Figure 2B, a deep learning computing device according to one embodiment can distribute the processing elements of a systolic array to multiple artificial neural networks via spatial multitasking and execute multiple artificial neural networks simultaneously.

[0064] The deep learning computing unit can execute only artificial neural network A 210 during the first time interval t0 to t1, execute artificial neural network A 210 and artificial neural network B 220 simultaneously during the second time interval t1 to t2, and execute artificial neural network A 210 and artificial neural network C 230 simultaneously during the third time interval t2 to t3.

[0065] Deep learning computing devices can improve the processing capacity of artificial neural networks by simultaneously executing multiple artificial neural networks on a single systolic array, and can ensure the real-time performance of high-priority artificial neural networks.

[0066] Figure 3A is a diagram illustrating a spatial multitasking operation method according to one embodiment.

[0067] Deep learning computing devices that support spatial multitasking can utilize the characteristics of systolic arrays, in which processing elements are arranged two-dimensionally, to distribute processing elements to multiple artificial neural networks in any desired ratio.

[0068] Referring to Figure 3A, when artificial neural networks A and B are executed simultaneously, the input data 310 for artificial neural network A may be input to the left side of the systolic array, and the input data 320 for artificial neural network B may be input to the right side of the systolic array. The input data 310 for artificial neural network A and the input data 320 for artificial neural network B may also be the input feature map data for artificial neural network A and the input feature map data for artificial neural network B, respectively.

[0069] The input data 310,320 provided from both sides of the systolic array may propagate the input data horizontally and the calculation results vertically, depending on the ratio in which processing elements are distributed between artificial neural networks A and B.

[0070] Specifically, the input data 310 of the artificial neural network A propagates from left to right, and multiplication and addition operations are performed with the input weight value by each clock. Output data 315 can be generated by propagating the output values ​​from top to bottom through the multiplication and addition operations of the input data 310 and weight value that are sequentially input, and accumulating and summing them up.

[0071] The input data 320 of the artificial neural network B propagates from right to left, and multiplication and addition operations are performed with the input weight value by each clock. Output data 325 can be generated by propagating the output values ​​from top to bottom through the multiplication and addition operations of the input data 320 and weight value that are sequentially input, and accumulating and summing them up.

[0072] Figure 3B is a diagram illustrating a spatial multitasking operation method according to another embodiment.

[0073] Referring to Figure 3B, when artificial neural networks A and B are executed simultaneously, all of the input data 330 for artificial neural network A and the input data 340 for artificial neural network B are input to the left side of the systolic array, and the input data can be propagated horizontally and the calculation results vertically according to the ratio in which the processing elements are distributed.

[0074] Specifically, the input data 330 of the artificial neural network A is propagated from right to left, and multiplication and addition operations are performed with the input weight value by each clock. Output data 335 can be generated by propagating the output values, which are generated through the multiplication and addition operations of the input data 330 and weight value that are sequentially input, from bottom to top and accumulating them.

[0075] The input data 340 of the artificial neural network B is propagated from left to right, and multiplication and addition operations are performed with the input weight value by each clock. Output data 345 can be generated by propagating the output values, which are generated through the multiplication and addition operations of the input data 340 and weight value that are sequentially input, from top to bottom and accumulating and summing them up.

[0076] To support spatial multitasking as shown in Figures 3A and 3B, a deep learning computing device according to one embodiment may include a processor. The processor can determine the direction (e.g., up / down, left / right) and distribution ratio of processing elements, and, taking this into consideration, provide input data to the systolic array. The processor may be a neural processing unit (NPU).

[0077] A deep learning computing device may require a structure in which each processing element of a systolic array can propagate input data bidirectionally, rather than unidirectionally. To this end, the deep learning computing device may include hardware units and an on-chip network (Network-on-a-Chip, NoC) that can propagate input data horizontally from the left and right sides of the systolic array, and hardware units and an on-chip network that can receive output data from the top and bottom sides of the systolic array. The following describes specific components of a deep learning computing device that performs multiple deep learning operations simultaneously, with reference to Figure 4.

[0078] Figure 4 shows an example of hardware implementation for a deep learning computing device that performs multiple deep learning operations simultaneously according to one embodiment.

[0079] Referring to Figure 4, a deep learning computing device according to one embodiment includes a main memory 410, a global buffer 415, a first systolic data setup module 420, a weight buffer 425, a systolic array 430, and an output result accumulation register 440.

[0080] A deep learning computing device can be a hardware-implemented computing device for performing neural network computations, such as a neural network device, neural network circuit, hardware accelerator, or processing unit. For example, a deep learning computing device may be implemented using various semiconductor devices such as a SoC (System on Chip), ASIC (application specific integrated circuit), CPU (Central Processing Unit), GPU (Graphics Processing Unit), VPU (Vision Processing Unit), and NPU (Neural Processing Unit).

[0081] The systolic array 430 may include a plurality of processing elements arranged along the left-right and up-down directions. The systolic array may perform calculations in accordance with a synchronization signal (e.g., a clock signal). The systolic array may be referred to as a processing element array (PE array).

[0082] The systolic array 430 may sequentially receive first input data and second input data from the first systolic data setup module 420 and buffer 425, respectively, via a clock signal. The first input data is input feature map data, and the second input data is weight values.

[0083] The systolic array 430 performs deep learning operations using input feature map data and weight values. The result of the systolic array 430's operations is a partial sum, which is an intermediate operation result for generating feature-map data. This partial sum propagates in a specific direction and is accumulated in the output result accumulation register 440.

[0084] The first systolic data setup module 420 may store input feature map data (e.g., 100 in Figure 1). The first systolic data setup module 420 can transmit the input feature map data to the left-hand side of the systolic array 430.

[0085] The weight buffer 425 stores the weight values ​​of the filters (for example, 110-1 to 110-n in Figure 1). The weight buffer 425 can transmit the weight values ​​to the upper edge of the systolic array 430.

[0086] In this embodiment, the first systolic data setup module 420 and the wait buffer 425 may be implemented in different memory devices, or in different areas of a single memory device.

[0087] The deep learning computing unit may further include a first on-chip network, a second systolic data setup module 445, second on-chip networks 460 and 460-1 to 460-n, third on-chip networks 450-1 to 450-n, and fourth on-chip networks 455-1 to 455-n to support spatial multitasking.

[0088] The deep learning computing device can perform data propagation in all directions (up, down, left, and right) between processing elements via the first on-chip network. Conventional deep learning computing devices only perform data propagation from top to bottom and from left to right between processing elements, but the deep learning computing device according to one embodiment can also perform data propagation from bottom to top and from right to left between processing elements via the first on-chip network.

[0089] The deep learning computing unit can transmit input feature map data to the right-hand side of the systolic array 430 via the second systolic data setup module 445, the second on-chip network 460, and 460-1 to 460-n. The second systolic data setup module 445 adjusts the timing of inputting the input feature map data to the right-hand side of the systolic array 430, and the second on-chip network 460 and 460-1 to 460-n can transmit the input feature map data to the right-hand side of the systolic array 430.

[0090] The deep learning computing device may transmit weight values ​​to the lower stage of the processing elements constituting the systolic array 430 via the third on-chip network 450-1 to 450-n. Conventional deep learning computing devices transmit weight values ​​only to the upper stage of the processing elements, but the deep learning computing device according to one embodiment can also transmit weight values ​​to the lower stage of the processing elements via the third on-chip network 450-1 to 450-n.

[0091] The deep learning arithmetic unit may modify the output result accumulation register 440 via the fourth on-chip network 455-1 to 455-n. In conventional deep learning arithmetic units, the partial sum is propagated only to the lower edge of the systolic array 430, and the propagated partial sum is transmitted to the upper stage of the output result accumulation register 440 and accumulated. On the other hand, in one embodiment of the deep learning arithmetic unit, the partial sum can also be propagated to the upper edge of the systolic array 430. Here, the deep learning arithmetic unit can transmit the partial sum propagated to the upper edge of the systolic array 430 via the fourth on-chip network 455-1 to 455-n to the lower stage of the output result accumulation register 440.

[0092] The deep learning computing unit generates instructions for controlling the main memory 410, global buffer 415, first systolic data setup module 420, wait buffer 425, systolic array 430, output result accumulation register 440, first on-chip network, second systolic data setup module 445, second on-chip networks 460 and 460-1 to 460-n, third on-chip networks 450-1 to 450-n, and fourth on-chip networks 455-1 to 455-n. For example, the processor can distribute the processing elements and set the propagation direction of input data and partial sums so that deep learning calculations can be performed simultaneously on multiple artificial neural networks based on the characteristics of the multiple artificial neural networks.

[0093] The first input data transmission module is a concept that includes a first systolic data setup module 420 and a second on-chip network 460 and 460-1 to 460-n; the second input data transmission module includes a wait buffer 425 and a third on-chip network 450-1 to 450-n; and the output data reception module is a concept that includes an output result accumulation register 440 and a fourth on-chip network 455-1 to 455-n.

[0094] The reason why the components are shown separately in the embodiment shown in Figure 4 is to explain each function separately. Therefore, when actually realizing the product, it is possible to configure all of these to be processed by the processor, or to configure only some of them to be processed by the processor.

[0095] In this embodiment, the positions of the weight buffer 425 and output result accumulation register 440, and the first systolic data setup module 420 and the second systolic data setup module 445 relative to the systolic array 430 are not limited to those shown in Figure 4. For example, the weight buffer 425 and the output result accumulation register 440 may be located on the left side, right side, left side or below, or above the systolic array 430, respectively, and the first systolic data setup module 420 and the second systolic data setup module 445 may be located above, below, or below, above, right, or left side of the systolic array 430, respectively.

[0096] Figures 5A to 5F are illustrative diagrams showing the specific computation execution process of a deep learning computing device according to one embodiment. The explanations shown in Figures 2B to 4 are also applicable to Figures 5A to 5F, so redundant explanations are omitted.

[0097] Figures 5A to 5B show a deep learning computing device that executes two artificial neural networks simultaneously by horizontally distributing the processing elements of a systolic array, Figures 5C to 5D show a deep learning computing device that executes two artificial neural networks simultaneously by vertically distributing the processing elements of a systolic array, and Figures 5E to 5F show a deep learning computing device that executes four artificial neural networks simultaneously by dividing the processing elements of a systolic array into four parts.

[0098] Referring to Figures 5A and 5B, the deep learning computing unit may later distribute the systolic array into a first region 530 and a second region 535, executing artificial neural network A in the first region 530 and artificial neural network B in the second region 535.

[0099] Referring to Figure 5A, the deep learning computing device may pre-propagate the weight values ​​of artificial neural network A to the first region 530 and the weight values ​​of artificial neural network B to the second region 535.

[0100] The weight buffer 525 of the deep learning computing unit may receive and store the weight values ​​of artificial neural network A from the main memory 510, and transmit the weight values ​​of artificial neural network A to the upper stage of the processing element in the first region 530 by a clock signal.

[0101] Furthermore, the weight buffer 525 of the deep learning computing unit may also receive and store the weight values ​​of the B artificial neural network from the main memory 510. The deep learning computing unit may also transmit the weight values ​​of the B artificial neural network to the lower stage of the processing element in the second region 535 via a clock signal through the third on-chip network.

[0102] Referring to Figure 5B, after propagating the weight values, the deep learning computing unit may propagate the input feature map data of artificial neural network A to the first region 530 and the input feature map data of artificial neural network B to the second region 535.

[0103] The aforementioned first systolic data setup module may include the 1-1 systolic data setup module 520-1 and the 1-2 systolic data setup module 520-2. However, the division of the first systolic data setup module into the 1-1 systolic data setup module 520-1 and the 1-2 systolic data setup module 520-2 is shown separately in the drawings to demonstrate that each module can be logically separated, and does not necessarily mean that they are physically separate components.

[0104] The first-1 systolic data setup module 520-1 of the deep learning computing unit may receive and store the input feature map data of artificial neural network A from the main memory 510, and transmit the input feature map data of artificial neural network A to the left side of the first region 530 by a clock signal. This allows the processing elements of the first region 530 to propagate the input feature map data of artificial neural network A from left to right.

[0105] The first-second systolic data setup module 520-2 of the deep learning computing unit may receive and store the input feature map data of the B artificial neural network from the main memory 510, and transmit the input feature map data of the B artificial neural network to the left side of the second region 535 by a clock signal. This allows the processing elements of the second region 535 to propagate the input feature map data of the B artificial neural network from left to right.

[0106] The processing elements in the first region 530 may propagate downwards a partial sum obtained by performing multiplication and addition operations on the input feature map data and weight values ​​of the sequentially input artificial neural network A. The deep learning computing unit can use the fourth on-chip network to transmit the partial sum propagated to the upper edge of the first region 530 to the lower stage of the output result accumulation register 540.

[0107] The processing element in the second region 535 performs operations such as multiplication and addition of the input feature map data and weight values ​​of the B artificial neural network that are sequentially input, and propagates the obtained partial sum from top to bottom. The partial sum propagated to the lower edge of the second region 535 can be transmitted to the upper stage of the output result accumulation register 540.

[0108] Referring to Figures 5C and 5D, the deep learning computing unit may vertically distribute the systolic array into a third region 550 and a fourth region 555, executing artificial neural network A in the third region 550 and artificial neural network B in the fourth region 555.

[0109] Referring to Figure 5C, the deep learning computing device may pre-propagate the weight values ​​of artificial neural network A to the third region 550 and the weight values ​​of artificial neural network B to the fourth region 555.

[0110] The weight buffer 525 of the deep learning computing unit receives and stores the weight values ​​of artificial neural networks A and B from the main memory 510, and can transmit the weight values ​​of artificial neural networks A to the upper stage of the processing element in the third region 550 and the weight values ​​of artificial neural networks B to the upper stage of the processing element in the fourth region 555 using a clock signal.

[0111] Referring to Figure 5D, after propagating the weight values, the deep learning computing unit can propagate the input feature map data of artificial neural network A to the third region 550 and the input feature map data of artificial neural network B to the fourth region 555.

[0112] The first systolic data setup modules 520-1 and 520-2 of the deep learning computing unit may receive and store input feature map data of artificial neural network A from the main memory 510, and transmit the input feature map data of artificial neural network A to the left side of the third region 550 by a clock signal. This allows the processing elements of the third region 550 to propagate the input feature map data of artificial neural network A from left to right.

[0113] The second systolic data setup modules 545-1 and 545-2 of the deep learning computing unit receive and store input feature map data of the B artificial neural network from the main memory 510. Similar to the first systolic data setup modules 545-1 and 545-2, the second systolic data setup module described above includes the 2-1 systolic data setup module 545-1 and the 2-2 systolic data setup module 545-2. However, the division of the second systolic data setup module into the 2-1 systolic data setup module 545-1 and the 2-2 systolic data setup module 545-2 is shown separately in the drawings to indicate that each module can be logically separated, and does not necessarily mean that they are physically separate components.

[0114] The deep learning computing unit may input the input feature map data of the B artificial neural network to the right-hand side of the fourth region 555 using the second on-chip network. This allows the processing element of the fourth region 555 to propagate the input feature map data of the B artificial neural network from right to left.

[0115] The processing element 550 in the third domain can propagate a partial sum obtained by sequentially multiplying and adding the input feature map data and weight values ​​of the artificial neural network A from bottom to top.

[0116] The processing element in the fourth region 555 performs operations of multiplication and addition of the input feature map data and weight values ​​of the B artificial neural network that are sequentially input, and propagates the obtained partial sum from top to bottom. The partial sum propagated to the lower edge of the fourth region 555 can be transmitted to the upper stage of the output result accumulation register 540.

[0117] Referring to Figures 5E and 5F, the deep learning computing unit may divide the systolic array into four regions: the fifth region 560, the sixth region 565, the seventh region 570, and the eighth region 575. Artificial neural network A may be executed in the fifth region 560, artificial neural network B in the sixth region 565, artificial neural network C in the seventh region 570, and artificial neural network D in the eighth region 575.

[0118] Referring to Figure 5E, the deep learning computing device may pre-propagate the weight values ​​of artificial neural network A to the fifth region 560, the weight values ​​of artificial neural network B to the sixth region 565, the weight values ​​of artificial neural network C to the seventh region 570, and the weight values ​​of artificial neural network D to the eighth region 575.

[0119] The weight buffer 525 of the deep learning computing unit receives and stores the weight values ​​of artificial neural networks A and B from the main memory 510, and can transmit the weight values ​​of artificial neural networks A and B to the upper stages of the processing elements in the fifth region 560 and sixth region 565, respectively, via a clock signal.

[0120] Furthermore, the weight buffer 525 of the deep learning computing unit may also receive and store the weight values ​​of the C artificial neural network and the D artificial neural network from the main memory 510. The deep learning computing unit can transmit the weight values ​​of the C artificial neural network and the D artificial neural network to the lower stages of the processing elements in the seventh region 570 and the eighth region 575 via a clock signal through the third on-chip network.

[0121] Referring to Figure 5F, after propagating the weight values, the deep learning computing unit may propagate the input feature map data of artificial neural network A to the fifth region 560, the input feature map data of artificial neural network B to the sixth region 565, the input feature map data of artificial neural network C to the seventh region 570, and the input feature map data of artificial neural network D to the eighth region 575.

[0122] The first-1 systolic data setup module 520-1 of the deep learning computing unit may receive and store the input feature map data of the artificial neural network A from the main memory 510, and transmit the input feature map data of the artificial neural network A to the left side of the fifth region 560 by a clock signal. This allows the processing elements of the fifth region 560 to propagate the input feature map data of the artificial neural network A from left to right.

[0123] The first-second systolic data setup module 520-2 of the deep learning computing unit may receive and store the input feature map data of the C artificial neural network from the main memory 510, and transmit the input feature map data of the C artificial neural network to the left side of the seventh region 570 by a clock signal. This allows the processing elements of the seventh region 570 to propagate the input feature map data of the C artificial neural network from left to right.

[0124] The second-first systolic data setup module 545-1 of the deep learning computing unit may receive and store the input feature map data of the B artificial neural network from the main memory 510. The deep learning computing unit may also input the input feature map data of the B artificial neural network to the right-hand side of the sixth region 565 using the second on-chip network. This allows the processing elements of the sixth region 565 to propagate the input feature map data of the B artificial neural network from right to left.

[0125] The 2-2 systolic data setup module 545-2 of the deep learning computing unit may receive and store the input feature map data of the D artificial neural network from the main memory 510. The deep learning computing unit may also input the input feature map data of the D artificial neural network to the right-hand side of the 8th region 575 using the 2nd on-chip network. This allows the processing elements of the 8th region 575 to propagate the input feature map data of the D artificial neural network from right to left.

[0126] The processing elements in the fifth region 560 can propagate partial sums obtained by sequentially multiplying and adding the input feature map data and weight values ​​of the artificial neural network A from bottom to top. The deep learning computing unit can use the fourth on-chip network to transmit the partial sum propagated to the upper edge of the fifth region 560 to the lower left section of the output result accumulation register 540.

[0127] The processing element in the seventh region 570 performs multiplication and addition operations on the input feature map data and weight values ​​of the C artificial neural network that are sequentially input, and propagates the obtained partial sum from top to bottom. The partial sum propagated to the bottom edge of the seventh region 570 can be transmitted to the upper left section of the output result accumulation register 540.

[0128] The processing elements in the sixth region 565 may propagate the obtained partial sum from bottom to top by performing multiplication and addition operations on the input feature map data and weight values ​​of the B artificial neural network that are sequentially input. The deep learning computing unit can use the fourth on-chip network to transmit the partial sum propagated to the upper edge of the sixth region 565 to the lower right section of the output result accumulation register 540.

[0129] The processing element in region 8 575 performs multiplication and addition operations on the input feature map data and weight values ​​of the D artificial neural network that are sequentially input, propagating the resulting partial sum from top to bottom. The partial sum propagated to the lower edge of region 8 575 can be transmitted to the upper right section of the output result accumulation register 540.

[0130] Figure 6 is a flowchart illustrating how to perform deep learning operations via spatial multitasking.

[0131] Referring to Figure 6, steps S610 to S655 may be performed by the deep learning computing device described above, referring to Figures 2B to 5F.

[0132] In step S610, the deep learning computing unit determines whether or not there is a first artificial neural network currently running.

[0133] In step S615, if there is a first artificial neural network currently running, the deep learning computing unit distributes processing elements based on the characteristics of the first artificial neural network and the characteristics of the newly running second artificial neural network, so that the deep learning calculations for the first artificial neural network and the deep learning calculations for the second artificial neural network can be performed simultaneously.

[0134] The deep learning computing unit may determine the distribution method and distribution ratio of processing elements based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network. The characteristics of the artificial neural network may include, for example, the number of artificial neural network layers, the input for each layer, the weight values, and the size of the output data.

[0135] The deep learning computing unit can preempt the deep learning computation of the first artificial neural network based on the determined allocation method and allocation ratio, reserve processing elements, and allocate the processing elements reserved through preemption to the deep learning computation of the second artificial neural network.

[0136] In step S620, the deep learning computing unit sets the propagation direction of the input data and partial sums based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network. The deep learning computing unit sets whether the input data of the first and second artificial neural networks is propagated in either the left-right or right-hand direction, and whether the partial sums are propagated in either the up-down or up-down direction.

[0137] In step S625, the deep learning computing unit simultaneously performs deep learning calculations for the first artificial neural network and the second artificial neural network using the distributed processing elements.

[0138] If there is no first artificial neural network currently running, the deep learning computing unit may run a second artificial neural network using all processing elements of the systolic array.

[0139] Furthermore, deep learning computing devices can separate a single artificial neural network into multiple sub-networks and run the sub-networks simultaneously, in order to improve the throughput and top-of-stress ratio (TOPS / Watt) even when running a single artificial neural network.

[0140] In step S630, the deep learning computing unit determines whether the second artificial neural network has multiple batches.

[0141] In step S635, if the second artificial neural network has multiple batches (for example, if there are several images to perform image recognition on), the deep learning computing unit separates the second artificial neural network into multiple sub-artificial neural networks. For example, the deep learning computing unit may separate it into two sub-artificial neural networks by dividing the batch in half.

[0142] In step S640, the deep learning computing unit distributes processing elements based on the characteristics of the sub-artificial neural network so that deep learning calculations for the sub-artificial neural network can be performed simultaneously.

[0143] In step S645, the deep learning computing unit sets the propagation direction of the input data and the partial sum based on the characteristics of the sub-artificial neural network. For example, the deep learning computing unit may distribute the processing elements of the systolic array identically to the two sub-artificial neural networks.

[0144] In step S650, the deep learning computing device simultaneously performs deep learning calculations on the sub-artificial neural network using the distributed processing elements.

[0145] The method of separating a single artificial neural network (AI) into multiple sub-AIs and running them simultaneously is highly effective when the various layers constituting the AI ​​are of extreme shapes. For example, in the case of a weight-stationary neural processing unit, if the number of output channels is smaller than the length of the processing element's side, it has the disadvantage of not being able to fully utilize computational resources. By running sub-AIs simultaneously, as in the example above, when the processing element cannot be fully utilized, the AI ​​can be divided into multiple sub-AIs and run simultaneously, achieving higher performance compared to running only a single AI network. Dividing the AI ​​network into multiple sub-AIs allows for efficient use when the size and shape of the various layers are extreme, as each layer is trained on different tasks.

[0146] In step S655, if the second artificial neural network has one batch, the deep learning computing unit can run the second artificial neural network using all the processing elements of the systolic array.

[0147] Figure 7 illustrates a method for utilizing a neural processing unit for spatial multitasking according to one embodiment.

[0148] Referring to Figure 7, a deep learning computing device according to one embodiment can simultaneously execute multiple artificial neural networks (for example, artificial neural network A 710-1 and artificial neural network B 710-2) in a multi-user environment such as a server or desktop equipped with a neural processing unit for spatial multitasking.

[0149] Multiple artificial neural networks can request the use of neural processing units via a Neural Network Framework 720 such as TensorFlow or PyTorch. Such requests can then be transmitted to a lower-level software called a Neural Network Scheduler 730.

[0150] Conventional neural processing units do not support spatial multitasking. Therefore, after sending one artificial neural network execution command to the neural processing unit, it did not send subsequent artificial neural network execution requests to the neural processing unit until the execution of that artificial neural network was completed.

[0151] On the other hand, the deep learning computing device according to one embodiment can simultaneously execute various artificial neural networks for spatial multitasking, so the artificial neural network scheduler 730, which takes spatial multitasking into consideration, can transmit multiple artificial neural network execution commands to the neural processing unit. Here, since the neural processing unit 750 is hardware and the artificial neural network scheduler 730 is software, the artificial neural network execution commands can be transmitted via a Neural Processing Unit Device Driver 740 that enables communication between the two.

[0152] The deep learning computing unit includes a neural processing unit 750 that supports spatial multitasking, and an artificial neural network scheduler 730 that takes spatial multitasking into consideration, which can simultaneously execute multiple artificial neural networks to which execution commands have been sent.

[0153] The embodiments described above are embodied in hardware components, software components, or combinations of hardware and software components. For example, the devices and components described in these embodiments are embodied using one or more general-purpose or special-purpose computers, such as a processor, controller, ALU (arithmetic logic unit), digital signal processor, microcomputer, FPA (field programmable array), PLU (programmable logic unit), microprocessor, or different devices that execute and respond to instructions. The processing device can run an operating system (OS) and one or more software applications run on the OS. The processing device can also access, store, manipulate, process, and generate data in response to software execution. For convenience of understanding, the processing device may sometimes be described as being used as a single unit, but a person with ordinary skill in the art will see that the processing device includes multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0154] Software includes computer programs, code, instructions, or a combination of one or more of these, which can configure a processing unit to operate as desired, or instruct the processing unit independently or in combination. Software and / or data can be permanently or temporarily embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, for interpretation by a processing unit or for providing instructions or data to a processing unit. Software can be distributed across a network of computer systems and stored and executed in a distributed manner. Software and data can be stored on a recording medium readable by one or more computers.

[0155] The method according to this embodiment is embodied in the form of program instructions that are implemented via various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, etc., individually or in combination. The recording medium and program instructions may be specifically designed and configured for the purposes of the present invention, or they may be known and usable by those skilled in the art who have technology in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like.

[0156] The hardware device described above may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0157] As described above, although embodiments have been illustrated with limited drawings, a person with ordinary skill in the art can apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or assembled in a different manner than described, or replaced or substituted with other components or equivalents, and still achieve suitable results.

[0158] Therefore, the scope of the present invention is not limited to the disclosed embodiments, but is defined by the claims and equivalents thereof.

Claims

1. In electronic devices, Includes a processor that performs multiple tasks simultaneously using a systolic array, The aforementioned processor, The systolic array, in which multiple processing elements (PEs) are arranged two-dimensionally vertically and / or horizontally, A first on-chip network that performs data propagation between the processing elements, An output data receiving module for receiving output data, which is the result of a calculation between the first input data and the second input data, from the upper and lower edges of the systolic array, Includes, The output data receiving module is Output accumulators and, A fourth on-chip network that transmits the partial sum propagated on the upper edge of the systolic array to the lower stage of the output result accumulation register, Includes, The partial sum propagated to the lower edge of the systolic array is then transmitted to the upper stage of the output result accumulation register. The aforementioned multiple tasks include an electronic device that includes deep learning operations involving one or more multiplication-addition operations.

2. The aforementioned processor, Based on the characteristics of multiple artificial neural networks, the processing elements are distributed so that deep learning calculations can be performed simultaneously on the multiple artificial neural networks. The aforementioned distribution is The electronic device according to claim 1, which is performed based on the characteristics of the plurality of artificial neural networks.

3. The electronic device according to claim 2, wherein the processor distributes the plurality of processing elements included in the systolic array.

4. The electronic device according to claim 1, wherein the processor sets the propagation direction of each input data and the propagation direction of the partial sum based on the characteristics of a plurality of artificial neural networks.

5. The electronic device according to claim 1, wherein the processor separates one artificial neural network into a plurality of sub-artificial neural networks and distributes the processing elements so that deep learning calculations can be performed simultaneously on the sub-artificial neural networks.

6. The electronic device according to claim 5, wherein the processor sets the propagation direction of the input data and the partial sum based on the characteristics of the sub-artificial neural network.

7. The electronic device according to any one of claims 1 to 6, wherein the processor further includes an input data transmission module for inputting data to another side of the systolic array.

8. The other sides of the systolic array are toward the left and right sides of the systolic array, The aforementioned input data transmission module is A first systolic data setup module adjusts the timing of inputting the first input data to the left-hand side of the systolic array and transmits the first input data to the left-hand side of the systolic array, A second systolic data setup module adjusts the timing of inputting the second input data to the right-hand side of the systolic array, A second on-chip network that transmits the second input data to the right-hand side of the systolic array, The electronic device according to claim 7, including the electronic device according to claim 7.

9. The other sides of the systolic array are toward the left and right sides of the systolic array, The first input data is input via the first on-chip network, and the second input data is input via the second on-chip network. The processor further includes an input data transmission module for inputting weight input data to the upper and lower stages of the systolic array, The aforementioned input data transmission module is A weight buffer adjusts the timing of inputting the first weight input data and the second weight input data to the systolic array and transmits the first weight input data to each first processing element via the upper stage of the systolic array. A third on-chip network transmits the second weight input data to each of the second processing elements among the processing elements via the lower stage of the systolic array, The electronic device according to claim 8, including the electronic device according to claim 8.

10. The electronic device according to any one of claims 1 to 9, wherein the processor further includes a second input data transmission module for inputting input data to the upper and lower stages of the processing element.

11. The aforementioned input data transmission module is A weight buffer adjusts the timing of inputting the first weight input data to the first processing element among the processing elements, and transmits the first weight input data to the upper stage of the first processing element, A second weight input data is transmitted to the lower stage of the second processing element among the processing elements via another on-chip network, The electronic device according to claim 9, including the electronic device according to claim 9.

12. The electronic device according to claim 11, wherein the weight buffer adjusts the timing of inputting the second weight input data to the second processing element.

13. A step to determine whether or not there is a first artificial neural network currently in operation, If there is a first artificial neural network currently running, the process involves distributing processing elements so that deep learning operations for the first artificial neural network and the second artificial neural network can be performed simultaneously, based on the characteristics of the first artificial neural network and the characteristics of the newly running second artificial neural network. The steps include setting the propagation direction of the input data and the partial sum based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network, The steps include simultaneously performing deep learning calculations for the first artificial neural network and the second artificial neural network using the distributed processing elements, Includes, A deep learning computation execution method comprising the step of distributing the processing elements, wherein the step of determining the distribution method and distribution ratio of the processing elements based on the characteristics of the first artificial neural network and the characteristics of the second artificial neural network.

14. The step of distributing the processing elements is: A step of preempting the deep learning calculations of the first artificial neural network based on the distribution method and the distribution ratio, The steps include: assigning processing elements secured via the preemption in order to perform deep learning calculations on the second artificial neural network; The steps include: assigning further processing elements from among the processing elements secured via the preemption in order to perform the deep learning operation of the first artificial neural network; A method for executing deep learning operations according to claim 13, including the following:

15. The deep learning operation execution method according to claim 14, wherein the processing element is a systolic array processing element.

16. If there is no first artificial neural network currently running, the step is to determine whether the second artificial neural network has multiple batches. If the second artificial neural network has multiple batches, the steps include separating the second artificial neural network into multiple sub-artificial neural networks, Based on the characteristics of the sub-artificial neural network, the processing elements are distributed so that deep learning calculations can be performed simultaneously on the sub-artificial neural network. The steps include setting the propagation direction of the input data and partial sum based on the characteristics of the aforementioned sub-artificial neural network, The steps include simultaneously performing deep learning calculations on the sub-artificial neural network using the distributed processing elements, The deep learning operation execution method according to claim 13, further comprising:

17. The deep learning computation execution method according to claim 16, wherein the step of distributing the processing elements includes determining the distribution method and distribution ratio of the processing elements based on the characteristics of the sub-artificial neural network.

18. The steps include dividing the second artificial neural network into a plurality of sub-artificial neural networks according to each batch of the second artificial neural network, The steps include: distributing the processing elements in order to simultaneously perform deep learning calculations on the sub-artificial neural network according to the characteristics of the sub-artificial neural network; The steps include setting the propagation direction of the input data and partial sum based on the characteristics of the aforementioned sub-artificial neural network, The steps include simultaneously performing deep learning calculations for the first artificial neural network and the sub-artificial neural network using the distributed processing elements, A method for executing deep learning operations according to any one of claims 13-17, including the following:

19. A computer program that causes a computer to execute the deep learning computation execution method described in any one of claims 13-18.

20. In an electronic device that performs deep learning calculations, Including the processor, The aforementioned processor, A systolic array in which multiple processing elements are arranged two-dimensionally vertically and / or horizontally, A first on-chip network that performs data propagation between the plurality of processing elements, An output data receiving module for receiving output data, which is the result of a calculation between the first input data and the second input data, from the upper and lower edges of the systolic array, Includes, The output data receiving module is Output accumulators and, A fourth on-chip network that transmits the partial sum propagated on the upper edge of the systolic array to the lower stage of the output result accumulation register, Includes, The partial sum propagated to the lower edge of the systolic array is then transmitted to the upper stage of the output result accumulation register. The processor is an electronic device that separates an artificial neural network into a plurality of sub-artificial neural networks and distributes processing elements so that it can simultaneously perform deep learning operations, including one or more multiplication-addition operations, on the sub-artificial neural networks.

21. The electronic device according to claim 20, wherein the separation of the artificial neural network into the sub-artificial neural networks is performed by the tasks of each different layer of the artificial neural network.

22. The electronic device according to claim 20, wherein the separation of the artificial neural network into the sub-artificial neural networks is performed according to different batches of the artificial neural networks.

23. The electronic device according to any one of claims 20-22, wherein the processor sets the propagation direction of input data and partial sums to the processing element based on the characteristics of the sub-artificial neural network.

24. The electronic device according to claim 23, wherein the processor determines the distribution method and distribution ratio of the processing elements based on the characteristics of the sub-artificial neural network and distributes the processing elements.

25. The electronic device according to claim 20, wherein the processor performs deep learning operations on a sub-artificial neural network using a processing element, and simultaneously performs deep learning operations on another artificial neural network using other processing elements from among a plurality of processing elements.

Images