Nerve processing unit, system-on-chip and electronic device

Abstract

One embodiment of the invention provides a neural processing unit, a system on chip and an electronic device. The neural processing unit may include: a plurality of PE groups which are set to be capable of performing an operation for an artificial neural network (ANN) model, and which are arranged to include a plurality of processing elements (PEs); and a second circuit arranged for a clock distributor that divides and generates a plurality of clock signals having different phases from each other based on an original clock signal from a clock source and supplies the plurality of clock signals to the plurality of PE groups. A first clock signal of the plurality of clock signals may be provided to a first PE group of the plurality of PE groups, and a second clock signal of the plurality of clock signals may be provided to a second PE group of the plurality of PE groups.

Description

Technical Field

[0001] The present disclosure relates to a technique for dispersing the peak power of a neural processing unit. Background Art

[0002] Artificial intelligence (AI) is developing gradually. AI refers to artificial intelligence that imitates human intelligence, that is, intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc.

[0003] Furthermore, in recent years, neural processing units (NPUs) have been developed to accelerate the speed of calculations used in artificial intelligence (AI).

[0004] There may be various types of artificial neural network (ANN) models depending on the purpose of the artificial intelligence service. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. within the image / video may be used for the artificial intelligence service.

[0005] Typically, an ANN model can have a different amount of computation in each layer.

[0006] In particular, if the amount of computation increases significantly in a particular layer, power consumption may increase instantaneously. Summary of the invention

[0007] Artificial neural network operations are data-intensive. In particular, artificial neural network operations require parallel processing operations. That is, artificial neural network operations have the characteristic of reducing processing speed when a large amount of data cannot be processed simultaneously and quickly in parallel.

[0008] Thus, the inventors of the present disclosure have developed a neural processing unit dedicated to artificial neural network operations. The inventors of the present disclosure attempt to improve the parallel processing performance of the neural processing unit by increasing the number of multiple processing elements of the neural processing unit. In addition, the inventors of the present disclosure also attempt to develop a neural processing unit that can achieve low-power operation.

[0009] In addition, the power supply capacity of the power supply unit of the neural processing unit dedicated to low-power operation provided in the edge device may be relatively low compared to the power supply unit of the graphics processing unit (GPU) operating in the server or personal computer (PC). In addition, the capacity of the capacitor of the power supply unit of the neural processing unit dedicated to low-power operation provided in the edge device may be insufficient to cope with the instantaneous power supply.

[0010] However, the inventors of the present disclosure have recognized that as the number of processing elements of a neural processing unit dedicated to low power operation increases, the degree of supply voltage transient oscillation may increase. Specifically, the amount of transient power supply of the neural processing unit required may be proportional to the number of processing elements in operation.

[0011] In addition, the computational amount of the artificial neural network model has a large deviation in each different layer. Therefore, the number of processing elements operating in parallel may be different depending on the computational amount of the layer of the artificial neural network model. That is, in the case where many processing elements are operating simultaneously, the voltage of the power supply part of the neural processing unit may oscillate or decrease instantaneously.

[0012] In addition, the amount of computation of a specific layer of the artificial neural network model may be very small. In this case, the inventors of the present disclosure recognize that even if the driving frequency of the neural processing unit is increased, the supply voltage stability of the neural processing unit can be guaranteed.

[0013] In addition, in the case where the supply voltage oscillates or decreases momentarily, it may be necessary to increase the supply voltage VDD for the sake of system safety. Therefore, if the system safety cannot be ensured, the problem of unnecessary increase in the supply voltage may occur. In the case where the supply voltage increases, the problem that the power consumption of the neural processing unit may increase sharply may occur. In addition, in the case of increased power consumption, the problem of increased heat generation of the neural processing unit may occur. In this case, the inventors of the present disclosure recognized that the voltage of the power supply of the neural processing unit can be stabilized by reducing the driving frequency of the neural processing unit.

[0014] On the contrary, as the supply voltage stabilizes, the supply voltage VDD can be reduced. Therefore, as long as the safety of the supply voltage of the neural processing unit is ensured, the supply voltage can be reduced, and as a result, the power consumption of the neural processing unit can be reduced.

[0015] Thus, the inventors of the present disclosure recognized that even if the peak power is increased by increasing the number of processing elements, there is a need to improve the stability of the supply voltage of a neural processing unit dedicated to artificial neural network operations.

[0016] Therefore, examples of the present disclosure are intended to provide a technical solution for stabilizing the jitter of the supply voltage of a power supply unit caused by excessive power in a specific operation step.

[0017] In order to achieve the above-mentioned purpose, an example of the present disclosure provides a neural processing unit. The neural processing unit may include: a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN: artificial neural network) model and is arranged for a plurality of PE groups including a plurality of processing elements (PE); and a second circuit, which is arranged for a clock distributor, and the clock distributor is based on the original clock signal from the clock source. The clock distributor generates a plurality of clock signals with different phases from each other and provides them to the plurality of PE groups. The first clock signal of the plurality of clock signals may be provided to the first PE group of the plurality of PE groups, and the second clock signal of the plurality of clock signals may be provided to the second PE group of the plurality of PE groups.

[0018] An example of the present disclosure provides a system on chip. The system on chip may include: a semiconductor substrate; a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN: artificial neural network) model and is arranged on the semiconductor substrate for a plurality of neural processing units (NPUs) including a plurality of processing elements (PEs); and a second circuit, which is arranged on the semiconductor substrate for a clock distributor, which divides and generates a plurality of clock signals with different phases from each other based on an original clock signal from a clock source and provides them to the plurality of NPUs. A first clock signal among the plurality of clock signals may be provided to a first NPU among the plurality of NPUs, and a second clock signal among the plurality of clock signals may be provided to a second NPU among the plurality of NPUs.

[0019] An example of the present disclosure provides an electronic device. The electronic device may include: a printed circuit substrate; a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN) model and is arranged on the printed circuit substrate for a plurality of neural processing units (NPUs) including a plurality of processing elements (PEs); and a second circuit, which is arranged on the printed circuit substrate for a clock distributor, the clock distributor splits and generates a plurality of clock signals with different phases from each other based on an original clock signal from a clock source and provides them to the plurality of NPUs. A first clock signal among the plurality of clock signals may be provided to a first NPU among the plurality of NPUs, and a second clock signal among the plurality of clock signals may be provided to a second NPU among the plurality of NPUs.

[0020] According to an example of the present disclosure, artificial neural network operations are dispersed according to a plurality of clock signals, thereby having an effect of being able to reduce peak power of specific operation steps.

[0021] According to an example of the present disclosure, artificial neural network operations are dispersed according to a plurality of clock signals, thereby having an effect of being able to improve the safety of a supply voltage supplied to a neural processing unit.

[0022] According to the examples of the present disclosure, even if the computational load of the artificial neural network is the same, the instantaneous peak power can be reduced, thereby having the effect of being able to provide relatively low peak power based on the same computational load. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.

[0024] Figure 2a This is a diagram used to illustrate the basic structure of a convolutional neural network (CNN).

[0025] Figure 2b is a comprehensive diagram that shows the operation of a convolutional neural network in an easy to understand manner.

[0026] Figure 3 is a schematic conceptual diagram illustrating a neural processing unit according to an example of the present disclosure.

[0027] Figure 4a is a schematic conceptual diagram illustrating one processing element among a plurality of processing elements that can be applied to an example of the present disclosure.

[0028] Figure 4b is a schematic conceptual diagram illustrating an SFU of an example that can be applied to the present disclosure.

[0029] Figure 5 It is shown Figure 3 An example diagram of a variation of a neural processing unit is shown.

[0030] Figure 6a is an example diagram showing the size of data in each layer of an exemplary artificial neural network model, Figure 6b It is shown Figure 6a An exemplary table showing the data size, etc., of each layer in the exemplary artificial neural network model shown.

[0031] Figure 7a is an exemplary diagram showing the structure of an NPU according to the first example of the present disclosure.

[0032] Figure 7b is an exemplary diagram showing a structure of an NPU according to a first modification example of the first example of the present disclosure.

[0033] Figure 7c is an exemplary diagram showing a structure of an NPU according to a second modification example of the first example of the present disclosure.

[0034] Figure 8a is an exemplary diagram showing a structure of an NPU according to a second example of the present disclosure.

[0035] Figure 8b is an exemplary diagram showing a structure of an NPU according to a first modification example of the second example of the present disclosure.

[0036] Figure 8c : is an exemplary diagram showing the structure of an NPU according to a second modification example of the second example of the present disclosure.

[0037] Figure 9a is an exemplary diagram showing the structure of a SoC according to a third example of the present disclosure.

[0038] Figure 9b is an exemplary diagram showing the structure of a SoC according to a first modification example of the third example of the present disclosure.

[0039] Fig.9c is an exemplary diagram showing the structure of a SoC according to a second modification example of the third example of the present disclosure.

[0040] Fig.10a is an exemplary diagram showing a structure of a system according to a fourth example of the present disclosure.

[0041] Fig.10b : is an exemplary diagram showing the structure of a system according to a first modification example of the fourth example of the present disclosure.

[0042] Fig.10c : is an exemplary diagram showing the structure of a system according to a second modification example of the fourth example of the present disclosure.

[0043] Fig.11a is the first example of a clock distributor.

[0044] Fig.11b is a second example of a clock distributor.

[0045] Fig.11c is a third example of a clock distributor.

[0046] Fig.11d is a fourth example of a clock distributor.

[0047] Fig.11e is an example diagram showing exemplary signal waveforms from a clock distributor.

[0048] Fig.12is an example diagram illustrating an operation method of an NPU according to an example of the present disclosure.

[0049] Fig.13 A comparative example is shown in which clock signals of different phases are not input to a plurality of PEs of an NPU.

[0050] Fig.14a An example of inputting clock signals of different phases to a plurality of PEs of two groups of an NPU in consideration of peak power according to an example of the present disclosure is shown.

[0051] Fig.14b An example of inputting clock signals of different phases to a plurality of PEs of three groups of an NPU by considering peak power according to an example of the present disclosure is shown.

[0052] Fig.14c An example of inputting clock signals of different phases to a plurality of PEs of four groups of an NPU by considering peak power according to an example of the present disclosure is shown.

[0053] Fig.15 A comparative example in which clock signals of different phases are not input to a plurality of NPUs is shown.

[0054] Fig.16a An example of inputting clock signals of different phases to two NPUs by considering peak power according to an example of the present disclosure is shown.

[0055] Fig.16b An example of inputting clock signals of different phases to three NPUs by considering peak power according to an example of the present disclosure is shown.

[0056] Fig.16c An example of inputting clock signals of different phases to four NPUs by considering peak power according to an example of the present disclosure is shown. DETAILED DESCRIPTION

[0057] The specific structure and even step description of the examples of the present disclosure are only used to illustrate the examples according to the concepts of the present disclosure. Therefore, the examples according to the concepts of the present disclosure can be implemented in various forms. The present disclosure should not be interpreted as being limited to the examples of the present disclosure.

[0058] The examples according to the concepts of the present disclosure may be subjected to various changes and may have various forms. Thus, specific examples will be shown in the drawings and the present disclosure or the present application will be described in detail. However, this is not intended to limit the examples according to the concepts of the present disclosure to specific disclosed forms. The examples according to the concepts of the present disclosure should be understood to include all changes, equivalents and even substitutes within the scope of the ideas and techniques of the present disclosure.

[0059] The terms first and / or second can be used to describe a variety of components, but these components are not limited by these terms. These terms are only used to distinguish one component from another component. These terms do not depart from the scope of rights of the concepts disclosed herein. The first component can be named as the second component, and similarly, the second component can also be named as the first component.

[0060] When a certain component is mentioned as being "connected" or "coupled" to another component, it may be directly connected or coupled to the other component. However, it should be understood that other components may exist between multiple components. On the contrary, when a certain component is mentioned as being "directly connected" or "directly coupled" to another component, it should be understood that there are no other components between the two. Other expressions used to describe the relationship between components, namely, "between" and "directly between" or "adjacent to" and "directly adjacent to", etc., should also be interpreted in the same manner.

[0061] The terms used in this disclosure are only used to illustrate specific examples and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the singular includes the plural. In this disclosure, terms such as "including" or "having" are used to specify the existence of the described features, numbers, steps, operations, constituent elements, components, or combinations thereof. Therefore, it should be understood that the existence or additional possibility of one or more other features or numbers, steps, operations, constituent elements, components, or combinations thereof is not precluded.

[0062] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs. Terms that are the same as those defined in commonly used dictionaries should be interpreted as having the same meanings as those in the context of the relevant technology, and should not be interpreted as ideal or overly formal meanings unless clearly defined in the present disclosure.

[0063] When describing the examples, the description of the technical contents that are well known in the art to which the present disclosure belongs and are not directly related to the present disclosure will be omitted. This is to convey the gist of the present disclosure more clearly without confusion by omitting unnecessary descriptions.

[0064] Definition of terms

[0065] Hereinafter, in order to help understanding of the disclosed inner cylinder proposed in the present disclosure, the terms used in the present disclosure will be briefly sorted out.

[0066] NPU: As the abbreviation of Neural Processing Unit, it can refer to a processor independent of the central processing unit (CPU) and specifically used for the calculation of artificial neural network models.

[0067] ANN: As an abbreviation of artificial neural network, it can refer to a network that connects nodes into a layer structure by imitating the connection of neurons in the human brain through synapses in order to imitate human intelligence.

[0068] DNN: As the abbreviation of Deep Neural Network, it can refer to increasing the number of hidden layers of artificial neural networks to achieve a higher level of artificial intelligence.

[0069] CNN: Convolutional Neural Network is a neural network that processes images in a similar way to the visual cortex of the human brain. Convolutional neural networks are known to be suitable for image processing and are easy to extract features of input data and grasp the patterns of features.

[0070] Hereinafter, preferred examples of the present disclosure will be described with reference to the accompanying drawings, thereby explaining the present disclosure in detail. Hereinafter, examples of the present disclosure will be described in detail with reference to the accompanying drawings.

[0071]

AI

[0072] Humans have intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc. Artificial intelligence (AI) refers to artificial imitation of human intelligence.

[0073] The human brain is composed of many nerve cells called neurons. Each neuron is connected to hundreds to thousands of other neurons through a connection part called a synapse. The model that models the operating principle of biological neurons and the connection relationship between neurons in order to imitate human intelligence is called an artificial neural network model. In other words, an artificial neural network is a system that connects nodes that imitate neurons into a layer structure.

[0074] This artificial neural network model is divided into "single-layer neural network" and "multi-layer neural network" according to the number of layers. A typical multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is a layer that receives external data. The number of neurons in the input layer is the same as the number of input variables. (2) The hidden layer is located between the input layer and the output layer, receives signals from the input layer, extracts features and passes them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them to the outside. The input signal between neurons is multiplied by each connection strength with a value between 0 and 1 and then summed. If the sum is greater than the threshold of the neuron, the neuron is activated and realized as an output value through the activation function.

[0075] In addition, increasing the number of hidden layers of an artificial neural network to achieve a higher level of artificial intelligence is called a deep neural network (DNN).

[0076] DNN is being developed in various structures. For example, it is known that a convolutional neural network (CNN) as an example of DNN is convenient for extracting features of input values ​​(images or pictures) and grasping the pattern of the extracted output values. CNN can be composed of convolution operations, activation function operations, pooling operations, etc. processed in a specific order.

[0077] For example, in each layer of a DNN, a parameter (i.e., input value, output value, weight or kernel, etc.) can be a matrix consisting of multiple channels. The parameter can be processed in the NPU using convolution or matrix multiplication. The output value processed by the operation is generated in each layer.

[0078] For example, a transformer is a DNN based on attention technology. The transformer uses multiple matrix multiplication operations. The transformer can obtain attention (Q, K, V) as the output value by using parameters such as input value and query vector (Q: query), key vector (K: key) and value vector (V: value). The transformer can process multiple reasoning operations based on the output value (i.e., attention (Q, K, V)). The transformer has a tendency to show better reasoning performance than CNN.

[0079] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.

[0080] The following describes the operation of an exemplary artificial neural network model 110 a that can be executed in the neural processing unit 100 .

[0081] Figure 1 The exemplary artificial neural network model 110a may be an artificial neural network trained to perform various reasoning functions such as object recognition and speech recognition.

[0082] The artificial neural network model 110 a may be a deep neural network (DNN: Deep Neural Network).

[0083] However, the artificial neural network model 110a according to the example of the present disclosure is not limited to a deep neural network.

[0084] For example, the artificial neural network model 110a can be implemented as LLM, Generative Adversarial Networks (GAN), Florence, DaViT, MobileViT, Swin-Transformer, Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3 and other models. However, the present disclosure is not limited to the above-mentioned models. In addition, the artificial neural network model 110a can also be an integrated model based on at least two different models.

[0085] In the following, the reasoning process performed by the exemplary artificial neural network model 110a will be described.

[0086] The artificial neural network model 110a is an exemplary deep neural network model, which includes an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5, a third connection network 110a-6 and an output layer 110a-7. However, the present disclosure is not limited to Figure 1 The artificial neural network model shown. The first hidden layer 110a-3 and the second hidden layer 110a-5 may also be referred to as a plurality of hidden layers.

[0087] Exemplarily, the input layer 110a-1 may include an x1 input node and an x2 input node. That is, the input layer 110a-1 may include information about 2 input values.

[0088] Exemplarily, the first connection network 110a-2 may include information about six weight values ​​for connecting each node of the input layer 110a-1 to each node of the first hidden layer 110a-3. Each weight value is multiplied by the input node value, and the accumulated value of the multiplied value is stored in the first hidden layer 110a-3. The weight value and the input node value may be referred to as parameters of the artificial neural network model.

[0089] For example, the first hidden layer 110a-3 may include an a1 node, an a2 node, and an a3 node. That is, the first hidden layer 110a-3 may include information about three node values.

[0090] Figure 1 The first processing element PE1 can process the operation of the a1 node.

[0091] Figure 1 The second processing element PE2 can process the operation of the a2 node.

[0092] Figure 1 The third processing element PE3 can process the operation of the a3 node.

[0093] Exemplarily, the second connection network 110a-4 may include information about nine weight values ​​for connecting each node of the first hidden layer 110a-3 to each node of the second hidden layer 110a-5. The weight values ​​of the second connection network 110a-4 are multiplied by the node values ​​input from the first hidden layer 110a-3, respectively, and the accumulated value of the multiplied values ​​is stored in the second hidden layer 110a-5.

[0094] For example, the second hidden layer 110a-5 may include a b1 node, a b2 node, and a b3 node. That is, the second hidden layer 110a-5 may include information on three node values.

[0095] Figure 1 The fourth processing element PE4 can process the operation of the b1 node.

[0096] Figure 1 The fifth processing element PE5 can process the operation of the b2 node.

[0097] Figure 1 The sixth processing element PE6 can process the operation of the b3 node.

[0098] Exemplarily, the third connection network 110a-6 may include information about six weight values ​​for connecting each node of the second hidden layer 110a-5 and each node of the output layer 110a-7. The weight values ​​of the third connection network 110a-6 are multiplied by the node values ​​input from the second hidden layer 110a-5, respectively, and the accumulated value of the multiplied values ​​is stored in the output layer 110a-7.

[0099] Exemplarily, the output layer 110a-7 may include a y1 node and a y2 node. That is, the output layer 110a-7 may include information about two node values.

[0100] Figure 1 The seventh processing element PE7 can process the operation of the y1 node.

[0101] Figure 1 The eighth processing element PE8 can process the operation of the y2 node.

[0102] Each node may correspond to an eigenvalue, and the eigenvalue may correspond to a feature graph.

[0103] Figure 2a This is a diagram used to illustrate the basic structure of a convolutional neural network (CNN).

[0104] Reference Figure 2a The input image may be represented as a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image may have a plurality of channels, where a channel may represent the number of color components of the input data image.

[0105] The convolution process involves traversing the input image at specified intervals and performing convolution operations with the kernel.

[0106] A convolutional neural network may have a structure in which the output value (convolution or matrix multiplication) of the current layer is passed to the input value of the next layer.

[0107] For example, convolution is defined by two main parameters: input feature map and kernel. Parameters can include input feature map, output feature map, activation map, weights, kernel, and attention (Q, K, V).

[0108] Convolution slides the kernel window onto the input feature map. The step size of sliding the kernel onto the input feature map is called the stride.

[0109] Pooling can be applied after convolution. In addition, a fully-connected (FC) layer can be arranged at the end of the convolutional neural network.

[0110] Figure 2b is a comprehensive diagram that shows the operation of a convolutional neural network in an easy to understand manner.

[0111] Reference Figure 2b , exemplarily showing the case where the input image is a two-dimensional matrix with a size of 6×6. Figure 2b The example shows the use of three nodes, namely, channel 1, channel 2, and channel 3.

[0112] First, the convolution operation is explained.

[0113] Input image (in Figure 2b exemplarily shown as a 6×6 size) in the first node with kernel 1 for channel 1 (in Figure 2b The convolution is performed as a result of the output feature Figure 1 (exist Figure 2b In addition, the input image (in Figure 2b 6×6 in the example shown in FIG. 1 ) in the second node with kernel 2 for channel 2 (in Figure 2b The convolution operation is performed on the feature map 2 (shown as a 3×3 size in the example), and as a result, the feature map 2 (in Figure 2b In addition, the input image is combined with kernel 3 for channel 3 in the third node (in Figure 2b The convolution is performed as a result of the output feature Figure 3 (exist Figure 2b 4×4 in size).

[0114] To process each convolution, the processing elements PE1 to PE12 of the neural processing unit 100 are configured to perform a MAC operation.

[0115] Next, the operation of the activation function is explained.

[0116] For the features output from the convolution operation Figure 1 , Feature Map 2 and Features Figure 3 (exist Figure 2b In the example, each size is exemplarily shown as 4×4) and an activation function can be applied. Exemplarily, the output after applying the activation function can be of size 4×4.

[0117] Next, the pooling operation is described.

[0118] The features output from the activation function Figure 1 ,Feature map 2, features Figure 3 (exist Figure 2b, each size is exemplarily shown as 4×4) input to 3 nodes. The feature map output from the activation function can be received as input to perform pooling. The pooling can reduce the size or emphasize specific values ​​in the matrix. As pooling methods, there are maximum pooling, average pooling and minimum pooling. Maximum pooling can be used to collect the maximum value of the values ​​in a specific area of ​​the matrix, and average pooling can be used to obtain the average in a specific area.

[0119] exist Figure 2b The example in FIG. 4 shows that a 4×4 feature map is reduced to a 2×2 size by pooling.

[0120] Specifically, the first node receives the feature for channel 1 Figure 1 As input and after performing pooling, for example, the output is a 2×2 matrix. The second node receives feature map 2 for channel 2 as input and after performing pooling, for example, the output is a 2×2 matrix. The third node receives feature map 2 for channel 3 as input and after performing pooling, for example, the output is a 2×2 matrix. Figure 3 After taking as input and performing pooling, for example, the output is a 2×2 matrix.

[0121] Repeat the above convolution, activation function and pooling, and finally, Figure 2a As shown, the output can be fully connected. The output can be input again to the artificial neural network for image recognition. However, the present disclosure is not limited to the size of the feature map and the kernel.

[0122] The CNN described so far is also the most commonly used method in the field of computer vision among various deep neural network (DNN) methods. In particular, CNN has shown amazing performance in various research fields performing various tasks such as image classification and object detection.

[0123]

Hardware resources required for ANN operations

[0124] Figure 3 is a schematic conceptual diagram illustrating a neural processing unit according to an example of the present disclosure.

[0125] Figure 3 The illustrated neural processing unit (NPU) 100 is a processor dedicated to performing operations for an artificial neural network.

[0126] An artificial neural network is a network of artificial neurons that multiply and add weights when various inputs or stimuli enter, and transform and transmit the values ​​that further add deviations through activation functions. The artificial neural network trained in this way can be used to output inference results from input data.

[0127] The neural processing unit 100 may be a semiconductor implemented using an electric / electronic circuit. The electric / electronic circuit may include many electronic components (eg, transistors, capacitors).

[0128] For the case of an artificial neural network model based on a transformer and / or CNN, the neural processing unit 100 can screen and process matrix multiplication operations, convolution operations, etc. according to the architecture of the artificial neural network.

[0129] For example, in each layer of a convolutional neural network (CNN), the input feature map corresponding to the input data and the kernel corresponding to the weight can be a tensor or matrix consisting of multiple channels. The convolution operation of the input feature map and the kernel is performed to generate a convolution operation and a pooled output feature map in each channel. An activation function is applied to the output feature map to generate an activation map of the channel. Then, pooling can be applied to the activation map. Here, the activation map can be collectively referred to as the output feature map. In the following, for ease of explanation, the activation map is referred to as the output feature map for explanation.

[0130] However, the examples of the present disclosure are not limited to this, and outputting a feature map means that a matrix multiplication operation or a convolution operation, etc., has been applied.

[0131] In detail, the output feature map according to the example of the present disclosure should be interpreted as a general meaning. For example, the output feature map can be the result value of a matrix multiplication operation or a convolution operation. Thus, the multiple processing elements 110 can also be implemented in a deformed manner to also include a processing circuit unit for an additional algorithm. That is, it can also be configured so that some circuit units of the SFU 150 described later are included in the multiple processing elements 110.

[0132] The neural processing unit 100 may be configured to include a plurality of processing elements 110 for processing the convolutions and matrix multiplications required for the artificial neural network operations described above.

[0133] The neural processing unit 100 can be constructed to include each processing circuit optimized for the matrix multiplication operation, convolution operation, activation function operation, pooling operation, stride operation, batch normalization operation, skip connection operation, connection operation, quantization operation, cropping operation, and padding operation required for the above-mentioned artificial neural network operation.

[0134] For example, the neural processing unit 100 may be configured to include an SFU 150 that is used to process at least one of an activation function operation, a pooling operation, a stride operation, a batch normalization operation, a skip connection operation, a connection operation, a quantization operation, a cropping operation, and a padding operation in the above-mentioned algorithm.

[0135] Specifically, the neural processing unit 100 may include a plurality of processing elements (PEs) 110, an SFU 150, an NPU internal memory 120, an NPU controller 130, and an NPU interface 140. Each of the plurality of processing elements 110, the SFU 150, the NPU internal memory 120, the NPU controller 130, and the NPU interface 140 may be a semiconductor circuit connected with a plurality of transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye, and can only be identified by operation.

[0136] For example, any circuit may operate as a plurality of processing elements 110, or may operate as an NPU controller 130. The NPU controller 130 may be configured to perform the functions of a control section configured to control the artificial neural network inference operation of the neural processing unit 100.

[0137] The neural processing unit 100 may include: an NPU internal memory 120, configured to store parameters of an artificial neural network model that can be inferred from multiple processing elements 110 and SFU 150; and an NPU controller 130, including a scheduler configured to control the operation plan of multiple processing elements 110, SFU 150 and NPU internal memory 120.

[0138] The neural processing unit 100 may be configured to process the feature map in accordance with the encoding and decoding method using scalable video coding (SVC) or scalable feature-map coding (SFC). The above method is a technology for variably changing the amount of data transmission according to the effective bandwidth and signal-to-noise ratio (SNR) of the communication channel or communication bus. That is, the neural processing unit 100 may also be configured to further include an encoder and a decoder.

[0139] The plurality of processing elements 110 may perform a portion of the operations for an artificial neural network.

[0140] The SFU 150 may perform another portion of the operations for the artificial neural network.

[0141] The neural processing unit 100 may be configured to accelerate the operation of an artificial neural network model on hardware by using a plurality of processing elements 110 and SFUs 150 .

[0142] The NPU interface 140 may communicate with various components (eg, memory) connected to the neural processing unit 100 through a system bus.

[0143] The NPU controller 130 may include a scheduler configured to control operations of the plurality of processing elements 110 for inference operations of the neural processing unit 100 , operations of the SFU 150 , and a read and write order of the NPU internal memory 120 .

[0144] The scheduler within the NPU controller 130 may be configured to control the plurality of processing elements 110 , the SFU 150 , and the NPU internal memory 120 based on the data locality information or the information on the structure of the artificial neural network model.

[0145] The scheduler within the NPU controller 130 may analyze the structure of the artificial neural network model to be run in the plurality of processing elements 110 and the SFU 150, or may receive information that has been analyzed. The analyzed information may be information generated by a compiler. For example, the data of the artificial neural network that the artificial neural network model may include may include at least a portion of the node data (i.e., feature map) of each layer, the arrangement data of the layer, regional information or information about the structure, and the weight data (i.e., weight kernel) of each of the connection networks connecting the nodes of each layer. The data of the artificial neural network may be stored in an internal memory provided to the NPU controller 130 or in an NPU internal memory 120. However, it is not limited thereto, and the data of the artificial neural network may be stored in a separate cache memory or register file configured in a SoC including the NPU or the NPU.

[0146] The scheduler within the NPU controller 130 may schedule the order of operations of the artificial neural network model to be executed by the neural processing unit 100 based on the data locality information or structure information about the artificial neural network model.

[0147] The scheduler in the NPU controller 130 may receive scheduling information of the operation sequence of the artificial neural network model to be executed by the neural processing unit 100 based on the compiled data locality information or structure-related information of the artificial neural network model. For example, the scheduling information may be information generated by a compiler. The scheduling information generated by the compiler may be referred to as machine code or binary code, etc.

[0148] That is, the scheduling information applied in the NPU controller 130 may be information generated by a compiler based on data locality information or structure of the artificial neural network model.

[0149] Specifically, the compiler can effectively schedule the NPU based on the degree of understanding and reconstruction of the locality of artificial neural network data, which is an inherent characteristic of the artificial neural network model.

[0150] In detail, the compiler can effectively schedule the NPU based on the understanding of the hardware structure and performance of the neural processing unit 100.

[0151] In detail, when the artificial neural network model is compiled by the compiler to run in the neural processing unit 100, the artificial neural network data locality can be rebuilt. The artificial neural network data locality can be rebuilt according to the algorithm applied to the artificial neural network model and the operating characteristics of the processor.

[0152] In detail, the artificial neural network data locality can be reconstructed according to the way the neural processing unit 100 processes the artificial neural network model (for example, feature map tiling, stationary mode of processing elements).

[0153] In detail, the artificial neural network data locality can be reconstructed according to the number of processing elements of the neural processing unit 100 and the capacity of the internal memory.

[0154] In detail, the artificial neural network data locality can be reconstructed based on the bandwidth of the memory communicating with the neural processing unit 100.

[0155] This is because, even if the same artificial neural network model is operated and processed according to the above-mentioned factors, the neural processing unit 100 can determine the order of data required at each moment differently in units of clocks.

[0156] The compiler may determine data locality based on an operation order of a layer of an artificial neural network, unit convolution, and / or matrix product as an order of data required for operation of an artificial neural network model, and generate compiled machine code.

[0157] The scheduler may be configured to apply scheduling information included in the machine code.

[0158] The scheduler within the NPU controller 130 may obtain a memory address value storing feature maps and weight data of a layer of an artificial neural network model based on the scheduling information.

[0159] For example, the scheduler in the NPU controller 130 can obtain the feature map of the layer of the artificial neural network model stored in the memory and the memory address value storing the weight data. Therefore, the scheduler in the NPU controller 130 can obtain the feature map and weight data of the layer of the artificial neural network model to be driven from the main memory and store them in the NPU internal memory 120.

[0160] The feature map of each layer may have a corresponding respective memory address value.

[0161] Each weight data may have a corresponding respective memory address value.

[0162] The scheduler in the NPU controller 130 may receive scheduling information about the operation order of the plurality of processing elements 110 based on the data locality information or the information about the structure of the artificial neural network model (e.g., the arrangement data of the layers of the artificial neural network of the artificial neural network model, the information about the locality information or the structure). The scheduling information may be generated in the compiling step.

[0163] Since the scheduler in the NPU controller 130 operates based on information scheduling based on data regional information or structure of the artificial neural network model, it can operate differently from the scheduling concept of the general CPU. The scheduling of the general CPU considers fairness, efficiency, stability, response time, etc. and operates to achieve the best efficiency. That is, it is scheduled to perform the most processing in the same time by considering priority, operation time, etc.

[0164] Conventional CPUs use an algorithm for scheduling tasks by taking into account data such as the priority of each process and the calculation processing time.

[0165] Different from this, the scheduler within the NPU controller 130 can control the neural processing unit 100 according to the determined processing order of the neural processing unit 100 based on the data locality information or structure information about the artificial neural network model.

[0166] Further, the scheduler within the NPU controller 130 may drive the neural processing unit 100 in a determined processing order based on information about the data locality information or structure of the artificial neural network model and / or information about the data locality information or structure of the neural processing unit 100 to be used.

[0167] However, the present disclosure is not limited to data locality information or information about the structure of the neural processing unit 100 .

[0168] The scheduler within the NPU controller 130 may be configured to store data locality information or information about the structure of the artificial neural network.

[0169] That is, the scheduler within the NPU controller 130 may determine the processing order by applying at least only the data locality information or the information on the structure of the artificial neural network of the artificial neural network model.

[0170] Further, the scheduler within the NPU controller 130 may determine the processing order of the neural processing unit 100 by considering the data locality information or structure information about the artificial neural network model and the data locality information or structure information about the neural processing unit 100. In addition, the processing of the neural processing unit 100 may also be optimized according to the determined processing order.

[0171] That is, the NPU controller 130 can be configured to operate based on machine code compiled by a compiler, but in another example, the NPU controller 130 can also be configured to have an embedded compiler built in. According to the above configuration, the neural processing unit 100 can be configured to receive files in the form of frameworks of various AI software and generate machine code. For example, the frameworks of AI software include TensorFlow, PyTorch, Keras, XGBoost, mxnet, DARKNET, ONNX, etc.

[0172] The plurality of processing elements 110 refers to a configuration in which a plurality of processing elements PE1 to PE12 configured to operate feature maps and weight data of an artificial neural network are arranged. Each processing element may include a multiplication and accumulation (MAC: multiply and accumulate) operator and / or an arithmetic logic unit (ALU: Arithmetic Logic Unit) operator. However, the examples according to the present disclosure are not limited thereto.

[0173] Each processing element may be configured to selectively further include additional special function units to process additional special functions.

[0174] For example, the processing element PE may also be implemented in a modified manner to further include a batch normalization unit, an activation function unit, an interpolation unit, etc.

[0175] The SFU 150 may include a circuit portion configured to select and process activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, padding operations, etc. according to the architecture of the artificial neural network. That is, the SFU 150 may include a plurality of special function operation processing circuit units.

[0176] exist Figure 3 Although multiple processing elements are shown as an example in FIG, it is also possible to arrange arithmetic units implemented by multiple multipliers and adder trees in parallel instead of MAC inside a processing element. In this case, the multiple processing elements 110 can also be referred to as at least one processing element including multiple operators.

[0177] The plurality of processing elements 110 is configured to include a plurality of processing elements PE1 to PE12 . Figure 3 The multiple processing elements PE1 to PE12 shown are only examples for ease of explanation, and the number of the multiple processing elements PE1 to PE12 is not limited. The size or number of the multiple processing elements 110 can be determined by the number of the multiple processing elements PE1 to PE12. The size of the multiple processing elements 110 can be implemented in an N×M matrix form. Here, N and M are integers greater than 0. The multiple processing elements 110 may include N×M processing elements. That is, there may be more than one processing element.

[0178] The sizes of the plurality of processing elements 110 may be designed by taking into consideration the characteristics of the artificial neural network model run by the neural processing unit 100 .

[0179] The plurality of processing elements 110 are configured to perform functions such as addition, multiplication, accumulation, etc. required for artificial neural network operations. In other words, the plurality of processing elements 110 may be configured to perform multiplication and accumulation (MAC) operations.

[0180] Hereinafter, the first processing element PE1 among the plurality of processing elements 110 is described as an example.

[0181] Figure 4a is a schematic conceptual diagram illustrating one processing element among a plurality of processing elements that can be applied to an example of the present disclosure.

[0182] According to an example of the present disclosure, a neural processing unit 100 may include a plurality of processing elements 110, an NPU internal memory 120 configured to store an artificial neural network model that can be inferred from the plurality of processing elements 110, and an NPU controller 130 configured to control the plurality of processing elements 110 and the NPU internal memory 120. The plurality of processing elements 110 may be configured to perform MAC operations, and the plurality of processing elements 110 may be configured to quantize and output MAC operation results. However, the examples of the present disclosure are not limited thereto.

[0183] The NPU internal memory 120 may store all or part of the artificial neural network model based on the memory size and the data size of the artificial neural network model.

[0184] The first processing element PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. However, the example of the present disclosure is not limited thereto, and the plurality of processing elements 110 may also be implemented in a modified manner in consideration of the operation characteristics of the artificial neural network.

[0185] The multiplier 111 multiplies the received (N)-bit ((N)bit) data and (M)-bit ((M)bit) data. The operation value of the multiplier 111 is output as (N+M)-bit ((N+M)bit) data.

[0186] The multiplier 111 may be configured to receive a variable and a constant.

[0187] The accumulator 113 uses the adder 112 to accumulate the operation value of the multiplier 111 and the operation value of the accumulator 113 for (L) loops. Therefore, the bit width of the data of the output part and the input part of the accumulator 113 can be output as (N+M+log2(L)) bits ((N+M+log2(L)) bits). Wherein, L is an integer greater than 0.

[0188] The accumulator 113 may receive an initialization signal (initialization reset) when the accumulation is finished and initialize the data stored in the accumulator 113 to 0. However, the examples according to the present disclosure are not limited thereto.

[0189] The bit quantization unit 114 can reduce the bit width of the data output from the accumulator 113. The bit quantization unit 114 can be controlled by the NPU controller 130. The bit width of the quantized data can be output as (X) bits ((x)bit). Wherein, X is an integer greater than 0. According to the above-mentioned configuration, the plurality of processing elements 110 are configured to perform MAC operations, and the plurality of processing elements 110 have the effect of quantizing the MAC operation results and outputting them. In particular, this quantization has the effect of further reducing power consumption as the (L) loops ((L)loops) increase. In addition, if power consumption is reduced, heat generation can also be reduced. In particular, if heat generation is reduced, the possibility of erroneous operation caused by the high temperature of the neural processing unit 100 can be reduced.

[0190] The output data (X) bits ((x)bit) of the bit quantization unit 114 can become the node data of the next layer or the input data of the convolution. If the artificial neural network model is quantized, the bit quantization unit 114 can be configured to receive quantized information from the artificial neural network model. However, not limited to this, the NPU controller 130 can also be configured to extract quantized information by analyzing the artificial neural network model. Therefore, the output data (X) bits ((x)bit) can be converted into a quantized bit width and output to correspond to the quantized data size. The output data (X) bits ((x)bit) of the bit quantization unit 114 can be stored in the NPU internal memory 120 with a quantized bit width.

[0191] According to an example of the present disclosure, the plurality of processing elements 110 of the neural processing unit 100 include a multiplier 111, an adder 112, and an accumulator 113. The bit quantization unit 114 can make a choice based on whether quantization is applied. In another example, the bit quantization unit can also be configured to be included in the SFU 150.

[0192] Figure 4b is a schematic conceptual diagram illustrating an SFU of an example that can be applied to the present disclosure.

[0193] Reference Figure 4b The SFU 150 includes various functional units. Each functional unit can be selectively operated. Each functional unit can be selectively turned on or off. That is, each functional unit can be set.

[0194] In other words, the SFU 150 may include various circuit units required for artificial neural network reasoning operations.

[0195] For example, the circuit units of SFU 150 may include functional units for skip-connection operations, functional units for activation function operations, functional units for pooling operations, functional units for quantization operations, functional units for non-maximum suppression (NMS) operations, functional units for integer and floating-point conversion (INT to FP32) operations, functional units for batch-normalization operations, functional units for interpolation operations, functional units for concatenation operations, and functional units for bias operations, etc.

[0196] The functional units of the SFU 150 can be selectively turned on or off by the data locality information of the artificial neural network model. When performing an operation for a specific layer, the data locality information of the artificial neural network model may include control information related to turning on or off the functional unit.

[0197] The activated units in the functional units of the SFU 150 may be turned on. As described above, by selectively disconnecting some of the functional units of the SFU 150, the power consumption of the neural processing unit 100 may be reduced. In addition, in order to disconnect some of the functional units, power gating may be utilized. Alternatively, in order to disconnect some of the functional units, clock gating may also be performed.

[0198] Figure 5 It is shown Figure 3 An example diagram of a variation of neural processing unit 100 is shown.

[0199] Figure 5 The neural processing unit 100 shown is Figure 3 Compared with the neural processing unit 100 exemplarily shown in , except for the multiple processing elements 110, they are essentially the same. Therefore, the following is only for the convenience of explanation and repeated explanation may be omitted.

[0200] In addition to a plurality of processing elements PE1 to PE12, Figure 5 The plurality of processing elements 110 exemplarily shown in FIG. 1 may also include each register file RF1 to RF12 corresponding to each processing element PE1 to PE12 .

[0201] Figure 5The plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 shown are merely examples for convenience of description, and the number of the plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 is not limited.

[0202] The size or number of the plurality of processing elements 110 may be determined by the number of the plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12. The size of the plurality of processing elements 110 and the plurality of register files RF1 to RF12 may be implemented in an N×M matrix form, where N and M are integers greater than 0.

[0203] The array size of the plurality of processing elements 110 may be designed in consideration of the characteristics of the artificial neural network model run by the neural processing unit 100. Specifically, the memory size of the register file may be determined by considering the data size of the artificial neural network model to be run, the required operation speed, the required power consumption, etc.

[0204] The register files (RF1 to RF12) of the neural processing unit 100 are static memory units directly connected to the processing elements (PE1 to PE12). For example, the register files (RF1 to RF12) may be composed of triggers and / or latches, etc. The register files RF1 to RF12 may be configured to store the MAC operation values ​​of the corresponding processing elements (PE1 to PE12). The register files RF1 to RF12 may be configured to provide weight data and / or node data to the NPU internal memory 120 or to receive weight data and / or node data from the NPU internal memory 120.

[0205] The register files RF1 to RF12 may be configured to function as temporary memories of accumulators when performing MAC operations.

[0206] [Technical difficulties discovered by the inventors of this disclosure]

[0207] Depending on the purpose of the artificial intelligence service, there may be various types of artificial neural network (ANN) models. For example, in the case where the input data is an image or video, for the artificial intelligence service, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. within the image / video may be used.

[0208] Usually, the amount of computation in each layer of an artificial neural network model can be different. Figure 6a and Figure 6b This is explained.

[0209] Figure 6a is an example diagram showing the size of data at each layer within an exemplary artificial neural network model. Figure 6b It is shown Figure 6a An exemplary table showing the data size, etc., of each layer in the exemplary artificial neural network model shown.

[0210] Figure 6a The exemplary artificial neural network model shown is denoted as Mobilenet V1. Figure 6a The horizontal axis shown sequentially represents layers in the exemplary artificial neural network model, and the vertical axis represents the size of data.

[0211] Reference Figure 6a As shown in layer 1, the size of the output feature map OFMAP_SIZE is larger than the size of the input feature map IFMAP_SIZE.

[0212] The output feature map of layer 1 is passed to layer 2, thereby becoming the input feature map of layer 2. If the operation of layer 2 is completed, the output feature map will be output. The output feature map of layer 2 is passed to layer 3 again, thereby becoming the input feature map of layer 3.

[0213] In this way, the size of the input data input to each layer can be different from the size of the output feature map output from each layer. As a result, the amount of calculation in any layer may be small, but the amount of calculation in another layer may be very large. As described above, since the amount of calculation in each layer changes greatly, the problem of difficulty in power control occurs.

[0214] Due to the number of processing elements of the neural processing unit and the capacity limitation of the NPU internal memory 120, each layer can be divided into multiple operation steps for processing. Thus, the neural processing unit 100 can divide each layer into multiple blocks and schedule them in a manner of processing multiple operation steps. The scheduling can be performed by a compiler. For example, a layer can be divided into 4 blocks. Each block can be processed in sequence by the neural processing unit. The compiled artificial neural network model can store the operation order information determined when it is compiled in the scheduler. At this time, information on PE utilization can be provided in each operation step.

[0215] The amount of computation of each layer can be known through MAC. It can be seen that the amount of computation of each layer differs by up to 227 times.

[0216] The neural processing unit 100 may determine the number of processing elements used among the plurality of processing elements 110 in proportion to the MAC operation amount in each operation step of each layer. Also, the power consumption amount may increase in proportion to the number of processing elements used.

[0217] Here, if the number of utilized processing elements and the total number of processing elements are known, the utilization rate (%) of the neural processing unit 100 can be calculated.

[0218] Therefore, the PE utilization can be calculated in each operation step of each layer of the artificial neural network model processed by the neural processing unit 100. In addition, unless the structure of the artificial neural network model processed by the neural processing unit 100 changes, the PE utilization in each operation step can be repeatedly and accurately predicted. That is, the neural processing unit 100 can repeatedly infer a specific artificial neural network model. In this case, the inventors of the present disclosure recognize that the neural processing unit 100 can reuse the same weight parameters and the same network layer structure.

[0219] In addition, as the difference in the amount of MAC operations of adjacent layers increases, the deviation of the peak power between adjacent layers may increase. And, as the deviation of the peak power of adjacent layers increases, the supply voltage VDD may further oscillate. At this point, the inventors of the present disclosure recognize that the driving frequency can be changed by considering the stability of the supply voltage VDD.

[0220] In particular, the inventors of the present disclosure have recognized that if the amount of computation in a computation step of a specific layer increases significantly, instantaneous power consumption will increase, and thus a problem of reduced system stability may occur.

[0221] For example, multiple processing elements can be started simultaneously for the operation of the operation steps of a specific layer. In order to drive each processing element, a predetermined power supply is required. If a considerable number of processing elements are driven at the same time, the required power will surge instantly. In the case where the neural processing unit is designed to be dedicated to low-power operation, its power supply capacity may be relatively insufficient than the neural processing unit for server use. Therefore, this neural processing unit for edge devices may be more susceptible to instantaneous power supply problems. In the case of a surge in power supply, the supply voltage VDD may oscillate. In particular, when the supply voltage VDD drops below the threshold voltage of the transistor, the data stored in the transistor may be lost. In other words, when the supply voltage VDD is reduced, the operating speed of the transistor is reduced, resulting in setup / hold violations, which may cause misoperation. In particular, this phenomenon may become more and more serious as the process of semiconductor foundry is reduced to 3nm, 4nm, 5nm, and 7nm.

[0222] As another example, the inventors of the present disclosure recognize that the operations used for artificial neural networks, i.e., PE 110 performing, for example, addition, multiplication, accumulation, etc., use a large amount of power consumption in an instant, and thus may not be able to supply sufficient power to other components within the neural processing unit (e.g., internal memory 120). Specifically, the inventors of the present disclosure recognize that if sufficient power cannot be provided to the internal memory 120, there may be a problem that the possibility of the stored data bits being damaged (compromised) cannot be ruled out.

[0223] Furthermore, the inventors of the present disclosure have recognized that, when the utilization rate of PE in a specific operation step is low, the voltage stability of the neural processing unit can be guaranteed even if the driving frequency of the neural processing unit is increased.

[0224] [Disclosure of this patent]

[0225] The inventors of the present disclosure have recognized that the above-mentioned problem occurs because the operation of the artificial neural network model is performed on the hardware based on a clock signal.

[0226] Therefore, the inventor of this patent invented a technology that divides the multiple PEs in the NPU into a first group of PEs and a second group of PEs, and operates them according to different clock signals, thereby reducing peak power.

[0227] Hereinafter, various examples of implementing the technology invented by the inventors of the present disclosure will be described with reference to the drawings.

[0228] Figure 7a is an exemplary diagram showing the structure of an NPU according to the first example of the present disclosure.

[0229] exist Figure 7a , a situation in which the NPU 100 includes a first group of PEs 110 - 1 , a second group of PEs 110 - 2 , an internal memory 120 , an SFU 150 , a clock source 170 , and a clock distributor 190 is shown.

[0230] In several examples, such as Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0231] Figure 7a The first group of PE 110-1 and the second group of PE 110-2 shown in FIG. Figure 3 or Figure 5 The plurality of PEs 110 are shown as being partitioned. Figure 7a10-1 and 110-2. The example in FIG. 10-2 shows that the number of PEs belonging to each group is 8, but it is obvious that the number of PEs belonging to each group can be changed. In addition, the number of PEs belonging to each group can be changed semi-statically or dynamically. For example, it can be set that the PE 110-1 of the first group includes 10 PEs, and the PE 110-2 of the second group includes 6 PEs. This change can be performed according to the control of the NPU controller 130.

[0232] Each PE or each group of PEs may also be referred to as an NPU core, an NPU engine, an NPU thread, etc. The NPU core, the NPU engine, the NPU thread, etc. may be a semiconductor circuit having many transistors connected thereto.

[0233] The first group of PE 110-1, the second group of PE 110-2, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150, the clock source 170, and the clock distributor 190 may be semiconductor circuits connected with many transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye and can only be identified by operation.

[0234] For example, the PE 110 - 1 of the first group and the PE 110 - 2 of the second group may be referred to as a first circuit.

[0235] For example, the internal memory 120 may be referred to as a second circuit.

[0236] For example, SFU 150 may be referred to as a third circuit.

[0237] For example, the clock source 170 may be referred to as a fourth circuit configured to output at least one clock signal.

[0238] For example, the clock distributor 190 may be referred to as a fifth circuit configured to convert the at least one clock signal and output it as a plurality of clock signals. The clock distributor 190 may calibrate or adjust the phase of a clock signal output by the clock source 170 and output a plurality of clock signals.

[0239] However, examples of the present disclosure are not limited thereto, and any circuit may operate as a plurality of PEs 110 , or may operate as the NPU controller 130 .

[0240] If the clock source 170 generates and outputs an original clock signal, the clock distributor 190 may directly (bypass) output the original clock signal, or may delay or shift the phase of the original clock signal and output it.

[0241] For example, the clock distributor 190 may directly (bypass) transmit the original clock signal from the clock source 170 to the PE 110-1 of the first group. Alternatively, the clock distributor 190 may transmit the first delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-1 of the first group. In addition, the clock distributor 190 may transmit the second delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-2 of the second group. The internal memory 120 may also be configured to directly (bypass) receive the original clock signal from the clock distributor 190.

[0242] In addition, although Figure 7a Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0243] in addition, Figure 7a The internal memory 120 and the SFU 150 shown may also operate according to a clock signal (eg, an original clock signal or a phase-delayed clock signal) output from the clock distributor 190 .

[0244] On the other hand, if Figure 7a As shown, the PE 110-1 of the first group and the PE 110-2 of the second group perform operations on any layer of the artificial neural network model, and the PE 110-1 of the first group and the PE 110-2 of the second group can respectively output output feature maps according to the clock signals respectively received from the clock distributor 190. Specifically, the PE 110-1 of the first group can output a first output feature map according to any clock signal (e.g., the original clock signal or the first delayed clock signal) received from the clock distributor 190. Similarly, the PE 110-2 of the second group can output a second output feature map according to any clock signal (e.g., the second delayed clock signal) received from the clock distributor 190.

[0245] When the first output feature map output by the first group of PE 110-1 and the second output feature map output by the second group of PE 110-2 are independent of each other and not dependent on each other, the SFU 150 may first process the first received output feature map in a first-in-first-out (FIFO) manner.

[0246] In addition, if Figure 7aThe PE 110-1 of the first group and the PE 110-2 of the second group perform operations on any layer of the artificial neural network model, and the PE 110-1 of the first group and the PE 110-2 of the second group can match one of the first part and the second part of the clock to output the operation result (i.e., output feature map). To this end, at least one of the PE 110-1 of the first group and the PE 110-2 of the second group may also include a delay buffer (e.g., a shift register). The delay buffer (e.g., a shift register) may perform a delay so that the output matches the later part of the first part and the second part of the clock.

[0247] For example, the output port of the PE 110 - 2 of the second group may be connected to a delay buffer (eg, a shift register).

[0248] Alternatively, Figure 7a The PE 110-1 of the first group shown can match the first part of the arbitrary clock to output the first output feature map, and the PE 110-2 of the second group can match the second part of the arbitrary clock to output the second output feature map. In this case, the SFU 150 may include a delay buffer (e.g., a shift register). The SFU 150 may perform a delay to enable the output feature map received by matching the earlier part of the first and second parts of the arbitrary clock to be delayed so that it can be synchronized with the output feature map to be received by matching the later part of the first and second parts of the arbitrary clock.

[0249] Alternatively, in a case where a first output feature graph output by PE 110-1 of the first group and a second output feature graph output by PE 110-2 of the second group are independent of each other and not dependent on each other, the SFU 150 may first process the output feature graph received by matching the temporally earlier part of the first and second parts of the clock.

[0250] That is, a first output from the first group of PEs 110 - 1 and a second output from the second group of PEs 110 - 2 may be delivered with reference to the first portion of the arbitrary clock.

[0251] The reference phase for the second group of PEs 110 - 2 may be shifted from the second portion of the clock to the first portion.

[0252] exist Figure 7a Each element within the NPU 100 shown may be input with a supply voltage VDD.

[0253] exist Figure 7a2 shows a case where a common supply voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0254] In some examples, it may also be configured such that each component in the NPU 100 inputs an independent or separate supply voltage VDD.

[0255] In another example, it may also be configured such that each element in the NPU 100 receives an independent or separate supply voltage VDD.

[0256] In yet another example, a second supply voltage different from the first supply voltage may be input to a specific element in the NPU 100 .

[0257] As described above, in one example of the present disclosure, Figure 7a As shown, Figure 3 or Figure 5 After the multiple PEs 110 are divided into a first group of PEs 110-1 and a second group of PEs 110-2, the first group of PEs 110-1 and the second group of PEs 110-2 are operated in a dispersed manner according to different clock signals, thereby reducing the peak power of the multiple PE groups 110-1 and 110-2.

[0258] Furthermore, in an example of the present disclosure, in order to provide a plurality of clock signals having different phases from each other, a plurality of clock sources 170 may be provided without simply adding a phase converter, thereby reducing manufacturing costs.

[0259] Figure 7b is an exemplary diagram showing a structure of an NPU according to a first modification example of the first example of the present disclosure.

[0260] exist Figure 7b , a situation in which the NPU 100 includes a first group of PEs 110 - 1 , a second group of PEs 110 - 2 , a third group of PEs 110 - 3 , an internal memory 120 , an SFU 150 , a clock source 170 , and a clock distributor 190 is shown.

[0261] In some examples, such as Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0262] Figure 7b The PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group shown can be understood as Figure 3 or Figure 5 The plurality of PEs 110 are shown as being partitioned. Figure 7b10-1 includes 10 PEs, the second group PE 110-2 includes 8 PEs, and the third group PE 110-3 includes 6 PEs. Such a change can be performed according to the control of the NPU controller 130.

[0263] Each PE or each group of PEs may also be referred to as an NPU core, an NPU engine, an NPU thread, etc. The NPU core, the NPU engine, the NPU thread, etc. may be a semiconductor circuit having many transistors connected thereto.

[0264] The first group of PE 110-1, the second group of PE 110-2, the third group of PE 110-3, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150, the clock source 170, and the clock distributor 190 may be semiconductor circuits connected with many transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye and can only be identified by operation.

[0265] For example, the PE 110 - 1 of the first group, the PE 110 - 2 of the second group, and the PE 110 - 3 of the third group may be referred to as a first circuit.

[0266] For example, the internal memory 120 may be referred to as a second circuit.

[0267] For example, SFU 150 may be referred to as a third circuit.

[0268] For example, the clock source 170 may be referred to as a fourth circuit configured to output at least one clock signal.

[0269] For example, the clock distributor 190 may be referred to as a fifth circuit configured to convert the at least one clock signal and output it as a plurality of clock signals. The clock distributor 190 may calibrate or adjust the phase of a clock signal output by the clock source 170 and output a plurality of clock signals.

[0270] However, examples of the present disclosure are not limited thereto, and any circuit may operate as a plurality of PEs 110 , or may operate as the NPU controller 130 .

[0271] If the clock source 170 generates and outputs an original clock signal, the clock distributor 190 may directly (bypass) output the original clock signal, or may delay or shift the phase of the original clock signal and output it.

[0272] For example, the clock distributor 190 may directly (bypass) transmit the original clock signal from the clock source 170 to the PE 110-1 of the first group. Alternatively, the clock distributor 190 may transmit a first delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-1 of the first group. In addition, the clock distributor 190 may transmit a second delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-2 of the second group. In addition, the clock distributor 190 may transmit a third delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-3 of the third group. The internal memory 120 may also be configured to directly (bypass) receive the original clock signal from the clock distributor 190.

[0273] In addition, although Figure 7b Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0274] in addition, Figure 7b The internal memory 120 and the SFU 150 shown may also operate according to a clock signal (eg, an original clock signal or a phase-delayed clock signal) output from the clock distributor 190 .

[0275] On the other hand, if Figure 7bAs shown, the PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group perform operations on any layer of the artificial neural network model, and the PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group can output output feature maps respectively according to the clock signals respectively received from the clock distributor 190. Specifically, the PE 110-1 of the first group can output the first output feature map according to any clock signal (e.g., the original clock signal or the first delayed clock signal) received from the clock distributor 190. Similarly, the PE 110-2 of the second group can output the second output feature map according to any clock signal (e.g., the second delayed clock signal) received from the clock distributor 190. In addition, the PE 110-3 of the third group can output the third output feature map according to any clock signal (e.g., the third delayed clock signal) received from the clock distributor 190.

[0276] When the first output feature map output by the first group of PE 110-1, the second output feature map output by the second group of PE 110-2, and the third output feature map output by the third group of PE 110-3 are independent of each other and not dependent on each other, the SFU 150 can first process the first received output feature map in a first-in-first-out (FIFO) manner.

[0277] In addition, if Figure 7b As shown, the PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group perform operations on any layer of the artificial neural network model, and the PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group can match one of the first part, the second part, and the third part of the clock to output the operation result (i.e., output feature map). To this end, at least one of the PE 110-1 of the first group, the PE 110-2 of the second group, and the PE 110-3 of the third group can also include a delay buffer (e.g., a shift register). The delay buffer (e.g., a shift register) can perform a delay so that the output matches the execution of the later part of the first part, the second part, and the third part of the clock.

[0278] For example, the output port of the second group of PE 110-2 may be connected to a delay buffer (eg, a shift register). Similarly, the output port of the third group of PE 110-3 may be connected to a delay buffer (eg, a shift register).

[0279] Alternatively, Figure 7b The PE 110-1 of the first group shown can match the first part of the arbitrary clock to output the first output feature graph, and the PE 110-2 of the second group can match the second part of the arbitrary clock to output the second output feature graph. In addition, the PE 110-3 of the third group can match the third part of the arbitrary clock to output the third output feature graph. In this case, the SFU 150 may include a delay buffer (e.g., a shift register). The SFU 150 may perform a delay to enable the output feature graph received by matching the earlier part of the first part, the second part, and the third part of the arbitrary clock to be delayed so that it can be synchronized with the output feature graph to be received by matching the later part of the first part, the second part, and the third part of the arbitrary clock.

[0280] Alternatively, in the case where the first output feature graph output by the PE 110-1 of the first group, the second output feature graph output by the PE 110-2 of the second group, and the third output feature graph output by the PE 110-3 of the third group are independent of each other and not dependent on each other, the SFU 150 may first process the output feature graph received by matching the temporally earlier part of the first part, the second part, and the third part of the clock.

[0281] That is, a first output from the first group of PE 110 - 1 , a second output from the second group of PE 110 - 2 , and a third output from the third group of PE 110 - 3 may be delivered with reference to the first portion of the arbitrary clock.

[0282] The reference phase for the second group of PEs 110-2 may be switched from the second portion to the first portion of the clock. Likewise, the reference phase for the third group of PEs 110-3 may be switched from the third portion to the first portion of the clock.

[0283] exist Figure 7b Each element within the NPU 100 shown may be input with a supply voltage VDD.

[0284] exist Figure 7b 2 shows a case where a common supply voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0285] In some examples, it may also be configured such that each component in the NPU 100 inputs an independent or separate supply voltage VDD.

[0286] In another example, it may also be configured such that each element in the NPU 100 receives an independent or separate supply voltage VDD.

[0287] In yet another example, a second supply voltage different from the first supply voltage may be input to a specific element in the NPU 100 .

[0288] As described above, in one example of the present disclosure, Figure 7b As shown, Figure 3 or Figure 5 After the multiple PEs 110 are divided into a first group of PEs 110-1, a second group of PEs 110-2, and a third group of PEs 110-3, these groups are dispersedly operated according to different clock signals, thereby reducing peak power.

[0289] Furthermore, in an example of the present disclosure, in order to provide a plurality of clock signals having different phases from each other, a clock distributor 190 may be simply added without providing a plurality of clock sources 170 , thereby reducing manufacturing costs.

[0290] Figure 7c is an exemplary diagram showing a structure of an NPU according to a second modification example of the first example of the present disclosure.

[0291] exist Figure 7c , a situation in which the NPU 100 includes a first group of PEs 110 - 1 , a second group of PEs 110 - 2 , a third group of PEs 110 - 3 , a fourth group of PEs 110 - 4 , an internal memory 120 , an SFU 150 , a clock source 170 , and a clock distributor 190 is shown.

[0292] In some examples, such as Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0293] Figure 7b The PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group shown can be understood as Figure 3 or Figure 5 The plurality of PEs 110 are shown as being partitioned. Figure 7c10-1 includes 14 PEs, the second group PE 110-2 includes 8 PEs, the third group PE 110-3 includes 6 PEs, and the fourth group PE 110-4 includes 4 PEs. Such a change can be performed according to the control of the NPU controller 130.

[0294] Each PE or each group of PEs may also be referred to as an NPU core, an NPU engine, an NPU thread, etc. The NPU core, the NPU engine, the NPU thread, etc. may be a semiconductor circuit having many transistors connected thereto.

[0295] The first group of PE 110-1, the second group of PE 110-2, the third group of PE 110-3, the fourth group of PE 110-4, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150, the clock source 170, and the clock distributor 190 may be semiconductor circuits connected with many transistors, respectively. Therefore, some of them may be difficult to identify and distinguish with naked eyes, and can only be identified by operation.

[0296] For example, the PE 110 - 1 of the first group, the PE 110 - 2 of the second group, the PE 110 - 3 of the third group, and the PE 110 - 4 of the fourth group may be referred to as a first circuit.

[0297] For example, the internal memory 120 may be referred to as a second circuit.

[0298] For example, SFU 150 may be referred to as a third circuit.

[0299] For example, the clock source 170 may be referred to as a fourth circuit configured to output at least one clock signal.

[0300] For example, the clock distributor 190 may be referred to as a fifth circuit that converts the at least one clock signal and outputs a plurality of clock signals. The clock distributor 190 may calibrate or adjust the phase of a clock signal output by the clock source 170 and output a plurality of clock signals.

[0301] However, examples of the present disclosure are not limited thereto, and any circuit may operate as a plurality of PEs 110 , or may operate as the NPU controller 130 .

[0302] If the clock source 170 generates and outputs an original clock signal, the clock distributor 190 may directly (bypass) output the original clock signal, or may delay or shift the phase of the original clock signal and output it.

[0303] For example, the clock distributor 190 may directly (bypass) transmit the original clock signal from the clock source 170 to the PE 110-1 of the first group. Alternatively, the clock distributor 190 may transmit a first delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-1 of the first group. In addition, the clock distributor 190 may transmit a second delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-2 of the second group. In addition, the clock distributor 190 may transmit a third delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-3 of the third group. In addition, the clock distributor 190 may transmit a fourth delayed clock signal that delays or shifts the phase of the original clock signal from the clock source 170 to the PE 110-4 of the fourth group. The internal memory 120 may also be configured to directly receive the original clock signal from the clock distributor 190 (bypass).

[0304] In addition, although Figure 7c Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0305] in addition, Figure 7c The internal memory 120 and the SFU 150 shown may also operate according to a clock signal (eg, an original clock signal or a phase-delayed clock signal) output from the clock distributor 190 .

[0306] On the other hand, if Figure 7cAs shown, the PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group perform operations on any layer of the artificial neural network model, and the PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group can output output feature maps respectively according to the clock signals respectively received from the clock distributor 190. Specifically, the PE 110-1 of the first group can output a first output feature map according to any clock signal (e.g., the original clock signal or the first delayed clock signal) received from the clock distributor 190. Similarly, the PE 110-2 of the second group can output a second output feature map according to any clock signal (e.g., the second delayed clock signal) received from the clock distributor 190. In addition, the PE 110-3 of the third group can output a third output feature map according to any clock signal (e.g., the third delayed clock signal) received from the clock distributor 190. In addition, the PE 110 - 4 of the fourth group may output a fourth output characteristic pattern according to any clock signal (eg, the fourth delayed clock signal) received from the clock distributor 190 .

[0307] When the first output feature map output by the first group's PE 110-1, the second output feature map output by the second group's PE 110-2, the third output feature map output by the third group's PE 110-3, and the fourth output feature map output by the fourth group's PE 110-4 are independent of each other and not dependent on each other, the SFU 150 may first process the first received output feature map in a first-in-first-out (FIFO) manner.

[0308] In addition, if Figure 7bAs shown, the PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group perform operations on any layer of the artificial neural network model, then the PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group can match one of the first part, the second part, the third part, and the fourth part of the clock to output the operation result (i.e., output feature map). To this end, at least one of the PE 110-1 of the first group, the PE 110-2 of the second group, the PE 110-3 of the third group, and the PE 110-4 of the fourth group can also include a delay buffer (e.g., a shift register). The delay buffer (e.g., a shift register) can delay time so that the output matches the execution of the later part of the first part, the second part, the third part, and the fourth part of the clock.

[0309] For example, the output port of the second group of PE 110-2 may be connected to a delay buffer (e.g., a shift register). Similarly, the output port of the third group of PE 110-3 may be connected to a delay buffer (e.g., a shift register). Furthermore, the output port of the fourth group of PE 110-4 may also be connected to a delay buffer (e.g., a shift register).

[0310] Alternatively, Figure 7c The PE 110-1 of the first group shown can match the first part of the arbitrary clock to output the first output characteristic graph, and the PE 110-2 of the second group can match the second part of the arbitrary clock to output the second output characteristic graph. In addition, the PE 110-3 of the third group can match the third part of the arbitrary clock to output the third output characteristic graph. And, the PE 110-4 of the fourth group can match the fourth part of the arbitrary clock to output the fourth output characteristic graph. In this case, the SFU 150 may include a delay buffer (e.g., a shift register). The SFU 150 may perform a delay to enable the output characteristic graph received by matching the earlier part in time among the first part, the second part, the third part, and the fourth part of the arbitrary clock to be delayed so that it can be synchronized with the output characteristic graph received by matching the later part in time among the first part, the second part, the third part, and the fourth part of the arbitrary clock.

[0311] Alternatively, in the case where the first output characteristic graph output by PE 110-1 of the first group, the second output characteristic graph output by PE 110-2 of the second group, the third output characteristic graph output by PE 110-3 of the third group, and the fourth output characteristic graph output by PE 110-4 of the fourth group are independent of each other and not dependent on each other, the SFU 150 may first process the output characteristic graph received by matching the earlier part in time among the first part, the second part, the third part, and the fourth part of the clock.

[0312] That is, a first output from PE 110-1 of the first group, a second output from PE 110-2 of the second group, a third output from PE 110-3 of the third group, and a fourth output from PE 110-4 of the fourth group may be delivered with reference to the first portion of the arbitrary clock.

[0313] The reference phase for the PE 110-2 of the second group may be converted from the second portion to the first portion of the clock. Similarly, the reference phase for the PE 110-3 of the third group may be converted from the third portion to the first portion of the clock. And, the reference phase for the PE 110-4 of the fourth group may be converted from the fourth portion to the first portion of the clock.

[0314] exist Figure 7c Each element within the NPU 100 shown may be input with a supply voltage VDD.

[0315] exist Figure 7c 2 shows a case where a common supply voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0316] In some examples, it may also be configured such that each component in the NPU 100 inputs an independent or separate supply voltage VDD.

[0317] In another example, it may also be configured such that each element in the NPU 100 receives an independent or separate supply voltage VDD.

[0318] In yet another example, a second supply voltage different from the first supply voltage may be input to a specific element in the NPU 100 .

[0319] As described above, in one example of the present disclosure, Figure 7c As shown, Figure 3 or Figure 5After the multiple PEs 110 are divided into a first group of PEs 110-1, a second group of PEs 110-2, a third group of PEs 110-3, and a fourth group of PEs 110-4, these groups are dispersedly operated according to different clock signals, thereby reducing peak power.

[0320] Furthermore, in an example of the present disclosure, in order to provide a plurality of clock signals having different phases from each other, a clock distributor 190 may be simply added without providing a plurality of clock sources 170 , thereby reducing manufacturing costs.

[0321] Figure 8a is an exemplary diagram showing a structure of an NPU according to a second example of the present disclosure.

[0322] exist Figure 8a FIG. 4 shows a case where the NPU 100 includes a first group of PEs 110-1, a second group of PEs 110-2, an internal memory 120, and an SFU 150. However, in addition to this, Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0323] The first group of PE 110-1, the second group of PE 110-2, the internal memory 120, the NPU controller 130, the NPU interface 140 and the SFU 150 may be semiconductor circuits connected with many transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye and can only be identified by operation.

[0324] Reference Figure 8a , showing a situation where the clock source 170 and the clock distributor 190 are located outside the NPU 100 instead of inside. Figure 7a The situation shown is the same and therefore will not be treated separately Figure 8a Explain and directly quote the reference Figure 7a The content of the description.

[0325] Figure 8b is an exemplary diagram showing a structure of an NPU according to a first modification example of the second example of the present disclosure.

[0326] exist Figure 8b FIG. 4 shows a case where the NPU 100 includes a first group of PEs 110-1, a second group of PEs 110-2, a third group of PEs 110-3, an internal memory 120, and an SFU 150. However, in addition to this, Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0327] The first group of PE 110-1, the second group of PE 110-2, the third group of PE 110-3, the internal memory 120, the NPU controller 130, the NPU interface 140, and the SFU 150 may be semiconductor circuits connected with many transistors, respectively. Therefore, some of them may be difficult to identify and distinguish with the naked eye, and can only be identified by operation.

[0328] Reference Figure 8b , showing a situation where the clock source 170 and the clock distributor 190 are located outside the NPU 100 instead of inside. Figure 7b The situation shown is the same and therefore will not be treated separately Figure 8b Explain and directly quote references Figure 7b The content of the description.

[0329] Figure 8c : is an exemplary diagram showing the structure of an NPU according to a second modification example of the second example of the present disclosure.

[0330] exist Figure 8c 1 shows a case where the NPU 100 includes a first group of PEs 110-1, a second group of PEs 110-2, a third group of PEs 110-3, a fourth group of PEs 110-4, an internal memory 120, and an SFU 150. However, in addition to this, Figure 3 or Figure 5 As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140 .

[0331] The first group of PE 110-1, the second group of PE 110-2, the third group of PE 110-3, the fourth group of PE 110-4, the internal memory 120, the NPU controller 130, the NPU interface 140, and the SFU 150 may be semiconductor circuits connected with many transistors, respectively. Therefore, some of them may be difficult to identify and distinguish with naked eyes, and can only be identified by operation.

[0332] Reference Figure 8c , the clock source 170 and the clock distributor 190 are located outside the NPU 100 instead of inside. Figure 7c The situation shown is the same and therefore will not be treated separately Figure 8c Explain and directly quote references Figure 7c The content of the description.

[0333] So far, the NPU has been described, but the NPU may also be implemented in the form of a SoC.

[0334] Figure 9a is an exemplary diagram showing a structure of a SoC according to a third example of the present disclosure.

[0335] Reference Figure 9a , the exemplary SoC 1000 may include a plurality of NPUs, a plurality of central processing units (CPUs), and a plurality of memories. For example, the plurality of NPUs may include a first NPU 100-1 and a second NPU 100-2. And, for example, the plurality of CPUs may include a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.

[0336] like Figures 8a to 8c As shown, each of the plurality of NPUs (eg, the first NPU 100 - 1 and the second NPU 100 - 2 ) may include a plurality of groups of PEs.

[0337] exist Figure 9a , the number of the multiple NPUs, the multiple CPUs, and the multiple memories is shown as 2 respectively, but the present invention is not limited thereto, and the number may be variously transformed into 4, 6, 8, etc.

[0338] The exemplary SoC 1000 may include a memory controller 350 , a clock source 170 , the clock distributor 190 , a system bus 500 , and an input / output (I / O) interface 800 .

[0339] The system bus 500 may be implemented by an electrically conductive pattern formed on a semiconductor die. The system bus is capable of high-speed communication. For example, the multiple NPUs, the multiple CPUs, the multiple memories, and the memory controller 350 may communicate with each other via the system bus 500.

[0340] The multiple NPUs and the multiple CPUs make requests to the memory controller 350 through the system bus 500. Thus, the memory controller 350 can read data from at least one of the multiple memories or write data to at least one of the multiple memories.

[0341] Figure 9a The clock source 170 and the clock distributor 190 are shown in FIG. Figures 7a to 8c Same as shown.

[0342] Therefore, the clock distributor 190 may provide the original clock signal from the clock source 170 to the first NPU 100-1, and may provide a first phase-delayed clock signal whose phase is delayed by the original clock signal to the second NPU 100-2. Alternatively, the clock distributor 190 may provide a first phase-delayed clock signal whose phase is delayed by the original clock signal provided from the clock source 170 to the first NPU 100-1, and may provide a second phase-delayed clock signal to the second NPU 100-2.

[0343] Figure 9b is an exemplary diagram showing the structure of a SoC according to a first modification example of the third example of the present disclosure.

[0344] Reference Figure 9b , the exemplary SoC 1000 may include a plurality of NPUs, a plurality of central processing units (CPUs), and a plurality of memories. For example, the plurality of NPUs may include a first NPU 100-1, a second NPU 100-2, and a third NPU 100-3. And, for example, the plurality of CPUs may include a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.

[0345] Each of the plurality of NPUs (eg, 100-1, 100-2, 100-3) may include Figure 3 or Figure 5 Shown are a plurality of PEs (PEs) 110 , an NPU internal memory 120 , an NPU controller 130 , an NPU interface 140 , and an SFU 150 .

[0346] If the clock source 170 generates and outputs an original clock signal, the clock distributor 190 may directly (bypass) output the original clock signal, or may output a plurality of phase-delayed signals generated by delaying or shifting the phase of the original clock signal.

[0347] For example, the clock distributor 190 may provide the original clock signal from the clock source 170 to the first NPU 100-1, and may provide a first phase-delayed clock signal whose phase is delayed from the original clock signal to the second NPU 100-2. Furthermore, the clock distributor 190 may provide a second phase-delayed clock signal to the third NPU 100-3.

[0348] Alternatively, the clock distributor 190 may provide a first phase-delayed clock signal that delays the phase of the original clock signal from the clock source 170 to the first NPU 100-1, and may provide a second phase-delayed clock signal to the second NPU 100-2. Furthermore, the clock distributor 190 may provide a third phase-delayed clock signal to the third NPU 100-3.

[0349] Fig.9c is an exemplary diagram showing the structure of a SoC according to a second modification example of the third example of the present disclosure.

[0350] Reference Fig.9c , the exemplary SoC 1000 may include a plurality of NPUs, a plurality of central processing units (CPUs), and a plurality of memories. For example, the plurality of NPUs may include a first NPU 100-1, a second NPU 100-2, a third NPU 100-3, and a fourth NPU 100-4. And, for example, the plurality of CPUs may include a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.

[0351] Each of the plurality of NPUs (eg, 100-1, 100-2, 100-3, 100-4) may include Figure 3 or Figure 5 Shown are a plurality of PEs (PEs) 110 , an NPU internal memory 120 , an NPU controller 130 , an NPU interface 140 , and an SFU 150 .

[0352] If the clock source 170 generates and outputs an original clock signal, the clock distributor 190 may directly (bypass) output the original clock signal, or may output a plurality of phase-delayed signals generated by delaying or shifting the phase of the original clock signal.

[0353] For example, the clock distributor 190 may provide the original clock signal from the clock source 170 to the first NPU 100-1, and may provide the first phase-delayed clock signal to the second NPU 100-2. Furthermore, the clock distributor 190 may provide the second phase-delayed clock signal to the third NPU 100-3, and provide the third phase-delayed clock signal to the fourth NPU 100-4.

[0354] Alternatively, the clock distributor 190 may provide a first phase-delayed clock signal to the first NPU 100-1, and may provide a second phase-delayed clock signal to the second NPU 100-2. Furthermore, the clock distributor 190 may provide a third phase-delayed clock signal to the third NPU 100-3, and provide a fourth phase-delayed clock signal to the fourth NPU 100-4.

[0355] exist Figures 9a to 9c The various elements (ie, NPU, memory, and CPU) within the SoC 1000 shown may be input with a supply voltage VDD.

[0356] exist Figures 9a to 9c 2 shows a case where a common supply voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0357] In some examples, it may also be configured to input an independent or separate supply voltage VDD to each element (ie, NPU, memory, and CPU) within the SoC 1000 .

[0358] In another example, an independent or separate supply voltage VDD may be input to some components (ie, NPU, memory, and CPU) within the SoC 1000 .

[0359] In yet another example, a second supply voltage different from the first supply voltage may be input to a specific element in the SoC 1000 .

[0360] As described above, in the third example of the present disclosure, a plurality of NPUs are made to perform decentralized operations according to different clock signals, thereby being able to reduce peak power.

[0361] Furthermore, in an example of the present disclosure, in order to provide a plurality of clock signals having different phases from each other, a clock distributor 190 may be simply added without providing a plurality of clock sources 170 , thereby reducing manufacturing costs.

[0362] Fig.10a is an exemplary diagram showing a structure of a system according to a fourth example of the present disclosure.

[0363] The following will only explain Figure 9a If the contents are different, the same contents will refer to the previous contents.

[0364] First, in Fig.10aIn the embodiment, an electrically conductive pattern that functions as a system bus 500 may be formed on a board substrate that is not a SoC. Furthermore, a first semiconductor chip provided for the first NPU 100-1 and a second semiconductor chip provided for the second NPU 100-2 may be mounted on the board substrate. In addition, a semiconductor chip for a first memory 300-1 and a semiconductor chip for a second memory 300-2 may be mounted on the board substrate. In addition, a memory controller 350 may be mounted on the board substrate. In addition, a semiconductor chip for a first CPU 200-1 and a semiconductor chip for a second CPU 200-2 may be mounted on the board substrate. In addition, a semiconductor chip for an I / O interface 800 may be mounted on the board substrate. The board substrate may be a printed circuit board (PCB).

[0365] Fig.10a The clock source 170 and the clock distributor 190 are shown in FIG. Figures 7a to 8c Same as shown.

[0366] Fig.10b : is an exemplary diagram showing the structure of a system according to a first modification example of the fourth example of the present disclosure.

[0367] The following will only explain Figure 9b If the contents are different, the same contents will refer to the previous contents.

[0368] First, in Fig.10bIn the embodiment, an electrically conductive pattern that functions as a system bus 500 may be formed on a board substrate that is not a SoC. Furthermore, a first semiconductor chip provided for the first NPU 100-1, a second semiconductor chip provided for the second NPU 100-2, and a third semiconductor chip provided for the third NPU 100-3 may be mounted on the board substrate. In addition, a semiconductor chip for a first memory 300-1 and a semiconductor chip for a second memory 300-2 may be mounted on the board substrate. In addition, a memory controller 350 may be mounted on the board substrate. In addition, a semiconductor chip for a first CPU 200-1 and a semiconductor chip for a second CPU 200-2 may be mounted on the board substrate. In addition, a semiconductor chip for an I / O interface 800 may be mounted on the board substrate. The board substrate may be a printed circuit board (PCB: Printed circuit board).

[0369] Fig.10b The clock source 170 and the clock distributor 190 are shown in FIG. Figures 7a to 8c Same as shown.

[0370] Fig.10c : is an exemplary diagram showing the structure of a system according to a second modification example of the fourth example of the present disclosure.

[0371] The following will only explain Fig.9c If the contents are different, the same contents will refer to the previous contents.

[0372] First, in Fig.10cIn the embodiment of the present invention, an electrically conductive pattern that functions as a system bus 500 may be formed on a board substrate that is not a SoC. Furthermore, a first semiconductor chip provided for the first NPU 100-1, a second semiconductor chip provided for the second NPU 100-2, a third semiconductor chip provided for the third NPU 100-3, and a fourth semiconductor chip provided for the fourth NPU 100-4 may be mounted on the board substrate. In addition, a semiconductor chip for the first memory 300-1 and a semiconductor chip for the second memory 300-2 may be mounted on the board substrate. In addition, a memory controller 350 may be mounted on the board substrate. In addition, a semiconductor chip for the first CPU 200-1 and a semiconductor chip for the second CPU 200-2 may be mounted on the board substrate. In addition, a semiconductor chip for an I / O interface 800 may be mounted on the board substrate. The board substrate may be a printed circuit board (PCB).

[0373] Fig.10c The clock source 170 and the clock distributor 190 are shown in FIG. Figures 7a to 8c Same as shown.

[0374] Fig.11a is a first example of the clock distributor 190 .

[0375] Reference Fig.11a The clock distributor 190 connected to the clock source 170 may include a distribution circuit (not shown) and a plurality of output circuits (not shown).

[0376] The clock distributor 190 may receive an original clock signal from the clock source 170, and generate and output a plurality of clock signals corresponding to the number of NPU or PE groups using the original clock signal. Each of the plurality of NPU or PE groups NPU_0, NPU_1, and NPU_2 may receive a clock signal with a different phase from the clock distributor 190. The clock distributor 190 may further include a division circuit (not shown) described later. Therefore, each of the plurality of NPU or PE groups NPU_0, NPU_1, and NPU_2 operates based on a different phase clock, thereby being able to reduce the peak power of a semiconductor chip including the plurality of NPU or PE groups NPU_0, NPU_1, and NPU_2 and / or a system including the semiconductor chip.

[0377] Fig.11b is a second example of the clock distributor 190 .

[0378] Reference Fig.11b The clock distributor 190 connected to the clock source 170 may include a plurality of flip-flops 193 a , 193 b , a multiplexer 194 , and a division circuit (eg, Div3 ) 195 .

[0379] exist Fig.11b , the case where the number of the multiple triggers 193a and 193b is 2 is exemplarily shown, but it can be changed according to the number of NPUs or the number of PE groups. For example, when the number of NPUs is 2 or the number of PE groups is 2, the number of the triggers can be 1. As another example, when the number of NPUs is 4 or the number of PE groups is 4, the number of the triggers can be 3.

[0380] The multiple triggers 193a, 193b are Fig.11b exemplarily shown as a D-type flip-flop. The D-type flip-flop acts as a delay element. The D-type flip-flop delays the input signal (for example, phase delays 180 degrees) until the next activation clock appears and then outputs it. Therefore, the D-type flip-flop can be used as a delay buffer.

[0381] Specifically, if the first flip-flop 193a of the plurality of flip-flops 193a and 193b receives the original clock signal from the clock source 170, the original clock signal can be output after being phase-delayed (e.g., phase-delayed by 180 degrees). The output of the first flip-flop 193a is transmitted to the multiplexer 194 and the second flip-flop 193b.

[0382] The second flip-flop 193 b may be connected to the first flip-flop 193 a and delay the phase of the clock signal output from the first flip-flop 193 a (eg, delay the phase by 180 degrees) before outputting the clock signal. The output of the second flip-flop 193 b is transferred to the multiplexer 194 .

[0383] The multiplexer 194 may multiplex the original clock signal from the clock source 170 and the output of each of the plurality of flip-flops 193 a , 193 b , and then pass them to the division circuit (eg, Div3 ) 195 .

[0384] The segmentation circuit 195 Fig.11b1 is shown as Div3 for three-partition, but it is not limited thereto and can be changed according to the number of NPUs or the number of PE groups. For example, when the number of NPUs is 2 or the number of PE groups is 2, the partitioning circuit 195 may be Div2 for two-partition. As another example, when the number of NPUs is 4 or the number of PE groups is 4, the partitioning circuit 195 may be Div4 for four-partition.

[0385] To explain the operation of the segmentation circuit 195, temporarily refer to Fig.11e To illustrate, the segmentation circuit 195 may segment the output signal from the multiplexer 194 into the NPU_0 signal, the NPU_1 signal, and the NPU_2 signal, and then transmit the signals to each NPU or each PE group.

[0386] according to Fig.11b The structure of the clock distributor 190 shown has the advantages that no additional setting by an expert is required and the phase can be always accurately separated. On the contrary, it has the disadvantage that the phase-locked loop (PLL: Phase Locked Loop; phase synchronization circuit) in the clock source 170 requires a high-frequency output.

[0387] Fig.11c is a third example of the clock distributor 190 .

[0388] As reference Fig.11c It can be seen that the clock distributor 190 connected to the clock source 170 may include a plurality of delay units 196 a , 196 b , 196 c , . . . , 196 n and a multiplexer 194 .

[0389] The plurality of delay units 196a, 196b, 196c, ..., 196n are connected in series in a chain form and are connected in parallel to the multiplexer 194. Each of the plurality of delay units 196a, 196b, 196c, ..., 196n can delay the phase of the clock signal by a predetermined offset and then output it.

[0390] The multiplexer 194 is connected to the output end of the clock source 170 and the output ends of the plurality of delay units 196a, 196b, 196c, ..., 196n. Therefore, the multiplexer 194 can multiplex and output the original clock signal from the clock source 170 and the clock signals output from the plurality of delay units 196a, 196b, 196c, ..., 196n.

[0391] In the examples of the present disclosure, the delay unit may also be referred to as a delay buffer.

[0392] according to Fig.11cThe advantages of the structure of the clock distributor 190 shown are that the degree of freedom of phase delay is high and the PLL in the clock source 170 can output at a low frequency. On the contrary, the disadvantage is that an appropriate delay value needs to be selected for each frequency.

[0393] Fig.11d is a fourth example of the clock distributor 190 .

[0394] As reference Fig.11d It can be known that the clock distributor 190 connected to the clock source 170 may include a plurality of flip-flops 193a, 193b and a division circuit (eg, Div3) 195. The clock distributor 190 may also selectively include at least one of a plurality of clock buffers 197a, 197b, 197c.

[0395] exist Fig.11d , the case where the number of the multiple triggers 193a and 193b is 2 is exemplarily shown, but it can be changed according to the number of NPUs or the number of PE groups. For example, when the number of NPUs is 2 or the number of PE groups is 2, the number of the triggers can be 1. As another example, when the number of NPUs is 4 or the number of PE groups is 4, the number of the triggers can be 3.

[0396] The multiple triggers 193a, 193b are Fig.11d exemplarily shown as a D-type flip-flop. The D-type flip-flop acts as a delay element. The D-type flip-flop delays the input signal (for example, phase delays 180 degrees) until the next activation clock appears and then outputs it. Therefore, the D-type flip-flop can be used as a delay buffer.

[0397] The segmentation circuit 195 Fig.11d 1 is shown as Div3 for three-partition, but it is not limited thereto and can be changed according to the number of NPUs or the number of PE groups. For example, when the number of NPUs is 2 or the number of PE groups is 2, the partitioning circuit 195 may be Div2 for two-partition. As another example, when the number of NPUs is 4 or the number of PE groups is 4, the partitioning circuit 195 may be Div4 for four-partition.

[0398] The clock distributor 190 may be configured to further selectively include a clock buffer. However, examples of the present disclosure are not limited to the clock buffer.

[0399] exist Fig.11d , multiple clock buffers 197a, 197b, and 197c are shown, but clock buffers may be selectively added or removed. The clock buffer may be configured to delay the phase of a clock signal to a specific phase.

[0400] The multiple clock buffers 197a, 197b, 197c are Fig.11d 3 are exemplarily shown in the figure, but may be changed according to the number of NPUs or the number of PE groups. For example, when the number of NPUs is 2 or the number of PE groups is 2, the number of the plurality of clock buffers may be 2. As another example, when the number of NPUs is 4 or the number of PE groups is 4, the number of the plurality of clock buffers may be 4.

[0401] For example, in Fig.11d In the clock distributor 190 shown, three clock signals having a phase difference of 120° are output for each clock signal. However, the physical distance between each NPU or PE group receiving the respective clock signals and the clock distributor 190 may be different. In the case where the physical distance of a particular NPU or PE group is particularly far compared to other NPUs or other PE groups, a portion of the phases of each clock signal output from the clock distributor 190 having a phase difference of 120° may substantially become similar or overlap. In this case, the clock buffer may be configured to be arranged on a signal line supplying a particular clock signal to compensate for phase changes according to physical distance. That is, at least one of the plurality of clock signals may be configured to be supplied to one of the plurality of PE groups after passing through at least one output buffer arranged in consideration of peak power reduction. That is, at least one clock buffer may be appropriately set in consideration of the physical layout or signal delay of the designed semiconductor chip.

[0402] Fig.11e is an exemplary diagram showing exemplary signal waveforms from the clock distributor 190 .

[0403] As reference Fig.11e It can be seen that compared with the original clock signal, the signal transmitted to each NPU or each PE group is not only delayed in phase, but also has a lower frequency. Fig.11e 2 shows that the signal transmitted to each NPU or each PE group is generated by dividing the original clock signal into three parts. Therefore, the frequency of the signal transmitted to each NPU or each PE group is 1 / 3 times lower than the frequency of the original clock signal.

[0404] Fig.12 is an example diagram illustrating an operation method of an NPU according to an example of the present disclosure.

[0405] Reference Fig.12 , a plurality of clock signals for driving a plurality of PEs may be generated ( S1210 ).

[0406] Furthermore, the plurality of PEs may be divided into a first group of PEs and a second group of PEs (S1220). It is apparent that the number of PEs belonging to each group may be changed. In addition, the number of PEs belonging to each group may be changed semi-statically or dynamically. Such a change may be performed according to the control of the NPU controller 130.

[0407] Next, the first group of PEs may be driven based on the first clock signal (S1030). In addition, the second group of PEs may be driven based on the second clock signal (S1040).

[0408] Fig.13 A comparative example is shown in which clock signals of different phases are not input to a plurality of PEs of an NPU.

[0409] Reference Fig.13 , the first clock CLK0 may be input to the PE 110-1 of the first group and the PE 110-2 of the second group. Here, the PEs 110-1 and 110-2 of each group are configured to receive clock signals of the same phase.

[0410] Fig.13 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0411] Fig.13 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0412] Fig.13 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0413] Reference Fig.13 , showing a tendency that as the number of activated processing elements of the neural processing unit 100 increases, the peak power increases and the supply voltage VDD decreases.

[0414] In more detail, the number of PEs operated in each clock cycle can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of PE. On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0415] Refer again Fig.13 The peak power of the first group of PE 110-1 and the second group of PE 110-2 operating with the first clock CLK0 as the reference may vary according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0416] In more detail, the NPU developed with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively low compared to the PCI Express interface. For example, in the case of the M.2 interface, there may be a current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, and as the peak power exceeds the power limit, the degree of reduction of the supply voltage VDD can be further increased in proportion to the exceeded information.

[0417] If the supply voltage VDD drops below the IR-drop margin, the data in the NPU operation may be erroneous. In detail, IR-drop may occur when a voltage drop (V=IR) occurs according to Ohm's law due to the resistance (R) of the metal interconnect and the current (I) flowing therethrough. If a large number of circuits associated with multiple PEs of the NPU are switched at the same time, the NPU may consume a large amount of current. This high current may cause a larger voltage drop to occur at the NPU and reduce the supply voltage at other parts of the SoC.

[0418] tidy Fig.13 For example, if you want to ensure stable operation of the NPU and prevent potential errors, you need to ensure sufficient IR-drop margin. The IR-drop margin can refer to the difference between the supply voltage VDD and the lowest voltage allowed at all points of the NPU during peak power conditions.

[0419] Fig.14a An example of inputting clock signals of different phases to a plurality of PEs of two groups of an NPU in consideration of peak power according to an example of the present disclosure is shown.

[0420] It should be noted that as the utilization of the processing element increases, the power can increase proportionally. Thus, the power of each operation step can be calculated based on the utilization information and the driving frequency information of the PE in each operation step. The power calculation can be calculated based on the scheduling information of a specific artificial neural network model. Among them, the power (Watt) can be calculated by dividing the energy consumed by the corresponding operation step by the value of the unit time (i.e., 1 watt (W) = 1 joule per second (J / s)). Therefore, the power of each operation step can be calculated based on the scheduling information.

[0421] Reference Fig.14a , the first clock CLK1 may be input to the PE 110-1 of the first group, and the second clock CLK2 may be input to the PE 110-2 of the second group. Here, the PEs 110-1 and 110-2 of each group are configured to receive clock signals of different phases.

[0422] Fig.14a 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0423] Fig.14a 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0424] Fig.14a 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0425] Reference Fig.14a , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the total peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to the PEs of each group.

[0426] Here, according to Fig.14a The average power of an example is based on Fig.13 The average power of the comparative example is substantially the same. Fig.14a The peak power of an example is less than Fig.13 Therefore, according to Fig.14a An example of Fig.13 Compared with the comparative example, the ratio of peak power to average power can be improved.

[0427] In detail, the number of PEs operated per clock can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of PE. On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0428] Refer again Fig.14a The peak power of the first group of PE 110-1 and the second group of PE 110-2 corresponding to the first clock CLK1 may vary according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0429] In detail, the NPU opened with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively low compared to the PCI Express interface. For example, in the case of the M.2 interface, there may be a current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, the more the peak power increases the power limit, and the greater the reduction in the supply voltage VDD can be.

[0430] If the supply voltage VDD drops below the IR-drop margin, the data in the NPU calculation may be erroneous.

[0431] In addition, the first clock CLK1 may be applied to the PE 110-1 of the first group, and the second clock CLK2 with a different phase from the first clock CLK1 may be applied to the PE 110-2 of the second group. Therefore, the peak power of the PEs of each group can be dispersed, and the peak power of the NPU can be compared to Fig.13 In this case, the Fig.13Compared with the comparative example shown, the voltage stability of the supply voltage VDD can be relatively more stable.

[0432] That is, according to an example of the present disclosure, the peak power of the NPU can be reduced, and the peak power of the SoC including the NPU can also be reduced.

[0433] Furthermore, in the case of a low power interface such as M.2, the power limit may be lower and thus may be more efficient in such a low power interface. However, the present disclosure is not limited to a particular interface.

[0434] Fig.14b An example of inputting clock signals of different phases to a plurality of PEs of three groups of an NPU by considering peak power according to an example of the present disclosure is shown.

[0435] It should be noted that as the utilization of the processing element increases, the power can increase proportionally. Thus, the power of each operation step can be calculated based on the utilization information and the driving frequency information of the PE in each operation step. The power calculation can be calculated based on the scheduling information of a specific artificial neural network model. Among them, the power (Watt) can be calculated by dividing the energy consumed by the corresponding operation step by the value of the unit time (i.e., 1 watt (W) = 1 joule per second (J / s)). Therefore, the scheduling information can be calculated based on the power of each operation step.

[0436] Reference Fig.14b , the first clock CLK1 may be input to the first group of PE 110-1, the second clock CLK2 may be input to the second group of PE 110-2, and the third clock CLK3 may be input to the third group of PE 110-3. Here, each group of PE 110-1, 110-2, 110-3 is configured to receive clock signals of different phases.

[0437] Fig.14b 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0438] Fig.14b 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0439] Fig.14b 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0440] Reference Fig.14b , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the total peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to the PEs of each group.

[0441] Here, according to Fig.14b The average power of an example is based on Fig.13 The average power of the comparative example is substantially the same. Fig.14b The peak power of an example is less than Fig.13 Therefore, according to Fig.14b An example of Fig.13 Compared with the comparative example, the ratio of peak power to average power can be improved.

[0442] In detail, the number of PEs operated per clock can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of PE. On the contrary, as the performance of the compiler is better, the peak power of the NPU can be increased by about 1%. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0443] Refer again Fig.14b The peak power of the first group PE 110-1, the second group PE 110-2, and the third group PE 110-3 corresponding to the first clock CLK1 may vary according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0444] In detail, the NPU developed with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively low compared to the PCI Express interface. For example, in the case of the M.2 interface, there may be a current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, the more the peak power increases the power limit, and the greater the reduction in the supply voltage VDD can be.

[0445] If the supply voltage VDD drops below the IR-drop margin, the data in the NPU calculation may be erroneous.

[0446] In addition, the first clock CLK1 may be applied to the PE 110-1 of the first group, the second clock CLK2 with a different phase from the first clock CLK1 may be applied to the PE 110-2 of the second group, and the third clock CLK3 with a different phase from the first clock CLK1 and the second clock CLK2 may be applied to the PE 110-3 of the third group. Therefore, the peak power of the PEs of each group can be dispersed, even if the peak power of the PEs of each group is smaller than that of the PEs of the first group. Fig.14a An example of more groups of PEs to operate on, Fig.14a Examples and Fig.14b The peak powers of the examples may also be the same.

[0447] That is, according to an example of the present disclosure, the peak power of the NPU can be reduced, and the peak power of the SoC including the NPU can also be reduced.

[0448] Furthermore, in the case of a low power interface such as M.2, the power limit may be lower and thus may be more efficient in such a low power interface. However, the present disclosure is not limited to a particular interface.

[0449] Fig.14c An example of inputting clock signals of different phases to a plurality of PEs of four groups of an NPU by considering peak power according to an example of the present disclosure is shown.

[0450] It should be noted that as the utilization of the processing element increases, the power can increase proportionally. Thus, the power of each operation step can be calculated based on the utilization information and the driving frequency information of the PE in each operation step. The power calculation can be calculated based on the scheduling information of a specific artificial neural network model. Among them, the power (Watt) can be calculated by dividing the energy consumed by the corresponding operation step by the value of the unit time (i.e., 1 watt (W) = 1 joule per second (J / s)). Therefore, the power of each operation step can be calculated based on the scheduling information.

[0451] Reference Fig.14c , the first clock CLK1 may be input to the PE 110-1 of the first group, the second clock CLK2 may be input to the PE 110-2 of the second group, the third clock CLK3 may be input to the PE 110-3 of the third group, and the fourth clock CLK4 may be input to the PE 110-4 of the fourth group. Here, the PEs 110-1, 110-2, 110-3, and 110-4 of each group are configured to receive clock signals of different phases.

[0452] Fig.14c100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0453] Fig.14c 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0454] Fig.14c 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0455] Reference Fig.14c , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the total peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to the PEs of each group.

[0456] Here, according to Fig.14c The average power of an example is based on Fig.13 The average power of the comparative example is substantially the same. Fig.14c The peak power of an example is less than Fig.13 Therefore, according to Fig.14c An example of Fig.13 Compared with the comparative example, the ratio of peak power to average power can be improved.

[0457] In detail, the number of PEs operated per clock can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of PE. On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0458] Refer again Fig.14c, the peak power of the first group of PE 110-1, the second group of PE 110-2, the third group of PE 110-3, and the fourth group of PE 110-4 corresponding to the first clock CLK1 may vary according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0459] In detail, the NPU developed with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively low compared to the PCI Express interface. For example, in the case of the M.2 interface, there may be a rated current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, the more the peak power increases the power limit, and the greater the reduction in the supply voltage VDD can be.

[0460] If the supply voltage VDD drops below the IR-drop margin, the data in the NPU calculation may be erroneous.

[0461] In addition, the first clock CLK1 may be applied to the PE 110-1 of the first group, the second clock CLK2 with a different phase from the first clock CLK1 may be applied to the PE 110-2 of the second group, the third clock CLK3 with a different phase from the first clock CLK1 and the second clock CLK2 may be applied to the PE 110-3 of the third group, and the fourth clock CLK4 with a different phase from the first clock CLK1, the second clock CLK2 and the third clock CLK3 may be applied to the PE 110-4 of the fourth group. Therefore, the peak power of the PEs of each group can be dispersed, even if the peak power of the PEs of each group is smaller than that of the PEs of the fourth group. Fig.14b An example of more groups of PEs to operate on, Fig.14b Examples and Fig.14c The peak powers of the examples may also be the same.

[0462] That is, according to an example of the present disclosure, the peak power of the NPU can be reduced, and the peak power of the SoC including the NPU can also be reduced.

[0463] Furthermore, in the case of a low power interface such as M.2, the power limit may be lower and thus may be more efficient in such a low power interface. However, the present disclosure is not limited to a particular interface.

[0464] Fig.15 A comparative example in which clock signals of different phases are not input to a plurality of NPUs is shown.

[0465] Reference Fig.15 , the first clock CLK0 can be input to multiple neural processing units 100-1, 100-2, 100-3, 100-4. Here, multiple neural processing units 100-1, 100-2, 100-3, 100-4 are configured to receive clock signals of the same phase. The multiple neural processing units may include the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4.

[0466] Fig.15 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0467] Fig.15 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0468] Fig.15 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0469] Reference Fig.15 , showing a tendency that as the number of activated processing elements of the neural processing unit 100 increases, the peak power increases and the supply voltage VDD decreases.

[0470] In detail, the number of PEs operated in each clock cycle can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of the PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of the PE. On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0471] Refer again Fig.15The peak power of the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 operating with the first clock CLK0 as the reference may be changed according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0472] In detail, the NPU developed with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively low compared to the PCI Express interface. For example, in the case of the M.2 interface, there may be a current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, and as the peak power exceeds the power limit, the degree of reduction of the supply voltage VDD can be further increased in proportion to the exceeded information.

[0473] If the supply voltage VDD drops below the IR-drop margin, data in the NPU operation may be erroneous. In detail, IR-drop may occur when a voltage drop (V=IR) occurs according to Ohm's law due to the resistance (R) of the metal interconnect and the current (I) flowing through it. If a large number of circuits associated with multiple PEs of the NPU are switched at the same time, the NPU may consume a large amount of current. This high current may cause a larger voltage drop in the NPU and reduce the supply voltage at other parts of the SoC.

[0474] tidy Fig.15 For example, if you want to ensure stable operation of the NPU and prevent potential errors, you need to ensure sufficient IR-drop margin. The IR-drop margin can refer to the difference between the supply voltage VDD and the lowest voltage allowed at all points of the NPU during peak power conditions.

[0475] Fig.16a An example of inputting clock signals of different phases to two NPUs by considering peak power according to an example of the present disclosure is shown.

[0476] Reference Fig.16a, a first clock CLK1 may be input to the first NPU 100-1, and a second clock CLK2 may be input to the second NPU 100-2. Here, the first NPU 100-1 and the second NPU 100-2 are configured to receive clock signals of different phases.

[0477] Fig.16a 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0478] Fig.16a 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0479] Fig.16a 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0480] Reference Fig.16a , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the total peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to multiple NPUs.

[0481] In detail, the number of PEs operated per clock can be changed according to the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be operated per unit clock. Here, the ratio of PEs operated among all PEs can be referred to as the utilization rate (utilization rate %) of PE. On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. That is, the peak power can be proportional to the utilization rate of the PE. Therefore, as the algorithm of the compiler becomes more sophisticated, there is a problem that the peak power of the NPU may further increase.

[0482] The peak power of the first NPU 100-1 corresponding to the first clock CLK1 may be changed according to the number of PEs operating at each clock. Similarly, the peak power of the second NPU 100-2 corresponding to the second clock CLK2 may be changed according to the number of PEs operating at each clock. That is, the peak power may increase in proportion to the number of PEs operating at one clock.

[0483] In detail, the NPU developed with low power can be developed to correspond to the M.2 interface or the PCI Express interface when applied to the edge device. In particular, in the case of the M.2 interface, the maximum power may be relatively lower than that of the PCI Express interface. For example, in the case of the M.2 interface, there may be a current of 3A and a power limit of 10W. If the peak power of the NPU of the comparative example exceeds 10W at a specific clock, the supply voltage VDD supplied to the NPU may oscillate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU can be increased, the more the peak power increases the power limit, and the greater the reduction in the supply voltage VDD can be.

[0484] If the supply voltage VDD drops below the IR-drop margin, the data in the NPU calculation may be erroneous.

[0485] In addition, the first NPU 100-1 may apply the first clock CLK1, and the second NPU 100-2 may apply the second clock CLK2 which has a different phase from the first clock CLK1. Therefore, the peak power of the first NPU 100-1 and the second NPU 100-2 can be dispersed, and the peak power of the NPU can be reduced to half. In this case, Fig.15 Compared with the comparative example shown, the voltage stability of the supply voltage VDD can be relatively more stable.

[0486] That is, according to an example of the present disclosure, the peak power of the NPU can be reduced, and the peak power of the SoC including the NPU can also be reduced.

[0487] Furthermore, in the case of a low power interface such as M.2, the power limit may be lower and thus may be more efficient in such a low power interface. However, the present disclosure is not limited to a particular interface.

[0488] Fig.16b An example of inputting clock signals of different phases to three NPUs by considering peak power according to an example of the present disclosure is shown.

[0489] Reference Fig.16b , the first clock CLK1 may be input to the first NPU 100-1, the second clock CLK2 may be input to the second NPU 100-2, and the third clock CLK3 may be input to the third NPU 100-3. Here, the first NPU 100-1, the second NPU 100-2, and the third NPU 100-3 are configured to receive clock signals of different phases.

[0490] Fig.16b100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0491] Fig.16b 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0492] Fig.16b 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0493] Reference Fig.16b , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the maximum peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to multiple NPUs.

[0494] A first clock CLK1 may be applied to the first NPU 100 - 1 , a second clock CLK2 may be applied to the second NPU 100 - 2 , and a third clock CLK3 may be applied to the third NPU 100 - 3 .

[0495] like Fig.16b As shown in FIG. 1 , the peak power intervals of each NPU can be dispersed on the time axis, and the maximum peak power can be significantly reduced. Fig.15 Compared with the comparative example shown, the voltage stability of the supply voltage VDD can be relatively more stable.

[0496] Fig.16c An example of inputting clock signals of different phases to four NPUs by considering peak power according to an example of the present disclosure is shown.

[0497] Reference Fig.16c , a first clock CLK1 may be input to the first NPU 100-1, a second clock CLK2 may be input to the second NPU 100-2, a third clock CLK3 may be input to the third NPU 100-3, and a fourth clock CLK4 may be input to the fourth NPU 100-4. Here, the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 are configured to receive clock signals of different phases.

[0498] Fig.16c 100x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when one hundred processing elements are activated.

[0499] Fig.16c 200x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when two hundred processing elements are activated.

[0500] Fig.16c 500x of PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when five hundred processing elements are activated.

[0501] Reference Fig.16c , showing a tendency that the peak power increases and the supply voltage VDD decreases as the number of activated processing elements of the neural processing unit 100 increases. However, it also shows a tendency that the total peak power decreases and the fluctuation of the supply voltage VDD also decreases as signal clocks with different phases are input to multiple NPUs.

[0502] The first NPU 100-1 may apply a first clock CLK1, the second NPU 100-2 may apply a second clock CLK2 having a different phase from the first clock CLK1, the third NPU 100-3 may apply a third clock CLK3, and the fourth NPU 100-4 may apply a fourth clock CLK4.

[0503] Therefore, the peak power occurrence areas of the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 can be dispersed on the time axis.

[0504] like Fig.16c As shown in FIG. 1 , the peak power intervals of each NPU can be dispersed on the time axis, and the maximum peak power can be significantly reduced. Fig.15 Compared with the comparative example shown, the voltage stability of the supply voltage VDD can be relatively more stable.

[0505] [Brief summary of examples disclosed in this disclosure]

[0506] The contents of the present disclosure described so far are organized and explained as follows.

[0507] An example of the present disclosure provides a neural processing unit. The neural processing unit may include: a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN: artificial neural network) model and is arranged for a plurality of PE groups including a plurality of processing elements (PE); and a second circuit, which is arranged for a clock distributor, which divides and generates a plurality of clock signals with different phases based on an original clock signal from a clock source and provides them to the plurality of PE groups. A first clock signal among the plurality of clock signals may be provided to a first PE group among the plurality of PE groups, and a second clock signal among the plurality of clock signals may be provided to a second PE group among the plurality of PE groups.

[0508] At least one of the plurality of clock signals may be configured to be supplied to one of the plurality of PE groups after passing through at least one output buffer arranged in consideration of peak power reduction.

[0509] The frequencies of the plurality of clock signals may be lower than the frequency of the original clock signal.

[0510] The frequencies of the plurality of clock signals may be determined based on the number of the plurality of PE groups.

[0511] The frequencies of the plurality of clock signals may be lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of PE groups.

[0512] The phase of the second clock signal may be slower than the phase of the first clock signal.

[0513] The second circuit arranged for the clock distributor may include: a plurality of flip-flops for delaying the original clock signal; a multiplexer connected in parallel to the plurality of flip-flops; and a divider for dividing the frequency of an output signal from the multiplexer.

[0514] The second circuit arranged for the clock distributor may include: a plurality of delay units; and a multiplexer connected in parallel to the plurality of delay units.

[0515] The second circuit arranged for the clock distributor may include: a divider connected to the original clock signal and dividing the frequency of the original clock signal; and a first D-type flip-flop connected to the output of the divider. The outputs of the divider and the first D-type flip-flop may be connected to the clock frequency of the original clock signal, respectively.

[0516] An example of the present disclosure provides a system on chip. The system on chip may include: a semiconductor substrate; a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN: artificial neural network) model and is arranged on the semiconductor substrate for a plurality of neural processing units (NPUs) including a plurality of processing elements (PEs); and a second circuit, which is arranged on the semiconductor substrate for a clock distributor, which divides and generates a plurality of clock signals with different phases from each other based on an original clock signal from a clock source and provides them to the plurality of NPUs. A first clock signal among the plurality of clock signals may be provided to a first NPU among the plurality of NPUs, and a second clock signal among the plurality of clock signals may be provided to a second NPU among the plurality of NPUs.

[0517] At least one of the plurality of clock signals may be configured to be supplied to one of the plurality of NPUs after passing through at least one output buffer arranged in consideration of reduction in peak power.

[0518] The frequencies of the plurality of clock signals may be lower than the original clock signal.

[0519] The frequencies of the plurality of clock signals may be determined based on the number of the plurality of NPUs.

[0520] The frequencies of the plurality of clock signals may be lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of PE groups.

[0521] The phase of the second clock signal may be slower than that of the first clock signal.

[0522] An example of the present disclosure provides an electronic device. The electronic device may include: a printed circuit substrate; a first circuit, which is configured to be able to perform operations for an artificial neural network (ANN) model and is arranged on the printed circuit substrate for a plurality of neural processing units (NPUs) including a plurality of processing elements (PEs); and a second circuit, which is arranged on the printed circuit substrate for a clock distributor, the clock distributor splits and generates a plurality of clock signals with different phases from each other based on an original clock signal from a clock source and provides them to the plurality of NPUs. A first clock signal among the plurality of clock signals may be provided to a first NPU among the plurality of NPUs, and a second clock signal among the plurality of clock signals may be provided to a second NPU among the plurality of NPUs.

[0523] At least one of the plurality of clock signals may be configured to be supplied to one of the plurality of NPUs after passing through at least one output buffer arranged in consideration of peak power reduction.

[0524] The frequencies of the plurality of clock signals may be lower than the original clock signal.

[0525] The frequencies of the plurality of clock signals may be determined based on the number of the plurality of NPUs.

[0526] The frequencies of the plurality of clock signals may be lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of PE groups.

[0527] The phase of the second clock signal may be slower than that of the first clock signal.

[0528] The examples of the present disclosure shown in this specification and the drawings are only specific examples proposed to easily illustrate the technical content of the present disclosure and help understand the present disclosure, and are not intended to limit the scope of the present disclosure. In addition to the examples described so far, other variations may also be implemented, which is obvious to ordinary technicians in the technical field to which the present disclosure belongs.

[0529]

National R&D Project to Support This Invention

[0530]

Project unique number

[0531]

Project number

[0532]

Department name

[0533]

Project Management (Professional) Organization Name

[0534] [Research Project Name] PIM Artificial Intelligence Semiconductor Core Technology Development (Design)

[0535]

Research Topic Name

[0536]

Project Execution Organization Name

[0537]

Study period

Claims

1. A neural processing unit, comprising: A first circuit is configured to be capable of performing operations for an ANN model and is arranged for a plurality of PE groups including a plurality of processing elements; as well as A second circuit is arranged for a clock distributor, the clock distributor generates a plurality of clock signals having different phases from each other based on an original clock signal from a clock source and provides the clock signals to the plurality of PE groups. The first clock signal among the multiple clock signals is provided to a first PE group among the multiple PE groups, and the second clock signal among the multiple clock signals is provided to a second PE group among the multiple PE groups.

2. The neural processing unit according to claim 1, wherein: At least one of the plurality of clock signals is configured to be supplied to one of the plurality of PE groups after passing through at least one output buffer arranged in consideration of reduction in peak power.

3. The neural processing unit according to claim 1, wherein: The frequencies of the multiple clock signals are lower than that of the original clock signal.

4. The neural processing unit according to claim 1, wherein: The frequencies of the plurality of clock signals are determined based on the number of the plurality of PE groups.

5. The neural processing unit according to claim 1, wherein: The frequencies of the plurality of clock signals are lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of PE groups.

6. The neural processing unit according to claim 1, wherein: The phase of the second clock signal is slower than that of the first clock signal.

7. The neural processing unit according to claim 1, wherein: The second circuit arranged for the clock distributor comprises: A plurality of flip-flops, used for delaying the original clock signal; a multiplexer connected in parallel with the plurality of flip-flops; and A divider divides the frequency of an output signal from the multiplexer.

8. The neural processing unit according to claim 1, wherein: The second circuit arranged for the clock distributor comprises: a plurality of delay units; and A multiplexer is connected in parallel with the plurality of delay units.

9. The neural processing unit according to claim 1, wherein: The second circuit arranged for the clock distributor comprises: a divider connected to the original clock signal and dividing the frequency of the original clock signal; and A first D-type flip-flop connected to the output terminal of the divider, Wherein, the output ends of the divider and the first D-type flip-flop are respectively connected to the clock frequency of the original clock signal.

10. A system on chip, comprising: Semiconductor substrates; A first circuit configured to be capable of performing operations for an ANN model and arranged on the semiconductor substrate for a plurality of neural processing units including a plurality of processing elements; as well as A second circuit is arranged on the semiconductor substrate for a clock distributor, wherein the clock distributor generates a plurality of clock signals having different phases based on an original clock signal from a clock source and provides the clock signals to the plurality of NPUs. The first clock signal among the multiple clock signals is provided to a first NPU among the multiple NPUs, and the second clock signal among the multiple clock signals is provided to a second NPU among the multiple NPUs.

11. The system on chip according to claim 10, wherein: At least one of the plurality of clock signals is configured to be supplied to one of the plurality of NPUs after passing through at least one output buffer arranged in consideration of reduction in peak power.

12. The system on chip according to claim 10, wherein: The frequencies of the multiple clock signals are lower than that of the original clock signal.

13. The system on chip according to claim 10, wherein: The frequencies of the plurality of clock signals are determined based on the number of the plurality of NPUs.

14. The system on chip according to claim 10, wherein: The frequencies of the plurality of clock signals are lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of NPUs.

15. The system on chip according to claim 10, wherein: The phase of the second clock signal is slower than that of the first clock signal.

16. An electronic device comprising: Printed circuit substrates; A first circuit configured to perform operations for an ANN model and arranged on the printed circuit substrate for a plurality of neural processing units including a plurality of processing elements; and a second circuit arranged on the printed circuit substrate for a clock distributor, the clock distributor generating a plurality of clock signals having different phases from each other based on an original clock signal from a clock source and providing the clock signals to the plurality of NPUs by dividing the clock signals. The first clock signal among the multiple clock signals is provided to a first NPU among the multiple NPUs, and the second clock signal among the multiple clock signals is provided to a second NPU among the multiple NPUs.

17. The electronic device according to claim 16, wherein: At least one of the plurality of clock signals is configured to be supplied to one of the plurality of NPUs after passing through at least one output buffer arranged in consideration of reduction in peak power.

18. The electronic device according to claim 16, wherein: The frequencies of the multiple clock signals are lower than that of the original clock signal.

19. The electronic device according to claim 16, wherein: The frequencies of the plurality of clock signals are lower than the frequency of the original clock signal by a multiple corresponding to the number of the plurality of NPUs.

20. The electronic device according to claim 16, wherein: The phase of the second clock signal is slower than that of the first clock signal.

Images