Neural processor driven according to dynamically phase-adjusted clock signal

Abstract

An example of the present disclosure provides a neural processor that is driven according to a clock signal that dynamically adjusts a phase. The neural processor may include: a first circuit configured to be capable of performing an operation on an ANN (artistic network) model, the first circuit being configured for a plurality of PE groups including a plurality of processing units (PEs); a second circuit configured to output a plurality of clock signals to the first circuit; a third circuit configured to measure at least a ratio of a peak power to an average power of the first circuit; and a fourth circuit configured to dynamically calibrate at least one phase of the plurality of clock signals of the second circuit based on a ratio of peak power to average power measured by the third circuit.

Description

Technical Field

[0001] The present disclosure relates to a technology for reducing the instantaneous power consumption of a neural processor. Background Art

[0002] Artificial intelligence (AI) is also developing gradually. AI refers to artificially imitating human intelligence, that is, intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc.

[0003] In addition, recently, a neural processing unit (NPU) has been developed to increase the speed of calculations used for artificial intelligence (AI).

[0004] There are various types of artificial neural network (ANN) models depending on the purpose of artificial intelligence services. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. in the image / video can be used for artificial intelligence services.

[0005] Generally speaking, the amount of computation required for each layer of an artificial neural network model varies.

[0006] Especially in a specific layer, if the amount of calculation increases significantly, the power consumption will increase instantly. Summary of the invention

[0007] Artificial neural network operations are data-intensive. In particular, artificial neural network operations require parallel processing operations. In other words, artificial neural network operations have the characteristic of being slow in processing speed if a large amount of data cannot be processed simultaneously and quickly in parallel.

[0008] Therefore, the inventors of the present disclosure have developed a neural processor specifically for artificial neural network operations. The inventors of the present disclosure intend to increase the number of multiple processing units of the neural processor to improve the parallel processing performance of the neural processor. In addition, the inventors of the present disclosure intend to develop a neural processor that can operate at low power.

[0009] On the other hand, the power supply unit of the low-power operation dedicated neural processor installed in the peripheral device may have a relatively low power supply capacity compared to the power supply unit of the graphics processing unit (GPU) running in the server or personal computer (PC). In addition, the capacity of the capacitor of the power supply unit of the low-power operation dedicated neural processor installed in the peripheral device may be difficult to bear the instantaneous power supply.

[0010] However, the inventors of the present disclosure have realized that the more the number of processing units of the low-power operation dedicated neural processor increases, the greater the degree of instantaneous fluctuation of the supply voltage will be. Furthermore, the required instantaneous power supply of the neural processor will be proportional to the number of processing units running.

[0011] In addition, the computational load of the artificial neural network model varies greatly in each layer. Therefore, the number of processing units running in parallel will vary depending on the computational load of the layers of the artificial neural network model. In other words, when a large number of processing units are running simultaneously, the voltage of the power supply of the neural processor may fluctuate or drop instantly.

[0012] In addition, the amount of computation of a specific layer of the artificial neural network model may be very small. The inventors of the present disclosure recognize that even if the driving frequency of the neural processor is increased, the supply voltage stability of the neural processor can be guaranteed.

[0013] In addition, when the supply voltage fluctuates or drops instantaneously, the supply voltage VDD may need to be increased in order to ensure system safety. Therefore, if the system safety cannot be ensured, the supply voltage may increase unnecessarily. When the supply voltage increases, the power consumption of the neural processor increases sharply. The inventors of the present disclosure recognize that reducing the driving frequency of the neural processor at this time can stabilize the voltage of the power supply unit of the neural processor.

[0014] On the contrary, the more stable the supply voltage is, the lower the supply voltage VDD can be. Therefore, if the safety of the supply voltage of the neural processor is ensured, the supply voltage can be lowered, and ultimately the power consumption of the neural processor can be reduced.

[0015] Therefore, the inventors of the present disclosure recognized that there is a need for a technology that can improve the stability of the supply voltage of a neural processor dedicated to artificial neural network operations even if the peak power increases due to an increase in the number of processing units.

[0016] Therefore, the present disclosure aims to provide a technical solution for stabilizing the supply voltage fluctuation of the power supply unit caused by excessive power in the operation step.

[0017] In order to achieve the above-mentioned purpose, an example of the present disclosure provides a neural processor. The neural processor may include: a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured for multiple PE groups including multiple processing units PE; a second circuit, the second circuit is configured to output multiple clock signals to the first circuit; a third circuit, the third circuit is configured to at least measure the ratio of peak power to average power of the first circuit; and a fourth circuit, the fourth circuit is configured to dynamically calibrate at least one phase of the multiple clock signals of the second circuit based on the ratio of peak power to average power measured by the third circuit.

[0018] It can be configured such that: when the peak power value measured by the third circuit is higher than a critical power value, the fourth circuit dynamically adjusts at least one phase of the plurality of clock signals.

[0019] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0020] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit remeasures a second power value of the first circuit, and the fourth circuit determines whether the second power value is less than the first power value.

[0021] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0022] The one or more clock signals may include: a first clock signal for operating PEs of a first group among the plurality of PE groups; and a second clock signal for operating PEs of a second group among the plurality of PE groups.

[0023] The plurality of clock signals may include: a first clock signal having a first phase; and a second clock signal having a second phase that is later than a first phase of the first clock signal.

[0024] The plurality of clock signals may include: a first clock signal generated based on an original clock signal; and a second clock signal generated by shifting or drifting the original clock signal.

[0025] The second circuit may include: an original clock signal source; one or more phase converters, the one or more phase converters are connected in a chain form, and a first phase converter among the one or more phase converters is connected to the original clock signal source; and a selector, the selector is connected to the output of the original clock signal source and the output of the one or more phase converters, and selects at least one output from the output of the original clock signal source and the output of the one or more phase converters.

[0026] In order to achieve the above-mentioned purpose, an example of the present disclosure provides a system on chip. The system on chip may include: a semiconductor substrate; a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured on the semiconductor substrate for multiple PE groups including multiple processing elements (PE); a second circuit, the second circuit is configured on the semiconductor substrate to output multiple clock signals to the first circuit; a third circuit, the third circuit is configured on the semiconductor substrate to at least measure the ratio of peak power to average power of the first circuit; and a fourth circuit, the fourth circuit is configured on the semiconductor substrate to dynamically calibrate at least one phase of the multiple clock signals of the second circuit based on the ratio of peak power to average power measured by the third circuit.

[0027] When the power value measured by the third circuit is higher than a critical power value, the fourth circuit may be configured to dynamically adjust at least one phase of the plurality of clock signals.

[0028] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0029] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit measures a second power value of the first circuit again, and the fourth circuit determines whether the second power value is less than the first power value.

[0030] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0031] The one or more clock signals may include: a first clock signal for operating PEs of a first group among the plurality of PE groups; and a second clock signal for operating PEs of a second group among the plurality of PE groups.

[0032] In order to achieve the aforementioned objectives, an example of the present disclosure provides an electronic device. The electronic device may include: a printed circuit board; a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured on the printed circuit board for multiple PE groups including multiple processing units (PEs); a second circuit, the second circuit is configured on the printed circuit board so as to output multiple clock signals to the first circuit; a third circuit, the third circuit is configured on the printed circuit board so as to at least measure the ratio of peak power to average power of the first circuit; and a fourth circuit, the fourth circuit is configured on the printed circuit board so as to dynamically calibrate at least one phase of the multiple clock signals of the second circuit based on the ratio of peak power to average power measured by the third circuit.

[0033] When the power value measured by the third circuit is higher than a critical power value, the fourth circuit may be configured to dynamically adjust at least one phase of the plurality of clock signals.

[0034] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0035] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit measures a second power value of the first circuit again, and the fourth circuit determines whether the second power value is less than the first power value.

[0036] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0037] According to an example of the present disclosure, artificial neural network operations are distributed and run according to multiple clock signals, which has the effect of reducing the power of specific operation steps.

[0038] According to an example of the present disclosure, artificial neural network operations are dispersedly executed according to multiple clock signals, which has the effect of improving the safety of the supply voltage supplied to the neural processor.

[0039] According to the examples disclosed herein, artificial neural network operations are dispersedly performed based on multiple clock signals, which has the effect of reducing the supply voltage supplied to the neural processor and significantly reducing the power consumption of the neural processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A conceptual diagram describing an exemplary artificial neural network model.

[0041] Figure 2A A diagram describing the basic architecture of a convolutional neural network (CNN).

[0042] Figure 2B A comprehensive diagram showing the operation of a convolutional neural network in an easily understandable manner.

[0043] Figure 3 A conceptual diagram of a neural processor for describing an example of the present disclosure.

[0044] Figure 4A The present invention is a conceptual diagram for describing one processing unit among a plurality of processing units to which an example of the present disclosure may be applied.

[0045] Figure 4B The present invention is a conceptual diagram of an SFU to which an example of the present invention is applicable.

[0046] Figure 5 To show Figure 3 An example diagram of a variation of the neural processor 100 is shown.

[0047] Fig. 6A For an example diagram showing the data size of each layer in an exemplary artificial neural network model, Figure 6B To show Fig. 6A An example table showing data sizes, etc. for each layer in the exemplary artificial neural network model shown.

[0048] Fig. 7A The figure is an example diagram showing an NPU architecture of an example of the present disclosure.

[0049] Figure 7B The following is an example diagram showing an NPU architecture of another example of the present disclosure.

[0050] Fig. 8A To illustrate in detail according to an example Fig. 7A or Figure 7B An example diagram of clock source 170 is shown.

[0051] Figure 8B is a diagram showing an example of multiple clock signals output by the clock source 170 .

[0052] Fig. 9 The figure is an example diagram showing the SoC architecture of an example of the present disclosure.

[0053] Fig.10 The following is an exemplary diagram showing an operation method of an NPU according to another example of the present disclosure.

[0054] Fig.11 This shows an example in which clock signals having different phases from each other are not input to a plurality of PEs of the NPU.

[0055] Fig.12 According to an example of the present disclosure, an example of inputting clock signals with different phases into a plurality of PEs of an NPU in consideration of peak power is shown. DETAILED DESCRIPTION

[0056] The specific architecture and even the step-by-step description of the examples of the present disclosure are only used to describe the examples of the concepts of the present disclosure. Therefore, the examples of the concepts of the present disclosure can be implemented in various forms. It should not be interpreted that the present disclosure is limited to the examples of the present disclosure.

[0057] Various changes may be applied to the examples of the concepts of the present disclosure, and various forms may be provided. Therefore, specific examples are exemplarily shown in the accompanying drawings, and the present disclosure or application is described in detail. However, this is not intended to limit the examples of the concepts of the present disclosure to specific disclosed forms. It should be understood that the examples of the concepts of the present disclosure include all changes, equivalents, and even substitutes included in the ideas and technical scope of the present disclosure.

[0058] The terms first and / or second may be used to describe various constituent elements, but the constituent elements shall not be limited by the terms. The terms may only be used to distinguish one constituent element from another constituent element. The terms may be used to describe a first constituent element and a second constituent element, and similarly, a second constituent element may be named a first constituent element without departing from the scope of the present disclosure.

[0059] When it is mentioned that a certain component is "connected" or "connected to" another component, it should be understood that it can be directly connected to or connected to the other component, or other components can exist between multiple components. On the contrary, when it is mentioned that a certain component is "directly connected" or "directly connected to" another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, that is, "between" or "directly between" or "adjacent to" or "directly adjacent to" etc. should also be interpreted in the same way.

[0060] The terms used in this disclosure are only used to describe specific examples and are not intended to limit the meaning of this disclosure. As long as the context does not clearly indicate otherwise, the singular expression includes the plural expression. In this disclosure, the terms "including" or "having" are intended to specify the presence of the described features, numbers, steps, actions, constituent elements, components, or combinations thereof. It should therefore be understood that the presence or possibility of adding one or more other features or numbers, steps, actions, constituent elements, components, or combinations thereof is not precluded.

[0061] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the relevant technical context. Terms defined in commonly used dictionaries should not be idealized or overly interpreted as formal meanings unless explicitly defined in the present disclosure.

[0062] In describing the examples, the description of the technical contents that are well known in the technical field to which the present disclosure belongs and are not directly related to the present disclosure is omitted. The reason is that by omitting the redundant description, the main idea of ​​the present disclosure can be more clearly conveyed to avoid confusion.

[0063] <Definition of terms>

[0064] The terms used in the present disclosure are briefly summarized below to help understand the disclosure proposed in the present disclosure.

[0065] NPU: The abbreviation of Neural Processing Unit, which may mean a processor independent of CPU (Central Processing Unit) and dedicated to artificial neural network model calculations.

[0066] ANN is the abbreviation of artificial neural network, which means a network that connects nodes into layers by simulating neurons in the human brain through synapses in order to imitate human intelligence.

[0067] DNN: The abbreviation of Deep Neural Network, which can refer to a neural network with increased number of hidden layers of artificial neural network in order to achieve higher artificial intelligence.

[0068] CNN: Abbreviation for Convolutional Neural Network, a type of neural network that functions similarly to the image processing in the visual cortex of the human brain. Convolutional neural networks are known to be suitable for image processing, extracting features from input data and easily identifying characteristic patterns.

[0069] Hereinafter, preferred examples of the present disclosure are described with reference to the accompanying drawings to describe the present disclosure in detail. Hereinafter, examples of the present disclosure are described in detail with reference to the accompanying drawings.

[0070] <Artificial Intelligence>

[0071] Humans have intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc. Artificial intelligence (AI) means artificially imitating human intelligence.

[0072] The human brain is composed of a large number of nerve cells called neurons. Each neuron is connected to hundreds to thousands of other neurons through a connection site called a synapse. In order to imitate human intelligence, the working principle of biological neurons and the connection relationship between neurons are modeled, which is called an artificial neural network model. That is, an artificial neural network is a system that connects nodes that imitate neurons into a layer architecture.

[0073] This artificial neural network model is divided into "single-layer neural network" and "multi-layer neural network" according to the number of layers. An ordinary multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is the layer that obtains external data. The number of neurons in the input layer is the same as the number of input parameters. (2) The hidden layer is located between the input layer and the output layer. It receives signals from the input layer, extracts features, and transmits them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them to the outside. The input signals between neurons are multiplied by the connection strengths with values ​​between 0 and 1 and then added. If the sum is greater than the threshold of the neuron, the neuron is activated and the output value is realized through the activation function.

[0074] On the other hand, a neural network with an increased number of hidden layers in order to achieve higher artificial intelligence is called a deep neural network (DNN).

[0075] DNN is being developed in various architectures. For example, it is known that a convolutional neural network (CNN), which is an example of DNN, can easily extract features of input values ​​(images or pictures) and recognize patterns of the extracted output values. CNN can be composed of convolution operations, activation function operations, pooling operations, etc., which are processed in a specific order.

[0076] For example, in each layer of a DNN, parameters (i.e., input values, output values, weights, or kernels, etc.) may be matrices consisting of multiple channels. The parameters may be processed in the NPU through convolution or matrix multiplication. After the operations are performed in each layer, the output value is generated.

[0077] For example, Transformer is a DNN based on attention technology. Transformer uses multiple matrix multiplication operations. Transformer can obtain attention (Q, K, V) as the output value using parameters such as input value and query vector (query: Q), key vector (key: K) and value vector (value: V). Transformer can process various reasoning operations based on the output value (i.e., attention (Q, K, V)). Transformer tends to show better reasoning performance than CNN.

[0078] Figure 1 A conceptual diagram describing an exemplary artificial neural network model.

[0079] The operation of an exemplary artificial neural network model 110 a that may be executed in the neural processor 100 is described below.

[0080] Figure 1 The exemplary artificial neural network model 110a may be an artificial neural network trained to perform various reasoning functions such as object recognition and speech recognition.

[0081] The artificial neural network model 110 a may be a deep neural network (DNN).

[0082] However, the artificial neural network model 110a of the present disclosure example is not limited to a deep neural network.

[0083] For example, the artificial neural network model 110a can be implemented by models such as DaViT, MobileViT, Swin-Transformer, Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3, etc. However, the present disclosure is not limited to the above models. In addition, the artificial neural network model 110a can also be an integrated model based on at least two different models.

[0084] The following describes the inference process performed by the exemplary artificial neural network model 110a.

[0085] The artificial neural network model 110a is an exemplary deep neural network model including an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5, a third connection network 110a-6 and an output layer 110a-7. However, the present disclosure is not limited to Figure 1 The artificial neural network model shown. The first hidden layer 110a-3 and the second hidden layer 110a-5 may also be referred to as multiple hidden layers.

[0086] The input layer 110a-1 may illustratively include x1 and x2 input nodes. That is, the input layer 110a-1 may include information of 2 input values.

[0087] The first connection network 110a-2 may exemplarily include information of six weight values ​​for connecting each node of the input layer 110a-1 to each node of the first hidden layer 110a-3. Each weight value is multiplied by the input node value, and the accumulated value of the product value is stored in the first hidden layer 110a-3. The weight value and the input node value may be referred to as parameters of the artificial neural network model.

[0088] The first hidden layer 110a-3 may exemplarily include nodes a1, a2, and a3. That is, the first hidden layer 110a-3 may include information of three node values.

[0089] Figure 1 The first processing unit PE1 can process the operation of the a1 node.

[0090] Figure 1 The second processing unit PE2 can process the operation of the a2 node.

[0091] Figure 1 The third processing unit PE3 of the first hidden layer 110a-3 can process the operation of the node a3. The second connection network 110a-4 can exemplarily include information of 9 weight values ​​for connecting each node of the first hidden layer 110a-3 to each node of the second hidden layer 110a-5. The weight values ​​of the second connection network 110a-4 are multiplied by the node values ​​input from the first hidden layer 110a-3, and the accumulated value of the product values ​​is stored in the second hidden layer 110a-5.

[0092] The second hidden layer 110a-5 may exemplarily include b1, b2, and b3 nodes. That is, the second hidden layer 110a-5 may include information of 3 node values.

[0093] Figure 1 The fourth processing unit PE4 can process the operation of the b1 node.

[0094] Figure 1 The fifth processing unit PE5 can process the operation of the b2 node.

[0095] Figure 1 The sixth processing unit PE6 can process the operation of the b3 node.

[0096] The third connection network 110a-6 may include information of six weight values ​​for connecting each node of the second hidden layer 110a-5 with each node of the output layer 110a-7. The weight values ​​of the third connection network 110a-6 are multiplied by the node values ​​input from the second hidden layer 110a-5, and the accumulated value of the product values ​​is stored in the output layer 110a-7.

[0097] The output layer 110a-7 may illustratively include y1 and y2 nodes. That is, the output layer 110a-7 may include information of two node values.

[0098] Figure 1 The seventh processing unit PE7 can process the operation of the y1 node.

[0099] Figure 1 The eighth processing unit PE8 can process the operation of the y2 node.

[0100] Each node may correspond to a feature value, and the feature value may correspond to a feature graph.

[0101] Figure 2A A diagram describing the basic architecture of a convolutional neural network (CNN).

[0102] Reference Figure 2A The input image may be displayed as a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image may have a plurality of channels, where a channel may represent the number of color components of the input data image.

[0103] The convolution process means performing kernel and convolution operations while traversing the input image at specified intervals.

[0104] A convolutional neural network may have an architecture in which the output value (convolution or matrix multiplication) of the current layer is passed as the input value of the next layer.

[0105] For example, convolution is defined by two main parameters: input feature map and kernel. Parameters may include input feature map, output feature map, activation map, weights, kernel, and attention (Q, K, V).

[0106] Convolution is to slide the kernel window on the input feature map. The single size of the kernel sliding on the input feature map is called the stride.

[0107] After convolution, pooling can be applied. In addition, at the end of the convolutional neural network, a fully-connected (FC) layer can be configured.

[0108] Figure 2B A comprehensive diagram showing the operation of a convolutional neural network in an easily understandable manner.

[0109] Reference Figure 2B , exemplarily showing that the input image is a two-dimensional matrix with a size of 6×6. Figure 2B In the figure, the use of three nodes, namely channel 1, channel 2, and channel 3, is shown as an example.

[0110] First, the convolution operation is described.

[0111] Input image (in Figure 2B , exemplarily shown as 6×6 size) at the first node, with kernel 1 for channel 1 (in Figure 2B , exemplarily shown as a 3×3 size) convolution, as a result of which the output feature Figure 1 (exist Figure 2B In the example, the size is 4×4. Figure 2B , exemplarily shown as 6×6 size) at the second node with kernel 2 for channel 2 (in Figure 2B , exemplarily shown as a 3×3 size) convolution, as a result, the output feature map 2 (in Figure 2B In the example, the size is 4×4. In addition, the input image is combined with the kernel 3 for channel 3 at the third node (in Figure 2B , exemplarily shown as a 3×3 size) convolution, as a result of which the output feature Figure 3 (exist Figure 2B , exemplarily shown as a 4×4 size).

[0112] To process each convolution, the processing units PE1-PE12 of the neural processor 100 are configured to perform MAC operations.

[0113] Then, the operation of the activation function is described.

[0114] Features that can be output by self-convolution operations Figure 1 , Feature map 2 and features Figure 3 (exist Figure 2B In the example, the size of each is exemplarily shown as 4×4) and the activation function is applied. The output after applying the activation function can be exemplarily of 4×4 size.

[0115] Then, the pooling operation is described.

[0116] The features output by the activation function Figure 1 ,Feature map 2, features Figure 3 (exist Figure 2B , the size of each is exemplarily shown as 4×4) is input to 3 nodes. The feature map input output by the self-activation function can be accepted and pooling can be performed. The so-called pooling can reduce the size or strengthen specific values ​​in the matrix. The pooling methods include maximum pooling, average pooling, and minimum pooling. Maximum pooling is used to converge the maximum value of the value to a specific area of ​​the matrix, and average pooling can be used to obtain the mean value in a specific area.

[0117] exist Figure 2B In the example of , it is shown that the feature map of size 4×4 is reduced to size 2×2 with the help of pooling.

[0118] Specifically, the first node accepts the features for channel 1 Figure 1 After input and pooling, the output is, for example, a 2×2 matrix. The second node accepts feature map 2 for channel 2 as input and pooling is performed, for example, the output is a 2×2 matrix. The third node accepts feature map 3 for channel 4 as input. Figure 3 After input and pooling, the output is, for example, a 2×2 matrix.

[0119] Repeat the above convolution, activation function and pooling, and finally Fig. 8A As shown, the output can be fully connected. The corresponding output can be input again into the artificial neural network for image recognition. However, the present disclosure is not limited to the size of the feature map and the kernel.

[0120] The CNN described so far is the most widely used method among various deep neural network (DNN) methods in the field of computer vision. In particular, CNN has shown amazing performance in various research fields for performing various tasks such as image classification and object detection.

[0121] <Hardware resources required for ANN operations>

[0122] Figure 3 Conceptual schematic diagram of a neural processor for describing an example of the present disclosure.

[0123] Figure 3 The neural processor (NPU: neural processing unit) 100 shown is a processor optimized for performing operations for artificial neural networks.

[0124] An artificial neural network means a network formed by aggregating artificial neurons. After multiple inputs or stimuli come in, each weight is multiplied and added, and then the value added with the bias is transformed through an activation function and transmitted. The artificial neural network trained in this way can be used to output an inference result from input data.

[0125] The neural processor 100 can be a semiconductor implemented by an electrical / electronic circuit. The so-called electrical / electronic circuit can mean one including a large number of electronic components (such as transistors, capacitors).

[0126] For Transformer and / or CNN-based artificial neural network models, the neural processor 100 can identify and process matrix multiplication operations, convolution operations, etc. according to the architecture of the artificial neural network.

[0127] For example, in each layer of a convolutional neural network CNN, the input feature map corresponding to the input data and the kernel corresponding to the weight can be a tensor or matrix composed of multiple channels. The convolution operation of the input feature map and the kernel is performed to generate a convolution operation and a pooling output feature map in each channel. An activation function is applied to the output feature map to generate an activation map for the corresponding channel. Then, pooling of the activation map can be applied. Among them, the activation map can generally be called the output feature map. For the convenience of description below, the activation map will be referred to as the output feature map for description.

[0128] However, the example of the present disclosure is not limited to this, and the output feature map means that matrix multiplication operations, convolution operations, etc. are applied.

[0129] Furthermore, the output feature graph of the example disclosed herein should be interpreted as a general meaning. For example, the output feature graph may be a matrix multiplication operation or a convolution operation result value. Therefore, the multiple processing units 110 may also be implemented in a modified manner so as to also include a processing circuit unit for an additional algorithm. That is, a part of the circuit units of the SFU 150 described later may also be configured to be included in the multiple processing units 110.

[0130] The neural processor 100 may be configured to include a plurality of processing units 110 for processing the convolution and matrix multiplication required for the above-mentioned artificial neural network operations.

[0131] The neural processor 100 can be configured to include various processing circuits optimized for the matrix multiplication operations, convolution operations, activation function operations, pooling operations, stride operations, batch normalization operations, skip connection operations, cascade operations, quantization operations, cropping operations, and padding operations required for the above-mentioned artificial neural network operations.

[0132] For example, the neural processor 100 may be configured to include an SFU 150 for processing at least one of an activation function operation, a pooling operation, a stride operation, a batch normalization operation, a skip connection operation, a cascade operation, a quantization operation, a cropping operation, and a padding operation in the above algorithm.

[0133] Specifically, the neural processor 100 may include a plurality of PEs 110, an SFU 150, an NPU internal memory 120, an NPU controller 130, and an NPU interface 140. The plurality of processing units 110, the SFU 150, the NPU internal memory 120, the NPU controller 130, and the NPU interface 140 may be semiconductor circuits connected with a large number of transistors. Therefore, some of them may be difficult to identify and distinguish by naked eyes, and can only be identified based on operation.

[0134] For example, any circuit may operate as both the plurality of processing units 110 and the NPU controller 130. The NPU controller 130 may be configured to perform the functions of a control section configured to control the artificial neural network inference operation of the neural processor 100.

[0135] The neural processor 100 may include: an NPU internal memory 120, which is configured to store parameters of an artificial neural network model that can be used for inference in multiple processing units 110 and SFU 150; and an NPU controller 130, which includes a scheduler configured to control the operation scheduling of multiple processing units 110, SFU 150 and NPU internal memory 120.

[0136] The neural processor 100 may be configured to process the feature map in accordance with an encoding and decoding method using SVC (scalable video coding) or SFC (scalable feature-map coding). The method is a technology that can variably change the amount of data transmission according to the actual effective bandwidth and signal to noise ratio (SNR) of a communication channel or communication bus. That is, the neural processor 100 may also be configured to further include an encoder and a decoder.

[0137] The plurality of processing units 110 may perform a portion of the operations for the artificial neural network.

[0138] The SFU 150 may perform another portion of the operations for the artificial neural network.

[0139] The neural processor 100 may be configured to accelerate the operation of an artificial neural network model on hardware using a plurality of processing units 110 and SFUs 150 .

[0140] The NPU interface 140 can communicate with various components connected to the neural processor 100 through the system bus, such as a memory.

[0141] The NPU controller 130 may include a scheduler configured to control operations of the plurality of processing units 110 of the neural processor 100 for inference operations, operations of the SFU 150 , and a read and write order of the NPU internal memory 120 .

[0142] The scheduler in the NPU controller 130 may be configured to control the plurality of processing units 110 , the SFU 150 , and the NPU internal memory 120 based on data locality information or architecture information of the artificial neural network model.

[0143] The scheduler in the NPU controller 130 may analyze the architecture of the artificial neural network model to be run in the plurality of processing units 110 and the SFU 150 or obtain analyzed information. The analyzed information may be information generated by a compiler. For example, the data of the artificial neural network that may be included in the artificial neural network model may include at least a portion of the node data (i.e., feature graph) of each layer, the batch data of the layer, the locality information or the architecture information, and the weight data (i.e., the weight kernel) of each connection network connecting the nodes of each layer. The data of the artificial neural network may be stored in a memory equipped inside the NPU controller 130 or in the NPU internal memory 120. However, it is not limited thereto, and the data of the artificial neural network may be stored in an independent cache memory or register stack equipped in the NPU or the SoC including the NPU.

[0144] The scheduler in the NPU controller 130 can schedule the order of operations of the artificial neural network model to be executed by the neural processor 100 based on the data locality information or architecture information of the artificial neural network model.

[0145] The scheduler in the NPU controller 130 may obtain scheduling information of the operation order of the artificial neural network model to be executed by the neural processor 100 based on the data locality information or architecture information of the compiled artificial neural network model. For example, the scheduling information may be information generated by a compiler. The scheduling information generated by the compiler may be referred to as machine code or binary code, etc.

[0146] That is, the scheduling information used by the NPU controller 130 may be information generated by a compiler based on data locality information or architecture of the artificial neural network model.

[0147] Furthermore, the compiler can efficiently schedule the NPU based on the unique characteristics of the artificial neural network model, namely the understanding and reconstruction of the locality of the artificial neural network data.

[0148] Furthermore, the compiler can efficiently schedule the NPU based on its understanding of the hardware architecture and performance of the neural processor 100.

[0149] Furthermore, when the artificial neural network model is compiled by the compiler to run in the neural processor 100, the artificial neural network data locality can be reconstructed. The artificial neural network data locality can be reconstructed according to the algorithm applied by the artificial neural network model and the operating characteristics of the processor.

[0150] Furthermore, the locality of the artificial neural network data can be reconstructed according to the way in which the neural processor 100 processes the corresponding artificial neural network model, such as feature map tiling and the stationary mode of the processing unit.

[0151] Furthermore, the locality of the artificial neural network data can be reconstructed according to the number of processing units of the neural processor 100 and the capacity of the internal memory.

[0152] Furthermore, artificial neural network data locality may be reconstructed based on the bandwidth of the memory communicating with the neural processor 100 .

[0153] This is because, due to the above-mentioned factors, even if the same artificial neural network model is operated and processed, the neural processor 100 will determine the order of the data required at each moment differently in clock units.

[0154] The compiler may determine data locality based on the order of data required for the operation of the artificial neural network model, the layers of the artificial neural network, the order of operations of unit convolution and / or matrix multiplication, and generate compiled machine code.

[0155] The scheduler may be configured to utilize scheduling information included in the machine code.

[0156] The scheduler within the NPU controller 130 may obtain a memory address value storing feature maps and weight data of a layer of an artificial neural network model based on the scheduling information.

[0157] For example, the scheduler in the NPU controller 130 may obtain the memory address value of the feature map and weight data of the layer of the artificial neural network model stored in the memory. Therefore, the scheduler in the NPU controller 130 may export the feature map and weight data of the layer of the artificial neural network model to be driven from the autonomous memory and store them in the NPU internal memory 120.

[0158] The feature maps of each layer may have corresponding memory address values.

[0159] Each weight data may have a corresponding memory address value.

[0160] The scheduler in the NPU controller 130 may obtain scheduling information of the operation order of the plurality of processing units 110 based on data locality information or architecture information of the artificial neural network model, such as batch data, locality information or architecture information of a layer of the artificial neural network of the artificial neural network model. The scheduling information may be generated in the compilation step.

[0161] The scheduler in the NPU controller 130 operates based on the information for scheduling based on the data locality information or architecture information of the artificial neural network model, and thus can operate differently from the scheduling concept of an ordinary CPU. The scheduling of an ordinary CPU takes into account fairness, efficiency, stability, response time, etc. to achieve the highest efficiency. That is, scheduling is performed taking into account priority, operation time, etc. so that the most processing can be performed in the same time.

[0162] Conventional CPUs use an algorithm that takes into account data such as the priority of each process and the processing time to schedule tasks.

[0163] Different from this, the scheduler in the NPU controller 130 may control the neural processor 100 according to the processing order of the neural processor 100 determined based on the data locality information or architecture information of the artificial neural network model.

[0164] Furthermore, the scheduler within the NPU controller 130 may drive the NPU 100 according to a processing order determined based on data locality information or architecture information of the artificial neural network model and / or data locality information or architecture information of the neural processor 100 to be used.

[0165] However, the present disclosure is not limited to data locality information or architectural information of the neuroprocessor 100 .

[0166] The scheduler within the NPU controller 130 may be configured to store data locality information or architecture information of the artificial neural network.

[0167] That is, the scheduler in the NPU controller 130 can determine the processing order even if it only uses at least the data locality information or architecture information of the artificial neural network of the artificial neural network model.

[0168] Furthermore, the scheduler in the NPU controller 130 may determine the processing order of the neural processor 100 by taking into account the data locality information or architecture information of the artificial neural network model and the data locality information or architecture information of the neural processor 100. In addition, the processing of the neural processor 100 may also be optimized according to the determined processing order.

[0169] That is, the NPU controller 130 can be configured to run based on the machine code compiled by the compiler, but in another example, the NPU controller 130 can also be configured to have a built-in embedded compiler. According to the above configuration, the neural processor 100 can be configured to obtain files in the form of frameworks of various AI software to generate machine code. For example, the frameworks of AI software include TensorFlow, PyTorch, Keras, XGBoost, mxnet, DARKNET, ONNX, etc.

[0170] The plurality of processing units 110 means a configuration including a plurality of processing units PE1-PE12 configured to operate on feature maps and weight data of an artificial neural network. Each processing unit may include a multiplication and accumulation (MAC) operator and / or an arithmetic logic unit (ALU) operator. However, the examples disclosed herein are not limited thereto.

[0171] Each processing unit may be configured to further selectively include a supplementary special function unit to process supplementary special functions.

[0172] For example, the processing unit PE may also be implemented in a deformed manner to further include a batch normalization unit, an activation function unit, an interpolation unit, etc.

[0173] The SFU 150 may include a circuit portion configured to identify and process activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, padding operations, etc. according to the architecture of the artificial neural network. That is, the SFU 150 may include a plurality of special function operation processing circuit units.

[0174] Figure 3 , multiple processing units are shown as an example, however, within a processing unit, multiple operators implemented by multipliers and adder trees can be configured in parallel instead of MAC. In this case, the multiple processing units 110 can also be referred to as at least one processing unit including multiple operators.

[0175] The plurality of processing units 110 are configured to include a plurality of processing units PE1 ˜ PE12 . Figure 3 The multiple processing units PE1 to PE12 shown are only examples for ease of description, and the number of the multiple processing units PE1 to PE12 is not limited. The size or number of the multiple processing units 110 can be determined according to the number of the multiple processing units PE1 to PE12. The size of the multiple processing units 110 can be implemented in the form of an N×M matrix. Wherein N and M are integers greater than 0. The multiple processing units 110 may include N×M processing units. That is, the number of processing units may be more than one.

[0176] The sizes of the multiple processing units 110 may be designed taking into account the characteristics of the artificial neural network model run by the neural processor 100 .

[0177] The plurality of processing units 110 are configured to perform functions such as addition, multiplication, accumulation, etc. required for artificial neural network operations. In other words, the plurality of processing units 110 may be configured to perform multiplication and accumulation (MAC) operations.

[0178] The following description is made by taking the first processing unit PE1 among the multiple processing units 110 as an example.

[0179] Figure 4A The present invention is a conceptual diagram for describing one processing unit among a plurality of processing units to which an example of the present disclosure may be applied.

[0180] A neural processor 100 of an example of the present disclosure may include: a plurality of processing units 110; an NPU internal memory 120, the NPU internal memory 120 being configured to store an artificial neural network model that can be inferred in the plurality of processing units 110; and an NPU controller 130, the NPU controller 130 being configured to control the plurality of processing units 110 and the NPU internal memory 120; wherein the plurality of processing units 110 may be configured to perform MAC operations, and the plurality of processing units 110 may be configured to quantize and output MAC operation results. However, the examples of the present disclosure are not limited thereto.

[0181] The NPU internal memory 120 can store all or part of the artificial neural network model according to the memory size and the data size of the artificial neural network model.

[0182] The first processing unit PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. However, the examples disclosed herein are not limited thereto, and the plurality of processing units 110 may also be implemented in a modified manner taking into account the computational characteristics of the artificial neural network.

[0183] The multiplier 111 can multiply the input (N) bit data and (M) bit data. The operation value of the multiplier 111 is output as (N+M) bit data.

[0184] Multiplier 111 may be configured to accept a parameter and a constant input.

[0185] The accumulator 113 can use the adder 112 for (L) loops to add the operation value of the multiplier 111 and the operation value of the accumulator 113. Therefore, the bit width of the data of the output part and the input part of the accumulator 113 can be output as (N+M+log2(L)) bits, where L is an integer greater than 0.

[0186] After the accumulator 113 finishes accumulating, an initialization signal (initialization reset) may be input to initialize the data stored inside the accumulator 113 to 0. However, the examples of the present disclosure are not limited thereto.

[0187] The bit quantization unit 114 can reduce the bit width of the data output by the accumulator 113. The bit quantization unit 114 can be controlled by the NPU controller 130. The bit width of the quantized data can be output as (X)bit. Here, X is an integer greater than 0. According to the above structure, the multiple processing units 110 can be configured to perform MAC operations, and the multiple processing units 110 have the effect of quantizing and outputting the results of the MAC operations. In particular, this quantization has the effect of saving power as the number of (L)loops increases. In addition, if the power consumption is reduced, it can also reduce the heat generation. In particular, if the heat generation is reduced, it can reduce the possibility of malfunction of the neural processor 100 due to high temperature.

[0188] The output data (X) bit of the bit quantization unit 114 may become the node data of the next layer or the input data of the convolution. If the artificial neural network model has been quantized, the bit quantization unit 114 may be configured to obtain quantized information from the artificial neural network model. However, it is not limited to this, and the NPU controller 130 may also be configured to analyze the artificial neural network model and extract quantized information. Therefore, the output data (X) bit may be converted into a quantized bit width and output in a manner corresponding to the size of the quantized data. The output data (X) bit of the bit quantization unit 114 may be stored in the NPU internal memory 120 according to the quantized bit width.

[0189] The multiple processing units 110 of the neural processor 100 of an example of the present disclosure include a multiplier 111, an adder 112, and an accumulator 113. The bit quantization unit 114 can make a selection based on whether to apply quantization. In another example, the bit quantization unit can also be configured to be included in the SFU 150.

[0190] Figure 4B The present invention is a conceptual diagram of an SFU to which an example of the present invention is applicable.

[0191] refer to Figure 4B The SFU 150 includes a plurality of functional units. Each functional unit can be selectively operated. Each functional unit can be selectively turned on or off. That is, each functional unit can be set.

[0192] In other words, the SFU 150 may include various circuit units required for artificial neural network reasoning operations.

[0193] For example, the circuit units of SFU 150 may include functional units for skip-connection operations, functional units for activation function operations, functional units for pooling operations, functional units for quantization operations, functional units for non-maximum suppression (NMS) operations, functional units for integer to floating-point conversion (INT to FP32) operations, functional units for batch-normalization operations, functional units for interpolation operations, functional units for concatenation operations, and functional units for bias operations, etc.

[0194] The functional units of the SFU 150 can be selectively turned on or off according to the data locality information of the artificial neural network model. The data locality information of the artificial neural network model may include the turning off of the corresponding functional units or control information related to the turning off when performing operations of a specific layer.

[0195] The activated units in the functional units of the SFU 150 can be turned on. When a portion of the functional units of the SFU 150 is selectively turned off in this way, the power consumption of the neural processor 100 can be saved. On the other hand, in order to turn off a portion of the functional units, power gating can be used. Alternatively, in order to turn off a portion of the functional units, clock gating can also be performed.

[0196] Figure 5 To show Figure 3 An example diagram of a variation of the neural processor 100 is shown.

[0197] Figure 5 The neural processor 100 is shown with Figure 3 Compared with the processor 100 exemplarily shown in the figure, except for the multiple processing units 110, they are essentially the same, so they are not repeated below for the convenience of description.

[0198] Figure 5 The multiple processing units 110 exemplarily shown in FIG. 1 may include, in addition to the multiple processing units PE1 ˜ PE12 , respective register files RF1 ˜ RF12 corresponding to the respective processing units PE1 ˜ PE12 .

[0199] Figure 5The multiple processing units PE1-PE12 and the multiple register files RF1-RF12 shown are just examples for convenience of description, and the number of the multiple processing units PE1-PE12 and the multiple register files RF1-RF12 is not limited.

[0200] The size or number of the processing units 110 may be determined according to the number of the processing units PE1-PE12 and the register files RF1-RF12. The size of the processing units 110 and the register files RF1-RF12 may be implemented in the form of an N×M matrix, where N and M are integers greater than 0.

[0201] The array size of the plurality of processing units 110 may be designed taking into account the characteristics of the artificial neural network model run by the neural processor 100. Furthermore, the memory size of the register file may be determined taking into account the data size of the artificial neural network model to be run, the required running speed, the required power consumption, etc.

[0202] The register stacks RF1 to RF12 of the neural processor 100 are static memory units directly connected to the processing units PE1 to PE12. The register stacks RF1 to RF12 may be composed of, for example, triggers and / or latches. The register stacks RF1 to RF12 may be configured to store the MAC operation values ​​of the corresponding processing units PE1 to PE12. The register stacks RF1 to RF12 may be configured to send and receive weight data and / or node data with the NPU internal memory 120.

[0203] Register files RF1-RF12 may also be configured to perform a temporary storage function of an accumulator during MAC operation.

[0204] <Technical Difficulties Discovered by the Inventors of the Present Disclosure>

[0205] There are various types of artificial neural network (ANN) models depending on the purpose of artificial intelligence services. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. in the image / video can be used for artificial intelligence services.

[0206] Generally speaking, the amount of computation in each layer of an artificial neural network model is different, which is described with reference to FIG6 .

[0207] Fig. 6A An example diagram showing the data size of each layer in an exemplary artificial neural network model. Figure 6B To show Fig. 6A An example table showing data sizes, etc. for each layer in the exemplary artificial neural network model shown.

[0208] Fig. 6AThe exemplary artificial neural network model shown is shown as Mobilenet V1. Fig. 6A The horizontal axis represents the layers in the exemplary artificial neural network model in sequence, and the vertical axis represents the data size.

[0209] refer to Fig. 6A As shown in layer 1, the size of the output feature map (OFMAP_SIZE) is larger than the size of the input feature map (IFMAP_SIZE).

[0210] The output feature map of layer 1 is passed to layer 2 and becomes the input feature map of layer 2. After the operation of layer 2 is completed, the output feature map is output. The output feature map of layer 2 is then passed to layer 3 and becomes the input feature map of layer 3.

[0211] Therefore, the size of the input data to each layer and the size of the output feature map output by each layer may be different. Therefore, the amount of computation may be small in any layer, but it may be large in other layers. Therefore, as the amount of computation of each layer changes greatly, the problem of difficulty in power control occurs.

[0212] Each layer can be divided into multiple operation steps for processing according to the number of processing units of the neural processor and the capacity limit of the NPU internal memory 120. Therefore, the neural processor 100 can be scheduled to divide each layer into multiple tiles to process multiple operation steps. The scheduling can be performed by the compiler. For example, a layer can be divided into 4 tiles. Each tile can be processed by the neural processor in sequence. The compiled artificial neural network model can store the operation order information determined at compile time to the scheduler. At this time, each operation step can provide PE utilization information.

[0213] The amount of computation of each layer can be obtained through MAC. It can be seen that the maximum difference between the amount of computation of each layer is 227 times.

[0214] The neural processor 100 may determine the number of processing units used in the plurality of processing units 110 in proportion to the MAC computation amount in each computation step of each layer. Furthermore, power consumption increases in proportion to the number of processing units used.

[0215] If the number of processing units used and the total number of processing units are known, the utilization rate (%) of the neural processor 100 can be calculated.

[0216] Therefore, the PE utilization rate can be calculated for each operation step of each layer of the artificial neural network model processed in the neural processor 100. In addition, as long as the architecture of the artificial neural network model processed by the neural processor 100 does not change, the PE utilization rate of each operation step can be repeatedly and accurately predicted. That is, the neural processor 100 can repeatedly infer a specific artificial neural network model. The inventors of the present disclosure recognize that at this time, the neural processor 100 can repeatedly use the same weight parameters and the same network layer architecture.

[0217] In addition, the greater the difference in MAC operation amount between adjacent layers, the greater the peak power deviation between adjacent layers. Moreover, the greater the peak power deviation between adjacent layers, the more the supply voltage VDD will fluctuate. The inventors of the present disclosure recognize that the driving frequency can be changed by taking into account the stability of the supply voltage VDD.

[0218] In particular, the inventors of the present disclosure have recognized that, in a calculation step of a specific layer, if the amount of calculation increases significantly, the power consumption increases instantaneously, thereby causing a problem of reduced system stability.

[0219] For example, a large number of processing units can be started simultaneously for the operation of a specific layer of operation steps. A predetermined power supply is required to drive each processing unit. If a considerable number of processing units are driven at the same time, the required power will increase sharply in an instant. In the case where the neural processor is specially designed for low-power operation, the power supply capacity will be relatively insufficient compared to the neural processor for server use. Therefore, this type of neural processor for peripheral devices will be relatively weak in instantaneous power supply, and when the power supply increases sharply, the supply voltage VDD will fluctuate. In particular, when the supply voltage VDD drops below the critical voltage of the transistor, the data stored in the transistor will be damaged. In other words, when the supply voltage VDD decreases, the operating speed of the transistor is slow, setup / hold violations occur, and thus misoperation will occur. In particular, the smaller the process of the semiconductor foundry, such as 3nm, 4nm, 5nm, and 7nm, the more serious this phenomenon will be.

[0220] For another example, the inventors of the present disclosure recognize that the PE 110 that performs artificial neural network operations, such as addition, multiplication, and accumulation, consumes a large amount of power instantly, so that other components in the neural processor, such as the internal memory 120, cannot obtain sufficient power supply. Specifically, the inventors of the present disclosure recognize that if the internal memory 120 cannot obtain sufficient power supply, there will be a problem that it cannot be ruled out that the stored data bits may be damaged (compromised).

[0221] Furthermore, the inventors of the present disclosure have recognized that when the PE utilization rate in a specific operation step is low, the voltage stability of the neural processor can be guaranteed even if the driving frequency of the neural processor is increased.

[0222] <Disclosure of this Patent>

[0223] The inventors of the present disclosure have recognized that the aforementioned problem occurs because the operation of the artificial neural network model is performed on hardware based on a clock signal.

[0224] Therefore, the inventor of this patent divides the multiple PEs in the NPU into a first group of PEs and a second group of PEs so that they operate according to different clock signals, thereby inventing a method for reducing peak power.

[0225] An example of implementing the method invented by the inventors of the present disclosure is described below with reference to the accompanying drawings.

[0226] Fig. 7A The figure is an example diagram showing an NPU architecture of an example of the present disclosure.

[0227] exist Fig. 7A , it is shown that the NPU 100 includes a first group of PEs 110 - 1 , a second group of PEs 110 - 2 , an internal memory 120 , an SFU 150 , a clock source 170 , a power management integrated circuit (PMIC) 180 , and a phase determination circuit 190 .

[0228] In some examples, the NPU 100 may be configured as follows: Figure 3 or Figure 5 As shown, an NPU controller 130 and an NPU interface 140 may also be included.

[0229] The plurality of PEs may be divided into a first group of PEs 110 - 1 and a second group of PEs 110 - 2 .

[0230] The plurality of PEs may also be referred to as NPU cores, NPU engines, NPU threads, etc. The NPU cores, NPU engines, NPU threads, etc. may be semiconductor circuits connected with a large number of transistors.

[0231] The first group of PEs 110-1, the second group of PEs 110-02, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150, the clock source 170, the PMIC 180, and the phase determination circuit 190 may be semiconductor circuits connected with a large number of transistors. Therefore, some of them are difficult to identify by naked eyes and can only be identified by operation. The clock source 170 is as follows: Fig. 8A As shown, a clock signal generating circuit 171 and a phase converter 175 may be included, which will be described later.

[0232] For example, the first group of PEs 110 - 1 and the second group of PEs 110 - 2 may be referred to as a first circuit.

[0233] For example, the clock source 170 including the phase converter 175 may be referred to as a second circuit.

[0234] For example, the PMIC 180 may be referred to as a third circuit.

[0235] For example, the phase determination circuit 190 may be referred to as a fourth circuit.

[0236] However, examples of the present disclosure are not limited thereto, and any circuit may operate as a plurality of PEs 110 , or may also operate as an NPU controller 130 .

[0237] Fig. 7A The first group of PE 110-1 and the second group of PE 110-2 shown in FIG. Figure 3 or Figure 5 The number of PEs in each group is Fig. 7A 8 PEs are shown as an example, but it goes without saying that the number of PEs in each group may be different. In addition, the number of PEs in each group may be semi-fixed or dynamically changed. For example, it may be set that the first group of PEs 110-1 includes 10 PEs and the second group of PEs 110-2 includes 6 PEs. Such changes may be performed according to the control of the NPU controller 130.

[0238] The PMIC 180 may be a circuit configured to measure power of at least one circuit of the NPU 100 .

[0239] For example, the PMIC 180 may be configured to measure the power of the first group of PEs 110-1 during operation. For example, the PMIC 180 may be configured to measure the power of the second group of PEs 110-1 during operation. For example, the PMIC 180 may be configured to measure the power of the internal memory 120 during operation. For example, the PMIC 180 may be configured to measure the power of the SFU 150 during operation. However, the examples of the present disclosure are not limited thereto, and may be configured to measure the power of various circuits.

[0240] The PMIC 180 may be configured to measure instantaneous power in real time. At this time, the instantaneous power with the maximum value may be defined as the peak power. The PMIC 180 may be configured to measure the average power within a specific period.

[0241] Further, PMIC 180 may be configured to measure peak power and average power. PMIC 180 may store peak power values ​​occurring over a short period of time. In addition, average power may be measured based on an integral value of power measured over a given period. PMIC 180 may provide power values ​​in a graphical form along a time axis during operation.

[0242] When the average power is the same and the peak power is relatively large during a specific period, the stability of the supply voltage VDD will be relatively unstable.

[0243] Therefore, the phase determination circuit 190 may be a circuit designed to calculate the ratio of the average power measured by the PMIC 180 during a specific period to the average power of the peak power during the corresponding period. However, the examples of the present disclosure are not limited thereto, and the ratio may also be calculated by the PMIC 180 or other circuits.

[0244] If the peak power is 1 W and the average power is 1 W, the ratio of the peak power to the average power may be 1: 1. That is, the ratio of the peak power to the average power in a specific period may be 100%.

[0245] If the peak power is 1 W and the average power is 2 W, the ratio of the peak power to the average power may be 1: 2. That is, the ratio of the peak power to the average power in a specific period may be 200%.

[0246] If the peak power is 1 W and the average power is 3 W, the ratio of the peak power to the average power may be 1: 3. That is, the ratio of the peak power to the average power in a specific period may be 300%.

[0247] That is, the more the ratio of the peak power to the average power increases, the more unstable the stability of the supply voltage VDD is determined by the phase determination circuit 190 .

[0248] That is, the PMIC 180 may provide the phase determination circuit 190 with data of a ratio of average power to peak power during a specific period while the NPU 100 is operating.

[0249] The average power value and the peak power value measured by the PMIC 180 may be stored in a specific memory, a register file, etc. For example, the measured average power value, peak power value, and / or a ratio value of peak power to average power may be stored in the phase determination circuit 190. However, examples of the present disclosure are not limited thereto.

[0250] In some embodiments, the power available to multiple PEs may increase with the utilization of multiple PEs. Further, the power available to PEs may increase in direct proportion to the utilization of PEs. This characteristic will be Figure 11 to Figure 12Detailed description is given in . Therefore, the phase determination circuit 190 may also derive multiple PE utilizations based on the power measured by the PMIC 180. In addition, when the number of PEs and the power consumption information of each PE are obtained, the phase determination circuit 190 may more accurately predict the PE utilization based on the power measured by the PMIC 180. Therefore, the phase determination circuit 190 may adjust at least one phase of the multiple clock signals supplied to each group of PEs to reduce the peak power as the utilization of the PE group increases.

[0251] For example, the PMIC 180 may measure a first peak power value of the plurality of PE groups 110-1 and 110-2, and the PMIC 180 may provide the first peak power value to the phase determination circuit 190. Therefore, the phase determination circuit 190 may calibrate at least one phase of the plurality of clock signals. Then, the PMIC 180 may be configured to measure a second peak power value of the plurality of PE groups 110-1 and 110-2 again, and the phase determination circuit 190 determines whether the second peak power value is less than the first peak power value.

[0252] In some embodiments, the phase determination circuit 190 may obtain information about the amount of operation of a specific operation step of a specific ANN model processed by the multiple PE groups 110-1 and 110-2 or the utilization rate of the multiple PE groups 110-1 and 110-2. The phase determination circuit 190 may be configured to generate a control signal based on the amount of operation of a specific operation step of a specific ANN model processed by the multiple PE groups 110-1 and 110-2 or the utilization rate of the multiple PE groups 110-1 and 110-2. That is, the IR-drop margin may be predicted based on the utilization rate of the multiple PE groups 110-1 and 110-2 of the specific operation step of the specific artificial neural network model. Therefore, when the IR-drop margin is expected to drop below a threshold, a control signal may be determined based on the utilization rate of the multiple PE groups 110-1 and 110-2 and the IR-drop margin to ensure the IR-drop margin.

[0253] In some embodiments, the phase determination circuit 190 may dynamically adjust the phase of at least one of the plurality of clock signals.

[0254] In some examples, the phase determination circuit 190 may adjust at least one phase of the plurality of clock signals based on a preset critical power value so as to have a preset phase difference. The critical power value may be set by taking into account the peak power characteristic corresponding to the PE utilization and the peak power reduction characteristic corresponding to the phase difference. For example, when the peak power value measured by the PMIC 180 is higher than the critical power value, the phase determination circuit 190 may be configured to dynamically adjust at least one phase of the plurality of clock signals. For example, at least one phase of the plurality of clock signals may be adjusted based on the critical power value, which is preset based on the utilization of the plurality of PE groups.

[0255] The clock source 170 may be a circuit configured to output one or more clock signals.

[0256] In order to reduce the peak power of the NPU 100, the phase determination circuit 190 may generate a control signal based on the ratio of the peak power to the average power, and the control signal is used to calibrate or adjust at least one phase of the one or more clock signals to be output by the clock source 170. The control signal may be input to the phase determination circuit 190.

[0257] In the following Fig.11 and Fig.12 In the following, an example and a comparative example of the present disclosure in which at least one phase of a plurality of clock signals is calibrated to reduce peak power even if average power is the same will be described.

[0258] The clock source 170 may dynamically adjust and output at least one phase of the multiple clock signals corresponding to the control signal of the phase determination circuit 190. At least one phase of the multiple clock signals may be adjusted based on the ratio of the peak power to the average power of the multiple PEs.

[0259] For example, when the ratio of peak power to average power is measured to be higher than before, in order to reduce the peak power, the clock source 170 may output a phase delayed clock signal that shifts or drifts the phase of the original clock signal according to a control signal from the phase determination circuit 190.

[0260] After outputting the phase delayed clock signal, the phase determination circuit 190 may compare the stored peak power value with the peak power value measured after phase calibration.

[0261] If the peak power value measured after the phase calibration is lower than the original peak power value, the phase determination circuit 190 may determine that the phase calibration is successful. Therefore, the phase determination circuit 190 may continue to maintain the calibrated phase difference.

[0262] If the peak power value measured after the phase calibration is higher than the original peak power value, the phase delay determination circuit 190 may determine that the phase calibration has failed. Therefore, the phase determination circuit 190 may restore the previous phase.

[0263] The phase calibration unit may be executed as a one-time inference unit. If described in detail, when a specific artificial neural network model is inferred once, the measured average power may be the same regardless of whether the phases of the multiple clocks input to the multiple PE groups are the same or different. However, the peak power may vary depending on the phase. Therefore, the phase determination circuit 190 may input the control signal used to generate various phase differences into the clock source 170, and derive the phase difference with the lowest peak power value.

[0264] Here, one inference may refer to processing all operations from the input layer to the output layer of the artificial neural network model. That is, one inference may be referred to as one image frame inference. Therefore, the phase calibration may be performed in at least one image frame unit or one inference unit.

[0265] For example, 10 control signals having 10 different phases may be input to the clock source 170 per image frame. Therefore, the phase determination circuit 190 may measure the peak power of each phase during the 10 frames. As a result, the phase determination circuit 190 may derive the optimal phase difference of the plurality of clock signals based on the results measured during the 10 frames. Therefore, the optimal phase difference result value may be stored in the phase determination circuit 190.

[0266] The phase determination circuit 190 may maintain the phase difference until the artificial neural network model processed by the NPU 100 is changed. If the peak power increases, the phase determination circuit 190 may also perform phase calibration of at least one clock among the multiple clock signals again. If the artificial neural network model is changed, the phase determination circuit 190 may be configured to perform the phase calibration step again.

[0267] The clock source 170 is as follows Fig. 8A As shown, it may include a clock signal generating circuit 171 and a phase converter 175. The clock signal generating circuit 171 may include an oscillator. In addition, the oscillator may include a PWM (Pulse Width Modulator). The PWM may change the clock cycle into a duty cycle, thereby adjusting the ratio of the high level signal to the low level signal.

[0268] If the clock signal generating circuit 171 in the clock source 170 generates and outputs an original clock signal, then according to the control signal from the phase determining circuit 190, the phase converter 175 in the clock source 170 may output the original clock signal as is (bypass), or may output the original clock signal with a phase delay (delay) or shift (shift).

[0269] For example, the phase converter 175 in the clock source 170 may operate according to the control signal from the phase determination circuit 190. The phase converter 175 may pass the original clock signal generated by the clock signal generation circuit 171 to the first group of PEs 110-1 as it is (bypass). Or the phase converter 175 may pass the first delayed clock signal that delays or shifts the phase of the original clock signal generated by the clock signal generation circuit 171 to the first group of PEs 110-1. In addition, according to the control signal from the phase determination circuit 190, the phase converter 175 may pass the second delayed clock signal that delays or shifts the phase of the original clock signal generated by the clock signal generation circuit 171 to the second group of PEs 110-2.

[0270] As mentioned above, an example of the present disclosure will Figure 3 or Figure 5 The multiple PE110 shown, such as Fig. 7A After being divided into the first PE group 110-1 and the second PE group 110-2, the first PE group 110-1 and the second PE group 110-2 are operated in a distributed manner according to different clock signals, so that the peak power of the multiple PE groups 110-1 and 110-2 can be reduced.

[0271] In addition, in an example of the present disclosure, in order to provide a plurality of clock signals with different phases, a plurality of clock sources 170 may not be provided, but the manufacturing unit price may be reduced by simply adding a phase converter 175 .

[0272] Figure 7B The following is an example diagram showing an NPU architecture of another example of the present disclosure.

[0273] exist Figure 7B In FIG. 1 , the NPU 100 is shown to include a first group of PEs 110-1, a second group of PEs 110-2, an internal memory 120, and an SFU 150. However, in addition to this, the NPU 100 may also include a first group of PEs 110-1, a second group of PEs 110-2, an internal memory 120, and an SFU 150. Figure 3 or Figure 5 The NPU controller 130 and the NPU interface 140 are shown.

[0274] The first group of PEs 110-1, the second group of PEs 110-02, the internal memory 120, the NPU controller 130, the NPU interface 140 and the SFU 150 may be semiconductor circuits connected with a large number of transistors, and therefore, some of them may be difficult to identify by naked eyes and can only be identified by operation.

[0275] Reference Figure 7B , the clock source 170, the PMIC 180 and the phase determination circuit 190 are shown as being located outside the NPU 100 rather than inside. Fig. 7A The contents shown are the same, so it is incorrect Figure 7B Described separately, but directly quoted reference Fig. 7A The content of the description.

[0276] on the other hand, Fig. 7A and Figure 7B The internal memory 120 and the SFU 150 shown may also operate according to the clock signal (eg, the original clock signal or the phase-delayed clock signal) output from the phase converter 175 .

[0277] On the other hand, Fig. 7A and Figure 7B After the first group of PEs 110-1 and the second group of PEs 110-2 complete the operation of any layer of the artificial neural network model, the first group of PEs 110-1 and the second group of PEs 110-2 can respectively output output feature maps according to the clock signals respectively provided from the phase converter 175. Specifically, the first group of PEs 110-1 can output a first output feature map according to any clock signal (e.g., the original clock signal or the first delayed clock signal) provided from the phase converter 175. Similarly, the second group of PEs 110-2 can output a second output feature map according to any clock signal (e.g., the second delayed clock signal) provided from the phase converter 175.

[0278] When the first output feature map output by the first group of PEs 110-1 and the second output feature map output by the second group of PEs 110-2 are independent of each other, the SFU 150 may first process the output feature map transmitted in a FIFO (First Input First Output) manner.

[0279] On the other hand, although Fig. 7A and Figure 7B Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0280] Available Fig. 7Aand Figure 7B Each unit in the NPU 100 is shown as inputting a supply voltage VDD.

[0281] exist Fig. 7A and Figure 7B , a common supply voltage VDD is shown as being input, however examples of the present disclosure are not limited thereto.

[0282] In some embodiments, it may also be configured to input an independent or separate supply voltage VDD to each unit in the NPU 100 .

[0283] In some other embodiments, it may also be configured to input an independent or separate supply voltage VDD to each unit in the NPU 100 .

[0284] In some other examples, it may also be configured to input a second supply voltage different from the first supply voltage to a specific unit in the NPU 100 .

[0285] on the other hand, Fig. 7A and Figure 7B After the first group of PE 110-1 and the second group of PE 110-2 have completed the operation of any layer of the artificial neural network model, the first group of PE 110-1 and the second group of PE 110-2 may output the operation result, that is, output the feature map, according to any part of the first part and the second part of the clock. To this end, at least one of the first group of PE 110-1 and the second group of PE 110-2 may also include a time delay buffer (e.g., a shift register). The time delay buffer (e.g., a shift register) may perform a time delay so that the output is performed according to the later part of the first part and the second part of the clock.

[0286] For example, the output terminals of the second group of PEs 110 - 2 may be connected to a time delay buffer (eg, a shift register).

[0287] or, Fig. 7A and Figure 7B The first group of PEs 110-1 shown may output a first output feature map according to the first part of the clock, and the second group of PEs 110-2 may output a second output feature map according to the second part of the clock. In this case, the SFU 150 may include a time delay buffer (e.g., a shift register). The SFU 150 may time delay the output feature map received according to the earlier part of the first and second parts of the clock, so as to achieve time synchronization with the output feature map received according to the later part of the first and second parts of the clock.

[0288] Alternatively, when the first output feature graph output by the first group of PEs 110-1 and the second output feature graph output by the second group of PEs 110-2 are independent of each other, the SFU 150 may first process the output feature graph received according to the earlier part in time of the first and second parts of the clock.

[0289] That is, the first output from the first group of PEs 110 - 1 and the second output from the second group of PEs 110 - 2 may be transferred based on the first portion of the clock.

[0290] The reference phase for the second group of PEs 110-2 may be transformed from the second portion of the clock to the first portion.

[0291] on the other hand, Fig. 7A and Figure 7B Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0292] On the other hand, Fig. 7A and Figure 7B The NPU is described in , however, the NPU can also be implemented in a SoC form factor.

[0293] Fig. 8A To illustrate in detail according to an example Fig. 7A or Figure 7B An example diagram of a clock source 170 is shown, Figure 8B is a diagram showing an example of multiple clock signals output by the clock source 170 .

[0294] like Fig. 8A As shown, the clock source 170 may include a clock signal generating circuit 170 and a phase converter 175 .

[0295] The phase converter 175 may include a plurality of phase shifters S and a multiplexer / selector (MUX / Selector) 175-5. Fig. 8A , it is exemplarily shown that there are four phase shifters S (175-1, 175-2, 175-3, 175-4).

[0296] Please refer to the following Fig. 8A and Figure 8B Give a description.

[0297] The original clock signal output from the clock signal generating circuit 171 may be output as is (bypassed) in the phase converter 175 through the multiplexer / selector (MUX / Selector) 175 - 5 .

[0298] In addition, the phase of the original clock signal output from the clock signal generating circuit 171 is delayed once by the first phase shifter 175-1 in the phase converter 175, so that it can be converted into a first delayed clock signal and then output through the multiplexer / selector (MUX / Selector) 175-5.

[0299] Moreover, the phase of the original clock signal output from the clock signal generating circuit 171 is delayed twice by the first phase shifter 175-1 and the second phase shifter 175-2 in the phase converter 175, so that it can be output through the multiplexer / selector (MUX / Selector) 175-5 after being converted into a second delayed clock signal.

[0300] Moreover, the phase of the original clock signal output from the clock signal generating circuit 171 is delayed three times by the first phase shifter 175-1, the second phase shifter 175-2 and the third phase shifter 175-3 in the phase converter 175, so that it can be converted into a third delayed clock signal and output through the multiplexer / selector (MUX / Selector) 175-5.

[0301] Similarly, the phase of the original clock signal output from the clock signal generating circuit 171 is delayed four times by the first phase shifter 175-1, the second phase shifter 175-2, the third phase shifter 175-3 and the fourth phase shifter 175-4 in the phase converter 175, so that it can be converted into a fourth delayed clock signal and output through the multiplexer / selector (MUX / Selector) 175-5.

[0302] According to the control signal from the phase determination circuit 190, the multiplexer / selector (MUX / Selector) 175-5 is as follows Figure 8B As shown, after multiple clock signals (i.e., the original clock signal, the first delayed clock signal, the second delayed clock signal, the third delayed clock signal, and the fourth delayed clock signal) are multiplexed, a clock signal is selectively provided to each part within the NPU.

[0303] For example, according to the control signal from the phase determination circuit 190, the multiplexer / selector (MUX / Selector) 175-5 can provide the original clock signal to the internal memory 120, the first delayed clock signal to the first group of PEs 110-1, the second delayed clock signal to the second group of PEs 110-2, and the third delayed clock signal to the SFU 150. As described above, each part in the NPU operates according to a plurality of clock signals with different phases, thereby suppressing a sudden rise in peak power.

[0304] In some examples of the present disclosure, the clock source 170 may be a circuit including at least one of a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The clock source 170 may be a circuit in which at least two of the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0305] Fig. 9 The figure is an example diagram showing the SoC architecture of an example of the present disclosure.

[0306] Reference Fig. 9 , the exemplary SoC 1000 may include a plurality of NPUs, a plurality of CPUs (central processing units), and a plurality of memories. The plurality of NPUs may include, for example, a first NPU 100-1 and a second NPU 100-2. Moreover, the plurality of CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.

[0307] Each of the plurality of NPUs (eg, the first NPU 100-1 and the second NPU 100-2) may be configured as follows: Fig. 7A and Figure 7B The figure shows a first group of PEs and a second group of PEs.

[0308] exist Fig. 9 In the figure, it is shown that the number of the multiple NPUs, the multiple CPUs and the multiple memories is 2 respectively, but it is not limited to this, and the number can be varied in various ways, such as 4, 6, 8, etc.

[0309] The exemplary SoC 1000 may include a memory controller 350 , a clock source 170 , a phase converter 175 , a system bus 500 , and an I / O interface 800 .

[0310] The system bus 500 may be implemented by an electrically conductive pattern formed on a semiconductor die. The system bus may perform high-speed communication. For example, the plurality of NPUs, the plurality of CPUs, the plurality of memories, and the memory controller 350 may communicate with each other via the system bus 500.

[0311] The plurality of NPUs and the plurality of CPUs make requests to the memory controller 350 through the system bus 500. Therefore, the memory controller 350 may read data from or record data in at least one of the plurality of memories.

[0312] Fig. 9 The clock source 170, the PMIC 180 and the phase determination circuit 190 shown in FIG. Fig. 7A , Figure 7B , Fig. 8A The content shown is the same.

[0313] Therefore, according to the control signal from the phase determination circuit 190, the clock source 170 may provide the original clock signal to the first group of PEs in the first NPU 100-1, and at the same time, may provide the first phase-delayed clock signal after the phase of the original clock signal is delayed once to the second group of PEs in the first NPU 100-1. Alternatively, according to the control signal from the phase determination circuit 190, the clock source 170 may provide the first phase-delayed clock signal after the phase of the original clock signal from the clock source 170 is delayed by a specific phase difference to the first group of PEs in the first NPU 100-1, and at the same time, may provide the second phase-delayed clock signal after the phase of the original clock signal is delayed twice to the second group of PEs in the first NPU 100-1.

[0314] Similarly, according to the control signal from the phase determination circuit 190, the clock source 170 may provide the original clock signal to the first group of PEs in the second NPU 100-2, and at the same time, may provide the second phase-delayed clock signal after the phase of the original clock signal is delayed once to the second group of PEs in the second NPU 100-2. Alternatively, according to the control signal from the phase determination circuit 190, the clock source 170 may provide the first phase-delayed clock signal after the phase of the original clock signal is delayed once to the first group of PEs in the second NPU 100-2, and at the same time, provide the second phase-delayed clock signal after the phase of the original clock signal is delayed twice to the second group of PEs in the second NPU 100-2.

[0315] Available Fig. 9 Each unit (ie, NPU, memory, and CPU) within the SoC 1000 is shown to input a supply voltage VDD.

[0316] exist Fig. 9 , a common supply voltage VDD is shown as being input, however examples of the present disclosure are not limited thereto.

[0317] In some embodiments, it may also be configured to input an independent or separate supply voltage VDD to each unit (ie, NPU, memory, and CPU) within the SoC 1000 .

[0318] In some other examples, it may also be configured to input an independent or separate supply voltage VDD to some units (ie, NPU, memory, and CPU) within the SoC 1000 .

[0319] In some other examples, it may also be configured to input a second supply voltage different from the first supply voltage to a specific unit in the SoC 1000 .

[0320] Fig.10 The figure is an example diagram showing an operation method of an NPU according to an example of the present disclosure.

[0321] Reference Fig.10 , multiple clock signals for driving multiple PEs may be generated S1010. The multiple clock signals may include an original clock signal and multiple phase-delayed clock signals that are phase-delayed from the original clock signal.

[0322] To determine the phase delay, the power of the NPU may be measured. The power may vary with the utilization of multiple PEs. To reduce the peak power of the NPU, the phase of the original clock signal may be calibrated or adjusted based on the measured power.

[0323] Furthermore, the plurality of PEs may be divided into a first group of PEs and a second group of PEs S1020. It goes without saying that the number of PEs belonging to each group may be different. In addition, the number of PEs belonging to each group may be semi-fixed or dynamically changed. Such a change may be performed according to the control of the NPU controller 130.

[0324] Then, the first group of PEs may be driven based on the first clock signal S1030. In addition, the second group of PEs may be driven based on the second clock signal S1040.

[0325] According to one aspect of the present disclosure, a method for driving a neural processing unit (NPU) is provided. The NPU driving method may include: a step of calibrating at least one phase of one or more clock signals based on the measured power in order to reduce the peak power of the NPU; and a step of outputting the one or more clock signals. The one or more clock signals may include a first clock signal and a second clock signal. The NPU driving method may include: a step of running a first group of PEs (processing elements) based on the first clock signal; and a step of running a second group of PEs based on the second clock signal.

[0326] Fig.11 This shows an example in which clock signals with different phases from each other are not input to a plurality of PEs of the NPU.

[0327] Reference Fig.11 The first clock CLK0 can be input to the first group of PE 110-1 and the second group of PE 110-2. Each group of PE (110-1, 110-2) is configured to obtain clock signal inputs of the same phase.

[0328] Fig.11 100×PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when 100 processing units are activated.

[0329] Fig.11 200×PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when 200 processing units are activated.

[0330] Fig.11 500×PE utilization exemplarily shows the peak power and supply voltage VDD variation characteristics when 500 processing units are activated.

[0331] Reference Fig.11 , showing that the more the number of activated processing units of the neural processor 100 increases, the more the peak power increases and the more the supply voltage VDD decreases.

[0332] Furthermore, the number of PEs running per clock cycle will vary depending on the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be run per unit clock. Among them, the ratio of running PEs in all PEs can be called PE utilization (utilization rate%). On the contrary, the better the performance of the compiler, the more the peak power of the NPU will increase. That is, the peak power can be proportional to the PE utilization. Therefore, as the compiler algorithm becomes more sophisticated, there is a problem that the peak power of the NPU will further increase.

[0333] Refer again Fig.11 The peak power of the first group of PEs 110-1 and the second group of PEs 110-2 running based on the first clock CLK0 will vary with the number of running PEs in each clock. That is, the peak power will increase in proportion to the number of running PEs during one clock.

[0334] Furthermore, when the NPU developed for low power is applied to peripheral devices, it can be developed corresponding to the M.2 interface or the PCI Express interface. In particular, the maximum power of the M.2 interface will be relatively lower than that of the PCI Express interface. For example, the M.2 interface may have a 3A current and a 10W power limit. When the peak power of the NPU of the comparative example exceeds 10W during a specific clock period, the supply voltage VDD supplied to the NPU will fluctuate. At this time, the higher the utilization rate of the PE, the more the peak power of the NPU increases, and the more the peak power exceeds the power limit, the more the degree of decrease in the supply voltage VDD will further increase in proportion to the exceeding information.

[0335] When the supply voltage VDD drops below the IR-drop margin, data errors will occur in the neural processor operation. Further, this occurs due to the resistance R of the metal interconnect and the current I flowing through it. According to Ohm's law, IR-drop occurs when a voltage drop occurs (V=IR). If a large number of circuits associated with multiple PEs of the NPU are switched simultaneously, the NPU will consume a considerable amount of current. This high current will cause a larger voltage drop in the NPU, reducing the supply voltage in other parts of the SoC.

[0336] Summarize Fig.11 For example, to ensure stable operation of the NPU and prevent potential errors, sufficient IR-drop margin must be ensured. IR-drop margin can refer to the difference between the supply voltage VDD and the lowest voltage allowed at all locations of the NPU during peak power conditions.

[0337] Fig.12 According to an example of the present disclosure, an example of inputting clock signals with different phases into a plurality of PEs of an NPU in consideration of peak power is shown.

[0338] It should be noted that the more the utilization of the processing unit increases, the more the power will increase proportionally. Therefore, the power of each operation step can be calculated based on the PE utilization information and the driving frequency information of each operation step. The power calculation can be calculated based on the scheduling information of a specific artificial neural network model. Among them, the power (Watt) can be calculated as the value obtained by dividing the energy consumed by the corresponding operation step by the unit time (that is, 1watt (W) = 1joule per second (J / s)). Therefore, the power of each operation step can be calculated based on the scheduling information.

[0339] Reference Fig.12 The first clock CLK1 can be input to the first group of PE 110-1, and the second clock CLK2 can be input to the second group of PE 110-2. Each group of PE (110-1, 110-2) is configured to obtain clock signal inputs with different phases.

[0340] Fig.12 100×PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when 100 processing units are activated.

[0341] Fig.12 200×PE utilization exemplarily shows peak power and supply voltage VDD variation characteristics when 200 processing units are activated.

[0342] Fig.12 500×PE utilization exemplarily shows the peak power and supply voltage VDD variation characteristics when 500 processing units are activated.

[0343] Reference Fig.12 , showing that the more the number of activated processing units of the neural processor 100 increases, the more the peak power increases and the more the driving voltage VDD decreases. However, it is shown that as the signal clocks with different phases are input to each group of PE, the overall peak power tends to decrease, and the fluctuation of the supply voltage VDD also tends to decrease.

[0344] in, Fig.12 An example of the average power and Fig.11 The average power of the comparative example is substantially the same. However, Fig.12 The peak power of an example is less than Fig.11 Therefore, according to Fig.12 An example of Fig.11 Compared with the comparative example, the ratio of peak power to average power can be improved.

[0345] Furthermore, the number of PEs running per clock will vary depending on the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be run per unit clock. Among them, the ratio of running PEs in all PEs can be called PE utilization (utilization rate%). On the contrary, the better the performance of the compiler, the more the peak power of the NPU will increase. That is, the peak power can be proportional to the PE utilization. Therefore, as the compiler algorithm becomes more sophisticated, there is a problem that the peak power of the NPU will further increase.

[0346] Refer again Fig.12 The peak power of the first group of PEs 110-1 and the second group of PEs 110-2 corresponding to the first clock CLK1 will vary with the number of PEs running during each clock period. That is, the peak power will increase in proportion to the number of PEs running during one clock period.

[0347] Furthermore, when the NPU developed with low power is applied to peripheral devices, it can be developed corresponding to the M.2 interface or the PCI Express interface. In particular, the maximum power of the M.2 interface will be relatively lower than that of the PCI Express interface. For example, the M.2 interface may have a 3A current and a 10W power limit. When the peak power of the NPU of the comparative example exceeds 10W during a specific clock period, the supply voltage VDD supplied to the NPU will fluctuate. At this time, the higher the utilization rate of the PE, the greater the peak power increases, and the more the peak power exceeds the power limit, the greater the degree of decrease in the supply voltage VDD will be.

[0348] When the supply voltage VDD drops below the IR-drop margin, data errors may occur in the neural processor calculations.

[0349] On the other hand, the first clock CLK1 can be connected to the first group of PEs 110-1, and the second clock CLK2 with a different phase from the first clock CLK1 can be connected to the second group of PEs 110-2. Therefore, the peak power of each group of PEs can be dispersed. From the perspective of NPU peak power, the peak power is relatively Fig.11 The comparison example can be halved. At this time, the voltage stability of the supply voltage VDD is Fig.11 Compared with the comparative example shown, it is relatively more stable.

[0350] That is, according to an example of the present disclosure, the peak power of the NPU can be reduced, and the peak power of the SoC including the NPU can also be reduced.

[0351] In addition, when it is a low power interface such as M.2, since the power limit will be lower, it will be more efficient in such a low power interface. However, the present disclosure is not limited to a specific interface.

[0352] <Brief Summary of Examples of the Present Disclosure>

[0353] The contents of the present disclosure described so far are summarized as follows.

[0354] An example of the present disclosure provides a neural processor. The neural processor may include: a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured for multiple PE groups including multiple processing elements (PEs); a second circuit, the second circuit is configured to output multiple clock signals to the first circuit; a third circuit, the third circuit is configured to at least measure the ratio of peak power to average power of the first circuit; and a fourth circuit, the fourth circuit is configured to dynamically calibrate at least one phase of the multiple clock signals of the second circuit based on the ratio of peak power to average power measured by the third circuit.

[0355] When the power value measured by the third circuit is higher than a critical power value, the fourth circuit may be configured to dynamically adjust at least one phase of the plurality of clock signals.

[0356] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0357] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit measures a second power value of the first circuit again, and the fourth circuit determines whether the second power value is less than the first power value.

[0358] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0359] The one or more clock signals may include: a first clock signal for operating PEs of a first group among the plurality of PE groups; and a second clock signal for operating PEs of a second group among the plurality of PE groups.

[0360] The plurality of clock signals may include: a first clock signal having a first phase; and a second clock signal having a second phase that is later than the first phase of the first clock signal.

[0361] The plurality of clock signals may include: a first clock signal generated based on an original clock signal; and a second clock signal generated by shifting or drifting the original clock signal.

[0362] The second circuit may include: an original clock signal source; one or more phase converters, the one or more phase converters are connected in a chain form, and a first phase converter among the one or more phase converters is connected to the original clock signal source; and a selector, the selector is connected to the output of the original clock signal source and the output of the one or more phase converters, and selects at least one output from the output of the original clock signal source and the output of the one or more phase converters.

[0363] An example of the present disclosure provides a system on chip. The system on chip may include: a semiconductor substrate; a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured on the semiconductor substrate for a plurality of PE groups including a plurality of processing elements (PE); a second circuit, the second circuit is configured on the semiconductor substrate to output a plurality of clock signals to the first circuit; a third circuit, the third circuit is configured on the semiconductor substrate to at least measure the ratio of the peak power to the average power of the first circuit; and a fourth circuit, the fourth circuit is configured on the semiconductor substrate to dynamically calibrate at least one phase of the plurality of clock signals of the second circuit based on the ratio of the peak power to the average power measured by the third circuit.

[0364] When the power value measured by the third circuit is higher than a critical power value, the fourth circuit may be configured to dynamically adjust at least one phase of the plurality of clock signals.

[0365] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0366] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit measures a second power value of the first circuit again, and the fourth circuit determines whether the second power value is less than the first power value.

[0367] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0368] The one or more clock signals may include: a first clock signal for operating PEs of a first group among the plurality of PE groups; and a second clock signal for operating PEs of a second group among the plurality of PE groups.

[0369] An example of the present disclosure provides an electronic device. The electronic device may include: a printed circuit board; a first circuit, the first circuit is configured to perform operations on an ANN (artificial neural network) model, and is configured on the printed circuit board for multiple PE groups including multiple processing units (PEs); a second circuit, the second circuit is configured on the printed circuit board to output multiple clock signals to the first circuit; a third circuit, the third circuit is configured on the printed circuit board to at least measure the ratio of peak power to average power of the first circuit; and a fourth circuit, the fourth circuit is configured on the printed circuit board to dynamically calibrate at least one phase of the multiple clock signals of the second circuit based on the ratio of peak power to average power measured by the third circuit.

[0370] When the power value measured by the third circuit is higher than a critical power value, the fourth circuit may be configured to dynamically adjust at least one phase of the plurality of clock signals.

[0371] At least one phase of the plurality of clock signals may be adjusted based on a critical power value, wherein the critical power value is preset based on utilization rates of the plurality of PE groups.

[0372] It can be configured as follows: the third circuit measures a first power value of the first circuit, the third circuit provides the first power value to the fourth circuit, the fourth circuit calibrates at least one phase of the multiple clock signals, and then the third circuit measures a second power value of the first circuit again, and the fourth circuit determines whether the second power value is less than the first power value.

[0373] It can be configured as follows: the third circuit measures a first power value at a first time point, the fourth circuit compares the first power value at the first time point with a preset critical power value, and calibrates at least one phase of the multiple clock signals.

[0374] The examples of the present disclosure shown in this specification and the drawings are only used to easily describe the technical content of the present disclosure, and to disclose specific examples to help understand the present disclosure, and are not intended to limit the scope of the present invention. It goes without saying that a person of ordinary skill in the art to which the present disclosure belongs can implement other variations in addition to the examples described so far.

[0375]

National R&D Project to Support This Invention

[0376]

Project unique number

[0377]

Project number

[0378]

Department name

[0379]

Project Management (Professional) Organization Name

[0380] [Research Project Name] PIM Artificial Intelligence Semiconductor Core Technology Development (Design)

[0381]

Research Topic Name

[0382]

Project Execution Organization Name

[0383]

Study period

Claims

1. A neural processor, comprising: A first circuit configured to perform operations on an artificial neural network model and configured for a plurality of processing unit groups including a plurality of processing units; a second circuit configured to output a plurality of clock signals to the first circuit; a third circuit configured to measure at least a ratio of a peak power to an average power of the first circuit; and A fourth circuit is configured to dynamically calibrate at least one phase of the plurality of clock signals of the second circuit based on a ratio of peak power to average power measured by the third circuit.

2. The neural processor according to claim 1, wherein: When the peak power value measured by the third circuit is higher than the critical power value, The fourth circuit is configured to dynamically adjust at least one phase of the plurality of clock signals.

3. The neural processor according to claim 1, wherein: At least one phase of the plurality of clock signals is adjusted based on a critical power value, and the critical power value is preset based on utilization of the plurality of processing unit groups.

4. The neural processor of claim 1, wherein: Configured as: The third circuit measures a first peak power value of the first circuit, The third circuit provides the first peak power value to the fourth circuit, The fourth circuit calibrates at least one phase of the plurality of clock signals, Then, the third circuit remeasures the second peak power value of the first circuit, The fourth circuit determines whether the second peak power value is less than the first peak power value.

5. The neural processor of claim 1, wherein: Configured as: The third circuit measures a first peak power value at a first time point, The fourth circuit compares the first peak power value at the first time point with a preset critical power value to calibrate at least one phase of the plurality of clock signals.

6. The neural processor of claim 1, wherein: The multiple clock signals include: a first clock signal for operating a first group of processing units in the plurality of groups of processing units; and A second clock signal for operating processing units of a second group of the plurality of processing unit groups.

7. The neural processor of claim 1, wherein: The multiple clock signals include: a first clock signal having a first phase; and A second clock signal having a second phase, the second phase being later than a first phase of the first clock signal.

8. The neural processor of claim 1, wherein: The multiple clock signals include: a first clock signal generated based on the original clock signal; and A second clock signal is generated by shifting or drifting the original clock signal.

9. The neural processor of claim 1, wherein: The second circuit comprises: Original clock signal source; One or more phase converters, wherein the one or more phase converters are connected in a chain form, and a first phase converter among the one or more phase converters is connected to the original clock signal source; and A selector is connected to the output of the original clock signal source and the output of the one or more phase converters, and selects at least one output from the output of the original clock signal source and the output of the one or more phase converters.

10. A system on chip, comprising: Semiconductor substrates; A first circuit configured to perform operations on an ANN model and arranged on the semiconductor substrate for a plurality of processing unit groups including a plurality of processing units; a second circuit, the second circuit being arranged on the semiconductor substrate so as to output a plurality of clock signals to the first circuit; a third circuit, the third circuit being disposed on the semiconductor substrate so as to at least measure a ratio of a peak power to an average power of the first circuit; and A fourth circuit is configured on the semiconductor substrate to dynamically calibrate at least one phase of the plurality of clock signals of the second circuit based on a ratio of peak power to average power measured by the third circuit.

11. The system on chip according to claim 10, wherein: When the peak power value measured by the third circuit is higher than the critical power value, The fourth circuit is configured to dynamically adjust at least one phase of the plurality of clock signals.

12. The system on chip according to claim 10, At least one phase of the plurality of clock signals is adjusted based on a critical power value, and the critical power value is preset based on utilization rates of the plurality of PE groups.

13. The system on chip according to claim 10, wherein: Configured as: The third circuit measures a first peak power value of the first circuit, The third circuit provides the first peak power value to the fourth circuit, The fourth circuit calibrates at least one phase of the plurality of clock signals, Then, the third circuit measures the second peak power value of the first circuit again, The fourth circuit determines whether the second peak power value is less than the first peak power value.

14. The system on chip according to claim 10, wherein: Configured as: The third circuit measures a first peak power value at a first time point, The fourth circuit compares the first peak power value at the first time point with a preset critical power value to calibrate at least one phase of the plurality of clock signals.

15. The system on chip according to claim 10, wherein: The multiple clock signals include: a first clock signal for operating PEs of a first group of the plurality of PE groups; and A second clock signal for operating PEs of a second group of the plurality of PE groups.

16. An electronic device comprising: Printed circuit boards; A first circuit configured to perform operations on an artificial neural network model and arranged on the printed circuit board for a plurality of PE groups including a plurality of processing units PE; a second circuit, the second circuit being configured on the printed circuit board so as to output a plurality of clock signals to the first circuit; a third circuit, the third circuit being disposed on the printed circuit board so as to measure at least a ratio of a peak power to an average power of the first circuit; and A fourth circuit is configured on the printed circuit board to dynamically calibrate at least one phase of the plurality of clock signals of the second circuit based on a ratio of peak power to average power measured by the third circuit.

17. The electronic device according to claim 16, wherein: When the peak power value measured by the third circuit is higher than the critical power value, The fourth circuit is configured to dynamically adjust at least one phase of the plurality of clock signals.

18. The electronic device according to claim 16, wherein: At least one phase of the plurality of clock signals is adjusted based on a critical power value, and the critical power value is preset based on utilization of the plurality of processing unit groups.

19. The electronic device according to claim 16, wherein: Configured as: The third circuit measures a first peak power value of the first circuit, The third circuit provides the first peak power value to the fourth circuit, The fourth circuit calibrates at least one phase of the plurality of clock signals, Then, the third circuit measures the second peak power value of the first circuit again, The fourth circuit determines whether the second peak power value is less than the first peak power value.

20. The electronic device according to claim 16, wherein: Configured as: The third circuit measures a first peak power value at a first time point, The fourth circuit compares the first peak power value at the first time point with a preset critical power value to calibrate at least one phase of the plurality of clock signals.

Images