Techniques for reducing neural processor power with variable frequency

Abstract

One aspect of the present disclosure presents a system. The system may include a neural processor (NPU: artifact network), the NPU including a plurality of processing units (PE: processing agents) capable of performing operations on at least one artificial neural network (ANN: artifact network) in a model of the NPU, and at least one of the plurality of processing units (PE: processing agents) capable of performing operations on the at least one ANN in the model of the NPU, the plurality of processing units (PE: processing agents), the plurality of processing units (PE: processing agents), the plurality of processing units (PE: processing agents), the plurality of processing units (PE: processing agents), the plurality of processing units (PE: processing agents), the plurality of processing units (PE: processing agents), and the plurality of processing units (PE: processing agents). And a switching circuit. The switching circuit may select one of a plurality of clock signals having mutually different frequencies, and supply the selected one clock signal to the NPU. The one clock signal may be selected based on utilization of a given layer of a plurality of layers within the at least one ANN model by the PE.

Description

Technical Field

[0001] The present disclosure relates to a technique for reducing power of a neural processor using variable frequency. Background Art

[0002] Artificial intelligence (AI) is also developing gradually. AI refers to artificially imitating human intelligence, which can perform recognition, classification, inference, prediction, control / decision making, etc.

[0003] In addition, recently, a neural processing unit (NPU) has been developed to increase the speed of calculations used for artificial intelligence (AI).

[0004] Depending on the purpose of the artificial intelligence service, there may be various types of artificial neural network (ANN) models. For example, when the input data is an image or video, for the artificial intelligence service, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. in the image / video may be used.

[0005] Generally speaking, the amount of computation required for each layer of an artificial neural network model varies.

[0006] Especially in a specific layer, if the amount of calculation increases significantly, the power consumption will increase instantly. Summary of the invention

[0007] Technical issues

[0008] Artificial neural network operations are data-intensive. In particular, artificial neural network operations require parallel processing operations. In other words, artificial neural network operations have the characteristic of being slow if they cannot simultaneously and quickly process large amounts of data in parallel.

[0009] Therefore, the inventors of the present disclosure have developed a neural processor specifically for artificial neural network operations. The inventors of the present disclosure intend to increase the number of multiple processing units of the neural processor to improve the parallel processing performance of the neural processor. In addition, the inventors of the present disclosure intend to develop a neural processor that can operate at low power.

[0010] On the other hand, the power supply unit of the low-power operation dedicated neural processor installed in the edge device may have a relatively low power supply capacity compared to the power supply unit of the graphics processing unit (GPU) running in a server or a personal computer (PC). In addition, the capacity of the capacitor of the power supply unit of the low-power operation dedicated neural processor installed in the edge device may be insufficient for instantaneous power supply.

[0011] However, the inventors of the present disclosure have realized that the more the number of processing units of the low-power operation dedicated neural processor increases, the greater the degree of instantaneous fluctuation of the supply voltage will be. Furthermore, the required instantaneous power supply of the neural processor will be proportional to the number of processing units running.

[0012] In addition, the computational load of the artificial neural network model varies greatly in each layer. Therefore, the number of processing units running in parallel will vary depending on the computational load of the layers of the artificial neural network model. In other words, when a large number of processing units are running simultaneously, the voltage of the power supply of the neural processor may fluctuate or drop instantly.

[0013] In addition, the amount of computation of a specific layer of the artificial neural network model may be very small. The inventors of the present disclosure recognize that even if the driving frequency of the neural processor is increased, the supply voltage stability of the neural processor can be guaranteed.

[0014] In addition, when the supply voltage fluctuates or drops instantaneously, the supply voltage VDD may need to be increased in order to ensure system safety. Therefore, if the system safety cannot be ensured, the supply voltage may increase unnecessarily. When the supply voltage increases, the power consumption of the neural processor increases sharply. The inventors of the present disclosure recognize that reducing the driving frequency of the neural processor at this time can stabilize the voltage of the power supply unit of the neural processor.

[0015] On the contrary, the more stable the supply voltage is, the lower the supply voltage VDD can be. Therefore, if the safety of the supply voltage of the neural processor is ensured, the supply voltage can be lowered, and ultimately the power consumption of the neural processor can be reduced.

[0016] Therefore, the inventors of the present disclosure recognized that it is necessary to improve the stability of the supply voltage of a neural processor dedicated to artificial neural network operations by adjusting the driving frequency even if the amount of operations increases dramatically.

[0017] Therefore, the present disclosure aims to provide a technical solution to stabilize the supply voltage fluctuation of the power supply unit caused by excessive power in the operation step.

[0018] Technical Solution

[0019] In order to achieve the aforementioned objectives, an example of one aspect of the present disclosure provides a system. The system may include: a neural processor (NPU: artificial neural network), the NPU including a plurality of processing units (PE: processing element) capable of performing operations on at least one ANN in an artificial neural network (ANN: artificial neural network) model; and a switching circuit. The switching circuit may select one of a plurality of clock signals having mutually different frequencies, and supply the selected one clock signal to the NPU. The one clock signal may be selected based on the utilization of a given layer of a plurality of layers within the at least one ANN model by the PE.

[0020] The one clock signal may be selected further considering a utilization threshold of the PE.

[0021] In order to reduce the power of a particular computational step when performing computations in the given layer, a utilization threshold of the PE may be determined.

[0022] The one clock signal may be selected when the NPU completes operation on a layer above the given layer in the at least one ANN model.

[0023] The one clock signal may be selected for the given layer based on previous layer information output by the NPU by completing an operation on a layer previous to the given layer in the at least one ANN model.

[0024] The switch circuit may include: a generator that outputs a plurality of clock signals; and a selector that selects the one clock signal.

[0025] When the utilization rate of the given layer is higher than that of an upper layer, a clock signal selected for the given layer may have a second frequency that is lower than a first frequency of the upper layer.

[0026] When the utilization rate of the given layer is lower than that of an upper layer, a clock signal selected for the given layer may have a second frequency that is higher than a first frequency of the upper layer.

[0027] The utilization of the PE may be determined for each of a plurality of layers of the at least one ANN model.

[0028] The utilization of the PEs may be determined based on scheduling information, where the scheduling information is used to distribute operations of each layer of the at least one ANN model to the plurality of PEs.

[0029] To achieve the aforementioned objectives, one aspect of the present disclosure provides a system-on-chip (SoC). The SoC may include: a first circuit configured to enable a plurality of processing elements (PEs) to perform operations on at least one ANN model in an artificial neural network (ANN) model; and a switch circuit. The switch circuit may select one of a plurality of clock signals having different frequencies and provide the selected one clock signal to the NPU. The one clock signal may be selected based on the utilization of a given layer of a plurality of layers within the at least one ANN model by the PE.

[0030] To achieve the aforementioned objectives, one aspect of the present disclosure provides a method for driving a neural processing unit (NPU). The NPU driving method may include: a step of selecting one clock signal from a plurality of clock signals having different frequencies; and a step of supplying the selected one clock signal to the NPU. The NPU may include a plurality of processing elements (PEs) capable of performing operations on at least one ANN in an artificial neural network (ANN) model. The one clock signal may be selected based on the utilization of a given layer among a plurality of layers within the at least one ANN model by the PE.

[0031] Beneficial Effects

[0032] According to an example of the present disclosure, artificial neural network operations are dispersed and run according to different frequencies, which has the effect of reducing the power of specific operation steps.

[0033] According to an example of the present disclosure, artificial neural network operations are dispersedly performed according to different frequencies, which has the effect of improving the safety of the supply voltage supplied to the neural processor.

[0034] According to the examples disclosed herein, artificial neural network operations are dispersedly performed based on multiple clock signals, which has the effect of reducing the supply voltage supplied to the neural processor and significantly reducing the power consumption of the neural processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A conceptual diagram describing an exemplary artificial neural network model.

[0036] Figure 2a A diagram describing the basic architecture of a convolutional neural network (CNN).

[0037] Figure 2b A comprehensive diagram showing the operation of a convolutional neural network in an easily understandable manner.

[0038] Figure 3 A conceptual diagram of a neural processor for describing an example of the present disclosure.

[0039] Figure 4a The present invention is a conceptual diagram for describing one processing unit among a plurality of processing units to which an example of the present disclosure may be applied.

[0040] Figure 4b The present invention is a conceptual diagram of an SFU to which an example of the present invention is applicable.

[0041] Figure 5 To show Figure 3 An example diagram of a variation of the neural processor 100 is shown.

[0042] Figure 6a For an example diagram showing the data size of each layer in an exemplary artificial neural network model, Figure 6b To show Figure 6a An example table showing data sizes, etc. for each layer in the exemplary artificial neural network model shown.

[0043] Figure 7a The present invention briefly illustrates an exemplary method proposed in an example of the present disclosure.

[0044] Figure 7b The present invention is an exemplary diagram showing an example of a clock signal used by a method proposed in an example of the present disclosure.

[0045] Figure 7c The schematic diagram briefly shows the hardware structure for implementing the method proposed in an example of the present disclosure.

[0046] Figure 8a FIG. 1 is an exemplary diagram showing a SoC architecture of an example of the first disclosure of the present disclosure.

[0047] Figure 8b FIG. 1 is an exemplary diagram showing a SoC architecture of another example of the first disclosure of the present disclosure.

[0048] Figure 9a The diagram is an example of a system architecture showing an example of the second disclosure of the present disclosure.

[0049] Figure 9b The diagram is an example of a system architecture showing an example of the second disclosure of the present disclosure.

[0050] Fig.10a An example diagram showing a neural processor architecture of an example of the third disclosure of the present disclosure.

[0051] Fig.10bAn example diagram showing a neural processor architecture of another example of the third disclosure of the present disclosure.

[0052] Fig.11 An example is shown where peak power increases with increasing PE utilization.

[0053] Fig.12 An example is shown in which power is used differently when a clock signal is used according to PE utilization. DETAILED DESCRIPTION

[0054] The specific architecture and even the step-by-step description of the examples of the present disclosure are only used to describe the examples of the concepts of the present disclosure. Therefore, the examples of the concepts of the present disclosure can be implemented in various forms. It should not be interpreted that the present disclosure is limited to the examples of the present disclosure.

[0055] Various changes may be applied to the examples of the concepts of the present disclosure, and various forms may be provided. Therefore, specific examples are exemplarily shown in the accompanying drawings, and the present disclosure or application is described in detail. However, this is not intended to limit the examples of the concepts of the present disclosure to specific disclosed forms. It should be understood that the examples of the concepts of the present disclosure include all changes, equivalents, and even substitutes included in the ideas and technical scope of the present disclosure.

[0056] The terms first and / or second may be used to describe various constituent elements, but the constituent elements shall not be limited by the terms. The terms may only be used to distinguish one constituent element from another constituent element. The terms may be used to describe a first constituent element and a second constituent element, and similarly, a second constituent element may be named a first constituent element without departing from the scope of the present disclosure.

[0057] When it is mentioned that a certain component is "connected" or "connected to" another component, it should be understood that it can be directly connected to or connected to the other component, or other components can exist between multiple components. On the contrary, when it is mentioned that a certain component is "directly connected" or "directly connected to" another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, that is, "between" or "directly between" or "adjacent to" or "directly adjacent to" etc. should also be interpreted in the same way.

[0058] The terms used in this disclosure are only used to describe specific examples and are not intended to limit the meaning of this disclosure. As long as the context does not clearly indicate otherwise, the singular expression includes the plural expression. In this disclosure, the terms "including" or "having" are intended to specify the presence of the described features, numbers, steps, actions, constituent elements, components, or combinations thereof. It should therefore be understood that the presence or possibility of adding one or more other features or numbers, steps, actions, constituent elements, components, or combinations thereof is not precluded.

[0059] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the relevant technical context. Terms defined in commonly used dictionaries should not be idealized or overly interpreted as formal meanings unless explicitly defined in the present disclosure.

[0060] In describing examples, descriptions of technical contents that are well known in the technical field to which the present disclosure belongs and are not directly related to the present disclosure are omitted. The reason is that the gist of the present disclosure can be more clearly conveyed by omitting redundant descriptions to avoid confusion.

[0061] <Definition of terms>

[0062] The terms used in the present disclosure are briefly summarized below to help understand the disclosed content of the present disclosure.

[0063] NPU: The abbreviation of Neural Processing Unit, which may mean a processor independent of CPU (Central Processing Unit) and dedicated to artificial neural network model calculations.

[0064] ANN is the abbreviation of artificial neural network, which means a network that connects nodes into layers by simulating neurons in the human brain through synapses in order to imitate human intelligence.

[0065] DNN: The abbreviation of Deep Neural Network, which can refer to a neural network with increased number of hidden layers of artificial neural network in order to achieve higher artificial intelligence.

[0066] CNN: Abbreviation for Convolutional Neural Network, a type of neural network that functions similarly to the image processing in the visual cortex of the human brain. Convolutional neural networks are known to be suitable for image processing, extracting features from input data and easily identifying characteristic patterns.

[0067] Preferred examples of the present disclosure are described below with reference to the accompanying drawings to describe the present disclosure in detail.

[0068] <Artificial Intelligence>

[0069] Humans have intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc. Artificial intelligence (AI) means artificially imitating human intelligence.

[0070] The human brain is composed of a large number of nerve cells called neurons. Each neuron is connected to hundreds to thousands of other neurons through a connection site called a synapse. In order to imitate human intelligence, the working principle of biological neurons and the connection relationship between neurons are modeled, which is called an artificial neural network model. That is, an artificial neural network is a system that connects nodes that imitate neurons into a layer architecture.

[0071] This artificial neural network model is divided into "single-layer neural network" and "multi-layer neural network" according to the number of layers. An ordinary multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is the layer that obtains external data. The number of neurons in the input layer is the same as the number of input parameters. (2) The hidden layer is located between the input layer and the output layer. It receives signals from the input layer, extracts features, and transmits them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them to the outside. The input signals between neurons are multiplied by the connection strengths with values ​​between 0 and 1 and then added. If the sum is greater than the threshold of the neuron, the neuron is activated and the output value is realized through the activation function.

[0072] On the other hand, a neural network with an increased number of hidden layers in order to achieve the highest level of artificial intelligence is called a deep neural network (DNN).

[0073] DNN is being developed in various architectures. For example, it is known that a convolutional neural network (CNN), which is an example of DNN, can easily extract features of input values ​​(images or pictures) and recognize patterns of the extracted output values. CNN can be composed of convolution operations, activation function operations, pooling operations, etc., which are processed in a specific order.

[0074] For example, in each layer of a DNN, parameters (i.e., input values, output values, weights, or kernels, etc.) may be matrices consisting of multiple channels. The parameters may be processed in the NPU through convolution or matrix multiplication. After the operations are performed in each layer, the output value is generated.

[0075] For example, Transformer is a DNN based on attention technology. Transformer uses multiple matrix multiplication operations. Transformer can obtain attention (Q, K, V) as the output value using parameters such as input value and query vector (query: Q), key vector (key: K) and value vector (value: V). Transformer can process various reasoning operations based on the output value (i.e., attention (Q, K, V)). Transformer tends to show better reasoning performance than CNN.

[0076] Figure 1 A conceptual diagram describing an exemplary artificial neural network model.

[0077] The operation of an exemplary artificial neural network model 110 a that may be executed in the neural processor 100 is described below.

[0078] Figure 1 The exemplary artificial neural network model 110a may be an artificial neural network trained to perform various reasoning functions such as object recognition and speech recognition.

[0079] The artificial neural network model 110 a may be a deep neural network (DNN).

[0080] However, the artificial neural network model 110a of the present disclosure example is not limited to a deep neural network.

[0081] For example, the artificial neural network model 110a can be implemented by models such as DaViT, MobileViT, Swin-Transformer, Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3, etc. However, the present disclosure is not limited to the above models. In addition, the artificial neural network model 110a can also be an integrated model based on at least two different models.

[0082] The following describes the inference process performed by the exemplary artificial neural network model 110a.

[0083] The artificial neural network model 110a is an exemplary deep neural network model including an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5, a third connection network 110a-6 and an output layer 110a-7. However, the present disclosure is not limited to Figure 1 The artificial neural network model shown. The first hidden layer 110a-3 and the second hidden layer 110a-5 may also be referred to as multiple hidden layers.

[0084] The input layer 110a-1 may illustratively include x1 and x2 input nodes. That is, the input layer 110a-1 may include information of 2 input values.

[0085] The first connection network 110a-2 may exemplarily include information of six weight values ​​for connecting each node of the input layer 110a-1 to each node of the first hidden layer 110a-3. Each weight value is multiplied by the input node value, and the accumulated value of the product value is stored in the first hidden layer 110a-3. The weight value and the input node value may be referred to as parameters of the artificial neural network model.

[0086] The first hidden layer 110a-3 may exemplarily include nodes a1, a2, and a3. That is, the first hidden layer 110a-3 may include information of three node values.

[0087] Figure 1 The first processing unit PE1 can process the operation of the a1 node.

[0088] Figure 1 The second processing unit PE2 can process the operation of the a2 node.

[0089] Figure 1 The third processing unit PE3 of the first hidden layer 110a-3 can process the operation of the node a3. The second connection network 110a-4 can exemplarily include information of 9 weight values ​​for connecting each node of the first hidden layer 110a-3 to each node of the second hidden layer 110a-5. The weight values ​​of the second connection network 110a-4 are multiplied by the node values ​​input from the first hidden layer 110a-3, and the accumulated value of the product values ​​is stored in the second hidden layer 110a-5.

[0090] The second hidden layer 110a-5 may exemplarily include b1, b2, and b3 nodes. That is, the second hidden layer 110a-5 may include information of 3 node values.

[0091] Figure 1 The fourth processing unit PE4 can process the operation of the b1 node.

[0092] Figure 1 The fifth processing unit PE5 can process the operation of the b2 node.

[0093] Figure 1 The sixth processing unit PE6 can process the operation of the b3 node.

[0094] The third connection network 110a-6 may include information of six weight values ​​for connecting each node of the second hidden layer 110a-5 with each node of the output layer 110a-7. The weight values ​​of the third connection network 110a-6 are multiplied by the node values ​​input from the second hidden layer 110a-5, and the accumulated value of the product values ​​is stored in the output layer 110a-7.

[0095] The output layer 110a-7 may illustratively include y1 and y2 nodes. That is, the output layer 110a-7 may include information of two node values.

[0096] Figure 1 The seventh processing unit PE7 can process the operation of the y1 node.

[0097] Figure 1 The eighth processing unit PE8) can process the operation of the y2 node.

[0098] Each node may correspond to a feature value, and the feature value may correspond to a feature graph.

[0099] Figure 2a A diagram describing the basic architecture of a convolutional neural network (CNN).

[0100] Reference Figure 2a The input image may be displayed as a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image may have a plurality of channels, where a channel may represent the number of color components of the input data image.

[0101] The convolution process means performing kernel and convolution operations while traversing the input image at specified intervals.

[0102] A convolutional neural network may have an architecture in which the output value (convolution or matrix multiplication) of the current layer is passed as the input value of the next layer.

[0103] For example, convolution is defined by two main parameters: input feature map and kernel. Parameters may include input feature map, output feature map, activation map, weights, kernel, and attention (Q, K, V).

[0104] Convolution is to slide the kernel window on the input feature map. The single size of the kernel sliding on the input feature map is called the stride.

[0105] After convolution, pooling can be applied. In addition, at the end of the convolutional neural network, a fully-connected (FC) layer can be configured.

[0106] Figure 2b A comprehensive diagram showing the operation of a convolutional neural network in an easily understandable manner.

[0107] Reference Figure 2b , exemplarily showing that the input image is a two-dimensional matrix with a size of 6×6. Figure 2b In the figure, the use of three nodes, namely channel 1, channel 2, and channel 3, is shown as an example.

[0108] First, the convolution operation is described.

[0109] Input image (in Figure 2b , exemplarily shown as 6×6 size) at the first node, with kernel 1 for channel 1 (in Figure 2b , exemplarily shown as a 3×3 size) convolution, as a result of which the output feature Figure 1 (exist Figure 2b In the example, the size is 4×4. Figure 2b , exemplarily shown as 6×6 size) at the second node with kernel 2 for channel 2 (in Figure 2b , exemplarily shown as a 3×3 size) convolution, as a result, the output feature map 2 (in Figure 2b In the example, the size is 4×4. In addition, the input image is combined with the kernel 3 for channel 3 at the third node (in Figure 2b , exemplarily shown as a 3×3 size) convolution, as a result of which the output feature Figure 3 (exist Figure 2b , exemplarily shown as a 4×4 size).

[0110] To process each convolution, the processing units PE1-PE12 of the neural processor 100 are configured to perform MAC operations.

[0111] Then, the operation of the activation function is described.

[0112] Features that can be output by self-convolution operations Figure 1 , Feature map 2 and features Figure 3 (exist Figure 2b In the example, the size of each is exemplarily shown as 4×4) and the activation function is applied. The output after applying the activation function can be exemplarily 4×4 in size.

[0113] Then, the pooling operation is described.

[0114] Features output from the activation function Figure 1 feature map 2, features Figure 3 (In Figure 2b , the size of each is exemplarily shown as 4×4) are input to 3 nodes. A feature map output from the activation function can be accepted and pooling is performed. The so-called pooling can reduce the size or strengthen specific values within the matrix. Pooling methods include max pooling, average pooling, and min pooling. Max pooling is used to converge the maximum value of the values to a specific area of the matrix, and average pooling can be used to obtain the mean value within a specific area.

[0115] In Figure 2b example, the feature map shown as 4×4 size is reduced to 2×2 size by pooling.

[0116] Specifically, after the first node accepts the feature Figure 1 input for channel 1 and performs pooling, for example, the output is a 2×2 matrix. After the second node accepts the feature map 2 input for channel 2 and performs pooling, for example, the output is a 2×2 matrix. After the third node accepts the feature Figure 3 input for channel 3 and performs pooling, for example, the output is a 2×2 matrix.

[0117] The foregoing convolution, activation function, and pooling are repeatedly performed, and finally as Figure 8a shown, a fully connected output can be obtained. The corresponding output can be input again to an artificial neural network for image recognition. However, the present disclosure is not limited to the size of the feature map and the kernel.

[0118] The CNN described so far is the most widely applied method in the field of computer vision among various deep neural network (DNN) methods. In particular, CNN has shown amazing performance in various research fields performing various tasks such as image classification and object detection.

[0119] <Hardware resources required for ANN operations>

[0120] Figure 3 A conceptual schematic diagram of a neural processor for describing an example of the present disclosure.

[0121] Figure 3 The neural processor (NPU: neural processing unit) 100 shown is a processor optimized for performing operations for an artificial neural network.

[0122] An artificial neural network refers to a network composed of artificial neurons that multiply and add weights after multiple inputs or stimuli come in, and then transform and transmit the biased values ​​through activation functions. The artificial neural network trained in this way can be used to output inference results from the input data.

[0123] The neural processor 100 may be a semiconductor implemented as an electrical / electronic circuit, which may include a large number of electronic components (eg, transistors, capacitors).

[0124] For Transformer and / or CNN-based artificial neural network models, the neural processor 100 can identify and process matrix multiplication operations, convolution operations, etc. according to the architecture of the artificial neural network.

[0125] For example, in each layer of a convolutional neural network CNN, the input feature map corresponding to the input data and the kernel corresponding to the weight may be a tensor or matrix composed of multiple channels. The convolution operation of the input feature map and the kernel is performed to generate a convolution operation and a pooled output feature map in each channel. An activation function is applied to the output feature map to generate an activation map of the corresponding channel. Then, pooling of the activation map can be applied. Among them, the activation map can be generally referred to as the output feature map. For ease of description, the activation map is referred to as the output feature map for description below.

[0126] However, the disclosed examples are not limited thereto, and outputting a feature map means applying a matrix multiplication operation or a convolution operation, etc.

[0127] Furthermore, the output feature graph of the example disclosed herein should be interpreted as a general meaning. For example, the output feature graph may be a matrix multiplication operation or a convolution operation result value. Therefore, the multiple processing units 110 may also be implemented in a modified manner so as to also include a processing circuit unit for an additional algorithm. That is, a part of the circuit units of the SFU 150 described later may also be configured to be included in the multiple processing units 110.

[0128] The neural processor 100 may be configured to include a plurality of processing units 110 for processing the convolution and matrix multiplication required for the above-mentioned artificial neural network operations.

[0129] The neural processor 100 can be configured to include various processing circuits optimized for the matrix multiplication operations, convolution operations, activation function operations, pooling operations, stride operations, batch normalization operations, skip connection operations, cascade operations, quantization operations, cropping operations, and padding operations required for the above-mentioned artificial neural network operations.

[0130] For example, the neural processor 100 may be configured to include an SFU 150 for processing at least one of an activation function operation, a pooling operation, a stride operation, a batch normalization operation, a skip connection operation, a cascade operation, a quantization operation, a cropping operation, and a padding operation in the above algorithm.

[0131] Specifically, the neural processor 100 may include a plurality of PEs 110, an SFU 150, an NPU internal memory 120, an NPU controller 130, and an NPU interface 140. The plurality of processing units 110, the SFU 150, the NPU internal memory 120, the NPU controller 130, and the NPU interface 140 may be semiconductor circuits connected with a large number of transistors. Therefore, some of them may be difficult to identify and distinguish by naked eyes, and can only be identified based on operation.

[0132] For example, any circuit may operate as both the plurality of processing units 110 and the NPU controller 130. The NPU controller 130 may be configured to perform the functions of a control section configured to control the artificial neural network inference operation of the neural processor 100.

[0133] The neural processor 100 may include: an NPU internal memory 120, which is configured to store parameters of an artificial neural network model that can be used for inference in multiple processing units 110 and SFU 150; and an NPU controller 130, which includes a scheduler configured to control the operation scheduling of multiple processing units 110, SFU 150 and NPU internal memory 120.

[0134] The neural processor 100 may be configured to process the feature map in accordance with an encoding and decoding method using SVC (scalable video coding) or SFC (scalable feature-map coding). The method is a technology that can variably change the amount of data transmission according to the actual effective bandwidth and signal to noise ratio (SNR) of a communication channel or communication bus. That is, the neural processor 100 may also be configured to further include an encoder and a decoder.

[0135] The plurality of processing units 110 may perform a portion of the operations for the artificial neural network.

[0136] The SFU 150 may perform another portion of the operations for the artificial neural network.

[0137] The neural processor 100 may be configured to accelerate the operation of an artificial neural network model on hardware using a plurality of processing units 110 and SFUs 150 .

[0138] The NPU interface 140 can communicate with various components connected to the neural processor 100 through the system bus, such as a memory.

[0139] The NPU controller 130 may include a scheduler configured to control operations of the plurality of processing units 110 of the neural processor 100 for inference operations, operations of the SFU 150 , and a read and write order of the NPU internal memory 120 .

[0140] The scheduler in the NPU controller 130 may be configured to control the plurality of processing units 110 , the SFU 150 , and the NPU internal memory 120 based on data locality information or architecture information of the artificial neural network model.

[0141] The scheduler in the NPU controller 130 may analyze the architecture of the artificial neural network model to be run in the plurality of processing units 110 and the SFU 150 or obtain analyzed information. The analyzed information may be information generated by a compiler. For example, the data of the artificial neural network that may be included in the artificial neural network model may include at least a portion of the node data (i.e., feature graph) of each layer, the layout data of the layer, locality information or architecture information, and the weight data (i.e., weight kernel) of each connection network connecting the nodes of each layer. The data of the artificial neural network may be stored in a memory equipped inside the NPU controller 130 or in the NPU internal memory 120. However, it is not limited thereto, and the data of the artificial neural network may be stored in an independent cache memory or register file equipped in the NPU or a SoC including the NPU.

[0142] The scheduler in the NPU controller 130 can schedule the order of operations of the artificial neural network model to be executed by the neural processor 100 based on the data locality information or architecture information of the artificial neural network model.

[0143] The scheduler in the NPU controller 130 may obtain scheduling information of the operation order of the artificial neural network model to be executed by the neural processor 100 based on the data locality information or architecture information of the compiled artificial neural network model. For example, the scheduling information may be information generated by a compiler. The scheduling information generated by the compiler may be referred to as machine code or binary code, etc.

[0144] That is, the scheduling information used by the NPU controller 130 may be information generated by a compiler based on data locality information or architecture of the artificial neural network model.

[0145] Furthermore, the compiler can efficiently schedule the neural processor based on the unique characteristics of the artificial neural network model, namely the degree of understanding and reconstruction of the locality of the artificial neural network data.

[0146] Furthermore, the compiler can efficiently schedule the neural processor 100 based on its understanding of the hardware architecture and performance of the neural processor 100.

[0147] Furthermore, when the artificial neural network model is compiled by the compiler to run in the neural processor 100, the artificial neural network data locality can be reconstructed. The artificial neural network data locality can be reconstructed according to the algorithm applied by the artificial neural network model and the operating characteristics of the processor.

[0148] Furthermore, the locality of the artificial neural network data can be reconstructed according to the way in which the neural processor 100 processes the corresponding artificial neural network model, such as feature map tiling and the stationary mode of the processing unit.

[0149] Furthermore, the locality of the artificial neural network data can be reconstructed according to the number of processing units of the neural processor 100 and the capacity of the internal memory.

[0150] Furthermore, artificial neural network data locality may be reconstructed based on the bandwidth of the memory communicating with the neural processor 100 .

[0151] This is because, due to the above-mentioned factors, even if the same artificial neural network model is operated and processed, the neural processor 100 will determine the order of the data required at each moment differently in clock units.

[0152] The compiler may determine data locality based on the order of data required for the operation of the artificial neural network model, the layers of the artificial neural network, the order of operations of unit convolution and / or matrix multiplication, and generate compiled machine code.

[0153] The scheduler may be configured to utilize scheduling information included in the machine code.

[0154] The scheduler within the NPU controller 130 may obtain a memory address value storing feature maps and weight data of a layer of an artificial neural network model based on the scheduling information.

[0155] For example, the scheduler in the NPU controller 130 may obtain the memory address value of the feature map and weight data of the layer of the artificial neural network model stored in the memory. Therefore, the scheduler in the NPU controller 130 may export the feature map and weight data of the layer of the artificial neural network model to be driven from the autonomous memory and store them in the NPU internal memory 120.

[0156] The feature maps of each layer may have corresponding memory address values.

[0157] Each weight data may have a corresponding memory address value.

[0158] The scheduler in the NPU controller 130 may obtain information for scheduling the operation order of the plurality of processing units 110 based on data locality information or architecture information of the artificial neural network model, such as arrangement data, locality information or architecture information of the layers of the artificial neural network based on the artificial neural network model. The scheduling information may be generated in the compilation step.

[0159] The scheduler in the NPU controller 130 operates based on the information for scheduling based on the data locality information or architecture information of the artificial neural network model, and thus can operate differently from the scheduling concept of an ordinary CPU. The scheduling of an ordinary CPU takes into account fairness, efficiency, stability, response time, etc. to achieve the highest efficiency. That is, scheduling is performed taking into account priority, operation time, etc. so that the most processing can be performed in the same time.

[0160] Conventional CPUs use an algorithm that takes into account data such as the priority of each process and the processing time to schedule tasks.

[0161] Different from this, the scheduler in the NPU controller 130 may control the neural processor 100 according to the processing order of the neural processor 100 determined based on the data locality information or architecture information of the artificial neural network model.

[0162] Furthermore, the scheduler within the NPU controller 130 may drive the NPU 100 according to a processing order determined based on data locality information or architecture information of the artificial neural network model and / or data locality information or architecture information of the neural processor 100 to be used.

[0163] However, the present disclosure is not limited to data locality information or architectural information of the neuroprocessor 100 .

[0164] The scheduler within the NPU controller 130 may be configured to store data locality information or architecture information of the artificial neural network.

[0165] That is, the scheduler in the NPU controller 130 can determine the processing order even if it only uses at least the data locality information or architecture information of the artificial neural network of the artificial neural network model.

[0166] Furthermore, the scheduler in the NPU controller 130 may determine the processing order of the neural processor 100 by taking into account the data locality information or architecture information of the artificial neural network model and the data locality information or architecture information of the neural processor 100. In addition, the processing of the neural processor 100 may also be optimized according to the determined processing order.

[0167] That is, the NPU controller 130 can be configured to run based on the machine code compiled by the compiler, but in another example, the NPU controller 130 can also be configured to have a built-in embedded compiler. According to the above configuration, the neural processor 100 can be configured to obtain files in the form of frameworks of various AI software to generate machine code. For example, the frameworks of AI software include TensorFlow, PyTorch, Keras, XGBoost, mxnet, DARKNET, ONNX, etc.

[0168] The plurality of processing units 110 means a configuration including a plurality of processing units PE1-PE12 configured to operate on feature maps and weight data of an artificial neural network. Each processing unit may include a multiplication and accumulation (MAC) operator and / or an arithmetic logic unit (ALU) operator. However, the examples disclosed herein are not limited thereto.

[0169] Each processing unit may be configured to further selectively include a supplementary special function unit to process supplementary special functions.

[0170] For example, the processing unit PE may also be implemented in a deformed manner to further include a batch normalization unit, an activation function unit, an interpolation unit, etc.

[0171] The SFU 150 may include a circuit portion configured to identify and process activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, padding operations, etc. according to the architecture of the artificial neural network. That is, the SFU 150 may include a plurality of special function operation processing circuit units.

[0172] Figure 3 , multiple processing units are shown as an example, however, within a processing unit, multiple operators implemented by multipliers and adder trees can be configured in parallel instead of MAC. In this case, the multiple processing units 110 can also be referred to as at least one processing unit including multiple operators.

[0173] The plurality of processing units 110 are configured to include a plurality of processing units PE1 ˜ PE12 . Figure 3 The multiple processing units PE1 to PE12 shown are only examples for ease of description, and the number of the multiple processing units PE1 to PE12 is not limited. The size or number of the multiple processing units 110 can be determined according to the number of the multiple processing units PE1 to PE12. The size of the multiple processing units 110 can be implemented in the form of an N×M matrix. Wherein N and M are integers greater than 0. The multiple processing units 110 may include N×M processing units. That is, the number of processing units may be more than one.

[0174] The sizes of the multiple processing units 110 may be designed taking into account the characteristics of the artificial neural network model run by the neural processor 100 .

[0175] The plurality of processing units 110 are configured to perform functions such as addition, multiplication, accumulation, etc. required for artificial neural network operations. In other words, the plurality of processing units 110 may be configured to perform multiplication and accumulation (MAC) operations.

[0176] The following description is made by taking the first processing unit PE1 among the multiple processing units 110 as an example.

[0177] Figure 4a The present invention is a conceptual diagram for describing one processing unit among a plurality of processing units to which an example of the present disclosure may be applied.

[0178] A neural processor 100 of an example of the present disclosure may include: a plurality of processing units 110; an NPU internal memory 120, the NPU internal memory 120 being configured to store an artificial neural network model that can be inferred in the plurality of processing units 110; and an NPU controller 130, the NPU controller 130 being configured to control the plurality of processing units 110 and the NPU internal memory 120; wherein the plurality of processing units 110 may be configured to perform MAC operations, and the plurality of processing units 110 may be configured to quantize and output MAC operation results. However, the examples of the present disclosure are not limited thereto.

[0179] The NPU internal memory 120 can store all or part of the artificial neural network model according to the memory size and the data size of the artificial neural network model.

[0180] The first processing unit PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. However, the examples disclosed herein are not limited thereto, and the plurality of processing units 110 may also be implemented in a modified manner taking into account the computational characteristics of the artificial neural network.

[0181] The multiplier 111 can multiply the input (N) bit data and (M) bit data. The operation value of the multiplier 111 is output as (N+M) bit data.

[0182] Multiplier 111 may be configured to accept a parameter and a constant input.

[0183] The accumulator 113 can use the adder 112 for (L) loops to add the operation value of the multiplier 111 and the operation value of the accumulator 113. Therefore, the bit width of the data of the output part and the input part of the accumulator 113 can be output as (N+M+log2(L)) bits, where L is an integer greater than 0.

[0184] After the accumulator 113 finishes accumulating, an initialization signal (initialization reset) may be input to initialize the data stored inside the accumulator 113 to 0. However, the examples of the present disclosure are not limited thereto.

[0185] The bit quantization unit 114 can reduce the bit width of the data output by the accumulator 113. The bit quantization unit 114 can be controlled by the NPU controller 130. The bit width of the quantized data can be output as (X)bit. Here, X is an integer greater than 0. According to the above structure, the multiple processing units 110 can be configured to perform MAC operations, and the multiple processing units 110 have the effect of quantizing and outputting the results of the MAC operations. In particular, this quantization has the effect of saving power as the number of (L)loops increases. In addition, if the power consumption is reduced, it can also reduce the heat generation. In particular, if the heat generation is reduced, it can reduce the possibility of malfunction of the neural processor 100 due to high temperature.

[0186] The output data (X) bit of the bit quantization unit 114 can become the node data of the next layer or the input data of the convolution. If the artificial neural network model has been quantized, the bit quantization unit 114 can be configured to obtain quantized information from the artificial neural network model. However, it is not limited to this. The NPU controller 130 can also be configured to analyze the artificial neural network model and extract quantized information. Therefore, the output data (X) bit can be converted into a quantized bit width and output in a manner corresponding to the size of the quantized data. The output data (X) bit of the bit quantization unit 114 can be stored in the NPU internal memory 120 according to the quantized bit width.

[0187] The multiple processing units 110 of the neural processor 100 of an example of the present disclosure include a multiplier 111, an adder 112, and an accumulator 113. The bit quantization unit 114 can make a selection based on whether to apply quantization. In another example, the bit quantization unit can also be configured to be included in the SFU 150.

[0188] Figure 4b The present invention is a conceptual diagram of an SFU to which an example of the present invention is applicable.

[0189] refer to Figure 4b The SFU 150 includes a plurality of functional units. Each functional unit can be selectively operated. Each functional unit can be selectively turned on or off. That is, each functional unit can be set.

[0190] In other words, the SFU 150 may include various circuit units required for artificial neural network reasoning operations.

[0191] For example, the circuit units of SFU 150 may include functional units for skip-connection operations, functional units for activation function operations, functional units for pooling operations, functional units for quantization operations, functional units for non-maximum suppression (NMS) operations, functional units for integer to floating-point conversion (INT to FP32) operations, functional units for batch-normalization operations, functional units for interpolation operations, functional units for concatenation operations, and functional units for bias operations, etc.

[0192] The functional units of the SFU 150 can be selectively turned on or off according to the data locality information of the artificial neural network model. The data locality information of the artificial neural network model may include the turning off of the corresponding functional units or control information related to the turning off when performing operations of a specific layer.

[0193] The activated units in the functional units of the SFU 150 can be turned on. When a portion of the functional units of the SFU 150 is selectively turned off in this way, the power consumption of the neural processor 100 can be saved. On the other hand, in order to turn off a portion of the functional units, power gating can be used. Alternatively, in order to turn off a portion of the functional units, clock gating can also be performed.

[0194] Figure 5To show Figure 3 An example diagram of a variation of the neural processor 100 is shown.

[0195] Figure 5 The neural processor 100 is shown with Figure 3 Compared with the neural processor 100 exemplarily shown in the figure, except for the multiple processing units 110, they are essentially the same, so they are not repeated below for the convenience of description.

[0196] Figure 5 The multiple processing units 110 exemplarily shown in FIG. 1 may include, in addition to the multiple processing units PE1 ˜ PE12 , respective register files RF1 ˜ RF12 corresponding to the respective processing units PE1 ˜ PE12 .

[0197] Figure 5 The multiple processing units PE1-PE12 and the multiple register files RF1-RF12 shown are just examples for convenience of description, and the number of the multiple processing units PE1-PE12 and the multiple register files RF1-RF12 is not limited.

[0198] The size or number of the processing units 110 may be determined according to the number of the processing units PE1-PE12 and the register files RF1-RF12. The size of the processing units 110 and the register files RF1-RF12 may be implemented in the form of an N×M matrix, where N and M are integers greater than 0.

[0199] The array size of the plurality of processing units 110 may be designed taking into account the characteristics of the artificial neural network model run by the neural processor 10. Furthermore, the memory size of the register file may be determined taking into account the data size, required operating speed, required power consumption, etc. of the artificial neural network model to be run.

[0200] The register stacks RF1 to RF12 of the neural processor 100 are static memory units directly connected to the processing units PE1 to PE12. The register stacks RF1 to RF12 may be composed of, for example, triggers and / or latches. The register stacks RF1 to RF12 may be configured to store the MAC operation values ​​of the corresponding processing units PE1 to PE12. The register stacks RF1 to RF12 may be configured to send and receive weight data and / or node data with the NPU internal memory 120.

[0201] Register files RF1-RF12 may also be configured to perform a temporary storage function of an accumulator during MAC operation.

[0202] <Technical Difficulties Discovered by the Inventors of the Present Disclosure>

[0203] There are various types of artificial neural network (ANN) models depending on the purpose of artificial intelligence services. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. in the image / video can be used for artificial intelligence services.

[0204] Generally speaking, the amount of computation in each layer of an artificial neural network model is different, which is described with reference to FIG6 .

[0205] Figure 6a An example diagram showing the data size of each layer in an exemplary artificial neural network model. Figure 6b To show Figure 6a An example table showing data sizes, etc. for each layer in the exemplary artificial neural network model shown.

[0206] Figure 6a The exemplary artificial neural network model shown is shown as Mobilenet V1. Figure 6a The horizontal axis represents the layers in the exemplary artificial neural network model in sequence, and the vertical axis represents the data size.

[0207] refer to Figure 6a As shown in layer 1, the size of the output feature map (OFMAP_SIZE) is larger than the size of the input feature map (IFMAP_SIZE).

[0208] The output feature map of layer 1 is passed to layer 2 and becomes the input feature map of layer 2. After the operation of layer 2 is completed, the output feature map is output. The output feature map of layer 2 is then passed to layer 3 and becomes the input feature map of layer 3.

[0209] Therefore, the size of the input data to each layer and the size of the output feature map output by each layer may be different. Therefore, the amount of calculation may be small in any layer, but it may be large in other layers. Therefore, as the amount of calculation of each layer changes greatly, the problem of difficulty in power control occurs.

[0210] Each layer can be divided into multiple operation steps for processing according to the number of processing units of the neural processor and the capacity limit of the NPU internal memory 120. Therefore, the neural processor 100 can be scheduled to divide each layer into multiple tiles to process multiple operation steps. The scheduling can be performed by the compiler. For example, a layer can be divided into 4 tiles. Each tile can be processed by the neural processor in sequence. The compiled artificial neural network model can store the operation order information determined at compile time to the scheduler. At this time, each operation step can provide PE utilization information.

[0211] The amount of computation of each layer can be obtained through MAC. It can be seen that the maximum difference between the amount of computation of each layer is 227 times.

[0212] The neural processor 100 may determine the number of activated processing units in the plurality of processing units 110 in proportion to the MAC operation amount at each operation step of each layer. Moreover, the power consumption increases in proportion to the number of activated processing units.

[0213] Therefore, the PE utilization rate can be calculated for each operation step of each layer of the artificial neural network model processed in the neural processor 100. In addition, as long as the architecture of the artificial neural network model processed by the neural processor 100 does not change, the PE utilization rate of each operation step can be repeatedly and accurately predicted. That is, the neural processor 100 can repeatedly infer a specific artificial neural network model. The inventors of the present disclosure recognize that at this time, the neural processor 100 can repeatedly use the same weight parameters and the same network layer architecture.

[0214] In addition, the greater the difference in MAC operation amount between adjacent layers, the greater the peak power deviation between adjacent layers. Moreover, the greater the peak power deviation between adjacent layers, the more the supply voltage VDD will fluctuate. The inventors of the present disclosure recognize that the driving frequency can be changed by taking into account the stability of the supply voltage VDD.

[0215] In particular, the inventors of the present disclosure have recognized that, in a calculation step of a specific layer, if the amount of calculation increases significantly, the power consumption increases instantaneously, thereby causing a problem of reduced system stability.

[0216] For example, a large number of processing units can be started simultaneously for the calculation of the calculation steps of a specific layer. A predetermined power supply is required to drive each processing unit. If a considerable number of processing units are driven at the same time, the required power will increase sharply in an instant. In the case where the neural processor is specially designed for low-power operation, the power supply capacity will be relatively insufficient compared to the neural processor for server use. Therefore, this type of neural processor for edge devices will be relatively weak in instantaneous power supply, and when the power supply increases sharply, the supply voltage VDD will fluctuate. In particular, when the supply voltage VDD drops below the critical voltage of the transistor, the data stored in the transistor will be damaged. In other words, when the supply voltage VDD decreases, the operating speed of the transistor is slow, setup / hold violations occur, and thus misoperation will occur. In particular, the smaller the process of the semiconductor foundry, such as 3nm, 4nm, 5nm, and 7nm, the more serious this phenomenon will be.

[0217] For another example, the inventors of the present disclosure recognize that the PE 110 that performs artificial neural network operations, such as addition, multiplication, and accumulation, consumes a large amount of power instantly, so that other components in the neural processor, such as the internal memory 120, cannot obtain sufficient power supply. Specifically, the inventors of the present disclosure recognize that if the internal memory 120 cannot obtain sufficient power supply, there will be a problem that it cannot be ruled out that the stored data bits may be damaged (compromised).

[0218] Furthermore, the inventors of the present disclosure have recognized that when the PE utilization rate in a specific operation step is low, the voltage stability of the neural processor can be guaranteed even if the driving frequency of the neural processor is increased.

[0219] <Disclosure of this Patent>

[0220] As mentioned above, the inventors of the present disclosure recognize that the artificial neural network model processed by the neural processor may suffer from system stability problems due to the rapid changes in the amount of calculation in each layer and the large changes in power (W).

[0221] In the past, when designing a neural processor, the focus was only on reducing average power consumption, but the inventors of the present disclosure realized that reducing the power of a specific computing step is more important, that is, it is important to adjust the power of a specific computing step to stabilize the supply voltage VDD.

[0222] Therefore, the inventors of the present disclosure invented a method that reduces power deviation of each operation step even if the amount of operation of each operation step of the layer of the ANN model changes drastically.

[0223] An example of implementing the method invented by the inventors of the present disclosure is described below with reference to the accompanying drawings.

[0224] Figure 7a The present invention briefly illustrates an exemplary method proposed in an example of the present disclosure.

[0225] Figure 7b The present invention is an exemplary diagram showing an example of a clock signal used by a method proposed in an example of the present disclosure.

[0226] refer to Figure 7a According to an example of the present disclosure, a plurality of clock signals having different frequencies may be generated S710.

[0227] For example, multiple clock signals having different frequencies may be generated simultaneously. For example, multiple clock signals having different frequencies may be generated sequentially. For example, multiple clock signals having different frequencies may be generated simultaneously and / or sequentially.

[0228] For example, one of a plurality of clock signals having mutually different frequencies may be selected and provided. A plurality of clock signals of a plurality of clock signals having mutually different frequencies may be simultaneously selected and provided.

[0229] For example, Figure 7b As shown, the multiple clock signals may be a first clock signal, a second clock signal, and a third clock signal. Figure 7b As shown, the frequency of the first clock signal may be higher than that of the second clock signal. Figure 7b As shown, the frequency of the second clock signal may be higher than that of the third clock signal.

[0230] A high frequency means that the signal changes at a higher speed within the same time interval. In other words, the frequency of the second clock signal may be lower than that of the first clock signal. The frequency of the third clock signal may be lower than that of the second clock signal.

[0231] For example, the first clock signal may be 1.2 GHz. However, examples of the present disclosure are not limited thereto. For example, the second clock signal may be 1.0 GHz. However, examples of the present disclosure are not limited thereto. For example, the third clock signal may be 800 MHz. However, examples of the present disclosure are not limited thereto.

[0232] For example, the first clock signal may be 1.2 GHz. However, examples of the present disclosure are not limited thereto. For example, the second clock signal may be 1.1 GHz. However, examples of the present disclosure are not limited thereto. For example, the third clock signal may be 1.0 MHz. However, examples of the present disclosure are not limited thereto.

[0233] For example, the frequency of the first clock signal may be twice the frequency of the second clock signal. However, examples of the present disclosure are not limited thereto. For example, the frequency of the second clock signal may be twice the frequency of the third clock signal. However, examples of the present disclosure are not limited thereto.

[0234] In addition, the amount of computation or PE utilization of computation steps of a given layer among a plurality of layers within an artificial neural network (ANN) model processed by a neural processing unit (NPU) may be obtained S720.

[0235] The PE utilization rate means the ratio of the number of running PEs to the number of multiple PEs. For example, when the number of PEs is 1000 and the number of PEs mobilized by the corresponding operation step is 700, the utilization rate may be 70%. The PE utilization rate may vary according to the operation step. That is, the PE utilization rate of each operation step may be analyzed for at least one artificial neural network model. For example, the average of the PE utilization rates of multiple adjacent operation steps within a given period may be calculated.

[0236] The multiple PEs may also be referred to as NPU cores, NPU engines, NPU threads, etc. The NPU cores, NPU engines, NPU threads, etc. may be semiconductor circuits connected to a large number of transistors. Therefore, some of them may be difficult to identify and distinguish by naked eyes, and can only be identified based on operations.

[0237] The neural processor can also be configured to include multiple NPU cores, NPU engines, and NPU threads.

[0238] The plurality of PEs may include an adder, a multiplier, and an accumulator.

[0239] Then, based on the computation amount or PE utilization rate, one of the plurality of clock signals may be selected S730 and supplied S740.

[0240] The one clock signal may be selected by additionally considering the threshold of the computation amount or PE utilization. That is, the frequency of the one clock signal may be selected based on the threshold of the computation amount or PE utilization. In order to reduce the fluctuation of the supply voltage VDD when performing the computation in the computation step of the given layer, the computation amount of the computation step of the given layer or the threshold of PE utilization may be determined.

[0241] The PE utilization rate of each operation step can be calculated based on the parameter size and MAC operation amount corresponding to the operation step of each layer of the artificial neural network model processed by the neural processor. For example, when the parameter size of the input feature map of the operation step of a specific layer is large, the kernel size and the number of kernel channels are large, the PE utilization rate corresponding to the operation step of the corresponding layer will be high.

[0242] The characteristics of each layer of the artificial neural network model may include the shape of the input feature map (IF) and / or the weight. The shape of the input feature map (IF) and / or the weight may be determined by height, width, depth, and channel.

[0243] The input feature map of the artificial neural network can be divided into multiple small tiles, and each tile can extract features by operating with the kernel. According to each tile, the systolic array architecture that can minimize the MAC operation time can be different.

[0244] In a convolutional neural network, most of the computing time is occupied by the convolution operation. The number of MAC operations used for the convolution operation can be calculated by the algorithm of the following mathematical formula 1.

[0245]

Mathematical formula 1

[0246] for(m=0;m <M;m++) / / Num filters

[0247] for(e=0;e <E;e++) / / num_conv_window(height)

[0248] for(f=0;f <F;f++) / / num_conv_window(width)

[0249] for(c=0;c <C;c++) / / Elements per filter(channel)

[0250] for(r=0;r <R;r++) / / Elements per filter(height)

[0251] for(s=0;s <S;s++) / / Elements per filter(width)

[0252]

[0253] Referring to Mathematical Formula 1, m is a parameter representing the number of filters, e is a parameter representing the height of the output feature map, f is a parameter representing the width of the output feature map, c is a parameter representing the number of channels, r is a parameter representing the height of the filter, and s is a parameter representing the width of the filter. M is a constant representing the number of filters, E is a constant representing the height of the output feature map, F is a constant representing the width of the output feature map, C is a constant representing the number of channels, R is a constant representing the height of the filter, and S is a constant representing the width of the filter.

[0254] The algorithm of Mathematical Formula 1 can parallelize different for-loops according to each stationar method. For the systolic array architecture in the Output stationary mode, in the algorithm of Mathematical Formula 1, the first to third for-loops can be loop-unrolled to execute. Specifically, for the systolic array architecture in the Output stationary mode, the first for-loop (Num filters) can be parallelized according to the number of rows of the processing unit array, and the second for-loop (num_conv_window (height)) and the third for-loop (num_conv_window (width)) can be parallelized according to the number of columns of the processing unit array.

[0255] When a MAC operation with an input feature map and 1 kernel as input is called 1 cycle, a neural processor of an example of the present disclosure can calculate the total MAC operation time using the number of processing units, the input feature map and / or the weight. Therefore, the PE utilization can be calculated. The PE utilization analysis can be performed during the compilation step or runtime of the artificial neural network model.

[0256] For example, as shown in Table 1 below, when the PE utilization rate of a specific operation step of a specific layer exceeds a threshold value of 90%, a second clock signal may be provided.

[0257]

Table 1

[0258] PE utilization threshold Clock signal 50％ First clock signal 70％ Second clock signal 90％ The third clock signal

[0259] When the neural processor completes the operation of the previous layer of the given layer of the at least one ANN model, the one clock signal may be selected. That is, a clock signal with an optimal frequency taking into account the PE utilization of the next operation step may be selected. The frequency of the clock signal may be a frequency selected taking into account the power of a specific operation step. In addition, the frequency of the clock signal may be a frequency selected taking into account the stability of the supply voltage VDD.

[0260] The neural processor 100 can obtain the computational amount of the next computational step or the utilization rates of the plurality of PEs in advance based on the computational step information of the artificial neural network model stored in the scheduler of the NPU controller 130 .

[0261] When the computational complexity or the PE utilization rate of the computational steps of the given layer is higher than those of the computational steps of the previous layer, a clock signal selected for the computational steps of the given layer may have a second frequency that is lower than the first frequency of the computational steps of the previous layer.

[0262] For example, if the clock signal used for the operation step of the previous layer is Figure 7b The first clock signal shown, then the clock signal selected for the operation step of the given layer may be Figure 7b The second clock signal is shown.

[0263] When the computational complexity or the PE utilization rate of the computational steps of the given layer is lower than those of the computational steps of the previous layer, a clock signal selected for the computational steps of the given layer may have a second frequency that is higher than the first frequency of the computational steps of the previous layer.

[0264] For example, if the clock signal used for the operation step of the previous layer is Figure 7b The second clock signal shown, then the clock signal selected for the operation step of the given layer may be Figure 7b The first clock signal is shown.

[0265] The computation amount of the computation steps of the given layer or the PE utilization rate may be determined for each of the computation steps of the plurality of layers of the at least one ANN model.

[0266] The amount of computation of the computation steps of the given layer or the PE utilization rate may be determined based on scheduling information for distributing the computation of the computation steps of each layer of the at least one ANN model to the plurality of PEs.

[0267] For example, for at least one artificial neural network model, a PE utilization profile of each operation step may be used. Based on the profile, the frequency of the clock signal to be applied to the plurality of PEs in the corresponding operation step may be determined based on the threshold of each operation step. Information of each operation step of the artificial neural network model may be included in the scheduling information. Therefore, the profile information of the PE utilization of each operation step may be obtained based on the scheduling information.

[0268] Figure 7c The schematic diagram briefly shows the hardware structure for implementing the method proposed in an example of the present disclosure.

[0269] Reference Figure 7c , showing a switch circuit 400 and one or more neural processors. The one or more neural processors are Figure 7b In the figure, two (100-1, 100-2) are shown as examples.

[0270] The switch circuit 400 means a circuit designed to output a clock signal in a specific frequency region. For example, the switch circuit 400 may mean a circuit designed to select a frequency from 600 MHz to 2.8 GHz for output based on a selection signal. However, an example of the present disclosure is not limited to the output frequency of the switch circuit 400.

[0271] For example, the switch circuit 400 may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. For example, the switch circuit 400 may be a circuit that combines at least two of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator circuit, a phase-locked loop (PLL) circuit, and a selector. The technology of generating various frequencies by various combinations of various circuits is a well-known technology, and thus will not be described in detail.

[0272] For example, the switch circuit 400 may be a circuit designed to generate a clock signal of at least one frequency. For example, the switch circuit 400 may be a circuit designed to receive a clock signal input of at least one frequency.

[0273] Therefore, the switch circuit 400 can output clock signals of various frequencies according to the selection signal. For example, the switch circuit 400 can output a clock signal of at least one of 1.2 GHz, 1.1 GHz, 1.0 GHz, 900 MHz, and 800 MHz according to the selection signal. The switch circuit 400 can output a specific frequency among the preset specific frequencies according to the selection signal. However, the examples disclosed herein are not limited to the frequency of the switch circuit 400, and the switch circuit 400 can be designed to output various frequencies.

[0274] The switch circuit 400 may be designed to output a clock signal of a specific frequency based on the selection signal. The specific frequency may be a preset frequency. Alternatively, the specific frequency may be a variable frequency in a specific frequency band.

[0275] The selection signal may be determined based on information on PE utilization or MAC operation amount for an operation step of any given layer among operation steps of a plurality of layers in the ANN model.

[0276] The plurality of PEs may include an adder, a multiplier, and an accumulator.

[0277] The MAC operation amount of a given operation step of the given layer or the PE utilization information may be included in the scheduling information. Based on the scheduling information, the MAC operations of the operation steps of each layer of the at least one ANN model may be distributed to the plurality of PEs.

[0278] The switch circuit 400 may select one of the plurality of clock signals for output based on the selection signal. That is, the switch circuit 400 may output a clock signal having a specific frequency based on the selection signal.

[0279] For example, when the computational amount or the PE utilization rate of the computational steps of the given layer is higher than that of the computational steps of the previous layer, the switching circuit 400 may select a clock signal having a second operating frequency that is lower than the first operating frequency of the computational steps of the previous layer.

[0280] On the contrary, when the computational amount or the PE utilization rate of the computational steps of the given layer is lower than that of the computational steps of the previous layer, the switching circuit 400 may select a clock signal having a second operating frequency that is higher than the first operating frequency of the computational steps of the previous layer.

[0281] The neural processor 100 may obtain the computational load of all computational steps of a specific artificial neural network model or information on the utilization of multiple PEs. At this time, the neural processor 100 may preset a clock signal of an optimal frequency in each computational step.

[0282] The method disclosed in the present disclosure can be applied to the case where there are 2 neural processors, and can also be applied to the case where there are 3, 4, 6 or 8 neural processors. That is, the number of neural processors in an example of the present disclosure is not limited.

[0283] Figure 8a FIG. 1 is an exemplary diagram showing a SoC architecture of an example of the first disclosure of the present disclosure.

[0284] Reference Figure 8a , the exemplary SoC 1000 may include multiple neural processors, multiple CPUs, and multiple memories. The multiple neural processors may include, for example, a first NPU 100-1 and a second NPU 100-2. Moreover, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.

[0285] exist Figure 8a In the figure, it is shown that the number of the multiple neural processors, the multiple CPUs and the multiple memories is 2 respectively, but it is not limited to this, and the number can vary in various ways, such as 4, 6, 8, etc.

[0286] The exemplary SoC 1000 may include a memory controller 350 , a switch circuit 400 , a system bus 500 , and an I / O interface 800 .

[0287] The system bus 500 may be implemented by an electrically conductive pattern formed on a semiconductor die. The system bus may perform high-speed communication. For example, the plurality of neural processors, the plurality of CPUs, the plurality of memories, and the memory controller 350 may communicate with each other via the system bus 500.

[0288] The plurality of neural processors and the plurality of CPUs make requests to the memory controller 350 through the system bus 500. Therefore, the memory controller 350 may read data or record data from at least one of the plurality of memories.

[0289] Figure 8a Each unit (ie, NPU, memory, and CPU) in the SoC 1000 shown may be input with a driving voltage VDD.

[0290] exist Figure 8a, a common driving voltage VDD is shown as being input, but examples of the present disclosure are not limited thereto.

[0291] In some embodiments, it may also be configured to input an independent or separate driving voltage VDD to each unit (ie, NPU, memory, and CPU) within the SoC 1000 .

[0292] In some other examples, it may also be configured to input an independent or separate driving voltage VDD to some units (ie, NPU, memory, and CPU) within the SoC 1000 .

[0293] In some other examples, it may also be configured to input a second driving voltage different from the first driving voltage to a specific unit in the SoC 1000 .

[0294] Figure 8a The switch circuit 400 shown is configured to receive a selection signal input. There may be multiple selection signals. Each corresponding selection signal may be input to multiple neural processors (e.g., the first NPU 100-1 and the second NPU 100-2). The selection signal may be determined based on the amount of operation of a specific operation step of a specific ANN model processed by the multiple specific neural processors or PE utilization information within at least one neural processor.

[0295] The information may be provided from at least one of the plurality of neural processors. Alternatively, the information may be provided from at least one of the plurality of CPUs. At this time, a selection signal corresponding to the information may be generated. The selection signal may be generated by the plurality of CPUs or the plurality of neural processors. The generated selection signal may be transmitted from at least one of the plurality of neural processors or at least one of the plurality of CPUs to the switch circuit 400. The selection signal may be transmitted to the switch circuit 400 via the system bus 500.

[0296] After generating a plurality of clock signals, the switch circuit 400 may select a clock signal from the plurality of clock signals based on the obtained information, and the clock signal of the selected frequency may be transmitted to the corresponding neural processor.

[0297] To this end, the switch circuit 400 may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The switch circuit 400 may be a circuit in which at least two of the clock source, the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0298] That is, after the switch circuit 400 generates a clock signal for each of the plurality of neural processors (eg, the first NPU 100 - 1 and the second NPU 100 - 2 ), the clock signal is transmitted to the corresponding neural processor.

[0299] For example, the switch circuit 400 provides the first clock signal to the first NPU 100-1 and provides the second clock signal to the second NPU 100-2, so that the first NPU 100-1 operates according to the first clock signal and the second NPU 100-2 operates according to the second clock signal.

[0300] For example, when the amount of computation or PE utilization of a given operation step of any layer among the operation steps of multiple layers in the first ANN processed by the first NPU 100-1 becomes higher than the operation step of the previous layer, the selection signal may instruct to reduce the frequency. Therefore, the switch circuit 400 may select a first clock signal with a frequency lower than the frequency of the operation step of the previous layer and provide it to the first NPU 100-1. Alternatively, the switch circuit 400 may modulate the frequency of the second clock signal of the second frequency to generate a first clock signal of the first frequency and provide it to the first NPU 100-1.

[0301] On the contrary, when the amount of operation or PE utilization of the operation steps of a given arbitrary layer among the operation steps of the plurality of layers in the second ANN processed by the second NPU 100-2 becomes lower than the operation steps of the previous layer, the selection signal may instruct to increase the frequency. Therefore, the switch circuit 400 may select a second clock signal having a frequency higher than the frequency of the operation steps of the previous layer, and provide it to the second NPU 100-2. Alternatively, the switch circuit 400 may modulate the frequency of the first clock signal of the first frequency, generate a second clock signal of the second frequency, and provide it to the second NPU 100-2.

[0302] As described above, if the first NPU 100-1 and the second NPU 100-2 are operated in a distributed manner according to clock signals having different frequencies from each other, there are advantages in that power deviation between specific operation steps can be reduced and supply voltage stability can be improved.

[0303] Figure 8b FIG. 1 is an exemplary diagram showing a SoC architecture of another example of the first disclosure of the present disclosure.

[0304] Reference Figure 8b , the exemplary SoC 1000 may include multiple neural processors, multiple CPUs, and multiple memories. The multiple neural processors may include, for example, a first NPU 100-1 and a second NPU 100-2. Moreover, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.

[0305] exist Figure 8b In the figure, it is shown that the number of the multiple neural processors, the multiple CPUs and the multiple memories is 2 respectively, but it is not limited to this, and the number can vary in various ways, such as 4, 6, 8, etc.

[0306] The exemplary SoC 1000 may include a memory controller 350 , a switch circuit 400 , a plurality of buses, and an I / O interface 800 .

[0307] The plurality of buses may include a bus for a CPU, namely a CPU bus 500 - 1 , a bus for a neural processor, namely an NPU bus 500 - 2 , and a bus for peripheral components, namely a peripheral bus 500 - 3 .

[0308] The first CPU 200-1, the second CPU 200-2 and the first memory 300-1 may be connected to the CPU bus 500-1. The first NPU 100-1, the second NPU 100-2 and the second memory 300-2 may be connected to the NPU bus 500-2. The memory controller 350 and the switch circuit 400 may be connected to the peripheral bus 500-3.

[0309] Available Figure 8b Each unit (ie, NPU, memory, and CPU) within the SoC 1000 is shown to input a driving voltage VDD.

[0310] exist Figure 8b , it is shown that a common driving voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0311] In some embodiments, it may also be configured to input an independent or separate driving voltage VDD to each unit (ie, NPU, memory, and CPU) within the SoC 1000 .

[0312] In some other examples, it may also be configured to input an independent or separate driving voltage VDD to some units (ie, NPU, memory, and CPU) within the SoC 1000 .

[0313] In some other examples, it may also be configured to input a second driving voltage different from the first driving voltage to a specific unit in the SoC 1000 .

[0314] Figure 8b The switch circuit 400 shown is configured to receive a selection signal input. There may be multiple selection signals. Each corresponding selection signal may be input to multiple neural processors (e.g., the first NPU 100-1 and the second NPU 100-2). The selection signal may be determined based on the amount of operation of a specific operation step of a specific ANN model processed by the multiple specific neural processors or PE utilization information within at least one neural processor.

[0315] The information may be provided from at least one of the multiple neural processors. Alternatively, the information may be provided from at least one of the multiple CPUs. At this time, a selection signal corresponding to the information may be generated. The selection signal may be generated by the multiple CPUs or the multiple neural processors. The generated selection signal may be transmitted to the switch circuit 400 from at least one of the multiple neural processors or at least one of the multiple CPUs. The selection signal may be transmitted to the switch circuit 400 via at least one of the NPU bus 500-2, the CPU bus 500-1, and the peripheral bus 500-3.

[0316] Alternatively, the information may be provided from at least one of the plurality of CPUs. In this case, after the information is transmitted from at least one of the plurality of neural processors, it may be transmitted to at least one of the plurality of CPUs via the NPU bus 500-2 and the CPU bus 500-1. The selection signal generated by at least one of the plurality of CPUs may be transmitted to the switch circuit 400 via the CPU bus 500-1 and the peripheral bus 500-3.

[0317] After generating a plurality of clock signals, the switch circuit 400 may select a clock signal from the plurality of clock signals based on the obtained information, and the clock signal of the selected frequency may be transmitted to the corresponding neural processor.

[0318] To this end, the switch circuit 400 may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The switch circuit 400 may be a circuit in which at least two of the clock source, the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0319] That is, the switch circuit 400 may transmit the generated clock signal for each of the plurality of neural processors (eg, the first NPU 100 - 1 and the second NPU 100 - 2 ) to the corresponding neural processor.

[0320] For example, the switch circuit 400 provides the first clock signal to the first NPU 100-1 and provides the second clock signal to the second NPU 100-2, so that the first NPU 100-1 operates according to the first clock signal and the second NPU 100-2 operates according to the second clock signal.

[0321] For example, when the amount of computation or PE utilization of a given operation step of any layer among the operation steps of multiple layers in the first ANN processed by the first NPU 100-1 becomes higher than the operation step of the previous layer, the selection signal may instruct to reduce the frequency. Therefore, the switch circuit 400 may select a first clock signal having a frequency lower than the frequency of the operation step of the previous layer, and provide it to the first NPU 100-1. Alternatively, the switch circuit 400 may modulate the frequency of the second clock signal of the second frequency to generate a first clock signal of the first frequency, and provide it to the first NPU 100-1.

[0322] On the contrary, when the amount of operation or PE utilization of an operation step of a given arbitrary layer among the operation steps of multiple layers in the second ANN processed by the second NPU 100-2 becomes lower than the operation step of the previous layer, the selection signal may instruct to increase the frequency. Therefore, the switch circuit 400 may select a second clock signal having a frequency higher than the frequency of the operation step of the previous layer, and provide it to the second NPU 100-2. Alternatively, the switch circuit 400 may modulate the frequency of the first clock signal of the first frequency to generate a second clock signal of the second frequency, and provide it to the second NPU 100-2.

[0323] As described above, if the first NPU 100-1 and the second NPU 100-2 are operated in a distributed manner according to different clock signals, there are advantages in that the power deviation of adjacent operation steps can be reduced and the stability of the supply voltage can be improved.

[0324] The above description is mainly based on SoC, but the present disclosure is not limited to SoC. The content of the present disclosure can also be applied to board-level systems based on system-in-package (SiP) or printed circuit boards (PCB). For example, each functional component can be implemented by an independent semiconductor chip and connected through a system bus, which is implemented by means of an electrically conductive pattern formed on the PCB. Figure 9a and Figure 9b Give a description.

[0325] Figure 9a The diagram is an example of a system architecture showing an example of the second disclosure of the present disclosure.

[0326] Reference Figure 9a , the exemplary system 2000 may include multiple neural processors, multiple CPUs, and multiple memories attached to the substrate 900. The multiple neural processors may include, for example, a first NPU 100-1 and a second NPU 100-2. Moreover, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.

[0327] exist Figure 9a In the figure, it is shown that the number of the multiple neural processors, the multiple CPUs and the multiple memories is 2 respectively, but it is not limited to this, and the number can vary in various ways, such as 4, 6, 8, etc.

[0328] The exemplary system 2000 may include a memory controller 350 , a switch circuit 400 , a system bus 500 , and an I / O interface 800 .

[0329] The system bus 500 may be implemented by an electrically conductive pattern formed on the substrate 900. The system bus may perform high-speed communication. For example, the plurality of neural processors, the plurality of CPUs, the plurality of memories, and the memory controller 350 may communicate with each other through the system bus 500.

[0330] Available Figure 9aEach unit (ie, NPU, memory, and CPU) within the system 2000 is shown to input a driving voltage VDD.

[0331] exist Figure 9a , it is shown that a common driving voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0332] In some embodiments, it may also be configured to input an independent or separate driving voltage VDD to each unit (ie, NPU, memory, and CPU) within the system 2000 .

[0333] In other examples, it may also be configured to input an independent or separate driving voltage VDD to some units (ie, NPU, memory, and CPU) within the system 2000 .

[0334] In some other examples, it may also be configured to input a second driving voltage different from the first driving voltage to a specific unit in the system 2000 .

[0335] Figure 9a The switch circuit 400 shown is configured to receive a selection signal input. There may be multiple selection signals. Each corresponding selection signal may be input to multiple neural processors (e.g., the first NPU 100-1 and the second NPU 100-2). The selection signal may be determined based on the amount of operation of a specific operation step of a specific ANN model processed by the multiple specific neural processors or PE utilization information within at least one neural processor.

[0336] The information may be provided from at least one of the plurality of neural processors. Alternatively, the information may be provided from at least one of the plurality of CPUs. At this time, a selection signal corresponding to the information may be generated. The selection signal may be generated by the plurality of CPUs or the plurality of neural processors. The generated selection signal may be transmitted from at least one of the plurality of neural processors or at least one of the plurality of CPUs to the switch circuit 400. The selection signal may be transmitted to the switch circuit 400 via the system bus 500.

[0337] After generating a plurality of clock signals, the switch circuit 400 may select a clock signal from the plurality of clock signals based on the obtained information, and the clock signal of the selected frequency may be transmitted to the corresponding neural processor.

[0338] To this end, the switch circuit 400 may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The switch circuit 400 may be a circuit in which at least two of the clock source, the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0339] As another example, the switch circuit 400 may obtain the operation amount or PE utilization information of a specific operation step of a specific ANN model processed by at least one of the multiple neural processors. The switch circuit 400 may be configured to generate a selection signal based on the operation amount or PE utilization information of a specific operation step of a specific ANN model processed by at least one of the multiple neural processors.

[0340] That is, the switch circuit 400 may transmit the generated clock signal for each of the plurality of neural processors (eg, the first NPU 100 - 1 and the second NPU 100 - 2 ) to the corresponding neural processor.

[0341] For example, the switch circuit 400 provides the first clock signal to the first NPU 100-1 and provides the second clock signal to the second NPU 100-2, so that the first NPU 100-1 operates according to the first clock signal and the second NPU 100-2 operates according to the second clock signal.

[0342] For example, when the amount of computation or PE utilization of a given operation step of any layer among the operation steps of multiple layers in the first ANN processed by the first NPU 100-1 becomes higher than that of the operation step of the previous layer, the selection signal may command a frequency reduction. Therefore, the switch circuit 400 may select a first clock signal having a frequency lower than that of the operation step of the previous layer, and provide it to the first NPU 100-1. Alternatively, the switch circuit 400 may modulate the frequency of the second clock signal of the second frequency to generate a first clock signal of the first frequency, and provide it to the first NPU 100-1.

[0343] On the contrary, when the amount of operation or PE utilization of an operation step of a given arbitrary layer among the operation steps of multiple layers in the second ANN processed by the second NPU 100-2 becomes lower than the operation step of the previous layer, the selection signal may instruct to increase the frequency. Therefore, the switch circuit 400 may select a second clock signal having a frequency higher than the frequency of the operation step of the previous layer, and provide it to the second NPU 100-2. Alternatively, the switch circuit 400 may modulate the frequency of the first clock signal of the first frequency to generate a second clock signal of the second frequency, and provide it to the second NPU 100-2.

[0344] As described above, if the first NPU 100-1 and the second NPU 100-2 are operated in a distributed manner according to clock signals having different frequencies from each other, there are advantages in that power deviation of adjacent operation steps can be reduced and supply voltage stability can be improved.

[0345] Figure 9b The diagram is an example of a system architecture showing an example of the second disclosure of the present disclosure.

[0346] Reference Figure 9b , the exemplary system 2000 may include multiple neural processors, multiple CPUs, and multiple memories attached to the substrate 900. The multiple neural processors may include, for example, a first NPU 100-1 and a second NPU 100-2. Moreover, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.

[0347] exist Figure 9b In the figure, it is shown that the number of the multiple neural processors, the multiple CPUs and the multiple memories is 2 respectively, but it is not limited to this, and the number can vary in various ways, such as 4, 6, 8, etc.

[0348] The exemplary system 2000 may include a memory controller 350 , a switch circuit 400 , a plurality of buses, and an I / O interface 800 .

[0349] The plurality of bus bars may be implemented by electrically conductive patterns formed on the substrate 900 .

[0350] The plurality of buses may include a bus for a CPU, namely a CPU bus 500 - 1 , a bus for a neural processor, namely an NPU bus 500 - 2 , and a bus for peripheral components, namely a peripheral bus 500 - 3 .

[0351] The first CPU 200-1, the second CPU 200-2 and the first memory 300-1 may be connected to the CPU bus 500-1. The first NPU 100-1, the second NPU 100-2 and the second memory 300-2 may be connected to the NPU bus 500-2. The memory controller 350 and the switch circuit 400 may be connected to the peripheral bus 500-3.

[0352] Available Figure 9b Each unit (ie, NPU, memory, and CPU) within the system 2000 is shown to input a driving voltage VDD.

[0353] exist Figure 9b , it is shown that a common driving voltage VDD is input, but examples of the present disclosure are not limited thereto.

[0354] In some embodiments, it may also be configured to input an independent or separate driving voltage VDD to each unit (ie, NPU, memory, and CPU) within the system 2000 .

[0355] In other examples, it may also be configured to input an independent or separate driving voltage VDD to some units (ie, NPU, memory, and CPU) within the system 2000 .

[0356] In some other examples, it may also be configured to input a second driving voltage different from the first driving voltage to a specific unit in the system 2000 .

[0357] Figure 9a The switch circuit 400 shown is configured to receive a selection signal input. There may be multiple selection signals. Each corresponding selection signal may be input to multiple neural processors (e.g., the first NPU 100-1 and the second NPU 100-2). The selection signal may be determined based on the amount of operation of a specific operation step of a specific ANN model processed by the multiple specific neural processors or PE utilization information within at least one neural processor.

[0358] The information may be provided from at least one of the multiple neural processors. Alternatively, the information may be provided from at least one of the multiple CPUs. At this time, a selection signal corresponding to the information may be generated. The selection signal may be generated by the multiple CPUs or the multiple neural processors. The generated selection signal may be transmitted to the switch circuit 400 from at least one of the multiple neural processors or at least one of the multiple CPUs. The selection signal may be transmitted to the switch circuit 400 via at least one of the NPU bus 500-2, the CPU bus 500-1, and the peripheral bus 500-3.

[0359] Alternatively, the information may be provided from at least one of the plurality of CPUs. In this case, after being transmitted from at least one of the plurality of neural processors, the information may be transmitted to at least one of the plurality of CPUs via the NPU bus 500-2 and the CPU bus 500-1. The selection signal generated by at least one of the plurality of CPUs may be transmitted to the switch circuit 400 via the CPU bus 500-1 and the peripheral bus 500-3.

[0360] After generating a plurality of clock signals, the switch circuit 400 may select a clock signal from the plurality of clock signals based on the obtained information, and the clock signal of the selected frequency may be transmitted to the corresponding neural processor.

[0361] To this end, the switch circuit 400 may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The switch circuit 400 may be a circuit in which at least two of the clock source, the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0362] As another example, the switch circuit 400 may obtain the operation amount or PE utilization information of a specific operation step of a specific ANN model processed by at least one of the multiple neural processors. The switch circuit 400 may be configured to generate a selection signal based on the operation amount or PE utilization information of a specific operation step of a specific ANN model processed by at least one of the multiple neural processors.

[0363] That is, the switch circuit 400 may transmit the generated clock signal for each of the plurality of neural processors (eg, the first NPU 100 - 1 and the second NPU 100 - 2 ) to the corresponding neural processor.

[0364] For example, the switch circuit 400 provides the first clock signal to the first NPU 100-1 and provides the second clock signal to the second NPU 100-2, so that the first NPU 100-1 operates according to the first clock signal and the second NPU 100-2 operates according to the second clock signal.

[0365] For example, when the amount of computation or PE utilization of a given operation step of any layer among the operation steps of multiple layers in the first ANN processed by the first NPU 100-1 becomes higher than that of the operation step of the previous layer, the selection signal may command a frequency reduction. Therefore, the switch circuit 400 may select a first clock signal having a frequency lower than that of the operation step of the previous layer, and provide it to the first NPU 100-1. Alternatively, the switch circuit 400 may modulate the frequency of the second clock signal of the second frequency to generate a first clock signal of the first frequency, and provide it to the first NPU 100-1.

[0366] On the contrary, when the amount of operation or PE utilization of an operation step of a given arbitrary layer among the operation steps of multiple layers in the second ANN processed by the second NPU 100-2 becomes lower than the operation step of the previous layer, the selection signal may instruct to increase the frequency. Therefore, the switch circuit 400 may select a second clock signal having a frequency higher than the frequency of the operation step of the previous layer, and provide it to the second NPU 100-2. Alternatively, the switch circuit 400 may modulate the frequency of the first clock signal of the first frequency to generate a second clock signal of the second frequency, and provide it to the second NPU 100-2.

[0367] As described above, if the first NPU 100-1 and the second NPU 100-2 operate in a distributed manner according to different clock signals, there are advantages in that the power deviation of adjacent operation steps can be reduced and the stability of the supply voltage can be improved.

[0368] Fig.10a An example diagram showing a neural processor architecture of an example of the third disclosure of the present disclosure.

[0369] exist Fig.10a , the neural processor 100 is shown to include a plurality of PEs 110, an internal memory 120, an SFU 150, and a clock generator 190. However, the neural processor 100 is Figure 3 or Figure 5 As shown, it may also include an NPU controller 130 and an NPU interface 140. The multiple PEs 110, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150, and the clock generator 190 may be semiconductor circuits connected with a large number of transistors. Therefore, some of them are difficult to identify and distinguish with the naked eye, and can only be identified based on the operation. The switch circuit 190 can be called a clock signal supply circuit.

[0370] For example, any circuit may operate as a plurality of PEs 110 , or may operate as an NPU controller 130 .

[0371] The plurality of PEs 110 may include adders, multipliers, and accumulators.

[0372] Fig.10a The switch circuit 190 shown may be a circuit including at least one of a clock source, a clock skipper circuit, a clock divider circuit, a clock gating circuit, an oscillator, a phase-locked loop (PLL) circuit, and a selector. The switch circuit 190 may be a circuit in which at least two of the clock source, the clock skipper circuit, the clock divider circuit, the clock gating circuit, the oscillator circuit, the phase-locked loop (PLL) circuit, and the selector are combined. However, the examples of the present disclosure are not limited thereto.

[0373] The neural processor 100 may obtain the computational amount or PE utilization information of a specific computational step of a specific ANN model processed by the plurality of PEs 110. The neural processor 100 may be configured to generate a selection signal based on the computational amount or PE utilization information of a specific computational step of a specific ANN model processed by at least one neural processor among the plurality of neural processors.

[0374] As another example, the switch circuit 190 may obtain the computational amount or PE utilization information of a specific computational step of a specific ANN model processed by the neural processor 100. The switch circuit 190 may be configured to generate a selection signal based on the computational amount or PE utilization information of a specific computational step of a specific ANN model processed by the neural processor 100.

[0375] The switch circuit 190 can output a clock signal of a corresponding frequency in response to the selection signal.

[0376] The information may be provided by a scheduler within the NPU controller 130 .

[0377] When the amount of operation or PE utilization of the operation steps of a given arbitrary layer among the operation steps of the plurality of layers in the first ANN processed by the neural processor 100 becomes higher than the operation steps of the previous layer, the selection signal may instruct to reduce the frequency. Therefore, the switch circuit 190 may select a first clock signal having a frequency lower than the frequency of the operation steps of the previous layer, and provide it to the plurality of PEs 110. Alternatively, the switch circuit 190 may modulate the frequency of the second clock signal of the second frequency to generate a first clock signal of the first frequency, and provide it to the plurality of PEs 110.

[0378] On the contrary, when the amount of operation or PE utilization of the operation steps of a given arbitrary layer among the operation steps of the plurality of layers in the first ANN processed by the neural processor 100 becomes lower than the operation steps of the previous layer, the selection signal may instruct to increase the frequency. Therefore, the switch circuit 190 may select a second clock signal having a frequency higher than the frequency of the operation steps of the previous layer, and provide it to the plurality of PEs 110. Alternatively, the switch circuit 190 may modulate the frequency of the first clock signal of the first frequency, generate a second clock signal of the second frequency, and provide it to the plurality of PEs 110.

[0379] As described above, if the first NPU 100-1 and the second NPU 100-2 operate in a distributed manner according to different clock signals, there are advantages in that the power deviation of adjacent operation steps can be reduced and the stability of the supply voltage can be improved.

[0380] Fig.10b This is an example diagram showing a neural processor architecture of another example of the third disclosure of the present disclosure.

[0381] exist Fig.10b , the neural processor 100 is shown to include a plurality of PEs 110, an internal memory 120, and an SFU 150. However, in addition to this, the neural processor 100 is Figure 3 or Figure 5 As shown, it may also include an NPU controller 130 and an NPU interface 140. The plurality of PEs 110, the internal memory 120, the NPU controller 130, the NPU interface 140 and the SFU 150 may be semiconductor circuits connected with a large number of transistors. Therefore, some of them are difficult to identify and distinguish by naked eyes, and can only be identified according to the operation.

[0382] Reference Fig.10b , the switch circuit 400 is shown as being located outside the neural processor 100 rather than inside. Fig.10a The contents shown are the same, so it is incorrect Fig.10b Described separately, but directly quoted reference Fig.10a The content to be described.

[0383] On the other hand, although Fig.10a and Fig.10b Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .

[0384] On the other hand, Fig.10a and Fig.10b A neural processor is described in , however the neural processor may also be implemented in a SoC form factor.

[0385] Fig.11An example is shown where peak power increases with increasing PE utilization.

[0386] Reference Fig.11 , the clock signal CLK0 can be input to multiple neural processors (100-1, 100-2). Wherein, multiple neural processors (100-1, 100-2) are configured to accept input of the same phase clock signal. The multiple neural processors may include the first NPU 100-1 and the second NPU 100-2.

[0387] Fig.11 100×PE utilization exemplarily shows peak power and driving voltage VDD variation characteristics when 100 processing units are activated.

[0388] Fig.11 200×PE utilization exemplarily shows peak power and driving voltage VDD variation characteristics when 200 processing units are activated.

[0389] Fig.11 500×PE utilization exemplarily shows the peak power and driving voltage VDD variation characteristics when 500 processing units are activated.

[0390] Reference Fig.11 , showing a tendency that the more the number of activated processing units of the neural processor 100 increases, the more the peak power increases and the more the driving voltage VDD decreases.

[0391] Furthermore, the number of PEs running per clock cycle will vary depending on the performance of the compiler that compiles the artificial neural network model. That is, the better the performance of the compiler, the more PEs can be run per unit clock. Among them, the ratio of running PEs in all PEs can be called PE utilization (utilization rate%). On the contrary, the better the performance of the compiler, the more the peak power of the neural processor will increase. That is, the peak power may be proportional to the PE utilization. Therefore, as the compiler algorithm becomes more sophisticated, there is a problem that the peak power of the neural processor will further increase.

[0392] Refer again Fig.11 The peak power when running with the clock signal CLK0 as the reference will vary according to the number of PEs running per clock. In other words, the peak power will increase in proportion to the number of PEs running during one clock.

[0393] Furthermore, when a neural processor developed with low power is applied to edge devices, it can be developed corresponding to the M.2 interface or the PCI Express interface. In particular, the maximum power of the M.2 interface will be relatively lower than that of the PCI Express interface. For example, the M.2 interface can have a 3A current and a 10W power limit. When the peak power exceeds 10W during a specific clock period, the supply voltage VDD will fluctuate. At this time, the higher the utilization rate of PE, the more the peak power increases, and the more the peak power exceeds the power limit, the more the degree of decrease in the supply voltage VDD will increase in proportion to the exceeding information.

[0394] When the supply voltage VDD drops below the IR-drop margin, data errors will occur in the neural processor operation. Further, this occurs due to the resistance R of the metal interconnect and the current I flowing through it. According to Ohm's law, when a voltage drop occurs (V=IR), an IR-drop occurs. If a large number of circuits associated with multiple PEs of the neural processor are switched simultaneously, the neural processor will consume a considerable amount of current. This high current will cause a larger voltage drop in the neural processor, reducing the supply voltage in other parts of the SoC.

[0395] Summarize Fig.11 For example, to ensure stable operation of a neural processor and prevent potential errors, sufficient IR-drop margin must be ensured. IR-drop margin can refer to the difference between the supply voltage VDD and the lowest voltage allowed at all locations in the neural processor during peak power conditions.

[0396] According to an example of the present disclosure, the IR-drop margin can be predicted based on the PE utilization of a specific operation step of a specific artificial neural network model. Therefore, when the IR-drop margin is expected to drop below a threshold, a selection signal can be determined based on the PE utilization and the IR-drop margin to ensure the IR-drop margin.

[0397] Fig.12 An example is shown in which power is used differently when a clock signal is used according to PE utilization.

[0398] exist Fig.12 In FIG. 1 , three operation steps of a specific artificial neural network model are exemplarily shown along the time axis. In the corresponding example, the power characteristics of a processing unit PE at different driving frequencies are shown. Fig.12 , the power gradually increases in proportion to the frequency.

[0399] For example, as shown in Table 2 below, the switch circuit 400 may be designed to selectively output a clock signal of a preset frequency. Therefore, if a specific selection signal is input to the switch circuit 400, a clock signal corresponding to the selection signal may be supplied to the neural processor.

[0400]

Table 2

[0401] frequency Select Signal 600MHz 00 800MHz 01 1.0GHz 10 1.2GHz 11

[0402] Again Fig.11 And refer to Fig.12 It should be noted that the more the utilization of the processing unit increases, the more the power can be increased proportionally. Therefore, the power of each operation step can be calculated based on the PE utilization information and the driving frequency information of each operation step. The power calculation can be calculated based on the scheduling information of a specific artificial neural network model. Among them, the power (Watt) can be calculated as the value obtained by dividing the energy consumed by the corresponding operation step by the unit time (that is, 1watt (W) = 1joule per second (J / s)). Therefore, the power of each operation step can be calculated based on the scheduling information.

[0403] For example, the power in the 600 MHz interval is lower than that in the 800 MHz interval. In addition, the power in the 800 MHz interval is lower than that in the 1.0 GHz interval.

[0404] The switch circuit 400 may supply the first clock signal of 600 MHz to the neural processor according to the input first selection signal 000. Therefore, the first selection signal 000 is supplied during the first operation step according to the scheduling information. The first selection signal 000 may be selected based on the PE utilization of the corresponding operation step. The PE utilization of the first operation step may be higher than the PE utilization of the second operation step.

[0405] The switch circuit 400 supplies the second clock signal of 800 MHz to the neural processor according to the input second selection signal 001. The second selection signal 001 is supplied during the second operation step according to the scheduling information. The second selection signal 001 can be selected based on the PE utilization of the corresponding operation step. The PE utilization of the second operation step can be higher than the PE utilization of the third operation step.

[0406] The switch circuit 400 supplies the third clock signal of 1.0 GHz to the neural processor according to the input third selection signal 010. The third selection signal 010 is supplied during the third operation step according to the scheduling information. The third selection signal 010 can be selected based on the PE utilization of the corresponding operation step. The PE utilization of the third operation step can be lower than the PE utilization of the second operation step.

[0407] In some embodiments, the neural processor, the SoC including the neural processor, and the system including the SoC (including the neural processor) may further include a temperature sensor. In the example, the driving frequency may also be selectively set lower under conditions above a certain critical temperature.

[0408] As described above, in a specific operation step, as the PE utilization rate increases, a clock signal with a lower frequency is used, so that the power of the specific operation step may not increase instantaneously. If the PE utilization rate increases instantaneously, the peak power increases, and eventually the supply power VDD will drop below the IR-drop margin. That is, when the PE utilization rate increases, a clock signal with a lower frequency may be selectively used, so that the supply power VDD may not drop below the IR-drop margin.

[0409] <Brief summary of the disclosure of this patent>

[0410] The contents disclosed in this disclosure are organized as follows.

[0411] First, according to one aspect of the present disclosure, a system is disclosed. The system may include: a neural processor (NPU: artificial neural network), the NPU including a plurality of processing units (PE: processing element) capable of performing operations on at least one ANN in an artificial neural network (ANN: artificial neural network) model; and a switching circuit. The switching circuit may select one of a plurality of clock signals having mutually different frequencies, and supply the selected one clock signal to the NPU. The one clock signal may be selected based on the utilization rate of the PE for a given layer of a plurality of layers within the at least one ANN model.

[0412] In addition, according to one aspect of the present disclosure, a SoC (system-on-chip) is provided. The SoC may include: a first circuit configured to enable a plurality of processing elements (PEs) to perform operations on at least one ANN model in an artificial neural network (ANN) model; and a switch circuit. The switch circuit may select one of a plurality of clock signals having different frequencies and provide the selected one clock signal to the NPU. The one clock signal may be selected based on the utilization of a given layer of a plurality of layers within the at least one ANN model by the PE.

[0413] In addition, according to one aspect of the present disclosure, a method for driving a neural processor (NPU: neural processing unit) is provided. The NPU driving method may include: a step of selecting one clock signal from a plurality of clock signals having different frequencies; and a step of supplying the selected one clock signal to the NPU. The NPU may include a plurality of processing elements (PE: processing element) capable of performing operations on at least one ANN in an artificial neural network (ANN: artificial neural network) model. The one clock signal may be selected based on the utilization of the PE for a given layer of a plurality of layers within the at least one ANN model.

[0414] The applicable contents of the aforementioned disclosed system, SoC, and driving method are described as follows.

[0415] The one clock signal may be further selected taking into account the PE utilization threshold.

[0416] In order to reduce power when performing operations in the given layer, the PE utilization threshold may be determined.

[0417] The one clock signal may be selected when the NPU completes operation on a layer above the given layer in the at least one ANN model.

[0418] The one clock signal may be selected for the given layer based on previous layer information output by the NPU by completing an operation on a layer previous to the given layer in the at least one ANN model.

[0419] The switch circuit may include: a generator that outputs a plurality of clock signals; and a selector that selects the one clock signal.

[0420] When the utilization rate of the given layer is higher than that of an upper layer, a clock signal selected for the given layer may have a second frequency that is lower than a first frequency of the upper layer.

[0421] When the utilization rate of the given layer is lower than that of an upper layer, a clock signal selected for the given layer may have a second frequency that is higher than a first frequency of the upper layer.

[0422] The PE utilization may be determined for each of a plurality of layers of the at least one ANN model.

[0423] The PE utilization may be determined based on scheduling information, where the scheduling information is used to distribute operations of each layer of the at least one ANN model to the plurality of PEs.

[0424] The examples of the present disclosure shown in this specification and the drawings are only used to easily describe the technical content of the present disclosure, and specific examples are disclosed to help understand the present disclosure, and are not intended to limit the scope of the present invention. It goes without saying that a person skilled in the art of the present disclosure can implement other variations in addition to the examples described so far.

[0425]

National R&D Project to Support This Invention

[0426]

Project unique number

[0427]

Project number

[0428]

Department name

[0429]

Name of the project management (major) organization

[0430] [Research Project Name] PIM artificial intelligence semiconductor core technology development (design)

[0431]

Research Topic Name

[0432]

Project Execution Organization Name

[0433]

Study period

Claims

1. A system comprising: a neural processor comprising a plurality of processing units capable of performing operations on at least one artificial neural network in the artificial neural network model; and Switching circuit; wherein the switch circuit selects one of a plurality of clock signals having different frequencies, and provides the selected one clock signal to the neural processor, The one clock signal is selected based on utilization of a given layer of a plurality of layers within the at least one artificial neural network model by the processing unit.

2. The system according to claim 1, wherein: The one clock signal is further selected taking into account a utilization threshold of the processing unit.

3. The system according to claim 1, wherein: A utilization threshold of the processing unit is determined in order to reduce power when performing operations in the given layer.

4. The system according to claim 1, wherein: When the neural processor completes the operation of a layer above the given layer in the at least one artificial neural network model, the one clock signal is selected.

5. The system according to claim 1, wherein: The one clock signal is selected for the given layer based on previous layer information output by the neural processor by completing an operation on a layer previous to the given layer in the at least one artificial neural network model.

6. The system according to claim 1, wherein: The switch circuit comprises: a generator that outputs a plurality of clock signals; and A selector selects the one clock signal.

7. The system according to claim 1, wherein: When the utilization rate of the given layer is higher than that of the previous layer, One clock signal selected for the given layer has a second frequency that is lower than a first frequency of the previous layer.

8. The system according to claim 1, wherein: When the utilization rate of the given layer is lower than that of the previous layer, One clock signal selected for the given layer has a second frequency that is higher than a first frequency of the previous layer.

9. The system according to claim 1, wherein: The utilization of the processing unit is determined for each of a plurality of layers of the at least one artificial neural network model.

10. The system according to claim 1, wherein: The utilization of the processing units is determined based on scheduling information, and the scheduling information is used to distribute the operations of each layer of the at least one artificial neural network model to the multiple processing units.

11. A system on chip, comprising: a first circuit configured to be capable of performing a plurality of processing units on at least one of the artificial neural network models; and Switching circuit; wherein the switch circuit selects one clock signal from a plurality of clock signals having different frequencies, and provides the selected clock signal to the first circuit; The one clock signal is selected based on utilization of a given layer of a plurality of layers within the at least one artificial neural network model by the processing unit.

12. The system on chip according to claim 11, wherein: The one clock signal is further selected taking into account a utilization threshold of the processing unit.

13. The system on chip according to claim 11, wherein: A utilization threshold of the processing unit is determined in order to reduce power when performing operations in the given layer.

14. The system on chip according to claim 11, wherein: When the neural processor completes the operation of a layer above the given layer in the at least one artificial neural network model, the one clock signal is selected.

15. The system on chip according to claim 11, wherein: The one clock signal is selected for the given layer based on previous layer information output by the neural processor by completing an operation on a layer previous to the given layer in the at least one artificial neural network model.

16. The system on chip according to claim 11, wherein: When the utilization rate of the given layer is higher than that of the previous layer, One clock signal selected for the given layer has a second frequency that is lower than a first frequency of the previous layer.

17. The system on chip according to claim 11, wherein: When the utilization rate of the given layer is lower than that of the previous layer, One clock signal selected for the given layer has a second frequency that is higher than a first frequency of the previous layer.

18. A method, as a method of driving a neural processor, comprising: A step of selecting one clock signal from among a plurality of clock signals having mutually different frequencies; and The step of supplying said selected one clock signal to said neural processor; The neural processor includes a plurality of processing units capable of performing operations on at least one artificial neural network in the artificial neural network model. The one clock signal is selected based on utilization of a given layer of a plurality of layers within the at least one artificial neural network model by the processing unit.

19. The method according to claim 18, wherein: The one clock signal is further selected taking into account a utilization threshold of the processing unit.

20. The method according to claim 18, wherein: A utilization threshold of the processing unit is determined in order to reduce power when performing operations in the given layer.

Images