System including NPU, system on chip, and method for driving npu
By dividing the clock signal into multiple parts and making the artificial neural network model perform dispersed execution based on different parts of the clock signal, the instantaneous power consumption and instability of power supply caused by the difference in operation volume of the artificial neural network model at different levels are solved. Problem, achieving more stable power supply and reducing power consumption.
Patent Information
- Application Number
- CN202311416645.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-17
- Filing Date
- 2023-10-27
- Publication Date
- 2024-11-12
AI Technical Summary
The problems of instantaneous power consumption and instability of power supply caused by the difference in the operational volume of the artificial neural network model at different levels, especially in the absence of power supply capacity in edge devices, may lead to reduced system stability and data loss.
By dividing the clock signal into multiple parts, the operation of the artificial neural network model is performed dispersedly with different parts of the clock signal as the reference, and the operation is based on the rising and falling edges of the clock signal as the reference, and the delay is achieved by multiple processing elements. The buffer performs phase conversion to improve the stability of the power supply.
It reduces the instantaneous power consumption of the neural processing unit, improves the stability of the supply voltage, and avoids data loss and system stability caused by insufficient power supply.
Smart Images

Figure CN118940810A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a technique for reducing instantaneous power consumption of a neural processing unit. Background Art
[0002] Artificial intelligence (AI) is gradually developing. AI refers to artificial intelligence that imitates human intelligence, that is, intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc.
[0003] Furthermore, recently, in order to accelerate the calculation speed for artificial intelligence (AI), a neural processing unit (NPU) is being developed.
[0004] There may be various types of artificial neural network (ANN) models depending on the purpose of the artificial intelligence service. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. within the image / video may be used for the artificial intelligence service.
[0005] Typically, an artificial neural network model can have different amounts of computation in different layers.
[0006] In particular, if the amount of computation increases significantly in a particular layer, power consumption may increase instantaneously. Summary of the invention
[0007] Artificial neural network operations are data-intensive. In particular, artificial neural network operations require parallel processing operations. That is, the characteristic of artificial neural network operations is that if a large amount of data cannot be processed in parallel at the same time, the processing speed will be reduced.
[0008] Thus, the inventors of the present disclosure have developed a neural processing unit specifically for artificial neural network operations. The inventors of the present disclosure attempt to improve the parallel processing performance of the neural processing unit by increasing the number of multiple processing elements of the neural processing unit. In addition, the inventors of the present disclosure also attempt to develop a neural processing unit capable of low-power operation.
[0009] In addition, compared to the power supply of a graphics processing unit (GPU) operating in a server or a personal computer (PC), the power supply of a neural processing unit dedicated to low-power operation provided in an edge device may have a relatively low power supply capacity. In addition, the capacity of the capacitor of the power supply of the neural processing unit dedicated to low-power operation provided in the edge device may not be sufficient to meet the instantaneous power supply.
[0010] However, the inventors of the present disclosure recognized that as the number of processing elements of a neural processing unit dedicated to low-power operation increases, the degree of instantaneous oscillation of the supply voltage may increase. Specifically, the required instantaneous power supply of the neural processing unit may be proportional to the number of processing elements in operation. In addition, the amount of calculation of the artificial neural network model has a large deviation per layer. Therefore, depending on the amount of calculation of the layer of the artificial neural network model, the number of processing elements operating in parallel may be different. That is, when many processing elements are operating simultaneously, the voltage of the power supply unit of the neural processing unit may drop instantaneously.
[0011] Thus, the inventors of the present disclosure recognized that there is a need to improve the stability of the supply voltage of a neural processing unit dedicated to artificial neural network operations even if the number of processing elements increases.
[0012] Therefore, the disclosure of this specification aims to provide a technical solution for stabilizing the jitter of the supply voltage of a power supply unit.
[0013] In order to achieve the above-mentioned purpose, a disclosed content of the present specification provides a system. The system may include a neural processing unit (NPU), and the neural processing unit (NPU: neural processing unit) includes: at least one memory; and multiple processing elements (PE: processing element), which can perform operations for at least one of the artificial neural network (ANN: artificial neural network) models. The multiple processing units may include adders, multipliers, and accumulators. The multiple PEs may include a first group of processing units set to operate in a first part of a clock signal and a second group of processing units set to operate in a second part of the clock signal.
[0014] The first portion may be a rising edge portion of the clock signal, and the second portion may be a falling edge portion of the clock signal.
[0015] A first output from the first group of PEs and a second output from the second group of PEs may be communicated with reference to a first portion of the clock signal.
[0016] The reference phases of the first group of PEs and the second group of PEs may be switched from a rising edge of the clock signal to a falling edge of the clock signal, or from a falling edge of the clock signal to a rising edge of the clock signal.
[0017] The reference phase for the second group of PEs may be switched from the second portion to the first portion of the clock signal.
[0018] The output ports of the second group of PEs may be connected to a delay buffer, which may perform a conversion from a rising edge of the clock signal to a falling edge of the clock signal, or perform a conversion from a falling edge of the clock signal to a rising edge of the clock signal.
[0019] The output ports of the second group of PEs may be connected to delay buffers that may perform a conversion from the second portion of the clock signal to the first portion of the clock signal, or from the first portion of the clock signal to the second portion of the clock signal.
[0020] The NPU may further include a special function unit (SFU) connected to output ports of the first group of PEs and the second group of PEs.
[0021] The SFU may be connected to the at least one memory.
[0022] The first group of PEs and the second group of PEs may be time-variable.
[0023] In order to achieve the above-mentioned purpose, a disclosed content of the present specification provides a system-on-chip (SoC). The SoC may include: a first circuit arranged for at least one memory; and a second circuit arranged for a plurality of processing elements (PEs) capable of performing at least one operation for an artificial neural network (ANN) model, wherein the plurality of PEs may include an adder, a multiplier, and an accumulator, wherein the plurality of PEs may include: a first group of PEs set to operate in a first part of a clock signal; and a second group of PEs set to operate in a second part of the clock signal.
[0024] The first portion may be a rising edge portion of the clock signal, and the second portion may be a falling edge portion of the clock signal.
[0025] A first output from the first group of PEs and a second output from the second group of PEs may be delivered with reference to a first portion of the clock signal.
[0026] The reference phases of the first group of PEs and the second group of PEs are switched from a rising edge of the clock signal to a falling edge of the clock signal, or from a falling edge of the clock signal to a rising edge of the clock signal.
[0027] The reference phase for the second group of PEs may be switched from the second portion to the first portion of the clock signal.
[0028] The output ports of the second group of PEs may be connected to a delay buffer, which may perform a conversion from a rising edge of the clock signal to a falling edge of the clock signal, or perform a conversion from a falling edge of the clock signal to a rising edge of the clock signal.
[0029] The output ports of the second group of PEs may be connected to delay buffers that may perform a conversion from the second portion of the clock signal to the first portion of the clock signal, or from the first portion of the clock signal to the second portion of the clock signal.
[0030] In order to achieve the above-mentioned purpose, a disclosed content of the present specification provides a method for driving a neural processing unit (NPU). The driving method of the NPU may include the following steps: generating a clock signal for driving a neural processing unit NPU including a plurality of processing elements (PEs); causing a first group of PEs among the plurality of PEs to operate in a first part of the clock signal; and causing a second group of PEs among the plurality of PEs to operate in a second part of the clock signal, wherein the plurality of PEs may include an adder, a multiplier, and an accumulator.
[0031] The first portion may be a rising edge portion of the clock signal, and the second portion may be a falling edge portion of the clock signal.
[0032] A first output from the first group of PEs and a second output from the second group of PEs may be delivered with reference to a first portion of the clock signal.
[0033] According to the examples of the present disclosure, since artificial neural network operations are dispersedly performed based on different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0034] According to an example of the present disclosure, since artificial neural network operations are decentralized based on different parts of a clock, there is an advantage in that the safety of the supply voltage supplied to the neural processing unit can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.
[0036] Figure 2a This is a diagram used to illustrate the basic structure of a convolutional neural network (CNN).
[0037] Figure 2b is a comprehensive diagram showing the operation of a convolutional neural network for easy understanding.
[0038] Figure 3 is a schematic conceptual diagram illustrating a neural processing unit of the present disclosure.
[0039] Figure 4a is a schematic conceptual diagram illustrating a processing element that is one of a plurality of processing elements that may be applied to the present disclosure.
[0040] Figure 4b is a schematic conceptual diagram illustrating an SFU that can be applied to the present disclosure.
[0041] Figure 5 It is shown Figure 3 An example diagram of a modified example of the NPU 100 is shown.
[0042] Figure 6a is an example diagram showing the data size of each layer in an exemplary artificial neural network model, Figure 6b It is shown Figure 6a An exemplary table showing the data size, etc. of each layer in the exemplary artificial neural network model shown.
[0043] Figure 7a : is an exemplary diagram showing the structure of an NPU according to an example of the first disclosure of this specification.
[0044] Figure 7b is an exemplary diagram showing a structure of an NPU according to another example of the first disclosure of this specification.
[0045] Figure 8a is an exemplary diagram showing the structure of a SoC according to an example of the second disclosure of this specification.
[0046] Figure 8b is an exemplary diagram showing a structure of a SoC according to another example of the second disclosure of this specification.
[0047] Figure 9a : is an exemplary diagram showing the structure of a system according to an example of the third disclosure of this specification.
[0048] Figure 9b : is an exemplary diagram showing a system configuration according to an example of the third disclosure of this specification.
[0049] Fig.10 is an exemplary diagram illustrating an operation method of an NPU according to the disclosure of this specification.
[0050] Fig.11 is an exemplary diagram illustrating an operation method of an NPU according to the disclosure of this specification. DETAILED DESCRIPTION
[0051] The specific structure and step description of the examples of the present disclosure are only used to illustrate the examples according to the concepts of the present disclosure. Therefore, the examples according to the concepts of the present disclosure can be implemented in various forms. The examples according to the concepts of the present disclosure can be implemented in various forms. The present disclosure should not be interpreted as being limited to the examples of the present disclosure.
[0052] The examples according to the concepts of the present disclosure may be subjected to various changes and may have various forms. Thus, specific examples are shown in the accompanying drawings and the present disclosure or the present application is described in detail. However, this is not intended to limit the examples according to the concepts of the present disclosure to specific disclosed forms. The examples according to the concepts of the present disclosure should be understood to include all changes, equivalents and substitutes within the scope of the ideas and technologies of the present disclosure.
[0053] The terms first and / or second etc. may be used to describe a variety of constituent elements, but the constituent elements are not limited by the terms. The terms are only used to distinguish one constituent element from another constituent element. The terms do not depart from the scope of rights of the concepts disclosed herein, and the first constituent element may be named as the second constituent element, and similarly, the second constituent element may also be named as the first constituent element.
[0054] When it is mentioned that a constituent element is "connected" or "connected" to another constituent element, it can be directly connected or connected to the other constituent element. However, it should be understood that other constituent elements may exist between multiple constituent elements. On the contrary, when it is mentioned that a constituent element is "directly connected" or "directly connected" to another constituent element, it should be understood that there are no other constituent elements between the two. Other expressions used to describe the relationship between constituent elements, that is, "between" and "directly between" or "adjacent to" and "directly adjacent to", etc., should also be interpreted in the same manner.
[0055] The terms used in this disclosure are only used to illustrate specific examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In this disclosure, terms such as "including" or "having" are used to specify the existence of the described features, numbers, steps, operations, structural elements, components, or combinations thereof. Therefore, it should be understood that the existence or additional possibility of one or more other features or numbers, steps, operations, structural elements, components, or combinations thereof is not precluded.
[0056] In addition, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as those generally understood by ordinary technicians in the field to which the present invention belongs. Terms that are the same as those defined in commonly used dictionaries should be interpreted as having the same meaning as in the context of the relevant technology, and should not be interpreted as ideal or overly formal meanings unless clearly defined in this application.
[0057] When describing the examples, the description of the technical contents that are well known in the field to which the present disclosure belongs and are not directly related to the present disclosure will be omitted. This is to convey the gist of the present disclosure more clearly without confusion by omitting unnecessary descriptions.
[0058] Definition of terms
[0059] Hereinafter, in order to help understanding the disclosed contents proposed in the present disclosure, the terms used in the present disclosure will be briefly sorted out.
[0060] NPU: As the abbreviation of Neural Processing Unit, it can refer to a processor independent of the central processing unit (CPU) and specifically used for the calculation of artificial neural network models.
[0061] ANN: As the abbreviation of artificial neural network, it can refer to a network that imitates the connection of neurons in the human brain through synapses in order to imitate human intelligence, thereby connecting nodes into a layered structure.
[0062] DNN: As the abbreviation of Deep Neural Network, it can refer to increasing the number of hidden layers of artificial neural networks in order to achieve higher artificial intelligence.
[0063] CNN: Convolutional Neural Network is a neural network that functions similarly to the image processing in the visual cortex of the human brain. Convolutional neural networks are known to be suitable for image processing and are convenient for extracting features of input data and grasping the patterns of features.
[0064] Hereinafter, the present disclosure will be described in detail by describing preferred examples of the present disclosure with reference to the accompanying drawings. Hereinafter, examples of the present disclosure will be described in detail with reference to the accompanying drawings.
[0065]
AI
[0066] Humans have intelligence that can recognize, classify, infer, predict, control / decision making, etc. Artificial intelligence (AI) refers to artificial intelligence that imitates human intelligence.
[0067] The human brain is composed of many nerve cells called neurons. Each neuron is connected to hundreds or even thousands of other neurons through a connection site called a synapse. In order to imitate human intelligence, the working principle of biological neurons and the connection relationship between neurons are modeled and called an artificial neural network model. In other words, an artificial neural network is a system that connects nodes that imitate neurons into a layer structure.
[0068] The above-mentioned artificial neural network models are divided into "single-layer neural network" and "multi-layer neural network" according to the number of layers. A general multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is a layer that receives external data. The number of neurons in the input layer is the same as the number of input variables. (2) The hidden layer is located between the input layer and the output layer, receives signals from the input layer, extracts features and transmits them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them to the outside. The input signal between neurons is multiplied by the connection strength of each connection with a value between 0 and 1, and then summed. If the sum is greater than the threshold of the neuron, the neuron is activated and realized as an output value through the activation function.
[0069] In addition, increasing the number of hidden layers of an artificial neural network to achieve a higher level of artificial intelligence is called a deep neural network (DNN).
[0070] DNN is being developed in various structures. For example, it is known that a convolutional neural network (CNN) as an example of DNN is convenient for extracting features of input values (images or pictures) and grasping the pattern of the extracted output values. CNN can be composed of convolution operations, activation function operations, pooling operations, etc. processed in a specific order.
[0071] For example, in each layer of a DNN, a parameter (i.e., input value, output value, weight, or kernel, etc.) can be a matrix consisting of multiple channels. The parameter can be processed in the NPU through convolution or matrix multiplication. The processed output value is generated in each layer.
[0072] For example, a transformer is a DNN based on attention technology. The transformer uses multiple matrix multiplication operations. The transformer can obtain the attention (Q, K, V) as the output value by using the input value and the parameters such as the query vector (Q: query), the key vector (K: key), and the value vector (V: value). The transformer can process multiple inference operations based on the output value (i.e., attention (Q, K, V)). Transformers often show better inference performance than CNN.
[0073] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.
[0074] The following describes the operation of an exemplary artificial neural network model 110 a that can be operated in the neural network processing unit 100 .
[0075] Figure 1 The exemplary artificial neural network model 110a may be an artificial neural network that is learned in a manner capable of performing a variety of reasoning functions such as object recognition, speech recognition, etc.
[0076] The artificial neural network model 110 a may be a deep neural network (DNN: Deep Neural Network).
[0077] However, the artificial neural network model 110a according to the example of the present disclosure is not limited to a deep neural network.
[0078] For example, the artificial neural network model 110a can be implemented as a model of Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3, etc. However, the present disclosure is not limited to the above models. In addition, the artificial neural network model 110a can also be an integrated model based on at least two different models.
[0079] The following describes the reasoning process performed by the exemplary artificial neural network model 110a.
[0080] The artificial neural network model 110a is an exemplary deep neural network model, which includes an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5 and a third connection network 110a-6, and an output layer 110a-7. However, the present disclosure is not limited to Figure 1 The artificial neural network model shown in FIG. 1 . The first hidden layer 110 a - 3 and the second hidden layer 110 a - 5 may also be referred to as a plurality of hidden layers.
[0081] The input layer 110a-1 may illustratively include x1 and x2 input nodes. That is, the input layer 110a-1 may include information about two input values.
[0082] The first connection network 110a-2 may exemplarily include information about six weight values for connecting each node of the input layer 110a-1 to each node of the first hidden layer 110a-3. Each weight value is multiplied by the input node value, and the accumulated value of the multiplied value is stored in the first hidden layer 110a-3. The weight value and the input node value may be referred to as parameters of the artificial neural network model.
[0083] The first hidden layer 110a-3 may exemplarily include an a1 node, an a2 node, and an a3 node. That is, the first hidden layer 110a-3 may include information about three node values.
[0084] Figure 3 The first processing element PE1 can process the operation of the a1 node.
[0085] Figure 3 The second processing element PE2 can process the operation of the a2 node.
[0086] Figure 3 The third processing element PE3 of the first hidden layer 110a-3 can process the operation of the a3 node. The second connection network 110a-4 can exemplarily include information about nine weight values for connecting each node of the first hidden layer 110a-3 to each node of the second hidden layer 110a-5. The weight values of the second connection network 110a-4 are multiplied by the node values input from the first hidden layer 110a-3, respectively, and the accumulated values of the multiplied values are stored in the second hidden layer 110a-5.
[0087] The second hidden layer 110a-5 may exemplarily include b1, b2, and b3 nodes. That is, the second hidden layer 110a-5 may include information about three node values.
[0088] Figure 3 The fourth processing element PE4 can process the operation of the b1 node.
[0089] Figure 3 The fifth processing element PE5 can process the operation of the b2 node.
[0090] Figure 3 The sixth processing element PE6 can process the operation of the b3 node.
[0091] The third connection network 110a-6 may exemplarily include information about six weight values connecting each node of the second hidden layer 110a-5 and each node of the output layer 110a-7. The weight values of the third network 110a-6 are multiplied by the node values input from the second hidden layer 110a-5, respectively, and the accumulated values of the multiplied values are stored in the output layer 110a-7.
[0092] The output layer 110a-7 may illustratively include a y1 node and a y2 node. That is, the output layer 110a-7 may include information about two node values.
[0093] Figure 3 The seventh processing element PE7 can process the operation of the y1 node.
[0094] Figure 3 The eighth processing element PE8 can process the operation of the y2 node.
[0095] Figure 2a This is a diagram used to illustrate the basic structure of a convolutional neural network (CNN).
[0096] Reference Figure 2a , the input image can be represented as a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image can have multiple channels, where the channel can represent the number of color components of the input data image.
[0097] The convolution process means traversing the input image at specified intervals and performing convolution operations with the kernel.
[0098] A convolutional neural network may have a structure in which an output value (convolution or matrix multiplication) of a current layer is passed to an input value of a next layer.
[0099] For example, convolution is defined by two main parameters: input feature map and kernel. Parameters may include input feature map, output feature map, activation map, weights, kernel, and attention (Q, K, V).
[0100] Convolution slides the kernel window over the input feature map. The step difference between the kernel sliding the input feature map is called the stride.
[0101] Pooling can be used after convolution. In addition, a fully-connected (FC) layer can be arranged at the end of the convolutional neural network.
[0102] Figure 2b is a comprehensive diagram showing the operation of a convolutional neural network for easy understanding.
[0103] refer to Figure 2b , exemplarily showing the case where the input image is a two-dimensional matrix with a size of 6×6. Figure 2b The use of three nodes, namely channel 1, channel 2, and channel 3, is exemplarily shown in FIG.
[0104] First, the convolution operation is explained.
[0105] Input image (in Figure 2b) in the first node with kernel 1 for channel 1 (in Figure 2b The convolution is performed on the 3×3 size (shown as an example in the figure), and the result is the output feature Figure 1 (exist Figure 2b In addition, the input image (in Figure 2b ) in the second node with kernel 2 for channel 2 (in Figure 2b The convolution is performed on the output feature map 2 (in Figure 2b In the example, the size is 4×4. And, the input image is combined with kernel 3 for channel 3 at the third node (in Figure 2b The convolution is performed on the 3×3 size (shown as an example in the figure), and the result is the output feature Figure 3 (exist Figure 2b exemplarily shown as a size of 4×4).
[0106] To process each convolution, the processing units PE1 to PE12 of the NPU 100 are configured to perform a MAC operation.
[0107] Next, the operation of the activation function is explained.
[0108] For the features output from the convolution operation Figure 1 , feature map 2 and features Figure 3 (exist Figure 2b In the example, each size is exemplarily represented as 4×4) and an activation function can be applied. The output after applying the activation function can be exemplarily of a size of 4×4.
[0109] Next, the pooling operation is described.
[0110] The features output from the activation function Figure 1 ,Feature map 2, features Figure 3 (exist Figure 2b , the size of each is exemplarily represented as 4×4) is input to 3 nodes. The feature map output from the activation function can be received as input to perform pooling. The pooling can be performed by reducing the size or emphasizing specific values in the matrix. The pooling methods include maximum pooling, average pooling, and minimum pooling. Maximum pooling can be used to collect the maximum value of the values in a specific area of the matrix, and average pooling can be used to average the values in a specific area.
[0111] exist Figure 2b The example in FIG. 4 shows that a 4×4 feature map is reduced to a 2×2 size by pooling.
[0112] Specifically, the first node receives the characteristics for channel 1 Figure 1 The first node receives feature map 2 for channel 2 as input and performs pooling, then outputs, for example, a 2×2 matrix. The second node receives feature map 2 for channel 2 as input and performs pooling, then outputs, for example, a 2×2 matrix. The third node receives feature map 3 for channel 3 as input and performs pooling, then outputs, for example, a 2×2 matrix. Figure 3 Takes as input and performs pooling, then outputs, for example, a 2×2 matrix.
[0113] Repeat the above convolution, activation function and pooling, and finally, Figure 8a As shown, the output can be fully connected. The output can be input again to the artificial neural network for image recognition. However, the present disclosure is not limited to the size of the feature map and the kernel.
[0114] The CNN described so far is the most commonly used method in the field of computer vision among various deep neural network (DNN) methods. In particular, CNN has shown unexpected performance in various research fields such as image classification and object detection.
[0115]
Hardware resources required for ANN operations
[0116] Figure 3 is a schematic conceptual diagram illustrating a neural processing unit of the present disclosure.
[0117] Figure 3 The neural processing unit (NPU) 100 shown is a processor dedicated to performing operations for artificial neural networks.
[0118] An artificial neural network is a network of artificial neurons that multiply and add their respective weights when multiple inputs or stimuli are input, and transform and transmit the value of the additional bias through the activation function. The artificial neural network learned in this way can be used to output inference results from the input data.
[0119] The NPU 100 may be a semiconductor implemented using an electric / electronic circuit. The electric / electronic circuit may include many electronic components (eg, transistors, capacitors).
[0120] In the case of an artificial neural network model based on Transformer and / or CNN, the NPU 100 may filter and process matrix multiplication operations, convolution operations, etc. according to the architecture of the artificial neural network.
[0121] For example, in each layer of a convolutional neural network (CNN), the input feature map corresponding to the input data and the kernel corresponding to the weight value can be a matrix consisting of multiple channels. The convolution operation of the input feature map and the kernel is performed to generate a convolution operation and a pooled output feature map in each channel. The activation map of the channel is generated by applying the activation function to the output feature map. Then, pooling can be applied to the activation map. Among them, the activation map can be called the output feature map.
[0122] However, examples of the present disclosure are not limited thereto, and outputting a feature map means that a matrix multiplication operation or a convolution operation is applied.
[0123] Specifically, the output feature map according to the example of the present disclosure should be interpreted as a general meaning. For example, the output feature map can be the result value of a matrix multiplication operation or a convolution operation. Therefore, the plurality of processing elements 110 can be deformed and implemented to further include a processing circuit unit for additional algorithms.
[0124] The NPU 100 may be configured to include a plurality of processing elements 110 for processing convolution and matrix multiplication required for the above-mentioned artificial neural network operations.
[0125] NPU 100 can be constructed to include each processing circuit that optimizes the matrix multiplication operation, convolution operation, activation function operation, pooling operation, stride operation, batch normalization operation, skip connection operation, concatenation operation, quantization operation, pruning operation, and padding operation required for the above-mentioned artificial neural network operation.
[0126] For example, the NPU 100 may be configured to include an SFU 150 for processing at least one of an activation function operation, a pooling operation, a stride operation, a batch normalization operation, a skip connection operation, a concatenation operation, a quantization operation, a pruning operation, and a padding operation in the above algorithm.
[0127] Specifically, the NPU 100 may include a plurality of processing elements (PEs) 110, an SFU 150, an NPU internal memory 120, an NPU controller 130, and an NPU interface 140. Each of the plurality of processing elements 110, the SFU 150, the NPU internal memory 120, the NPU controller 130, and the NPU interface 140 may be a semiconductor circuit connected with a plurality of transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye, and can only be identified through operation.
[0128] For example, any circuit may operate as a plurality of processing elements 110, or may also operate as an NPU controller 130. The NPU controller 130 may be configured to perform the functions of a control unit configured to control the artificial neural network inference operation of the NPU 100.
[0129] The NPU 100 may include an NPU internal memory 120 and an NPU controller 130, wherein the NPU internal memory 120 is configured to store parameters of an artificial neural network model that can be inferred from multiple processing elements 110 and SFU 150, and the NPU controller 130 includes a scheduler configured to control multiple processing elements 110, SFU 150, and the NPU internal memory 120.
[0130] The NPU 100 may be configured to process feature maps corresponding to encoding and decoding schemes using SVC or SFC.
[0131] The plurality of processing elements 110 may perform a portion of the operations for an artificial neural network.
[0132] The SFU 150 may perform another portion of the operations for the artificial neural network.
[0133] The NPU 100 may be configured to accelerate the operation of an artificial neural network model in hardware using a plurality of processing elements 110 and SFUs 150 .
[0134] The NPU interface 140 can communicate with various components (eg, memory) connected to the NPU 100 through a system bus.
[0135] The NPU controller 130 may include a scheduler configured to schedule operations of the plurality of processing elements 110 for inference operations of the neural processing unit 100 , operations of the SFU 150 , and read and write sequences of the NPU internal memory 120 .
[0136] The scheduler within the NPU controller 130 may be configured to control the plurality of processing elements 110 , the SFU 150 , and the NPU internal memory 120 based on the data locality information or the information on the structure of the artificial neural network model.
[0137] The scheduler in the NPU controller 130 can analyze the structure of the artificial neural network model to be operated in the plurality of processing elements 110 and the SFU 150 or receive information that has been analyzed. For example, the data of the artificial neural network that the artificial neural network model can include can include at least a portion of the node data (i.e., feature map) of each layer, the arrangement data of the layer, the information about the regional information or structure, and the weight data (i.e., weight kernel) of each connection network connecting the nodes of each layer. The data of the artificial neural network can be stored in a memory provided to the inside of the NPU controller 130 or in the NPU internal memory 120.
[0138] The scheduler within the NPU controller 130 may schedule the order of operations of the artificial neural network model to be executed by the NPU 100 based on the data locality information or structure information about the artificial neural network model.
[0139] The scheduler in the NPU controller 130 can obtain the memory address value storing the feature map and weight data of the layer of the artificial neural network model based on the data regionality information or structure information of the artificial neural network model. For example, the scheduler in the NPU controller 130 can obtain the feature map of the layer of the artificial neural network model stored in the memory and the memory address value storing the weight data. Therefore, the scheduler in the NPU controller 130 can obtain the feature map and weight data of the layer of the artificial neural network model to be driven from the main memory and store them in the NPU internal memory 120.
[0140] The feature map of each layer may have a corresponding memory address value.
[0141] Each weight data may have a corresponding memory address value.
[0142] The scheduler within the NPU controller 130 can schedule the order of operations of multiple processing elements 110 based on data locality information or structure information about the artificial neural network model (for example, layout data, locality information or structure information about the layers of the artificial neural network of the artificial neural network model).
[0143] The scheduler in the NPU controller 130 performs scheduling based on the data locality information or structure information about the artificial neural network model, so its operation may be different from the scheduling concept of a conventional CPU. Conventional CPU scheduling operates in a manner that achieves optimal efficiency by taking into account fairness, efficiency, stability, response time, etc. That is, scheduling is performed in a manner that performs the most processing at the same time by taking into account priority, operation time, etc.
[0144] Existing CPUs use an algorithm for scheduling tasks, taking into account data such as the priority order of each process, the operation processing time, etc.
[0145] Different from this, the scheduler in the NPU controller 130 can control the NPU 100 according to the determined processing order of the NPU 100 based on the data locality information or structure information about the artificial neural network model.
[0146] In addition, the scheduler within the NPU controller 130 can drive the NPU 100 in a determined processing order based on information about the data locality information or structure of the artificial neural network model and / or information about the data locality information or structure of the neural processing unit 100 to be used.
[0147] However, the present disclosure is not limited to the data locality information or the information of the structure about the NPU 100 .
[0148] The scheduler within the NPU controller 130 may be configured to store data locality information or information about the structure of the artificial neural network.
[0149] That is, even using at least only the data locality information or the information on the structure of the artificial neural network of the artificial neural network model, the scheduler within the NPU controller 130 can determine the processing order.
[0150] Further, the scheduler within the NPU controller 130 may determine the processing order of the NPU 100 by considering the data locality information or structure information about the artificial neural network model and the data locality information or structure information about the NPU 100. In addition, the processing of the NPU 100 may also be optimized according to the determined processing order.
[0151] The plurality of processing elements 110 refers to a structure in which a plurality of processing elements PE1 to PE12 configured to operate feature maps and weight data of an artificial neural network are arranged. Each processing element may include a multiplication and accumulation (MAC: multiply and accumulate) operator and / or an arithmetic logic unit (ALU: Arithmetic Logic Unit) operator. However, the examples according to the present disclosure are not limited thereto.
[0152] Each processing element may be configured to selectively further include additional special function units to process additional special functions.
[0153] For example, the processing element PE may also be modified to further include a batch normalization unit, an activation function unit, an interpolation unit, etc.
[0154] The SFU 150 may include a circuit portion configured to select and process activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, padding operations, etc. according to the architecture of the artificial neural network. That is, the SFU 150 may include a plurality of special function operation processing circuit units.
[0155] Although in Figure 5 Although multiple processing units are shown as an example in FIG, operators implemented by multiple multipliers and adder trees may be arranged in parallel by replacing MAC within a processing unit. In this case, the multiple processing elements 110 may be referred to as at least one processing element including multiple operators.
[0156] The plurality of processing elements 110 is configured to include a plurality of processing elements PE1 to PE12 . Figure 5 The multiple processing elements PE1 to PE12 shown in the figure are only examples for convenience of explanation, and the number of the multiple processing elements PE1 to PE12 is not limited. The size or number of the multiple processing elements 110 can be determined by the number of the multiple processing elements PE1 to PE12. The size of the multiple processing elements 110 can be implemented in the form of an N×M matrix. Wherein N and M are integers greater than 0. The multiple processing elements 110 may include N×M processing elements. That is, there may be more than one processing element.
[0157] The sizes of the plurality of processing elements 110 may be designed by considering the characteristics of the artificial neural network model that the NPU 100 operates.
[0158] The plurality of processing elements 110 are configured to perform functions such as addition, multiplication, accumulation, etc. required for artificial neural network operations. In other words, the plurality of processing elements 110 may be configured to perform multiplication and accumulation (MAC) operations.
[0159] The following description will be made by taking the first processing element PE1 among the plurality of processing elements 110 as an example.
[0160] Figure 4a is a schematic conceptual diagram illustrating a processing element that is one of a plurality of processing elements that may be applied to the present disclosure.
[0161] According to an example of the present disclosure, the NPU 100 includes: a plurality of processing elements 110; an NPU internal memory 120 configured to store an artificial neural network model that can be inferred from the plurality of processing elements 110; and an NPU controller 130 configured to control the plurality of processing elements 110 and the NPU internal memory 120, wherein the plurality of processing elements 110 may be configured to perform MAC operations, and the plurality of processing elements 110 may be configured to quantize and output MAC operation results. However, the examples of the present disclosure are not limited thereto.
[0162] The NPU internal memory 120 can store all or part of the artificial neural network model according to the memory size and the data size of the artificial neural network model.
[0163] The first processing element PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantizer 114. However, the examples according to the present disclosure are not limited thereto, and the plurality of processing elements 110 may also be implemented in a modified manner by considering the operation characteristics of an artificial neural network.
[0164] The multiplier 111 multiplies the input (N)-bit data and (M)-bit data. The operation value of the multiplier 111 is output as (N+M)-bit data.
[0165] The multiplier 111 may be configured to receive a variable and a constant.
[0166] The accumulator 113 accumulates the operation value of the multiplier 111 and the operation value of the accumulator 113 in L loops by using the adder 112. Therefore, the bit width of the data of the output part and the input part of the accumulator 113 can be output as (N+M+log2(L)) bits, where L is an integer greater than 0.
[0167] If the accumulation is completed, the accumulator 113 may receive an initialization reset to initialize the data stored inside the accumulator 113 to 0. However, examples according to the present disclosure are not limited thereto.
[0168] The bit quantization unit 114 can reduce the bit width of the data output from the accumulator 113. The bit quantization unit 114 can be controlled by the NPU scheduler 130. The bit width of the quantized data can be output as (X) bits, where X is an integer greater than 0. According to the above structure, the multiple processing elements 110 are configured to perform MAC operations, and the multiple processing elements 110 have the effect of quantizing and outputting the MAC operation results. In particular, as the number of L cycles increases, this quantization has the effect of further reducing power consumption. In addition, if power consumption is reduced, it also has the effect of reducing heat generation. In particular, if heat generation is reduced, it has the effect of reducing the possibility of failure caused by the high temperature of NPU100.
[0169] The output data (X) bits of the bit quantization unit 114 can be the node data of the next layer or the input data of the convolution. If the artificial neural network model has been quantized, the bit quantization unit 114 can be configured to receive quantization information from the artificial neural network model. However, not limited to this, the NPU controller 130 can be configured to extract quantized information by analyzing the artificial neural network model. Therefore, the output data (X) bits can be converted into a quantized bit width and output to correspond to the quantized data size. The output data (X) bits of the bit quantization unit 114 can be stored in the NPU internal memory 120 with a quantized bit width.
[0170] The plurality of processing elements 110 of the NPU 100 according to an example of the present disclosure includes a multiplier 111, an adder 112, and an accumulator 113. The bit quantization unit 114 may perform a trade-off depending on whether to apply quantization.
[0171] Figure 4b is a schematic conceptual diagram illustrating an SFU that can be applied to the present disclosure.
[0172] refer to Figure 4b , SFU 150 includes a plurality of functional units. Each functional unit can be selectively operated. Each functional unit can be selectively turned on or off. That is, each functional unit can be set.
[0173] In other words, the SFU 150 may include various circuit units required in the artificial neural network reasoning operation.
[0174] For example, the circuit units of SFU 150 may include functional units for skip-connection operations, functional units for activation function operations, functional units for pooling operations, functional units for quantization operations, functional units for non-maximum suppression (NMS) operations, functional units for integer and floating-point conversion (INT to FP32) operations, functional units for batch-normalization operations, functional units for interpolation operations, functional units for concatenation operations, and functional units for bias operations, etc.
[0175] The functional units of the SFU 150 can be selectively turned on or off by the data locality information of the artificial neural network model. When performing an operation for a specific layer, the data locality information of the artificial neural network model may include control information related to turning on or off the functional unit.
[0176] The activated units among the functional units of the SFU 150 may be turned on. In the case where a portion of the functional units of the SFU 150 is selectively disconnected in this manner, the power consumption of the NPU 100 may be reduced. In addition, in order to disconnect a portion of the functional units, power gating may be utilized. In addition, in order to disconnect a portion of the functional units, clock gating may also be performed.
[0177] Figure 5 It is shown Figure 3 An example diagram of a modified example of the NPU 100 is shown.
[0178] Figure 5 The NPU 100 is shown with Figure 3 Compared with the processing unit 100 exemplarily shown in FIG. 1 , except for the multiple processing elements 110, they are substantially the same, and therefore, repeated descriptions may be omitted below only for the convenience of description.
[0179] In addition to a plurality of processing elements PE1 to PE12, Figure 5 The plurality of processing elements 110 exemplarily shown in FIG. 1 may also include register files RF1 to RF12 corresponding to each of the processing elements PE1 to PE12 .
[0180] Figure 5The plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 shown in FIG. 1 are merely examples for convenience of explanation, and the number of the plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 is not limited.
[0181] The size or number of processing elements 110 may be determined by the number of processing elements PE1 to PE12 and register files RF1 to RF12. The size of processing elements 110 and register files RF1 to RF12 may be implemented in the form of an N×M matrix, where N and M are integers greater than 0.
[0182] The array size of the plurality of processing elements 110 can be designed by considering the characteristics of the artificial neural network model operated by the NPU 100. In other words, the memory size of the register file can be determined by considering the data size of the artificial neural network model to be operated, the required operation speed, the required power consumption, etc.
[0183] The register files RF1 to RF12 of the NPU 100 are static memory units directly connected to the processing elements PE1 to PE12. The register files RF1 to RF12 may include, for example, triggers and / or latches, etc. The register files RF1 to RF12 may be configured to store MAC operation values of the corresponding processing elements PE1 to PE12. The register files RF1 to RF12 may be configured to provide weight data and / or node data to the NPU internal memory 120 or receive weight data and / or node data.
[0184] The register files RF1 to RF12 may also be configured to perform the function of a temporary memory of an accumulator during a MAC operation.
[0185] [Technical difficulties discovered by the inventor of this patent]
[0186] There may be various types of artificial neural network (ANN) models depending on the purpose of the artificial intelligence service. For example, when the input data is an image or video, a CNN type artificial neural network model for object classification, object detection, object tracking, etc. in images / videos may be used for artificial intelligence services.
[0187] Generally, the artificial neural network model may have different computational loads at each layer, which will be described with reference to FIG6 .
[0188] Figure 6a is an example diagram showing the data size of each layer in an exemplary artificial neural network model, Figure 6b It is shown Figure 6a An exemplary table showing the data size, etc. of each layer in the exemplary artificial neural network model shown.
[0189] Figure 6a The exemplary artificial neural network model shown in is denoted as Mobilenet V1. Figure 6a The horizontal axis shown in represents the layers in the exemplary artificial neural network model in sequence, and the vertical axis represents the size of the data.
[0190] Reference Figure 6a As can be seen from the layer 1 shown in , the size of the output feature map OFMAP_SIZE is larger than the size of the input feature map IFMAP_SIZE.
[0191] The output feature map of layer 1 is passed to layer 2, thereby becoming the input feature map of layer 2. If the operation of layer 2 is completed, the output feature map is output. The output feature map of layer 2 is passed to layer 3 again, thereby becoming the input feature map of layer 3.
[0192] In this way, the size of the input data to each layer can be different from the size of the output feature map output from each layer. As a result, the amount of calculation may be small in any layer, but it may be very large in other layers. In this way, as the amount of calculation of each layer changes greatly, the problem of power becoming difficult to control will occur.
[0193] In particular, the inventors of the present patent have recognized that if the amount of computation in a specific layer increases significantly, instantaneous power consumption will increase, which may cause a problem of reduced system stability.
[0194] For example, multiple processing elements can be started at the same time for operations at a specific layer. In order to drive each processing element, a specified power supply is required. If a large number of processing elements are driven at the same time, the required power will surge instantly. In the case where the neural processing unit is designed specifically for low-power operation, its power supply capacity may be relatively insufficient for a neural processing unit for a server. Therefore, such a neural processing unit for edge devices may be relatively easily affected by instantaneous power supply problems, and the supply voltage may fluctuate in the event of a surge in power supply. In particular, when the supply voltage drops below the threshold voltage of the transistor, the data stored in the transistor may be lost. In other words, when the supply voltage is reduced, the operating speed of the transistor will decrease, resulting in setup / hold violation problems, thereby causing failures.
[0195] As another example, the inventors of the present invention have recognized that the PE 110, which performs operations for artificial neural networks, such as addition, multiplication, and accumulation, instantly uses a large amount of power, so that other components in the NPU (e.g., the internal memory 120) may not be able to supply sufficient power. Specifically, the inventors of the present invention have recognized that if sufficient power cannot be supplied to the internal memory 120, there may be a problem in which the possibility of the stored data bits being damaged (compromised) cannot be ruled out.
[0196] [Disclosure of this patent]
[0197] The inventors of this patent realized that the above problem occurs because the operation of the artificial neural network model is performed on hardware based on a clock signal.
[0198] Therefore, the inventors of this patent invented a technology for solving the above-mentioned problem, which divides the clock signal into multiple parts and then makes the operation of the artificial neural network model be distributed and executed based on the multiple parts of the clock signal.
[0199] Hereinafter, multiple embodiments for realizing the technology invented by the inventor of this patent will be described with reference to the accompanying drawings.
[0200] Figure 7a : is an exemplary diagram showing the structure of an NPU according to an example of the first disclosure of this specification.
[0201] Although Figure 7a FIG. 1 shows that the NPU 100 includes a first group of PEs 110a, a second group of PEs 110b, an internal memory 120, an SFU 150, and a clock generator 190. Figure 3 or Figure 5 As shown, the NPU 100 may also include an NPU controller 130 and an NPU interface 140. The PE 110a of the first group, the PE 110b of the second group, the internal memory 120, the NPU controller 130, the NPU interface 140, the SFU 150 and the clock generator 190 may be semiconductor circuits connected with a plurality of transistors. Therefore, some of them may be difficult to identify and distinguish with the naked eye, and can only be identified through operation. The clock generator 190 may be called a clock signal supply circuit.
[0202] For example, any circuit may operate as a plurality of processing elements 110 , or may also operate as an NPU controller 130 .
[0203] Figure 7aThe first group of PE 110a and the second group of PE 110b shown can be understood as Figure 3 or Figure 5 The plurality of PEs 110 are shown as being partitioned. Figure 7a , it is shown that the number of PEs belonging to each group is 8, but it is obvious that the number of PEs belonging to each group can be changed. In addition, the number of PEs belonging to each group can be changed semi-statically or dynamically. For example, the PEs of the first group can be changed to include 10 PEs, and the PEs of the second group can be changed to include 6 PEs. This change can be performed according to the control of the NPU controller 130.
[0204] Figure 7a The clock generator 190 shown may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM). The PWM may adjust the ratio between the high signal and the low signal by changing the duty cycle of the clock cycle.
[0205] If the clock generator 190 generates and outputs a signal, the signal is transmitted to the PE 110 a of the first group and the PE 100 b of the second group.
[0206] At this time, the PE 100a of the first group may be set to operate based on the first part (eg, rising edge) of the clock, and the PE 100b of the second group may be set to operate based on the second part (eg, falling edge) of the clock.
[0207] like Figure 7a As shown, Figure 3 or Figure 5 The plurality of PEs 110 shown are divided into a first group of PEs 110a and a second group of PEs 110b. If the first group of PEs 110a and the second group of PEs 110b perform decentralized operations based on different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0208] Figure 7b is an exemplary diagram showing a structure of an NPU according to another example of the first disclosure of this specification.
[0209] Although Figure 7b FIG. 1 shows that the NPU 100 includes a first group of PEs 110 a, a second group of PEs 110 b, an NPU internal memory 120, and an SFU 150. In addition, Figure 3 or Figure 5As shown, the NPU 100 may further include an NPU controller 130 and an NPU interface 140. The first group of PEs 110a, the second group of PEs 110b, the NPU internal memory 120, the NPU controller 130, the NPU interface 140, and the SFU 150 may be semiconductor circuits connected with a plurality of transistors, respectively. Therefore, some of them may be difficult to identify and distinguish with the naked eye, and can only be identified through operation.
[0210] refer to Figure 7b , the clock generator 190 is shown as being located outside the NPU 100 rather than inside. Figure 7a The contents are the same as those described in Figure 7b No further explanation is given and reference is made directly Figure 7a The content of the description.
[0211] In addition, if Figure 7a and Figure 7b As shown in the figure, the PE 110a of the first group and the PE 100b of the second group complete the operation of any layer of the artificial neural network model, then the PE 110a of the first group and the PE 100b of the second group can match the operation result (i.e., feature map) to any one of the first part and the second part of the clock to output. To this end, at least one of the PE 110a of the first group and the PE 100b of the second group can further include a time delay buffer (e.g., a shift register). The time delay buffer (e.g., a shift register) can perform time delay so that the output can be performed by matching the later part of the first part and the second part of the clock.
[0212] For example, the output ports of the second group of PE 110b may be connected to a time delay buffer (e.g., a shift register). In this case, the delay buffer may perform a conversion from a first portion of the clock (e.g., a rising edge) to a second portion of the clock (e.g., a falling edge), or perform a conversion from a second portion of the clock (e.g., a falling edge) to a first portion of the clock (e.g., a rising edge).
[0213] Alternatively, Figure 7a and Figure 7bThe PE 110a of the first group shown in the figure can match the first output feature map to the first part of the clock and output it, and the PE 110b of the second group can match the second output feature map to the second part of the clock and output it. In this case, the SFU 150 may include a time delay buffer (e.g., a shift register). The SFU 150 may time delay the received output feature map by matching the faster part of the first and second parts of the clock, so that the output feature map to be received can be matched with the later part of the first and second parts of the clock to synchronize with the time.
[0214] Alternatively, in the case where the first output feature map output by the first group of PEs 110a and the second output feature map output by the second group of PEs 110b are independent of each other and not dependent on each other, in the first and second parts of the clock, the SFU 150 may pre-process the received output feature map in matching with the faster part in time.
[0215] That is, the first output from the first group of PEs 110a and the second output from the second group of PEs 110b may be delivered based on the first portion of the clock.
[0216] The reference phase for the second group of PEs 110b may be shifted from the second portion of the clock to the first portion.
[0217] The reference phase of the first group of PE 110a and the second group of PE 110b may be switched from the first part (i.e., rising edge) of the clock to the second part (i.e., falling edge) of the clock, or may be switched from the second part (i.e., falling edge) of the clock to the first part (i.e., rising edge) of the clock.
[0218] In addition, although Figure 7a and Figure 7b Although not shown, the SFU 150 may be directly connected to the NPU internal memory 120 .
[0219] In addition, although Figure 7a and Figure 7b The NPU is described in the figure, but the NPU can also be implemented in the form of a SoC.
[0220] Figure 8a is an exemplary diagram showing the structure of a SoC according to an example of the second disclosure of this specification.
[0221] Reference Figure 8a, the exemplary SoC 1000 may include multiple NPUs, multiple central processing units (CPUs), and multiple memories. The multiple NPUs may include, for example, a first NPU 100-1 and a second NPU 100-2. And, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.
[0222] exist Figure 8a Although the number of the multiple NPUs, the multiple CPUs, and the multiple memories is shown as 2 each, this is not limited to this, and the number can be changed to 4, 6, 8, etc.
[0223] The exemplary SoC 1000 may include a memory controller 350 , a clock generator 400 , a system bus 500 , and an input / output (I / O) interface 800 .
[0224] The system bus 500 may be implemented by an electrically conductive pattern formed on a semiconductor die. The system bus may implement high-speed communication. For example, the multiple NPUs, the multiple CPUs, the multiple memories, and the memory controller 350 may communicate with each other through the system bus 500.
[0225] The plurality of NPUs and the plurality of CPUs may request the memory controller 350 through the system bus 500 , and may read or record data from at least one of the plurality of memories.
[0226] Figure 8a The clock generator 400 shown may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM). The PWM may adjust the ratio of a high signal to a low signal by changing the duty cycle of a clock cycle.
[0227] If the clock generator 400 generates and outputs a signal, it is transmitted to the first NPU 100 - 1 and the second NPU 100 - 2 .
[0228] At this time, the first NPU 100-1 may be set to operate based on the first part (eg, rising edge) of the clock, and the second NPU 100-2 may be set to operate based on the second part (eg, falling edge) of the clock.
[0229] In this way, if the first NPU 100-1 and the second NPU 100-2 perform decentralized operations based on mutually different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0230] Figure 8b is an exemplary diagram showing a structure of a SoC according to another example of the second disclosure of this specification.
[0231] Reference Figure 8b , the exemplary SoC 1000 may include multiple NPUs, multiple central processing units (CPUs), and multiple memories. The multiple NPUs may include, for example, a first NPU 100-1 and a second NPU 100-2. And, the multiple CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The multiple memories may include a first memory 300-1 and a second memory 300-2.
[0232] exist Figure 8b , the number of each of the plurality of NPUs, the plurality of CPUs, and the plurality of memories is shown as 2, but the present invention is not limited thereto, and the number may be changed to 4, 6, 8, or the like.
[0233] The exemplary SoC 1000 may include a memory controller 350 , a clock generator 400 , a plurality of buses, and an input / output (I / O) interface 800 .
[0234] The plurality of buses may include a bus for a CPU, ie, a CPU bus 500 - 1 , a bus for an NPU, ie, an NPU bus 500 - 2 , and a bus for peripheral components, ie, a peripheral bus (PEipheral Bus) 500 - 3 .
[0235] The first CPU 200-1, the second CPU 200-2 and the first memory 300-1 may be connected to the CPU bus 500-1. The first NPU 100-1, the second NPU 100-2 and the second memory 300-2 may be connected to the NPU bus 500-2. The memory controller 350 and the clock generator 400 may be connected to the peripheral bus 500-3.
[0236] The clock generator 400 may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM).
[0237] If the clock generator 400 generates and outputs a signal, it is transmitted to the first NPU 100 - 1 and the second NPU 100 - 2 .
[0238] At this time, the first NPU 100-1 may be set to operate based on the first part (eg, rising edge) of the clock, and the second NPU 100-2 may be set to operate based on the second part (eg, falling edge) of the clock.
[0239] If the first NPU 100-1 and the second NPU 100-2 perform decentralized operations based on mutually different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0240] Although the above description is mainly based on SoC, the disclosure of this specification is not limited to SoC, and the disclosure of this specification can also be applied to board-level systems based on system-in-package (SiP) or printed circuit boards (PCB). For example, each functional component can be implemented by an independent semiconductor chip, and can be implemented in a form of interconnection through a system bus implemented by an electrically conductive pattern formed on a PCB. Specifically, it will be described with reference to FIG. 9.
[0241] Figure 9a : is an exemplary diagram showing the structure of a system according to an example of the third disclosure of this specification.
[0242] Reference Figure 9a , the exemplary system 2000 may include a plurality of NPUs and a plurality of central processing units (CPUs), and a plurality of memories mounted on a substrate 900. The plurality of NPUs may include, for example, a first NPU 100-1 and a second NPU 100-2. And, the plurality of CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.
[0243] exist Figure 9a In the embodiment, the number of the multiple NPUs, the multiple CPUs, and the multiple memories is 2 each, but is not limited thereto, and the number may be changed to 4, 6, 8, etc.
[0244] The exemplary system 2000 may include a memory controller 350 , a clock generator 400 , a system bus 500 , and an input / output (I / O) interface 800 .
[0245] The system bus 500 may be implemented by an electrically conductive pattern formed on the substrate 900. The system bus may implement high-speed communication. For example, the plurality of NPUs, the plurality of CPUs, the plurality of memories, and the memory controller 350 may communicate with each other through the system bus 500.
[0246] Figure 9a The clock generator 400 shown may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM). The PWM may adjust the ratio of a high signal to a low signal by changing the duty cycle of a clock cycle.
[0247] If the clock generator 400 generates and outputs a signal, it is transmitted to the first NPU 100 - 1 and the second NPU 100 - 2 .
[0248] At this time, the first NPU 100-1 may be set to operate based on the first part (eg, rising edge) of the clock, and the second NPU 100-2 may be set to operate based on the second part (eg, falling edge) of the clock.
[0249] In this way, if the first NPU 100-1 and the second NPU 100-2 perform decentralized operations based on mutually different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0250] Figure 9b : is an exemplary diagram showing a system configuration according to an example of the third disclosure of this specification.
[0251] Reference Figure 9b , the exemplary system 2000 may include a plurality of NPUs, a plurality of central processing units (CPUs), and a plurality of memories mounted on a substrate 900. The plurality of NPUs may include, for example, a first NPU 100-1 and a second NPU 100-2. And, the plurality of CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. The plurality of memories may include a first memory 300-1 and a second memory 300-2.
[0252] exist Figure 9b 2 shows a case where the number of the plurality of NPUs, the plurality of CPUs, and the plurality of memories is 2 each, but the present invention is not limited thereto and the number may be changed to various numbers such as 4, 6, 8, etc.
[0253] The exemplary system 2000 may include a memory controller 350 , a clock generator 400 , a plurality of buses, and an input / output (I / O) interface 800 .
[0254] The plurality of bus bars may be implemented by electrically conductive patterns formed on the substrate 900 .
[0255] The plurality of buses may include a bus for a CPU (ie, a CPU bus 500 - 1 ), a bus for an NPU (ie, an NPU bus 500 - 2 ), and a bus for peripheral components (ie, a peripheral bus 500 - 3 ).
[0256] The first CPU 200-1, the second CPU 200-2 and the first memory 300-1 may be connected to the CPU bus 500-1. The first NPU 100-1, the second NPU 100-2 and the second memory 300-2 may be connected to the NPU bus 500-2. The memory controller 350 and the clock generator 400 may be connected to the peripheral bus 500-3.
[0257] The clock generator 400 may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM).
[0258] If the clock generator 400 generates and outputs a signal, it is transmitted to the first NPU 100 - 1 and the second NPU 100 - 2 .
[0259] At this time, the first NPU 100 - 1 may be set to operate based on a first portion (eg, a rising edge) of the clock, and the second NPU 100 - 2 may be set to operate based on a second portion (eg, a falling edge) of the clock.
[0260] If the first NPU 100 - 1 and the second NPU 100 - 2 perform decentralized operations based on mutually different parts of the clock, there is an advantage that instantaneous power consumption can be reduced.
[0261] Fig.10 is an exemplary diagram illustrating an operation method of an NPU according to the disclosure of this specification.
[0262] refer to Fig.10 , a driving signal for driving a plurality of PEs may be generated ( S1010 ).
[0263] Furthermore, the plurality of PEs may be divided into a first group of PEs and a second group of PEs (S1020). It is apparent that the number of PEs belonging to each group may be changed. In addition, the number of PEs belonging to each group may be changed semi-statically or dynamically. Such a change may be performed according to the control of the NPU controller 130.
[0264] Next, the first group of PEs may be driven based on the first portion of the clock signal (S1030). In addition, the second group of PEs may be driven based on the second portion of the clock signal (S1040).
[0265] Fig.11 is an exemplary diagram illustrating an operation method of an NPU according to the disclosure of this specification.
[0266] Reference Fig.11 , a driving signal for driving a plurality of NPUs may be generated ( S1110 ).
[0267] Furthermore, the plurality of NPUs may be divided into a first group of NPUs and a second group of NPUs (S1120). It is apparent that the number of NPUs belonging to each group may be changed. In addition, the number of NPUs belonging to each group may be changed semi-statically or dynamically. Such a change may be performed according to the control of the CPU.
[0268] Next, the NPUs of the first group may be driven based on the first portion of the clock signal (S1130). In addition, the NPUs of the second group may be driven based on the second portion of the clock signal (S1140).
[0269] A disclosure of the present specification provides a system. The system may include: at least one memory; a neural processing unit (NPU), including a plurality of processing elements (PEs) capable of performing operations for at least one of an artificial neural network (ANN) model. The plurality of PEs may include an adder, a multiplier, and an accumulator. The plurality of PEs may include a first group of PEs configured to operate in a first portion of a clock signal and a second group of PEs configured to operate in a second portion of the clock signal.
[0270] A disclosure of the present specification provides a system-on-chip (SoC). The SoC may include: a first circuit arranged for at least one memory; and a second circuit arranged for a plurality of processing elements (PEs) capable of performing operations for at least one of an artificial neural network (ANN) model, wherein the plurality of PEs may include adders, multipliers, and accumulators. The plurality of PEs may include a first group of PEs configured to operate in a first portion of a clock signal and a second group of PEs configured to operate in a second portion of the clock signal.
[0271] A disclosure of the present specification provides a method for driving a neural processing unit (NPU). The driving method of the NPU may include the following steps: generating a clock signal for driving the NPU including a plurality of processing elements (PEs); causing a first group of PEs among the plurality of PEs to operate in a first part of the clock signal; and causing a second group of PEs among the plurality of PEs to operate in a second part of the clock signal. The plurality of PEs may include an adder, a multiplier, and an accumulator.
[0272] The examples of the present disclosure shown in this specification and the drawings are only specific examples proposed to easily illustrate the technical content of the present disclosure and help understand the present disclosure, and are not intended to limit the scope of the present disclosure. In addition to the examples currently described, other variations may also be implemented, which is obvious to ordinary technicians in the technical field to which the present disclosure belongs.
Claims
1. A system comprising a neural processing unit, comprising: The neural processing unit includes: at least one memory; and a plurality of processing elements capable of performing operations for at least one of the artificial neural network models, The plurality of processing units include adders, multipliers and accumulators, Wherein, the multiple processing units include: A first set of processing units configured to operate during a first portion of the clock signal; and The second group of processing units is configured to operate during a second portion of the clock signal.
2. The system according to claim 1, wherein: The first portion is a rising edge portion of the clock signal, and the second portion is a falling edge portion of the clock signal.
3. The system according to claim 1, wherein: A first output from the first group of processing elements and a second output from the second group of processing elements are communicated with reference to a first portion of the clock signal.
4. The system according to claim 1, wherein: The reference phases of the processing units of the first group and the processing units of the second group are switched from a rising edge of the clock signal to a falling edge of the clock signal, or from a falling edge of the clock signal to a rising edge of the clock signal.
5. The system according to claim 1, wherein: The reference phase for the second group of processing elements is switched from the second portion to the first portion of the clock signal.
6. The system according to claim 1, wherein: The output ports of the processing units of the second group are connected to delay buffers, The delay buffer performs a transition from a rising edge of the clock signal to a falling edge of the clock signal, or performs a transition from a falling edge of the clock signal to a rising edge of the clock signal.
7. The system according to claim 1, wherein: The output ports of the processing units of the second group are connected to delay buffers, The delay buffer performs a conversion from the second portion of the clock signal to the first portion of the clock signal or performs a conversion from the first portion of the clock signal to the second portion of the clock signal.
8. The system according to claim 1, wherein: The neural processing unit also includes: The special function unit is connected to the output ports of the first group of processing units and the second group of processing units.
9. The system according to claim 8, wherein: The special function unit is connected to the at least one memory.
10. The system according to claim 1, wherein: The processing units of the first group and the processing units of the second group are time-variable.
11. A system on chip, comprising: A first circuit arranged for at least one memory; as well as second circuitry, arranged to be capable of performing a plurality of processing elements for performing operations for at least one of the artificial neural network models, The plurality of processing elements include adders, multipliers and accumulators, Wherein, the plurality of processing elements include: A first set of processing elements configured to operate in a first portion of the clock signal; and A second group of processing units are configured to operate in a second portion of the clock signal.
12. The system on chip according to claim 11, wherein: The first portion is a rising edge portion of the clock signal, and the second portion is a falling edge portion of the clock signal.
13. The system on chip according to claim 11, wherein: A first output from the first group of processing elements and a second output from the second group of processing elements are delivered with reference to a first portion of the clock signal.
14. The system on chip according to claim 11, wherein: The reference phases of the processing units of the first group and the processing units of the second group are switched from a rising edge of the clock signal to a falling edge of the clock signal, or from a falling edge of the clock signal to a rising edge of the clock signal.
15. The system on chip according to claim 11, wherein: The reference phase for the second group of processing elements is switched from the second portion to the first portion of the clock signal.
16. The system on chip according to claim 11, wherein: The output ports of the processing units of the second group are connected to delay buffers, The delay buffer performs a transition from a rising edge of the clock signal to a falling edge of the clock signal, or performs a transition from a falling edge of the clock signal to a rising edge of the clock signal.
17. The system on chip according to claim 11, wherein: The output ports of the processing units of the second group are connected to delay buffers, The delay buffer performs a conversion from the second portion of the clock signal to the first portion of the clock signal or performs a conversion from the first portion of the clock signal to the second portion of the clock signal.
18. A method for driving a neural processing unit, comprising the following steps: generating a clock signal that drives a neural processing unit including a plurality of processing elements; causing a first group of processing units of the plurality of processing elements to operate during a first portion of the clock signal; and causing a second group of processing units of the plurality of processing elements to operate at a second portion of the clock signal, in, The plurality of processing elements include adders, multipliers, and accumulators.
19. The method for driving a neural processing unit according to claim 18, wherein: The first portion is a rising edge portion of the clock signal, and the second portion is a falling edge portion of the clock signal.
20. The method for driving a neural processing unit according to claim 18, wherein: A first output from the first group of processing elements and a second output from the second group of processing elements are delivered with reference to a first portion of the clock signal.
