System and system on chip

Abstract

The invention discloses a system and a system on chip. The SoC includes: a semiconductor substrate; the first circuit is arranged on the semiconductor substrate and is used for executing a first neural processing unit of an artificial neural network model; a second circuit provided on the semiconductor substrate, a second NPU provided for performing an operation of the ANN model; and a third circuit disposed on the semiconductor substrate and generating a control signal to selectively output at least one of the plurality of clock signals. Each of the first NPU and the second NPU includes a plurality of processing elements, and each of the plurality of PEs includes an adder, a multiplier, and an accumulator. The plurality of clock signals includes an original clock source or one or more phase-delayed clock signals that are phase-delayed relative to the original clock source. The first clock signal is applied to the first NPU among the plurality of clock signals according to the control signal. The second clock signal is applied to the second NPU in the plurality of clock signals according to the control signal.

Description

Technical Field

[0001] The present disclosure relates to a technique for reducing instantaneous power consumption of a System-On-Chip (SoC) of an artificial neural network. Background Art

[0002] Artificial intelligence (AI) is gradually developing. AI refers to human intelligence, that is, artificial imitation of intelligence that can perform recognition, classification, inference, prediction, control / decision making, etc.

[0003] In addition, neural processing units (NPUs) have recently been developed to speed up computing used for artificial intelligence (AI).

[0004] Depending on the purpose of the artificial intelligence service, there may be multiple types of artificial neural network ANN models. For example, when the input data is an image or video, a CNN type artificial neural network model can be used for object classification, object detection, and object tracking within the image / video for artificial intelligence services.

[0005] Generally speaking, the amount of computation required for each layer of an artificial neural network model may be different.

[0006] In particular, if the amount of operations at a particular layer increases significantly, the power consumption may increase instantaneously. Summary of the invention

[0007] (Technical Issues)

[0008] Artificial neural network operations are data-intensive. In particular, artificial neural network operations require parallel processing operations. In other words, artificial neural network operations have the characteristic that they cannot process huge amounts of data in parallel at high speed at the same time, which will reduce the processing speed.

[0009] In response to this, the inventors of the present disclosure have developed a neural processing unit specifically for artificial neural network operations. The inventors of the present disclosure hope to improve the parallel processing performance of the neural processing unit by increasing the number of processing elements of the neural processing unit. In addition, the inventors of the present disclosure hope to develop a neural processing unit that can operate at low power.

[0010] Meanwhile, the power supply unit of the neural processing unit installed in the edge device for low-power operation may have a relatively low power supply capability compared to the power supply unit of the graphics processing unit GPU running in a server or a personal computer PC. In addition, the capacitor capacity of the power supply unit of the neural processing unit installed in the edge device for low-power operation may not be sufficient to handle instantaneous power supply.

[0011] However, the inventors of the present disclosure recognized the fact that as the number of processing elements of a neural processing unit dedicated to low-power operation increases, the degree of instantaneous supply voltage fluctuation may increase. In detail, the required instantaneous power supply amount of the neural processing unit may be proportional to the number of processing elements in operation. In addition, the amount of computation required for each layer of the artificial neural network model varies greatly. Therefore, the number of processing elements running in parallel may vary depending on the amount of computation of the layer of the artificial neural network model. That is, when multiple processing elements are running simultaneously, the voltage of the power supply unit of the neural processing unit may fluctuate or drop instantaneously.

[0012] In addition, if the supply voltage fluctuates or drops instantaneously, the supply voltage VDD may need to be increased for system safety. Therefore, if system safety cannot be ensured, the supply voltage may be increased unnecessarily. If the supply voltage rises, the power consumption of the neural processing unit may rise rapidly.

[0013] On the contrary, as the supply voltage stabilizes, the supply voltage VDD may be reduced. Therefore, by ensuring the safety of the supply voltage of the neural processing unit, the supply voltage can be reduced, thereby reducing the power consumption of the neural processing unit.

[0014] Thus, the inventors of the present disclosure have recognized that even if the number of processing elements increases, it is necessary to improve the stability of the supply voltage of a neural processing unit dedicated to artificial neural network operations.

[0015] Therefore, the disclosure of this specification aims to propose a technical solution for stabilizing the fluctuation of the supply voltage supplied to the power supply unit due to excessive peak power.

[0016] (Technical solution)

[0017] In order to achieve the above-mentioned purpose, an example of the present specification provides a system-on-chip (SoC). The SoC includes: a semiconductor substrate; a first circuit, disposed on the semiconductor substrate, and a first neural processing unit (NPU) configured to perform operations of an artificial neural network (ANN) model; a second circuit, disposed on the semiconductor substrate, and a second NPU configured to perform operations of the ANN model; and a third circuit, disposed on the semiconductor substrate and generating a control signal to selectively output at least one of a plurality of clock signals. Each of the first NPU and the second NPU includes a plurality of processing elements (PEs), and each of the plurality of PEs includes an adder, a multiplier, and an accumulator. The plurality of clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, the first clock signal is applicable to the first NPU. According to the control signal, among the plurality of clock signals, the second clock signal is applicable to the second NPU.

[0018] A disclosure of the present specification also provides a system. The system includes: a substrate; a first semiconductor chip mounted on the substrate, a first neural processing unit (NPU) configured to perform operations of an artificial neural network (ANN) model; a second semiconductor chip mounted on the substrate, a second NPU configured to perform operations of the ANN model; and a third circuit configured on the substrate and generating a control signal to selectively output at least one of a plurality of clock signals. Each of the first NPU and the second NPU includes a plurality of processing elements (PEs), each of the plurality of PEs includes an adder, a multiplier, and an accumulator. The plurality of clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, the first clock signal is applicable to the first NPU. According to the control signal, among the plurality of clock signals, the second clock signal is applicable to the second NPU.

[0019] (Effects of the Invention)

[0020] According to an example of the present disclosure, artificial neural network operations are dispersedly executed according to a plurality of clock signals, thereby having the effect of reducing peak power.

[0021] According to an example of the present disclosure, artificial neural network operations are dispersedly executed according to a plurality of clock signals, thereby having the effect of improving the safety of the supply voltage supplied to the neural processing unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.

[0023] Figure 2A A diagram for explaining the basic structure of a convolutional neural network (CNN).

[0024] Figure 2B A comprehensive diagram to facilitate understanding of the operation of a convolutional neural network.

[0025] Figure 3 is a schematic conceptual diagram showing a neural processing unit according to an example of the present disclosure.

[0026] Figure 4A is a schematic conceptual diagram illustrating one processing element among a plurality of processing elements applicable to an example of the present disclosure.

[0027] Figure 4B is a schematic conceptual diagram showing an SFU of an example that may be applicable to the present disclosure.

[0028] Figure 5 It is shown Figure 3 An example diagram of a variation of neural processing unit 100 is shown.

[0029] Fig. 6A is an example diagram showing the size of data for each layer in an exemplary artificial neural network model, Figure 6B It is shown Fig. 6A An exemplary table showing the data size, etc., of each layer in the exemplary artificial neural network model shown.

[0030] Fig. 7A is an exemplary diagram showing a SoC structure according to an example disclosed in this specification.

[0031] Figure 7B It is briefly shown Fig. 7A An example diagram of a SoC is shown in FIG.

[0032] Figure 7C It is shown in detail according to an example Fig. 7A or Figure 7B An example diagram of clock source 170 is shown in FIG.

[0033] Fig.7D is a diagram showing an example of a plurality of clock signals output by the clock source 170 .

[0034] Fig. 8A is an exemplary diagram showing a SoC structure according to another example disclosed in the present specification.

[0035] Figure 8B It is shown in detail Fig. 8A Example diagram of any of the NPUs shown.

[0036] Fig. 9 An example is shown in which clock signals of different phases are not input to a plurality of NPUs.

[0037] Fig.10 An example of inputting clock signals of different phases to four NPUs in consideration of peak power according to an example of the present disclosure is shown. DETAILED DESCRIPTION

[0038] The specific structure and step description of the examples of the present disclosure are only used to illustrate the examples according to the concept of the present disclosure. Therefore, the examples according to the concept of the present disclosure can be implemented in various forms. The present disclosure should not be interpreted as being limited to the examples of the present disclosure.

[0039] In the examples according to the concept of the present disclosure, various changes can be added and various forms can be provided. Therefore, specific examples will be shown in the drawings and the present disclosure or application will be described in detail. However, this is not intended to limit the examples according to the concept of the present disclosure to specific public forms. The examples according to the concept of the present disclosure should be understood to include all changes, equivalents and substitutes included in the thought and technical scope of the present disclosure.

[0040] Terms such as first and / or second can be used to describe various constituent elements, but the constituent elements should not be limited by the terms. The terms can only be used to distinguish one constituent element from other constituent elements. The terms can be referred to as a first constituent element and a second constituent element can be referred to as a first constituent element without departing from the scope of rights of the concept according to the present disclosure.

[0041] When a constituent element is mentioned as being "connected" or "contacting" another constituent element, it may be directly connected or contacting the other constituent element. However, it should be understood that other constituent elements may also exist among the plurality of constituent elements. On the other hand, when it is mentioned that a constituent element is "directly connected" or "directly contacting" another constituent element, it should be understood that there are no other constituent elements in between. Other expressions describing the relationship between constituent elements, such as "between" and "just between" or "adjacent to" and "directly adjacent to", etc., should also be interpreted in the same way.

[0042] The terms used in this disclosure are only used to describe specific examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In this disclosure, terms such as "including" or "having" are intended to indicate the presence of the described features, numbers, steps, operations, constituent elements, parts or combinations thereof. Therefore, it should be understood that this does not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, constituent elements, parts or combinations thereof.

[0043] Unless otherwise defined, all technical or scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology. Unless clearly defined in the present disclosure, terms such as those defined in commonly used dictionaries should not be interpreted in an idealized or overly formal sense.

[0044] In the explanation examples, descriptions of technical contents that are well known in the technical field to which the present disclosure belongs and are not directly related to the present disclosure will be omitted. This is to convey the main idea of ​​the present disclosure more clearly by omitting unnecessary descriptions and not obscuring the main idea of ​​the present disclosure.

[0045] <Definition of terms>

[0046] Hereinafter, in order to help understand the examples presented in the present disclosure, the terms used in the present disclosure will be briefly sorted out.

[0047] NPU: As the abbreviation of Neural Processing Unit, it is different from the central processing unit (CPU) and can refer to a processor specifically used for the operation of artificial neural network models.

[0048] ANN: As the abbreviation of artificial neural network, in order to imitate human intelligence, it imitates the neurons in the human brain through synapse connection and connects the nodes into a layered network.

[0049] DNN: As the abbreviation of Deep Neural Network, it means increasing the number of hidden layers of artificial neural networks to achieve higher artificial intelligence.

[0050] CNN: Convolutional Neural Network is a neural network that functions similarly to the human visual cortex image processing. It is well known that convolutional neural networks are suitable for image processing and are easy to extract features of input data and grasp the pattern of features.

[0051] Hereinafter, the present disclosure will be described in detail by explaining preferred examples of the present disclosure with reference to the attached drawings. Hereinafter, examples of the present disclosure will be described in detail with reference to the attached drawings.

[0052] <Artificial Intelligence>

[0053] Humans have intelligence such as recognition, classification, inference, prediction, and control / decision making. Artificial intelligence (AI) refers to artificial imitation of human intelligence.

[0054] The human brain is composed of many nerve cells called neurons. Each neuron is connected to hundreds to thousands of other neurons through a connection site called a synapse. In order to imitate human intelligence, the operation principle of biological neurons and the connection relationship between neurons are modeled as artificial neural network models. In other words, an artificial neural network is a system that connects nodes that imitate neurons in a layer structure.

[0055] These artificial neural network models are divided into "single-layer neural network" and "multi-layer neural network" according to the number of layers. A typical multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is a layer that receives external data. The number of neurons in the input layer is the same as the number of input variables. (2) The hidden layer is located between the input layer and the output layer, and receives signals from the input layer, extracts features, and transmits them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them to the outside. The input signal between neurons is multiplied by the strength of each connection (whose value is between 0 and 1) and then summed. If the sum is greater than the threshold of the neuron, the neuron is activated and realized as an output value through the activation function.

[0056] At the same time, in order to achieve higher artificial intelligence, artificial neural networks with increased number of hidden layers are called deep neural networks (DNN).

[0057] DNN is being developed in various structures. For example, a convolutional neural network (CNN), which is an example of DNN, is known to be convenient for extracting features of input values ​​(video or image) and grasping the pattern of the extracted output values. CNN can construct a form of processing compound multiplication operations, activation function operations, pooling operations, etc. in a specific order.

[0058] For example, in each layer of a DNN, a parameter (i.e., input value, output value, weight, or kernel, etc.) can be a matrix consisting of multiple channels. The parameters can be processed in the NPU through complex multiplication or matrix multiplication. The output value processed by the operation is generated at each layer.

[0059] For example, the Transformer is a DNN based on the attention technique. The Transformer uses multiple matrix multiplication operations. The Transformer can use the input value and the parameters such as query (Q), key (K) and value (V) to obtain the output values ​​of attention Q, K, V. The Transformer can process various inference operations based on the output values ​​(i.e., attention Q, K, V). The Transformer tends to perform better inference performance than CNN.

[0060] Figure 1 is a schematic conceptual diagram illustrating an exemplary artificial neural network model.

[0061] Hereinafter, the operation of an exemplary artificial neural network model 110 a that may be operated in the neural network processing unit 100 will be described.

[0062] Figure 1 The exemplary artificial neural network model 110a may be an artificial neural network that has been learned to perform various reasoning functions such as object recognition and speech recognition.

[0063] The artificial neural network model 110a may be a deep neural network (DNN).

[0064] However, the artificial neural network model 110a according to the example of the present disclosure is not limited to a deep neural network.

[0065] For example, the artificial neural network model 110a includes DaViT, MobileViT, Swin-Transformer, Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, and Inception-v3. However, the present disclosure is not limited to the above models. In addition, the artificial neural network model 110a can be an integrated model based on at least two different models.

[0066] Hereinafter, the reasoning process performed by the exemplary artificial neural network model 110a will be described.

[0067] The artificial neural network model 110a includes an exemplary deep neural network model of an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5), a third connection network 110a-6 and an output layer 110a-7. However, the present disclosure is not limited to Figure 1 The artificial neural network model shown. The first hidden layer 110a-3 and the second hidden layer 110a-5 may also be referred to as a plurality of hidden layers.

[0068] The input layer 110a-1 may illustratively include x1 and x2 input nodes. That is, the input layer 110a-1 may include information about two input values.

[0069] The first connection network 110a-2 may exemplarily include information about six weight values ​​that connect each node of the input layer 110a-1 to each node of the first hidden layer 110a-3. Each weight value is multiplied by the input node value, and the accumulated value of the multiplied value is stored in the first hidden layer 110a-3. The weight value and the input node value may be referred to as parameters of the artificial neural network model.

[0070] The first hidden layer 110a-3 may exemplarily include nodes a1, a2, and a3. That is, the first hidden layer 110a-3 may include information about three node values.

[0071] Figure 1 The first processing element PE1 in can process the operation of the a1 node.

[0072] Figure 1 The second processing element PE2 in can process the operation of the a2 node.

[0073] Figure 1 The third processing element PE3 in the a3 node can process the operation of the a3 node. The second connection network 110a-4 can exemplarily include information of 9 weight values ​​that connect each node of the first hidden layer 110a-3 to each node of the second hidden layer 110a-5. The weight values ​​of the second connection network 110a-4 are multiplied by the node values ​​input from the first hidden layer 110a-3, respectively, and the accumulated value of the multiplied values ​​is stored in the second hidden layer 110a-5.

[0074] The second hidden layer 110a-5 may exemplarily include nodes b1, b2, and b3. That is, the second hidden layer 110a-5 may include information about three node values.

[0075] Figure 1 The fourth processing element PE4 in can process the operation of the b1 node.

[0076] Figure 1 The fifth processing element PE5 in can process the operation of the b2 node.

[0077] Figure 1 The sixth processing element PE6 in can process the operation of the b3 node.

[0078] The third connection network 110a-6 exemplarily includes information about six weight values ​​connecting each node of the second hidden layer 110a-5 and each node of the output layer 110a-7. The weight values ​​of the third connection network 110a-6 are multiplied by the node values ​​input from the second hidden layer 110a-5, respectively, and the accumulated values ​​of the multiplied values ​​are stored in the output layer 110a-7.

[0079] The output layer 110a-7 may illustratively include y1 and y2 nodes. That is, the output layer 110a-7 may include information about two node values.

[0080] Figure 1 The seventh processing element PE7 in can process the operation of the y1 node.

[0081] Figure 1 The eighth processing element PE8 in can process the operation of the y2 node.

[0082] Each node may correspond to an eigenvalue, and the eigenvalue may correspond to a feature graph.

[0083] Figure 2A A diagram for explaining the basic structure of a convolutional neural network (CNN).

[0084] refer to Figure 2A , the input image can be represented by a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image can have multiple channels, where the channel can represent the number of color components of the input data image.

[0085] The convolution process involves traversing the input image at specified intervals while performing compound multiplication operations with the kernel.

[0086] A convolutional neural network may have a structure in which the output value (complex multiplication or matrix multiplication) of the current layer is passed as the input value of the next layer.

[0087] For example, a compound multiplication (convolution) is defined by two main parameters: input feature map and kernel. Parameters may include input feature map, output feature map, activation map, weights, kernel, and attention (Q, K, V), etc.

[0088] The compound multiplication slides the kernel window on the input feature map. The difference in the size of the kernel sliding the input feature map is called the stride.

[0089] Pooling can be applied after the compound multiplication. In addition, a fully-connected (FC) layer can be configured at the end of the convolutional neural network.

[0090] Figure 2B A comprehensive diagram to facilitate understanding of the operation of a convolutional neural network.

[0091] refer to Figure 2B , the input image is exemplarily shown as a two-dimensional matrix of size 6×6. Figure 2B The exemplary representation uses three nodes, namely, channel 1, channel 2, and channel 3.

[0092] First, the compound multiplication operation is described.

[0093] Input image (in Figure 2B 6×6 size is shown in the example) in the first node with kernel 1 for channel 1 (in Figure 2B 3×3 size is shown in the example) to perform compound multiplication, and the output feature is Figure 1 (exist Figure 2B In addition, the input image (in Figure 2B 6×6 size is shown in the example) at the second node and core 2 for channel 2 (in Figure 2B 3×3 size is exemplarily shown in FIG) to perform compound multiplication, and as a result, feature map 2 is output (in Figure 2B In addition, the input image is combined with kernel 3 for channel 3 at the third node (in Figure 2B 3×3 size) is schematically shown in FIG, and the resulting output feature Figure 3 (exist Figure 2B 4×4 size is shown by way of example in FIG.

[0094] To process each complex multiplication, the processing elements PE1 to PE12 of the neural processing unit 100 are configured to perform a MAC operation.

[0095] Next, the operation of the activation function is described.

[0096] Activation functions can be applied to features output from the compound multiplication operation Figure 1 , feature map 2 and features Figure 3 (exist Figure 2B 4×4 is exemplarily shown in FIG. 1 ). The output after applying the activation function may exemplarily be of a size of 4×4.

[0097] Next, the pooling operation will be described.

[0098] The features output from the activation function Figure 1 , feature map 2 and features Figure 3 (exist Figure 2B Each size is 4×4, as shown in the example, and is input by three nodes. Pooling can be performed by receiving the feature map output from the activation function. The pooling can reduce the size of the matrix or emphasize specific values ​​in the matrix. Pooling methods include maximum pooling, average pooling, and minimum pooling. Maximum pooling can be used to collect the maximum value in a specific area of ​​the matrix, and average pooling can be used to find the average in a specific area.

[0099] exist Figure 2BThe example in Figure 2 shows that a 4×4 feature map is reduced to a 2×2 size through pooling.

[0100] Specifically, the first node receives the feature for channel 1 Figure 1 After pooling is performed, the output is, for example, a 2×2 matrix. The second node receives feature map 2 for channel 2. After pooling is performed, the output is, for example, a 2×2 matrix. The third node receives feature map 2 for channel 3. Figure 3 After pooling is performed, the output is, for example, a 2x2 matrix.

[0101] Repeat the above compound multiplication, activation function and pooling, and finally the output can be as follows Figure 2A The output can be input by an artificial neural network for image recognition again. However, the present disclosure is not limited to the size of the feature map or kernel.

[0102] CNNs are the most widely used method in the field of computer vision among various deep neural network (DNN) methods described so far. In particular, CNNs have shown excellent performance in various research fields for performing various tasks such as image classification and objection detection.

[0103] <Hardware resources required for ANN operations>

[0104] Figure 3 is a schematic conceptual diagram showing a neural processing unit according to an example of the present disclosure.

[0105] Figure 3 The neural processing unit (NPU) 100 shown is a specialized processor for executing operations of artificial neural networks.

[0106] An artificial neural network is a network of artificial neurons that multiply and add weights when multiple inputs or stimuli enter, and transform and transmit the value of the added deviation through an activation function. The artificial neural network learned in this way can be used to output inference results from input data.

[0107] The neural processing unit 100 may be a semiconductor implemented by an electric / electronic circuit. The electric / electronic circuit may mean including many electronic components (eg, transistors, capacitors).

[0108] In the case of a transformer and / or CNN-based artificial neural network model, the neural processing unit 100 can select and process matrix multiplication operations, compound multiplication operations, etc. according to the architecture of the artificial neural network.

[0109] For example, in each layer of a composite multiplication neural network CNN, the input feature map belonging to the input data and the kernel belonging to the weight can be a tensor or matrix consisting of multiple channels. A composite multiplication operation of the input feature map and the kernel is performed, and a composite multiplication operation and a pooled output feature map are generated in each channel. The activation map of the corresponding channel is generated by applying the activation function to the output feature map. After that, pooling of the activation map can be applied. Here, the activation map can be collectively referred to as the output feature map. For the convenience of the following explanation, the activation map will be referred to as the output feature map.

[0110] However, the examples of the present disclosure are not limited thereto, and outputting a feature map means that a matrix multiplication operation or a compound multiplication operation, etc., has been applied.

[0111] For detailed explanation, the output feature map according to the example of the present disclosure should be interpreted as a comprehensive meaning. For example, the output feature map can be the result value of a matrix multiplication operation or a compound multiplication operation. Therefore, the plurality of processing elements 110 can be modified to further include processing circuits for additional arithmetic operations. That is, some circuit units of the SFU 150 described later can be configured to be included in the plurality of processing elements 110.

[0112] The neural processing unit 100 may be configured to include a plurality of processing elements 110 for processing complex multiplications and matrix multiplications required for the above-mentioned artificial neural network operations.

[0113] The neural processing unit 100 can be configured to include each processing circuit that is optimized for matrix multiplication operations, compound multiplication operations, activation function operations, pooling operations, stripe operations, batch normalization operations, skip connection operations, concatenation operations, quantization operations, clipping operations, and padding operations required for the above-mentioned manual operations.

[0114] For example, the neural processing unit 100 may be configured to include an SFU 150 for processing at least one of an activation function operation, a pooling operation, a stripe operation, a batch normalization operation, a skip connection operation, a concatenation operation, a quantization operation, a clipping operation, and a padding operation in the above algorithm.

[0115] Specifically, the neural processing unit 100 may include a plurality of processing elements (PEs) 110, an SFU 150, an NPU internal memory 120, an NPU controller 130, and an NPU interface 140. Each of the plurality of processing elements 110, the SFU 150, the NPU internal memory 120, the NPU controller 130, and the NPU interface 140 may be a semiconductor circuit connected with a plurality of transistors. Therefore, some may be difficult to identify and distinguish with the naked eye and can only be identified by operation.

[0116] For example, any circuit may operate as a plurality of processing elements 110 or operate as an NPU controller 130. The NPU controller 130 may be configured to perform the functions of a control section configured to control the artificial neural network inference operation of the neural processing unit 100.

[0117] The neural processing unit 100 includes an NPU internal memory 120 and an NPD controller 130, wherein the NPU internal memory is configured to store parameters of an artificial neural network model inferred from multiple processing elements 110 and SFU 150, and the NPU controller 130 includes a scheduler configured to control the operation scheduling of multiple processing elements 110, SFU 150 and NPU internal memory 120.

[0118] The neural processing unit 100 may be configured to process the feature map in response to an encoding and decoding method using scalable video coding (SVC) or scalable feature-map coding (SFC). The above method is a technique for variably changing the amount of data transmission according to the effective bandwidth and signal to noise ratio (SNR) of a communication channel or communication bus. That is, the neural processing unit 100 may be configured to further include an encoder and a decoder.

[0119] The plurality of processing elements 110 may execute a portion of an artificial neural network operation.

[0120] The SFU 150 may perform other portions of the operation for the artificial neural network.

[0121] The neural processing unit 100 may be configured to accelerate the operation of an artificial neural network model through hardware using a plurality of processing elements 110 and SFU 150 .

[0122] The NPU interface 140 may communicate with various constituent elements (eg, memory) connected to the neural processing unit 100 via a system bus.

[0123] The NPU controller 130 may include a scheduler configured to control operations of the plurality of processing elements 110 for inference operations of the neural processing unit 100 , operations of the SFU 150 , and a read and write order of the NPU internal memory 120 .

[0124] The scheduler in the NPU controller 130 may be configured to control the plurality of processing elements 110 , the SFU 150 , and the NPU internal memory 120 based on data locality information of the artificial neural network model or information on the structure.

[0125] The scheduler within the NPU controller 130 may analyze the structure of the artificial neural network model to be operated in the plurality of processing elements 110 and the SFU 150 or receive information that has been analyzed. The analyzed information may be information generated by a compiler. For example, the artificial neural network data that may be included in the artificial neural network model may be at least a portion of the node data (i.e., feature graphs) of each layer, the configuration data, locality information or structural information of each layer, and each weight data (i.e., weight kernel) of the connection network connecting the nodes of each layer. The data of the artificial neural network may be stored in a memory provided inside the NPU controller 130 or in the NPU internal memory 120. However, without limitation thereto, the data of the artificial neural network may be stored in a separate cache memory or register file configured in the NPU or in the SoC including the NPU.

[0126] The scheduler in the NPU controller 130 can schedule the order of operations of the artificial neural network model to be executed by the neural processing unit 100 based on the data locality information or information about the structure of the artificial neural network model.

[0127] The scheduler in the NPU controller 130 may receive scheduling information of the operation order of the artificial neural network model to be executed by the neural processing unit 100 based on the data locality information of the compiled artificial neural network model or the information of the structure. For example, the scheduling information may be information generated by a compiler. The scheduling information generated by the compiler may be referred to as machine code or binary code, etc.

[0128] That is, the scheduling information used in the NPU controller 130 may be data locality information based on the artificial neural network model or information generated by a compiler for the structure.

[0129] Specifically, the compiler can effectively schedule the NPU according to the degree of understanding and reconstruction of the data locality of the artificial neural network, which is an inherent characteristic of the artificial neural network model.

[0130] In detail, the compiler can effectively schedule the NPU based on the degree of understanding of the hardware structure and performance of the neural processing unit 100.

[0131] In detail, when the artificial neural network model is compiled by the compiler to be executed in the neural processing unit 100, the artificial neural network data locality can be reconstructed. The artificial neural network data locality can be reconstructed according to the algorithm applicable to the artificial neural network model and the operating characteristics of the processor.

[0132] In detail, the locality of the artificial neural network data can be reconstructed according to the way in which the neural processing unit 100 processes the corresponding artificial neural network model (e.g., feature map tiling, stationary method of processing elements).

[0133] In detail, the artificial neural network data locality can be reconstructed based on the number of processing elements of the neural processing unit 100 and the capacity of the internal memory.

[0134] In detail, artificial neural network data locality can be reconstructed based on the bandwidth of the memory communicating with the neural processing unit 100.

[0135] This is because, according to the above-mentioned factors, even if the same artificial neural network model is operated and processed, the neural processing unit 100 can determine the order of data required at each moment differently based on the clock unit.

[0136] The compiler can generate compiled machine code, and the data order required for the operation of the artificial neural network model determines the data locality based on the operation order of the layers, unit complex multiplications and / or matrix multiplications of the artificial neural network.

[0137] The scheduler may be configured to utilize scheduling information included in the machine code.

[0138] The scheduler in the NPU controller 130 can obtain the memory address value storing the feature map and weight data of the layer of the artificial neural network model based on the scheduling information.

[0139] For example, the scheduler in the NPU controller 130 can obtain the feature map of the layer of the artificial neural network model stored in the memory and the memory address value where the weight data is stored. Therefore, the scheduler in the NPU controller 130 can retrieve the feature map and weight data of the layer of the artificial neural network model to be driven from the main memory and store them in the NPU internal memory 120.

[0140] The feature map of each layer may have corresponding respective memory address values.

[0141] Each weight data may have a corresponding respective memory address value.

[0142] The scheduler in the NPU controller 130 can schedule the operation order of multiple processing elements 110 based on the data locality information or structural information of the artificial neural network model (for example, the configuration data, locality information or structural information of the layers of the artificial neural network of the artificial neural network model).

[0143] The scheduler in the NPU controller 130 can perform scheduling based on data locality information or information about the structure of the artificial neural network model, and thus can operate differently from the scheduling concept of a general CPU. The scheduling of a general CPU considers fairness, effectiveness, stability, response time, etc., and thus operates to show the best efficiency. In other words, scheduling is performed considering priority, operation time, etc., so as to perform the most processing in the same time.

[0144] Conventional CPUs use an algorithm for scheduling jobs taking into consideration data such as the priority and operation time of each process.

[0145] Different from this, the scheduler in the NPU controller 130 can control the neural processing unit 100 according to the processing order of the neural processing unit 100 determined based on the data locality information of the artificial neural network model or the information about the structure.

[0146] In addition, the scheduler in the NPU controller 130 can drive the neural processing unit 100 according to the processing order determined based on the data locality information or information about the structure of the artificial neural network model and / or the data locality information or information about the structure of the neural processing unit 100 to be used.

[0147] However, the present disclosure is not limited to data locality information or information about the structure of the neural network unit 100.

[0148] The scheduler in the NPU controller 130 may be configured to store data locality information or information about the structure of the artificial neural network.

[0149] In other words, even if the scheduler in the NPU controller 130 uses at least only the data locality information of the artificial neural network of the artificial neural network model or the information on the structure, the processing order can be determined.

[0150] In addition, the scheduler in the NPU controller 130 considers the data locality information or information about the structure of the artificial neural network model and the data locality information or information about the structure of the neural processing unit 100 to determine the processing order of the neural processing unit 100. In addition, the processing of the neural processing unit 100 can also be optimized according to the determined processing order.

[0151] That is, the NPU controller 130 can be configured to run based on machine code compiled from a compiler, but in another example, the NPU controller 130 can be configured to have a built-in embedded compiler. According to the above configuration, the neural processing unit 100 can be configured to receive files in the framework format of various AI software to generate machine code. For example, AI software frameworks include TensorFlow, PyTorch, Keras, XGBoost, mxnet, DARKNET, ONNX, etc.

[0152] The plurality of processing elements 110 refers to a plurality of processing elements PE1 to PE12 configured to operate feature maps and weight data of an artificial neural network. Each processing element may include a multiply and accumulate (MAC) operator and / or an arithmetic logic unit (ALU) operator. However, examples according to the present disclosure are not limited thereto.

[0153] Each processing element may be configured to selectively also include additional special function units for processing additional special functions.

[0154] For example, the processing element PE may be modified to further include a batch normalization unit, an activation function unit, an interpolation unit, and the like.

[0155] The SFU 150 may include a circuit unit that selects and processes activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, and padding operations according to the architecture of the artificial neural network. That is, the SFU 150 may include a plurality of special function operation processing circuit units.

[0156] Despite Figure 3 Although multiple processing elements are shown in the example, the MAC may be replaced within one processing element, and multiple multipliers and adder trees may be configured in parallel to implement operators. In this case, the multiple processing elements 110 may also be referred to as at least one processing element including multiple operators.

[0157] The plurality of processing elements 110 are configured to include a plurality of processing elements PE1 to PE12 . Figure 3The multiple processing elements PE1 to PE12 shown are only examples for ease of explanation, and the number of the multiple processing elements PE1 to PE12 is not limited. The size or number of the multiple processing elements 110 can be determined according to the number of the multiple processing elements PE1 to PE12. The size of the multiple processing elements 110 can be implemented in the form of an N×M matrix. Here, N and M are integers greater than 0. The multiple processing elements 110 may include N×M processing elements. That is, the processing element may be one or more.

[0158] The sizes of the plurality of processing elements 110 may be designed in consideration of the characteristics of the artificial neural network model operating the neural processing unit 100 .

[0159] The plurality of processing elements 110 are configured to perform functions such as addition, multiplication, and accumulation required for artificial neural network operations. In other words, the plurality of processing elements 110 may be configured to perform multiplication and accumulation (MAC) operations.

[0160] Hereinafter, the first processing element PE1 among the plurality of processing elements 110 will be taken as an example for description.

[0161] Figure 4A is a schematic conceptual diagram illustrating one processing element among a plurality of processing elements applicable to an example of the present disclosure.

[0162] According to an example of the present disclosure, the neural processing unit 100 includes a plurality of processing elements 110, an NPU internal memory 120 storing an artificial neural network model inferable from the plurality of processing elements 110, and an NPU controller 130 controlling the plurality of processing elements 110 and the NPU memory 120. The plurality of processing elements 110 perform MAC operations, and the plurality of processing elements 110 quantize and output MAC operation results. However, the examples of the present disclosure are not limited thereto.

[0163] The NPU internal memory 120 can store all or part of the artificial neural network model according to the memory size and the data size of the artificial neural network model.

[0164] The first processing element PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. However, examples according to the present disclosure are not limited thereto, and the plurality of processing elements 110 may be modified and implemented in consideration of operation characteristics of an artificial neural network.

[0165] The multiplier 111 multiplies the received (N) bit data and (M) bit data. The operation value of the multiplier 111 is output as (N+M) bit data.

[0166] The multiplier 111 may be configured to receive a variable and a constant.

[0167] The accumulator 113 uses the adder 112 for (L) loops to accumulate the operation value of the multiplier 111 and the operation value of the accumulator 113. Therefore, the bit width of the data of the output and input of the accumulator 113 can be output by (N+M+log2(L)) bits. Here, L is an integer greater than 0.

[0168] When the accumulation is finished, the accumulator 113 may receive an initialization signal (initialization reset) and initialize the data stored inside the accumulator 113 to 0. However, examples according to the present disclosure are not limited thereto.

[0169] The bit quantization unit 114 can reduce the bit width of the data output from the accumulator 113. The bit quantization unit 114 can be controlled by the NPU controller 130. The bit width of the quantized data can be output by (X)bit. Here, X is an integer greater than 0. According to the above configuration, the plurality of processing elements 110 are configured to perform a MAC operation, and the plurality of processing elements 110 have the effect of quantizing and outputting the MAC operation results. In particular, as the (L)loops increase, this quantization has the effect of further reducing power consumption. In addition, reducing power consumption also has the effect of reducing heat generation. In particular, reducing heat generation has the effect of reducing the possibility of malfunction of the neural processing unit 100 due to high temperature.

[0170] The output data (X) bit of the bit quantization unit 114 can be the node data of the next layer or the input data of the composite multiplication. If the artificial neural network model is quantized, the bit quantization unit 114 can be configured to receive quantized information from the artificial neural network model. However, not limited to this, the NPU controller 130 can be configured to extract quantized information by analyzing the artificial neural network model. Therefore, the output data (X) bit can be transformed and output as a quantized bit width to correspond to the quantized data size. The output data (X) bit of the bit quantization unit 114 can be stored in the NPU internal memory 120 as a quantized bit width.

[0171] The plurality of processing elements 110 of the neural processing unit 100 according to an example of the present disclosure include a multiplier 111, an adder 112, and an accumulator 113. The bit quantization unit 114 may select a trade-off according to whether quantization is applied. In another example, the bit quantization unit may be configured to be included in the SFU 150.

[0172] Figure 4B is a schematic conceptual diagram showing an SFU of an example that may be applicable to the present disclosure.

[0173] Referring to FIG4b, SFU 150 includes a plurality of functional units. Each functional unit can be selectively operated. Each functional unit can be selectively turned on or off. That is, each functional unit can be set.

[0174] In other words, the SFU 150 may include various circuit units required for artificial neural network inference operations.

[0175] For example, the circuit units of SFU 150 may include functional units for skip-connection operation, functional units for activation function operation, functional units for pooling operation, functional units for quantization operation, functional units for non-maximum suppression (NMS) operation, functional units for integer and floating point conversion (INT to FP32) operation, functional units for batch normalization operation, functional units for interpolation operation, functional units for concatenation operation, and functional units for bias operation, etc.

[0176] The functional units of the SFU 150 may be selectively turned on or off based on the data locality information of the artificial neural network model. The data locality information of the artificial neural network model may include control information related to or related to the turning off of the corresponding functional units when performing operations of a specific layer.

[0177] In the functional units of the SFU 150, the activated units can be turned on. In this way, when a part of the functional units of the SFU 150 are selectively turned off, the power consumption of the neural processing unit 100 can be reduced. At the same time, power gating can be used to turn off a part of the functional units. Alternatively, clock gating can also be performed to turn off a part of the functional units.

[0178] Figure 5 It is shown Figure 3 An example diagram of a variation of neural processing unit 100 is shown.

[0179] Since in addition to the plurality of processing elements 110, Figure 5 The neural processing unit 100 shown in FIG. Figure 3 The processing units 100 exemplarily shown in FIG. 1 are substantially the same, and thus repeated descriptions may be omitted for ease of explanation.

[0180] Figure 5 The plurality of processing elements 110 schematically shown in FIG. 1 further include, in addition to the plurality of processing elements PE1 to PE12 , respective register files RF1 to RF12 corresponding to each of the processing elements PE1 to PE12 .

[0181] Figure 5 The plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 shown in FIG. 1 are merely examples for convenience of explanation, and the numbers of the plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 are not limited.

[0182] The size or number of the plurality of processing elements 110 may be determined according to the number of the plurality of processing elements PE1 to PE12 and the number of register files RF1 to RF12. The size of the plurality of processing elements 110 and the plurality of register files RF1 to RF12 may be implemented in the form of an N×M matrix, where N and M are integers greater than 0.

[0183] The array size of the plurality of processing elements 110 may be designed in consideration of the characteristics of the artificial neural network model operated by the neural processing unit 100. Specifically, the memory size of the register file may be determined according to the data size of the artificial neural network model to be operated, the required operating speed, the required power consumption, etc.

[0184] The register files RF1 to RF12 of the neural processing unit 100 are static memory units directly connected to the processing elements PE1 to PE12. The register files RF1 to RF12 may be composed of, for example, triggers and / or latches. The register files RF1 to RF12 may be configured to store MAC operation values ​​of the corresponding processing elements PE1 to PE12. The register files RF1 to RF12 may be configured to provide or receive the NPU internal memory 120, weight data and / or node data.

[0185] Register files RF1 to RF12 may also be configured to perform the function of temporary storage of accumulators during MAC operations.

[0186] <Technical Difficulties Discovered by the Inventors of the Present Application>

[0187] Depending on the purpose of the AI ​​service, there may be multiple types of artificial neural network (ANN) models. For example, when the input data is an image or video, a CNN type ANN model can be used for object classification, object detection, object tracking, etc. within the image / video to provide AI services.

[0188] Generally speaking, the amount of computation in each layer of an artificial neural network model may be different. This is explained with reference to FIG6 .

[0189] Fig. 6A is an example diagram showing the size of data for each layer in an exemplary artificial neural network model, Figure 6B It is shown Fig. 6A An exemplary table showing the data size, etc., of each layer in the exemplary artificial neural network model shown.

[0190] Fig. 6A The exemplary artificial neural network model shown is Mobilenet V1. Fig. 6A The horizontal axis shown in the figure exemplarily represents the layers in the artificial neural network model in sequence, and the vertical axis represents the size of the data.

[0191] refer to Fig. 6A In the layer 1 shown, it can be seen that the size of the output feature map (OFMAP_SIZE) is larger than the size of the input feature map (IFMAP_SIZE).

[0192] The output feature map of layer 1 is passed to layer 2 and becomes the input feature map of layer 2. When the operation of layer 2 is completed, the output feature map is output. The output feature map of layer 2 is passed to layer 3 again and becomes the input feature map of layer 3.

[0193] In this way, the size of the input data input to each layer and the size of the output feature map output at each layer may be different. Therefore, the amount of calculation may be small in any layer, but it may be very large in other layers. Since the amount of calculation in each layer changes significantly in this way, the problem of difficulty in controlling power occurs.

[0194] The amount of computation per layer can be known through MAC, and the amount of computation per layer can differ by up to 227 times.

[0195] The neural network processing unit 100 may determine the number of activated processing elements among the plurality of processing elements 110 in proportion to the MAC operation amount of each layer, and increase the power consumption in proportion to the number of activated processing elements.

[0196] In addition, as the difference in MAC operation amount between adjacent layers increases, the deviation of peak power between adjacent layers increases. And the greater the deviation of peak power, the further the supply voltage VDD may fluctuate.

[0197] In particular, the inventors of the present application have recognized that when the amount of computation at a specific layer increases significantly, the instantaneous power consumption increases, and thus this may lead to a problem of reduced system stability.

[0198] For example, multiple processing elements can be started simultaneously for operations at a specific layer. A certain amount of power is required to drive each processing element, and when a considerable number of processing elements are driven at the same time, the instantaneous power required increases dramatically. If the neural processing unit is specially designed for low-power operation, the power supply capacity may be relatively insufficient compared to the server neural processing unit. Therefore, the neural processing units of these edge devices may be relatively more vulnerable to instantaneous power supply issues, and the supply voltage VDD may fluctuate when the power supply increases dramatically. In particular, when the supply voltage VDD drops below the threshold voltage of the transistor, the data stored in the transistor may be lost. In other words, if the supply voltage VDD decreases, the operating speed of the transistor may decrease, resulting in setup / hold violation problems and causing malfunctions.

[0199] For another example, the inventors of the present application have recognized that the PE 110 used for artificial neural network operations, such as performing addition, multiplication, and accumulation, consumes a large amount of power instantaneously, thereby causing other components within the NPU, such as the internal memory 120, to be unable to supply sufficient power. Specifically, the inventors of the present application have recognized that if sufficient power is not supplied to the internal memory 120, there may be a problem in which the possibility of the stored data bits being damaged (compromised) cannot be ruled out.

[0200] <Disclosure of this Application>

[0201] The inventors of the present application have recognized that the above-mentioned problem occurs because the operation of the artificial neural network model is performed based on a single clock signal in hardware.

[0202] Therefore, the inventors of the present application have invented a technology in which multiple NPUs operate according to different clock signals to reduce peak power.

[0203] Various embodiments for implementing the technology invented by the inventors of the present application will be described below with reference to the drawings.

[0204] Fig. 7A is an exemplary diagram showing a SoC structure according to an example disclosed in this specification, Figure 7B It is briefly shown Fig. 7A An example diagram of a SoC is shown in FIG.

[0205] refer to Fig. 7A and Figure 7B , an exemplary SoC 1000 may include multiple NPUs, multiple central processing units (CPUs), and multiple memories.

[0206] refer to Fig. 7A and Figure 7B , the plurality of NPUs may include, for example, a first NPU 100 - 1 , a second NPU 100 - 2 , a third NPU 100 - 3 , and a fourth NPU 100 - 4 .

[0207] refer to Fig. 7A , the plurality of CPUs may include, for example, a first CPU 200-1 and a second CPU 200-2. And the plurality of memories may include a first memory 300-1 and a second memory 300-2. Fig. 7A , it is shown that the number of the plurality of CPUs and the number of the plurality of memories are both 2, but the number is not limited thereto and may be variously modified to 4, 6, 8, etc.

[0208] Fig. 7A and 7B Each of the plurality of NPUs 100-1, 100-2, 100-3 and 100-4 shown may include a Figure 3 or Figure 5 PE 110 , internal memory 120 , NPU controller 130 , NPU interface 140 , and SFU 150 are shown.

[0209] refer to Fig. 7A The exemplary SoC 1000 may include a memory controller 350 , a system bus 500 , and an input output (I / O) interface 800 .

[0210] The system bus 500 may be implemented by an electrically conductive pattern formed on a semiconductor die. The system bus may implement high-speed communication. For example, the plurality of NPUs, the plurality of CPUs, the plurality of memories, and the memory controller 350 may communicate with each other through the system bus 500.

[0211] The plurality of NPUs and the plurality of CPUs may request the memory controller 350 to read or record data from at least one of the plurality of memories through the system bus 500 .

[0212] At the same time, reference Fig. 7A and Figure 7B The exemplary SoC 1000 may include a clock source 170 and a phase determination circuit 190. A power management integrated circuit (PMIC) 180 may be connected to the phase determination circuit 190. Fig. 7A and Figure 7BIn the embodiment, the PMIC 180 is shown as being disposed outside the SoC 1000, but is not limited thereto. According to the implementation, the PMIC 180 may be integrated within the SoC 1000 or within a package including the SoC 1000. That is, the SoC 1000 and the PMIC 180 may be integrated in a System in Package (SiP) form. Alternatively, the SoC 1000 and the PMIC 180 may be integrated in a Chip On Wafer on Substrate (CoWoS) form.

[0213] The multiple NPUs 100-1, 100-2, 100-3, 100-4, the multiple CPUs 200-1, 200-2, the multiple memories 300-1, 300-2, the clock source 170, and the phase determination circuit 190 in the SoC 1000 may all be semiconductor circuits connected with multiple transistors. Therefore, some may be difficult to identify and distinguish with the naked eye and can only be identified through operation.

[0214] The clock source 170 may be a circuit configured to output one or more clock signals. Figure 7C As shown, the clock source 170 may include a clock signal generating circuit 171 and a phase converter 175 , which will be described later.

[0215] The PMIC 180 may be a circuit provided to measure power of the SoC 1000 or power of each of the plurality of NPUs. Here, power may be measured differently according to utilization rates of the plurality of NPUs.

[0216] In order to reduce the peak power of multiple NPUs or SoCs, the phase determination circuit 190 generates a control signal based on the power measured by the PMIC 180 and outputs it to the clock source 170, wherein the control signal is used to calibrate or adjust at least one phase of one or more clock signals to be output by the clock source 170.

[0217] According to the control signal from the phase determination circuit 190, the phase converter 175 in the clock source 170 outputs the original clock signal as is (bypass) or outputs the original clock signal with a delay or shift in phase.

[0218] Specifically, the phase converter 175 in the clock source 170 can dynamically adjust at least one phase of the one or more clock signals so that the measured power is lower than the peak power. At least one phase of the one or more clock signals can be adjusted based on the utilization of any NPU in the plurality of NPUs.

[0219] For example, when it is predicted that the utilization rate of any NPU among multiple NPUs is higher than before, in order to reduce peak power, the phase converter 175 in the clock source 170 can output the phase of the original clock signal that is shifted or drifted as a phase delayed clock signal according to the control signal of the phase determination circuit 190.

[0220] In a more specific example, according to the control signal from the phase determination circuit 190, the phase converter 175 in the clock source 170 passes the original clock signal generated by the clock signal generation circuit 171 to the first NPU 100-1 as it is (bypass), or can pass the first delayed clock signal whose phase is delayed or shifted from the original clock signal to the second NPU 100-2. In addition, according to the control signal from the phase determination circuit 190, the phase converter 175 passes the second delayed clock signal whose phase is delayed or shifted from the original clock signal generated by the clock signal generation circuit 171 to the third NPU 100-3.

[0221] Figure 7C It is shown in detail according to an example Fig. 7A or Figure 7B An example diagram of a clock source 170 is shown in FIG. Fig.7D is a diagram showing an example of a plurality of clock signals output by the clock source 170 .

[0222] like Figure 7C As shown, the clock source 170 may include a clock signal generating circuit 170 and a phase converter 175 .

[0223] The clock signal generating circuit 171 may include an oscillator. In addition, the oscillator may include a pulse width modulator (PWM). PWM may change the clock cycle into a duty cycle so as to adjust the ratio of the high signal to the low signal.

[0224] The phase converter 175 may include a plurality of shifters (S) and a multiplexer / selector (MUX / Selector) 175-5. Figure 7C Four deflectors (S) 175 - 1 , 175 - 2 , 175 - 3 , and 175 - 4 are exemplarily shown in FIG.

[0225] Next, we will refer to Figure 7C and Fig.7D Give a description.

[0226] The original clock signal output from the clock signal generating circuit 171 can be output as it is (bypassed) in the phase converter 175 through the multiplexer / selector (MUX / Selector) 175 - 5 .

[0227] In addition, the original clock signal output from the clock signal generating circuit 171 is converted into a first delayed clock signal by delaying the phase once through the first shifter 175-1 in the phase converter 175, and then output through the multiplexer / selector (MUX / Selector) 175-5.

[0228] In addition, the original clock signal output from the clock signal generating circuit 171 is phase-delayed twice by the first shifter 175-1 and the second shifter 175-2 in the phase converter 175, thereby being converted into a second delayed clock signal, and then output through the multiplexer / selector (MUX / Selector) 175-5.

[0229] In addition, the original clock signal output from the clock signal generating circuit 171 is phase-delayed three times through the first shifter 175-1, the second shifter 175-2 and the third shifter 175-3 in the phase converter 175, thereby being converted into a third delayed clock signal, and can then be output through the multiplexer / selector (MUX / Selector) 175-5.

[0230] Similarly, the original clock signal output from the clock signal generating circuit 171 is phase-delayed four times by the first shifter 175-1, the second shifter 175-2, the third shifter 175-3 and the fourth shifter 175-4 in the phase converter 175, thereby being converted into a fourth delayed clock signal, and then output through the multiplexer / selector (MUX / Selector) 175-5.

[0231] According to the control signal from the phase determination circuit 190, as Figure 7C After the multiplexer / selector (MUX / Selector) 175-5 shown multiplexes multiple clock signals (i.e., the original clock signal, the first delayed clock signal, the second delayed clock signal, the third delayed clock signal and the fourth delayed clock signal), a clock signal can be selectively provided to each of the multiple NPUs.

[0232] For example, the multiplexer / selector (MUX / Selector) 175-5 provides the original clock signal to the first NPU 100-1, the first delayed clock signal to the second NPU 100-2, the second delayed clock signal to the third NPU 100-3, and the third delayed clock signal to the fourth NPU 100-4 according to the control signal from the phase determination circuit 190. In this way, the plurality of NPUs operate according to various clock signals having different phases, thereby suppressing a sudden increase in peak power.

[0233] As described above, in order to provide a plurality of clock signals with different phases, the disclosure of this specification does not provide a plurality of clock sources 170 , but reduces manufacturing costs by simply adding a phase converter 175 .

[0234] Fig. 8A is an exemplary diagram showing a SoC structure according to another example disclosed in this specification, Figure 8B It is shown in detail Fig. 8A Example diagram of any of the NPUs shown.

[0235] refer to Fig. 8A , the exemplary SoC 1000 may include a plurality of NPUs, a clock source 170, and a phase determination circuit 190. A power measurement integrated circuit (PMIC) 180 may be connected to the phase determination circuit 190. A power measurement integrated circuit (power management integrated circuit, PMIC) 180 may be connected to the phase determination circuit 190. The PMIC 180 may be connected to the phase determination circuit 190. Fig. 8A 1000, but is not limited thereto. According to the implementation, PMIC 180 may be integrated inside SoC 1000 or integrated in a package including SoC 1000. That is, SoC 1000 and PMIC 180 may be integrated in a system-in-package (SiP) form. Alternatively, SoC 1000 and PMIC 180 may be integrated in a chip-on-wafer-on-substrate (CoWoS) form.

[0236] The plurality of NPUs may include, for example, a first NPU 100 - 1 , a second NPU 100 - 2 , a third NPU 100 - 3 , and a fourth NPU 100 - 4 .

[0237] The multiple NPUs 100-1, 100-2, 100-3, 100-4, the clock source 170, and the phase determination circuit 190 in the SoC 1000 may all be semiconductor circuits connected with multiple transistors. Therefore, some may be difficult to identify and distinguish with the naked eye and can only be identified through operation.

[0238] In addition, Fig. 8A The SoC 1000 shown may include Fig. 7A The system includes a plurality of CPUs 200 - 1 , 200 - 2 , a plurality of memories 300 - 1 , 300 - 2 , a memory controller 350 , a system bus 500 , and an input output (I / O) interface 800 .

[0239] exist Fig. 8A In the embodiment, each of the plurality of NPUs 100-1, 100-2, 100-3, and 100-4 includes: Figure 3 or Figure 5 PE 110 , internal memory 120 , NPU controller 130 , NPU interface 140 , and SFU 150 are shown.

[0240] In addition, each of the plurality of NPUs 100-1, 100-2, 100-3, 100-4 may further include: Fig. 8A and Figure 8B The phase converter 175 shown. That is, Fig. 7A The difference shown in Fig. 8A and Figure 8B , a phase converter 175 is shown as being included in each NPU.

[0241] The clock source 170 may be a circuit configured to output an original clock signal.

[0242] The PMIC 180 may be a circuit provided for measuring power of the SoC 1000 or power of each of the plurality of NPUs. Here, power may be measured differently according to utilization rates of the plurality of NPUs.

[0243] In order to reduce the peak power of multiple NPUs or SoCs, the phase determination circuit 190 generates a control signal based on the power measured by the PMIC 180 and outputs it to the phase converter 175, wherein the control signal is calibrated or adjusted by the phase converter 174 to the phase of the original clock signal output by the clock source 170.

[0244] According to the control signal from the phase determination circuit 190, the phase converter 175 in each NPU outputs the original clock signal as it is (bypass) or outputs it after delaying (delay) or shifting (shift) the phase of the original clock signal.

[0245] Detailed reference Figure 8B , the phase converter 175 may include a plurality of shifters (S) and a selector (Selector) 175-5. Figure 8B In the figure, four deflectors (S) 175-1, 175-2, 175-3, and 175-4 are shown as examples.

[0246] The original clock signal received from the clock signal generating circuit 171 is transmitted as is (bypassed) to the selector (Selector) 175 - 5 in the phase converter 175 .

[0247] The original clock signal received from the clock signal generating circuit 171 is converted into a first delayed clock signal by the first shifter 175 - 1 in the phase converter 175 after its phase is delayed once, and then transmitted to the selector 175 - 5 .

[0248] In addition, the original clock signal received from the clock signal generating circuit 171 is phase-delayed twice by the first shifter 175 - 1 and the second shifter 175 - 2 in the phase converter 175 , thereby being converted into a second delayed clock signal, and then transmitted to the selector 175 - 5 .

[0249] Furthermore, the original clock signal received from the clock signal generating circuit 171 is phase-delayed three times by the first shifter 175-1, the second shifter 175-2 and the third shifter 175-3 in the phase converter 175, thereby being converted into a third delayed clock signal, and then transmitted to the selector 175-5.

[0250] Similarly, the original clock signal received from the clock signal generating circuit 171 is phase-delayed four times by the first shifter 175-1, the second shifter 175-2, the third shifter 175-3 and the fourth shifter 175-4 in the phase converter 175, thereby being converted into a fourth delayed clock signal and then passed to the selector 175-5.

[0251] According to the control signal from the phase determination circuit 190, the selector (Selector) 175-5 selects any one of the multiple clock signals (i.e., the original clock signal, the first delayed clock signal, the second delayed clock signal, the third delayed clock signal and the fourth delayed clock signal), and then provides it to the NPU controller 130 or PE110.

[0252] For example, assume a situation where utilization of multiple NPUs is expected to be higher than before.

[0253] In this case, in order to reduce the peak power, the selector 175-5 in the phase converter 175 of the first NPU 100-1 selects the first delayed clock signal according to the control signal from the phase determination circuit 190. Also, the selector 175-5 in the phase converter 175 of the second NPU 100-2 may select the second delayed clock signal according to the control signal from the phase determination circuit 190. In addition, the selector 175-5 in the phase converter 175 of the third NPU 100-3 may select the third delayed clock signal according to the control signal from the phase determination circuit 190. Similarly, the selector 175-5 in the phase converter 175 of the fourth NPU 100-4 may select the fourth delayed clock signal according to the control signal from the phase determination circuit 190.

[0254] In this way, the plurality of NPUs operate according to various clock signals of different phases, thereby making it possible to suppress a sudden increase in peak power.

[0255] That is, according to one disclosure of this specification, there is an advantage that the peak power can be reduced while maintaining the average power or energy used by the SoC to a certain extent.

[0256] In addition, according to one disclosure of this specification, in order to provide a plurality of clock signals with different phases, a plurality of clock sources 170 are not provided, and the manufacturing cost is reduced by simply adding a phase converter 175 .

[0257] So far, the description has been mainly based on the system on chip (SoC), but the disclosure of this specification is not limited to SoC, and the content of the present disclosure can also be applied to a board-level system based on a system in package (SiP) or a printed circuit board (PCB). For example, each functional component can be implemented by an independent semiconductor chip and connected to each other by a system bus implemented by an electrically conductive pattern formed on a PCB.

[0258] For example, an electrically conductive pattern that functions as a system bus 500 may be formed on a board substrate instead of an SoC. In addition, a first semiconductor chip 100-1 provided for a first NPU and a second semiconductor chip 100-2 provided for a second NPU may be mounted on a board substrate. In addition, a semiconductor chip for a first memory 300-1 and a semiconductor chip for a second memory 300-2 may be mounted on a board substrate. In addition, a memory controller 350 may be mounted on a board substrate. In addition, a semiconductor chip for a first CPU 200-1 and a semiconductor chip for a second CPU 200-2 may be mounted on a board substrate. In addition, a semiconductor chip for an I / O interface 800 may be mounted on a board substrate. The board substrate may be a printed circuit board (PCB).

[0259] Fig. 9 An example is shown in which clock signals of different phases are not input to a plurality of NPUs.

[0260] refer to Fig. 9 , the first clock CLK0 may be input to a plurality of neural processing units 100-1, 100-2, 100-3, 100-4. Among them, the plurality of neural processing units 100-1, 100-2, 100-3, 100-4 are configured to receive clock signals of the same phase. The plurality of neural processing units may include a first NPU 100-1, a second NPU 100-2, a third NPU 100-3, and a fourth NPU 100-4.

[0261] Fig. 9 100x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when one hundred processing elements are activated.

[0262] Fig. 9 200x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when two hundred processing elements are activated.

[0263] Fig. 9 500x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when five hundred processing elements are activated.

[0264] refer to Fig. 9, as the number of activated processing elements of the neural processing unit 100 increases, the peak power increases and the driving voltage VDD tends to decrease.

[0265] In detail, the number of PEs running in each clock cycle may vary depending on the performance of the compiler that compiles the artificial neural network model. In other words, the better the performance of the compiler, the more PEs can be run per unit clock. Among them, the proportion of PEs running among all PEs can be called PE utilization (utilization rate%). On the contrary, the better the performance of the compiler, the more the peak power of the NPU can be increased. In other words, the peak power can be proportional to the utilization rate of the PE. Therefore, there is a problem that as the compiler algorithm becomes more sophisticated, the peak power of the NPU may further increase.

[0266] Reference again Fig. 9 The peak power of the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 operating with the first clock CLK0 as the reference may be different according to the number of PEs operating in each clock. That is, the peak power may increase in proportion to the number of PEs operating in one clock.

[0267] In detail, when the NPU developed by low power consumption is applied to the edge device, it can be developed corresponding to the M.2 interface or the PCI Express interface. In particular, compared with the PCI Express interface, the maximum power of the M.2 interface may be relatively low. For example, the M.2 interface may have a current of 3A and a power limit of 10W. If the peak power of the NPU in the comparative example exceeds 10W at a specific clock, the supply voltage VDD provided to the NPU may be fluctuated. At this time, as the utilization rate of the PE increases, the peak power of the NPU further increases, and when the peak power exceeds the power limit, the degree of decrease in the supply voltage VDD may increase in proportion to the exceeded information.

[0268] If the supply voltage VDD drops below the IR-drop margin, data in the NPU operation may be erroneous. In detail, IR-drop is caused by the resistance R of the metal interconnect and the current I flowing through it, and IR-drop occurs when a voltage drop (V=IR) occurs according to Ohm's law. If multiple circuits related to multiple PEs of the NPU are switched at the same time, the NPU may consume a large amount of current. This higher current produces a larger voltage drop on the NPU, which can reduce the supply voltage in other parts of the SoC.

[0269] Summarize Fig. 9In the comparative example, sufficient IR-drop margin must be ensured to ensure stable operation of the NPU and prevent potential errors. IR-drop margin can refer to the difference between the supply voltage VDD and the lowest voltage allowed at all locations of the NPU during peak power conditions.

[0270] Fig.10 An example of inputting clock signals of different phases to four NPUs in consideration of peak power according to an example of the present disclosure is shown.

[0271] refer to Fig.10 , the first clock CLK1 is input to the first NPU 100-1, the second clock CLK2 is input to the second NPU 100-2, the third clock CLK3 is input to the third NPU 100-3, and the fourth clock CLK4 is input to the fourth NPU 100-4. The first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 are configured to receive clock signals of different phases from each other.

[0272] Fig.10 100x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when one hundred processing elements are activated.

[0273] Fig.10 200x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when one hundred processing elements are activated.

[0274] Fig.10 500x of PE utilization in FIG. 1 exemplarily shows the peak power and driving voltage VDD variation characteristics when one hundred processing elements are activated.

[0275] refer to Fig.10 As the number of activated processing elements of the neural processing unit 100 increases, the peak power increases and the driving voltage VDD tends to decrease. However, as signal clocks with different phases are input to multiple NPUs, the overall peak power tends to decrease, and the fluctuation of the supply voltage VDD also tends to decrease.

[0276] A first clock CLK1 may be applied to the first NPU 100-1, a second clock CLK2 having a different phase from the first clock CLK1 may be applied to the second NPU 100-2, a third clock CLK3 may be applied to the third NPU 100-3, and a fourth clock CLK4 may be applied to the fourth NPU 100-4.

[0277] Therefore, the intervals in which the peak power of the first NPU 100-1, the second NPU 100-2, the third NPU 100-3, and the fourth NPU 100-4 occurs can be dispersed on the time axis. Therefore, the entire peak power can be linearized, such as Fig.10 Therefore, the supply voltage VDD does not decrease and the linearity can be maintained.

[0278] like Fig.10 As shown in FIG. 1 , the intervals where the peak power of each NPU occurs can be dispersed on the time axis, thereby greatly reducing the overall peak power. Fig. 9 Compared with the comparative example shown, the voltage stability of the supply voltage VDD can be relatively more stable.

[0279] <Brief Summary of the Disclosure of the Present Application>

[0280] The contents of this manual described so far are summarized and explained as follows.

[0281] A disclosure of the present specification provides a system-on-chip (SoC). The SoC includes a semiconductor substrate; a first circuit, disposed on the semiconductor substrate, and a first neural processing unit (NPU) configured to perform operations of an artificial neural network (ANN) model; a second circuit, disposed on the semiconductor substrate, and a second NPU configured to perform operations of the ANN model; and a third circuit, disposed on the semiconductor substrate and generating a control signal to selectively output at least one of a plurality of clock signals. Each of the first NPU and the second NPU includes a plurality of processing elements (PEs), and each of the plurality of PEs includes an adder, a multiplier, and an accumulator. The plurality of clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, the first clock signal is applicable to the first NPU. According to the control signal, among the plurality of clock signals, the second clock signal is applicable to the second NPU.

[0282] A disclosure of the present specification also provides a system. The system includes: a substrate; a first semiconductor chip mounted on the substrate, a first neural processing unit (NPU) configured to perform operations of an artificial neural network (ANN) model; a second semiconductor chip mounted on the substrate, a second NPU configured to perform operations of the ANN model; and a third circuit configured on the substrate and generating a control signal to selectively output at least one of a plurality of clock signals. Each of the first NPU and the second NPU includes a plurality of processing elements (PEs), each of the plurality of PEs includes an adder, a multiplier, and an accumulator. The plurality of clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, the first clock signal is applicable to the first NPU. According to the control signal, among the plurality of clock signals, the second clock signal is applicable to the second NPU.

[0283] The SoC or system may further include a power management integrated circuit configured to measure power.

[0284] The first clock signal or the second clock signal is delayed by a preset phase compared to the phase of the original clock signal.

[0285] The SoC or system may further include a phase conversion circuit for generating, according to the control signal, at least one phase-delayed clock signal whose phase is delayed compared to the original clock source.

[0286] The phase conversion circuit includes one or more phase converters connected in a chain form. Among the one or more phase converters, a first phase converter receives the original clock signal. The phase conversion circuit includes a selector, which receives the original clock signal and is connected to the output of the one or more phase converters, so as to select at least one output from the original clock signal and the output of the one or more phase converters.

[0287] The first circuit for the first NPU or the second circuit for the second NPU includes a phase conversion circuit, which generates at least one phase-delayed clock signal whose phase is delayed compared to the original clock source according to the control signal.

[0288] In order to measure a power lower than a peak power, the phase of the one or more phase delayed signals is dynamically adjusted.

[0289] When the utilization rate of the first NPU or the second NPU is expected to be higher than before, the phase of the one or more phase delayed signals is shifted or drifted compared to the original clock signal in order to reduce peak power.

[0290] The first clock signal has a first phase, and the second clock signal has a second phase later than the first phase of the first clock signal.

[0291] The phase of the first clock signal is the same as that of the original clock signal, and the phase of the second clock signal is later than that of the first clock signal.

[0292] The examples of the present disclosure shown in this specification and the drawings are provided only as specific examples to easily explain the technical content of the present disclosure and help understand the present disclosure, and are not intended to limit the scope of the present disclosure. It is obvious to those skilled in the art that other modifications can be implemented in addition to the examples described so far.

[0293]

National R&D Project to Support This Invention

[0294]

Project unique number

[0295]

Project Number

[0296]

Department name

[0297]

Project Management (Professional) Organization Name

[0298] [Research Project Name] PIM Artificial Intelligence Semiconductor Core Technology Development

[0299] [Research Topic Name] Development of PIM Semiconductor Technology for Distributed Chip Memory-Computing Unit Fusion for Edge Use

[0300]

Project Execution Organization Name

[0301]

Study period

Claims

1. A system on chip, comprising: Semiconductor substrate; A first circuit, disposed on the semiconductor substrate, is a first neural processing unit configured to execute operations of an artificial neural network model; A second circuit, disposed on the semiconductor substrate, is a second neural processing unit configured to execute operations of an artificial neural network model; and a third circuit disposed on the semiconductor substrate and generating a control signal to selectively output at least one of the plurality of clock signals, Each of the first neural processing unit and the second neural processing unit includes a plurality of processing elements, each of the plurality of processing units includes an adder, a multiplier, and an accumulator, The multiple clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, a first clock signal is applicable to the first neural processing unit, and According to the control signal, among the multiple clock signals, a second clock signal is applicable to the second neural processing unit.

2. The system on chip according to claim 1, further comprising: The power measurement integrated circuit is configured on the semiconductor substrate and is used for measuring power.

3. The system on chip according to claim 1, wherein: The first clock signal or the second clock signal is delayed by a preset phase compared to the phase of the original clock signal.

4. The system on chip according to claim 1, further comprising: The phase conversion circuit generates at least one phase-delayed clock signal whose phase is delayed compared to the original clock source according to the control signal.

5. The system on chip according to claim 4, wherein: The phase conversion circuit comprises: one or more phase converters connected in a chain; and A selector receives the original clock signal and is connected to the output of the one or more phase converters, so as to select at least one output from the original clock signal and the output of the one or more phase converters, Among the one or more phase converters, a first phase converter receives the original clock signal.

6. The system on chip according to claim 1, wherein: A first circuit for the first neural processing unit or a second circuit for the second neural processing unit comprises: The phase conversion circuit generates at least one phase-delayed clock signal whose phase is delayed compared to the original clock source according to the control signal.

7. The system on chip according to claim 1, wherein: In order to measure a power lower than a peak power, the phase of the one or more phase delayed signals is dynamically adjusted.

8. The system on chip according to claim 1, wherein: When the utilization rate of the first neural processing unit or the second neural processing unit is expected to be higher than before, in order to reduce the peak power, the phase of the one or more phase delayed signals is offset or shifted compared to the original clock signal.

9. The system on chip according to claim 1, wherein: The first clock signal has a first phase, The second clock signal has a second phase that is later than a first phase of the first clock signal.

10. The system on chip according to claim 1, wherein: The phase of the first clock signal is the same as the phase of the original clock signal, The phase of the second clock signal is later than the phase of the first clock signal.

11. A system comprising: substrate; A first semiconductor chip, mounted on the substrate, is used to execute a first neural processing unit configured to perform operations of an artificial neural network model; A second semiconductor chip, mounted on the substrate, is used to execute a second neural processing unit configured to perform operations of the artificial neural network model; and a third circuit disposed on the substrate and generating a control signal to selectively output at least one of the plurality of clock signals, Each of the first neural processing unit and the second neural processing unit includes a plurality of processing elements, each of the plurality of processing units includes an adder, a multiplier, and an accumulator, The multiple clock signals include an original clock source or one or more phase-delayed clock signals whose phases are delayed relative to the original clock source. According to the control signal, among the plurality of clock signals, a first clock signal is applicable to the first neural processing unit, and According to the control signal, among the multiple clock signals, a second clock signal is applicable to the second neural processing unit.

12. The system according to claim 11, further comprising: The power measurement integrated circuit is configured on the substrate and is used for measuring power.

13. The system according to claim 11, wherein: The first clock signal or the second clock signal is delayed by a preset phase compared to the phase of the original clock signal.

14. The system of claim 11, further comprising: The phase conversion circuit generates at least one phase-delayed clock signal whose phase is delayed compared to the original clock source according to the control signal.

15. The system of claim 14, wherein: The phase conversion circuit comprises: one or more phase converters connected in a chain; and A selector receives the original clock signal and is connected to the output of the one or more phase converters, so as to select at least one output from the original clock signal and the output of the one or more phase converters, Among the one or more phase converters, a first phase converter receives the original clock signal.

16. The system according to claim 11, wherein: A first circuit for the first neural processing unit or a second circuit for the second neural processing unit comprises: The phase conversion circuit generates at least one phase-delayed clock signal whose phase is delayed compared to the original clock source according to the control signal.

17. The system of claim 11, wherein: In order to measure a power lower than a peak power, the phase of the one or more phase delayed signals is dynamically adjusted.

18. The system of claim 11, wherein: When the utilization rate of the first neural processing unit or the second neural processing unit is expected to be higher than before, in order to reduce the peak power, the phase of the one or more phase delayed signals is offset or shifted compared to the original clock signal.

19. The system of claim 11, wherein: The first clock signal has a first phase, The second clock signal has a second phase that is later than a first phase of the first clock signal.

20. The system of claim 11, wherein: The phase of the first clock signal is the same as the phase of the original clock signal, The phase of the second clock signal is later than the phase of the first clock signal.

Images