Neural processing unit, neural processing system, and application system

Abstract

A neural processing unit, a neural processing system, and an application system are provided. The neural processing unit performs an application job including a first neural network operation, the neural processing unit including a first processing core configured to perform the first neural network operation, a hardware block reconfigurable as a hardware core configured to perform a hardware block job, and at least one processor configured to execute computer-readable instructions to distribute a portion of the application job as the hardware block job to the hardware block based on a first workload of the first processing core.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2018-0137345, filed on November 9, 2018, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to a neural processing unit, a neural processing system, and an application system, and more specifically, to a neural processing unit, a neural processing system, and an application system comprising reconfigurable hardware blocks. Background Technology

[0004] Processing cores such as CPUs (Central Processing Units) or GPUs (Graphics Processing Units) can be used to perform DNN (Deep Neural Network) operations. As the throughput of DNN operations increases, NPUs (Neural Processing Units) can be used to perform DNN operations quickly.

[0005] DNN operations can be categorized into CNN (Convolutional Neural Network) operations, RNN (Recurrent Neural Network) operations, and so on. Depending on the type of DNN operation, NPUs with different hardware architectures can be used. For example, the hardware architecture of an NPU used to perform CNN operations may differ from that of an NPU used to perform RNN operations.

[0006] Multiple DNN operations can be performed in a specific application (e.g., application job). For example, both CNN and RNN operations can be performed in a single application.

[0007] When an NPU has only the hardware architecture for performing one type of DNN operation, it may not perform another type of DNN operation, or it may perform another type of DNN operation with high latency. Furthermore, even when the NPU has the hardware architecture for performing multiple DNN operations, the time consumed in executing the application task can be delayed depending on the amount of DNN operations included in the application task. For example, an NPU may have one CNN processing core and one RNN processing core. If the amount of CNN operations and RNN operations included in the application task executed by the NPU differs from each other (e.g., many CNN operations and a few RNN operations), resources may be wasted. That is, one processing core may complete its operation earlier than the other and become idle. The completion of the application task may be delayed until the other processing core completes its operation.

[0008] Therefore, in order to process application tasks quickly and prevent or reduce resource waste, it is desirable to realize neural processing units, neural processing systems and application systems that are heterogeneous in processing two or more neural network operations simultaneously and are reconfigurable, which is the ability of hardware to be reconfigured to process two or more neural network operations. Summary of the Invention

[0009] Various aspects of this disclosure provide a neural processing unit, a neural processing system, and an application system that are heterogeneous and reconfigurable to rapidly process application tasks and prevent or reduce resource waste.

[0010] However, the aspects of this disclosure are not limited to those described herein. These and other aspects of this disclosure will become clearer to those skilled in the art upon reference to the detailed description of this disclosure provided below.

[0011] According to one aspect of this disclosure, a neural processing unit is provided, configured to perform application work including a first neural network operation. The neural processing unit includes: a first processing core configured to perform the first neural network operation; a hardware block reconfigurable to be configured to perform hardware block work on the hardware core; and at least one processor configured to execute computer-readable instructions to distribute a portion of the application work as hardware block work to the hardware block based on a first workload of the first processing core.

[0012] According to one aspect of this disclosure, a neural processing unit is provided, the neural processing unit comprising: a hardware block reconfigurable as: a first hardware core configured to perform operations of a first neural network, or a second hardware core configured to perform operations of a second neural network different from the first neural network; and an internal memory storing function data for performing operations of the first neural network or the second neural network.

[0013] According to one aspect of this disclosure, a neural processing system is provided, the neural processing system comprising: an external memory storing metadata available for reconstructing hardware blocks; and a neural processing unit including a first processing core and hardware blocks, the first processing core being configured to perform operations of a first neural network.

[0014] According to one aspect of this disclosure, an application system is provided, the application system comprising: a central processing unit (CPU) configured to execute computer-readable instructions to perform one or more operations for executing an application, and distributing a first application task among a set of application tasks to be executed for executing the application, the first application task including a first neural network operation; a neural processing unit including a first processing core and a hardware block, the neural processing unit being configured to execute the first application task using the first processing core and the hardware block, the first processing core being configured to execute the first neural network operation; and a reconstruction loader configured to pass metadata to the neural processing unit, the metadata being available for reconstructing the hardware block. Attached Figure Description

[0015] The above and other aspects and features of this disclosure will become clearer from the following detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 This is a block diagram of a neural processing unit according to some embodiments;

[0017] Figure 2 This is a block diagram of a neural processing unit according to some embodiments;

[0018] Figure 3 This is a block diagram of a neural processing unit according to some embodiments;

[0019] Figure 4 This is a block diagram of a neural processing unit according to some embodiments;

[0020] Figure 5 This is a block diagram illustrating DNN (Deep Neural Network) operations according to some embodiments;

[0021] Figure 6 This is a block diagram of DNN operations according to some embodiments;

[0022] Figure 7 This is a diagram showing the quantized weighted data;

[0023] Figure 8 This is a block diagram of a neural processing unit according to some embodiments;

[0024] Figure 9 This is a block diagram of a neural processing unit according to some embodiments;

[0025] Figure 10 It is a graph showing the function data;

[0026] Figure 11 This is a block diagram of a neural processing system according to some embodiments;

[0027] Figure 12This is a block diagram of a neural processing system according to some embodiments;

[0028] Figure 13 This is a block diagram of a neural processing system according to some embodiments;

[0029] Figure 14 This is a block diagram of an application system according to some embodiments;

[0030] Figure 15 This is a block diagram illustrating an application workflow according to some embodiments; and

[0031] Figure 16 This is a block diagram illustrating data flowing in a memory according to some embodiments. Detailed Implementation

[0032] Reference Figures 1 to 4 Describes a neural processing unit according to some embodiments of this application. Figures 1 to 4 This is a block diagram of a neural processing unit according to some embodiments of this application.

[0033] Reference Figure 1 According to some embodiments of this application, the neural processing unit 100 may include a first processing core 110, a working distributor 120, a hardware block 130, and / or a bus 140.

[0034] The neural processing unit 100 can execute application work 150 (e.g., neural network operations to be performed in a particular application). According to some example embodiments, application work 150 may include one or more machine code and / or native instructions that may specify specific registers and / or algorithms, addressing and / or control functions corresponding to the registers and / or functions of the neural processing unit 100. Application work 150 may include a first processing core work 152 or a hardware block work 154. The first processing core work 152 may be executed by a first processing core 110, which will be described below. Hardware block work 154 may be executed by a hardware core, which will be described below. Here, a hardware core may refer to a later reference reconstructed from hardware block 120. Figures 2 to 4 The CNN processing hardware core 132, RNN processing hardware core 134, and MAC (multiply-add, i.e., multiplication and addition) processing hardware core 136 for performing MAC operations are described, but the scope of this disclosure is not limited thereto.

[0035] The first processing core 110 can perform a first neural network operation. That is, the first processing core operation 152 can include a first neural network operation. For example, the first neural network operation can be a CNN (Convolutional Neural Network) operation, and the first processing core 110 can be a CNN IP (Intellectual Property) core for processing CNN operations. Alternatively, the first neural network operation can be an RNN (Recurrent Neural Network) operation, and the first processing core 110 can be an RNN IP core for processing RNN operations. Thus, in this specification, CNN operations and RNN operations have been described as examples of first neural network operations, but the scope of this disclosure is not limited thereto.

[0036] Work distributor 120 can distribute a portion of application work 150 as hardware block work 154 to hardware block 130 (distributing hardware block work 154 to hardware block 130 is also referred to herein as distributing "for hardware block work 154"). Work distributor 120 can distribute a portion of application work 150 as hardware block work 154 according to a first workload of the first processing core 110. For example, work distributor 120 can receive the first workload from the first processing core 110 via bus 140. Additionally, work distributor 120 can distribute a portion of application work 150 as hardware block work 154 according to the first workload.

[0037] In some embodiments, the work distributor 120 may distribute application work 150 input to the neural processing unit 100 to the first processing core 110 and / or hardware block 130. For example, the work distributor 120 may distribute a first neural network operation to the first processing core 110 as a first processing core operation 152 (distributing to the first processing core 110 as the first processing core operation 152 is also referred to herein as distribution "for first processing core operation 152"), and may distribute the remaining operations of application work 150 to hardware block operation 154.

[0038] In some embodiments, the work distributor 120 may distribute a portion of the first processing core operation 152 as hardware block operation 154 based on whether the first workload exceeds a predetermined (or limited) value. For example, if the first workload exceeds the predetermined value, the work distributor 120 may distribute a portion of the first processing core operation 152 as hardware block operation 154. In an embodiment, a portion of the first neural network operation included in the first processing core operation 152 may be distributed as hardware block operation 154. Alternatively, if the first workload does not exceed the predetermined value, the work distributor 120 may not distribute a portion of the first processing core operation 152 as hardware block operation 154. In an embodiment, the work distributor 120 may distribute other operations (not the first processing core operation 152) included in the application operation 150 as hardware block operation 154.

[0039] The work distributor 120 can be hardware. For example, the work distributor 120 can be (but is not limited to) a discrete IP core. Alternatively, the work distributor 120 can be software. For example, the neural processing unit 100 can include a general-purpose processing core that performs the general work in application work 150 other than the first processing core work 152 and the hardware block work 154, and the work distributor 120 can be software executed by the general-purpose processing core. According to some example embodiments, operations described herein as being performed by either or both of the neural processing unit 100 and the work distributor 120 can be executed by at least one processor (e.g., a general-purpose processing core) that executes program code including instructions corresponding to the operations. The instructions can include one or more machine code and / or native instructions that can specify particular registers and / or algorithms, addressing and / or control functions of registers and / or functions corresponding to the neural processing unit 100 and / or the work distributor 120. The instructions can be stored in the memory of the neural processing unit 100 (e.g., internal memory 160 discussed below). For example, the term 'processor' as used in this disclosure may refer to a hardware-implemented data processing device having circuitry physically structured to perform desired operations, including, for example, operations represented as code and / or instructions included in a program. In at least some example embodiments, the aforementioned hardware-implemented data processing device may include (but is not limited to) a microprocessor, a central processing unit (CPU), a processor core, a multi-core processor; a multiprocessor, an application-specific integrated circuit (ASIC), and a field-programmable gate array (FPGA).

[0040] Hardware block 130 can be reconfigured into a hardware core for performing hardware block operation 154. That is, hardware block 130 can be reconfigured into a hardware core with a structure for performing hardware block operation 154, based on hardware block operation 154. If the work distributor 120 distributes the first neural network operation as hardware block operation 154, then hardware block 130 can be reconfigured into a hardware core for processing the first neural network operation. If the work distributor 120 distributes the second neural network operation, which is different from the first neural network, as hardware block operation 154, then hardware block 130 can be reconfigured into a hardware core for processing the second neural network operation.

[0041] As an example, when hardware block 154 performs a CNN (Convolutional Neural Network) operation, hardware block 130 can be reconstructed as follows: Figure 2 The CNN processing hardware core 132 described herein. As another example, when hardware block operation 154 is an RNN (Recurrent Neural Network) operation, hardware block 130 can be reconstructed to later perform... Figure 3 The RNN processing hardware core 134 is described in the text. Furthermore, as another example, when hardware block 154 performs a MAC (multiply-accumulate) operation, hardware block 130 can be reconfigured to later perform... Figure 4The MAC processing hardware core 136 described herein is used to perform MAC operations.

[0042] In some embodiments, while the neural processing unit 100 performs application tasks, the hardware block 130 can be reconfigured into a hardware core for performing hardware block tasks 154. For example, while the neural processing unit 100 performs application tasks, the task distributor 120 can sequentially distribute a first neural network operation and a second neural network operation different from the first neural network into hardware block tasks 154. In this case, the hardware block 130 can be sequentially reconfigured into a first hardware core for processing the first neural network operation and a second hardware core for processing the second neural network operation.

[0043] According to some example embodiments, the operations described herein as being performed by any one or both of the first processing core 110 and hardware block 130 (and reconstructed hardware cores of hardware block 130, such as CNN processing hardware core 132, RNN processing hardware core 134, and / or MAC processing hardware core 136 discussed below) can be executed by at least one processor executing program code, which includes instructions corresponding to the operations. The instructions may include one or more machine code and / or native instructions that may specify particular registers and / or algorithms, addressing and / or control functions corresponding to the registers and / or functions of the first processing core 110 and / or hardware block 130. The instructions may be stored in memory. For example, any one or both of the first processing core 110 and hardware block 130 may include logic blocks programmed in a hardware description language (e.g., Verilog). As an example, any one or both of the first processing core 110 and hardware block 130 may be an FPGA (Field Programmable Gate Array). As another example, the neural processing unit may be an eFPGA (embedded FPGA), and either or both of the first processing core 110 and hardware block 130 may be FPGA blocks included in the eFPGA. In some example embodiments, operations described herein as being performed by either or both of the neural processing unit 100 and the work distributor 120 may be performed by at least one first processor executing first program code, the first program code including instructions corresponding to the operation; operations described herein as being performed by the first processing core 110 may be performed by at least one second processor executing second program code, the second program code including instructions corresponding to the operation; and operations described herein as being performed by hardware block 130 may be performed by at least one third processor executing third program code, the third program code including instructions corresponding to the operation. The first program code, the second program code, and / or the third program code may be stored in the memory of the neural processing unit 100 (e.g., internal memory 160). In embodiments, the second program code may be stored in the memory of the first processing core 110, and the third program code may be stored in the memory of the hardware block 130.

[0044] Bus 140 can be a path for moving data between the work distributor 120, the first processing core 110, and / or the hardware block 130. Bus 140 can be a path for transferring application work 150 of the neural processing unit 100 to the work distributor 120. Bus 140 can be a path for transferring first processing core work 152 and / or hardware block work 154 distributed from the work distributor 120 to the first processing core 110 and / or the hardware block 130.

[0045] Reference Figure 2 , Figure 1 Hardware block 130 can be reconfigured into a CNN (Convolutional Neural Network) processing hardware core 132. For example, if hardware block operation 154 distributed from work distributor 120 is a CNN operation, then hardware block 130 can be reconfigured into a CNN processing hardware core 132 for processing CNN operations. For example, CNN operations may include neural network operations related to image recognition processing. For example, CNN operations may include neural network operations for recognizing characters, people, objects, etc., in an image.

[0046] Reference Figure 3 , Figure 1 Hardware block 130 can be reconfigured into an RNN (Recurrent Neural Network) processing hardware core 134. For example, if hardware block operation 154 distributed from work distributor 120 is an RNN operation, then hardware block 130 can be reconfigured into an RNN processing hardware core 134 for processing RNN operations. For example, RNN operations may include neural network operations related to speech recognition processing. For example, RNN operations may include neural network operations for recognizing characters, music, sound sources, etc., from speech (e.g., detected and / or recorded audio).

[0047] Reference Figure 4Hardware block 130 can be reconfigured into a MAC processing hardware core 136 that performs MAC operations. In some embodiments, the first neural network operation may include a first MAC (multiply-accumulate) operation. Work distributor 120 distributes a portion of the first MAC operation as hardware block work based on the ratio of the first MAC operation to the first workload of the first processing core 110. In an embodiment, the ratio of the first MAC operation to the first workload is the ratio of the amount of operations and / or instructions included in the first MAC operation to the amount of operations and / or instructions included in the first workload. In an embodiment, the ratio of the first MAC operation to the first workload is the ratio of the data size of the first MAC operation to the data size of the first workload. As an example, if the ratio of the first MAC operation to the first workload of the first processing core 110 exceeds a predetermined (or limited) value, work distributor 120 may distribute a portion of the first MAC operation as hardware block work 154. For example, when the predetermined value is 0.4 and the ratio of the first MAC operation to the first workload is 0.6, work distributor 120 may divide a portion of the first MAC operation into hardware block work 154. In one embodiment, hardware block 130 may be reconfigured into a MAC processing hardware core 136 for performing the first MAC operation. As another example, when the ratio of the first MAC operation to the first workload of the first processing core 110 does not exceed a predetermined (or limited) value, the workload distributor 120 may not distribute a portion of the first MAC operation to hardware block workload 154. In another embodiment, hardware block 130 may not be reconfigured into a MAC processing hardware core 136.

[0048] For example, the MAC processing hardware core 136 can perform operations to process a series of input values ​​(X1, X2, ..., X...). m ) and a series of weighted values ​​(W X1 W X2 ... W Xm The weighted sum of input values ​​(X1×W) is calculated in the input. X1 +X2×W X2 +……+X m ×W Xm The MAC processing hardware core 136 may include a multiplier performing multiplication and / or an adder performing addition, according to some example embodiments. Operations described herein as being performed by either or both of the multiplier and adder may be executed by at least one processor executing program code including instructions corresponding to the operation. The instructions may be stored in memory.

[0049] The multiplier can take input values ​​(X1, X2, ..., X...) m ) and a series of weighted values ​​(W X1 W X2 ... WXm The weighted input value (X1×W) is calculated (e.g., determined and / or computed) in the calculation. X1 X2×W X2 ... X m ×W Xm As an example, the multiplier of the MAC processing hardware core 136 may include multiplying a series of input values ​​(X1, X2, ..., X...). m ) and a series of weighted values ​​(W X1 W X2 ... W Xm A multiplier that performs direct multiplication. As another example, the multiplier in the MAC processing hardware core 136 may include a lookup table, which includes, for example... Figures 5 to 7 The quantitative weighted data described in detail in the document.

[0050] The adder can take the weighted input value (X1×W) X1 X2×W X2 ... X m ×W Xm The sum of the weighted input values ​​(X1×W) is calculated (e.g., determined and / or computed). X1 +X2×W X2 +……+X m ×W Xm ).

[0051] Reference Figure 5 and Figure 6 This application describes a MAC processing hardware core 136 of a neural processing unit according to some embodiments of the present application. For ease of explanation, the description will primarily focus on... Figures 1 to 4 The differences described. Figure 5 and Figure 6 This is a block diagram illustrating DNN (Deep Neural Network) operations according to some embodiments.

[0052] Reference Figure 5 DNN operations include MAC operations 210 and / or activation function operations 220.

[0053] MAC operation 210 can be performed from input values ​​(X1, X2, ..., X...). m ) and a series of weighted values ​​(W X1 W X2 ... W Xm The sum of the weighted input values ​​(X1×W) is calculated (e.g., determined and / or computed). X1 +X2×W X2 +……+X m ×W Xm According to some embodiments of this application, MAC operation 210 may be performed by the MAC processing hardware core 136 of the neural processing unit 100.

[0054] Activation function operation 220 can be based on input value ΣX i W i Calculate the output value f(ΣX) of the activation function. i W i For example, the activation function used for activation function operation 220 may include the Sigmoid function, the tanh (hyperbolic tangent) function, the ReLU (corrected linear unit) function, etc., but the scope of this disclosure is not limited thereto. According to some embodiments of this application, the MAC processing hardware core 136 can use the following references Figure 9 The information described in memory 160 is used to perform activation function operation 220.

[0055] Reference Figure 6 According to some embodiments of this application, the MAC processing hardware core 136 may include a first signal splitter to an m-th signal splitter (multiplexer) (D1, ..., Dm), a first lookup table to an m-th lookup table (T1, ..., Tm), and / or an adder A1. According to some example embodiments, operations described herein as being performed by any one or all of the first signal splitter to an m-th signal splitter and the adder A1 may be executed by at least one processor executing program code, which includes instructions corresponding to the operations. The instructions and / or the first lookup table to an m-th lookup table (T1, ..., Tm) may be stored in memory.

[0056] The first to the m-th multiplexers (D1, ..., Dm) can determine the index of the corresponding weighting value. For example, the first multiplexer D1 can determine the first weighting value W of the first input value X1. X1 First index (Id) X1 =2). The second multiplexer D2 can determine the second weighting value W of the second input value X2. X2 Index (Id) X2 =0). The m-th multiplexer Dm can determine the m-th input value X. m The m-th weighted value W Xm Index (Id) Xm =1). The following will be... Figure 7 The index of the weighted value is described in detail.

[0057] The first lookup table to the m-th lookup table (T1, ..., Tm) can include the product of the input data and a series of quantized weighted values. For example, the first lookup table T1 can include the first input data X1 and a series of quantized weighted values ​​(W'0, W'1, ..., W'). 15 The product values ​​of (X1×W'0, X1×W'1, ..., X1×W') 15The second lookup table T2 may include the second input data X2 and a series of quantized weighted values ​​(W'0, W'1, ..., W'). 15 Multiplication values ​​(X2×W'0, X2×W'1, ..., X2×W') 15 The m-th lookup table Tm can include the m-th input data X. m With a series of weighted values ​​(W'0, W'1, ..., W') quantified 15 The product of (X) m ×W'0、X m ×W'1、……、X m ×W' 15 (See below for reference) Figure 7 Describe in detail a series of quantified weighted values.

[0058] The first to the m-th multiplexers (D1, ..., Dm) can send quantized data corresponding to a defined index of the first to the m-th lookup tables (T1, ..., Tm) to the adder A1. For example, the first multiplexer D1 can send quantized data X1 by multiplying it by the first index (Id) of the first lookup table T1. X1 =2) The value (X1×W'2) obtained by the quantized weighted data W'2 corresponding to the input data is sent to adder A1. The second multiplexer D2 can multiply the input data X2 by the second index (Id) of the second lookup table T2. X2 The value (X2×W'0) obtained by quantizing and weighting the data W'0 corresponding to (=0) is sent to adder A1. The m-th multiplexer Dm can send the input data X... m Multiply by the m-th index (Id) of the m-th lookup table Tm Xm =1) and the value (X) obtained from the quantized weighted data W'1. m ×W'1) is sent to adder A1.

[0059] Adder A1 can compute (e.g., determine and / or calculate) the weighted data (X1×W'2, X2×W'0, ..., X) sent from the first multiplexer to the m-th multiplexer (D1, ..., Dm). m The sum of (X1×W'1) and (X2×W'0+……+X) m ×W'1). The result value from adder A1 can be input into activation function operation 220.

[0060] Thus, when the hardware block 130 of the neural processing unit 100 according to some embodiments of this application is reconstructed into a MAC processing hardware core 136 performing MAC operations using a lookup table including quantized weighted data, the multiplication operation of the input value and the weighted value performed according to the general MAC operation can be omitted. Therefore, the neural processing unit 100 according to some embodiments of this application can quickly process the application task 150 and save resources.

[0061] Reference Figure 7 Describes the index and weighting values ​​of quantized weighted data. Figure 7 It is a graph used to interpret quantized weighted data. Figure 7 In the graph shown, the horizontal axis represents the magnitude of the weighted values, and the vertical axis represents the distribution of the weighted values ​​(e.g., density). Figure 7 The weighted values ​​are shown to conform to a normal distribution, but the embodiments are not limited to this.

[0062] Reference Figure 7 The distribution of weighted values ​​can include multiple zones. For example, the distribution of weighted values ​​can include zones 0 to 15 (R0, R1, ..., R...) with the same width. 15 Additionally, the distribution of weighted values ​​can include 128 zones with the same width.

[0063] Multiple zones (R0, R1, ..., R 15 Each of the data points may include a quantized weighted data set (W'0, W'1, ..., W'). 15 As an example, quantized weighted data (W'0, W'1, ..., W') 15 ) can be the midpoint of each zone. That is, when the horizontal axis of zone R1 is from W' 1i To W' 1f When within the range, from W' 1i The width of the region from W'1 to W' can be the same as that from W'1 to W' 1f The widths of the regions are the same. As another example, quantized weighted data (W'0, W'1, ..., W') 15 () can be the average value in each zone. That is, when the horizontal axis of zone R1 is from W' 1i To W' 1f When within the range, W'1 can be (W' 1i +W' 1f ) / 2. Refer again. Figure 6 The MAC processing hardware core 136's first to m-th multiplexers (D1, ..., Dm) can determine the input value (X1, X2, ..., Xm). m The weighted value (W) of each in ) X1 W X2 ... W XmThe index of ). For example, the first multiplexer D1 can be based on Figure 7 Which of the plurality of regions in the distribution of weighted values ​​shown includes the first weighted value W of the first input value X1? X1 To determine the index. For example, when the first weighted value W is included in the second region R2. X1 At that time, the first index Id X1 It can be determined to be 2. When the second weighted value W is included in region 0 R0. X2 At that time, the second index Id X2 It can be determined to be 0. When the first region R1 includes the m-th weighted value W Xm At that time, the m-th index Id Xm It can be determined as 1.

[0064] In the following text, reference will be made to Figure 8 This application describes neural processing units according to some embodiments. For ease of explanation, the description will primarily focus on... Figures 1 to 7 The differences between those mentioned. Figure 8 This is a block diagram of a neural processing unit according to some embodiments.

[0065] Reference Figure 8 The neural processing unit 102 according to some embodiments of this application may include a first processing core 110, a second processing core 112, a work distributor 120, a hardware block 130, and / or a bus 140. According to some example embodiments, operations described herein as being performed by the second processing core 112 may be executed by at least one processor executing program code, which includes instructions corresponding to the operations. The instructions may include one or more machine code and / or native instructions that may specify particular registers and / or algorithms, addressing and / or control functions of the registers and / or functions of the second processing core 112. The instructions may be stored in memory. For example, the second processing core 112 may include logic blocks programmed in a hardware description language (e.g., Verilog). As an example, the second processing core 112 may be an FPGA (Field-Programmable Gate Array). As another example, the neural processing unit may be an eFPGA (Embedded FPGA), and the second processing core 112 may be an FPGA block included in the eFPGA. In some example embodiments, the operations described herein as being performed by the second processing core 112 may be executed by at least one fourth processor executing fourth program code, which includes instructions corresponding to the operations. The fourth program code may be stored in the memory of the neural processing unit 100 (e.g., internal memory 160) and / or the memory of the second processing core 112.

[0066] The neural processing unit 102 can execute application work 150. Application work 150 may include a first processing core work 152, a second processing core work 156, and / or hardware block work 154. The second processing core work 156 may be executed by a second processing core 112, which will be described below.

[0067] The second processing core 112 can perform a second neural network operation. That is, the second processing core operation 156 can include a second neural network operation. For example, the second neural network operation can be a CNN operation, and the second processing core 112 can be a CNN IP core for processing CNN operations. In an embodiment, the second neural network operation can be an RNN operation, and the second processing core 112 can be an RNN IP core for processing RNN operations. Thus, in this specification, CNN operations and / or RNN operations have been described as examples of second neural network operations, but the scope of this disclosure is not limited thereto.

[0068] The second neural network operation can be a neural network operation of a different type than the first neural network operation. For example, when the first neural network operation is a CNN operation, the second neural network operation can be an RNN operation. As another example, when the first neural network operation is an RNN operation, the second neural network operation can be a CNN operation.

[0069] The work distributor 120 can distribute a portion of application work 150 into hardware block work 154. The work distributor 120 can distribute a portion of application work 150 into hardware block work 154 based on a first workload of the first processing core 110 and / or a second workload of the second processing core 112. For example, the work distributor 120 can receive a first workload from the first processing core and a second workload from the second processing core 112 via bus 140. Additionally, the work distributor 120 can distribute a portion of application work 150 into hardware block work 154 based on the received first and second workloads.

[0070] In some embodiments, the work distributor 120 may distribute the application work 150 input to the neural processing unit 102 to the first processing core 110, the second processing core 112, and / or the hardware block 130. For example, the work distributor 120 may distribute the first neural network operation as the first processing core operation 152, distribute the second neural network operation as the second processing core operation 156, and distribute the remaining operations in the application work 150 to the hardware block 130 as the hardware block operation 154.

[0071] In some embodiments, the work distributor 120 may distribute a portion of the first processing core job 152 and / or a portion of the second processing core job 156 as hardware block jobs 154 based on a first workload and / or a second workload. For example, the work distributor 120 may compare the amount of the first workload with the amount of the second workload to distribute a portion of the first processing core job 152 or a portion of the second processing core job 156 as hardware block jobs 154. In embodiments, the amount of each workload compared by the work distributor 120 includes at least one of the amount of operations and / or instructions included in the workload, and / or the data size of each workload. As an example, if the work distributor 120 determines that the amount of the first workload is greater than the amount of the second workload, the work distributor 120 may distribute a portion of the first processing core job 152 as hardware block jobs 154. In embodiments, a portion of a first neural network operation included in the first processing core job 152 may be distributed as hardware block jobs 154. As another example, if the workload distributor 120 determines that the amount of the first workload is less than the amount of the second workload, the workload distributor 120 may distribute a portion of the second processing core workload 156 as hardware block workload 154. In an embodiment, a portion of the second neural network operation included in the second processing core workload 156 may be distributed as hardware block workload 154.

[0072] In the following text, reference will be made to Figure 5 and Figure 9 This application describes neural processing units according to some embodiments. For ease of explanation, the description will primarily focus on... Figures 1 to 8 The differences between those mentioned. Figure 9 This is a block diagram of a neural processing unit according to some embodiments.

[0073] Reference Figure 9 The neural processing unit 104 according to some embodiments of this application may include a first processing core 110, a work distributor 120, a hardware block 130, a bus 140, and / or internal memory 160. According to some example embodiments, operations described herein as being performed by the internal memory 160 may be executed by at least one processor (e.g., the at least one first processor, the at least one second processor, and / or the at least one third processor) that executes program code including instructions corresponding to the operations. These instructions may be stored in the memory of the neural processing unit 100 (e.g., internal memory 160), the memory of the first processing core 110, and / or the memory of the hardware block 130.

[0074] Internal memory 160 may store function data used in neural network operations. For example, internal memory 160 may store function data used in first neural network operations and / or second neural network operations, including those in first processing core operation 152 and / or hardware block operation 154.

[0075] More specifically, the internal memory 160 can store Figure 5 The activation function data used in the activation function operation 220 includes the activation function key and value. The key is the input data (e.g., k) of the activation function, and the value can be the output data (e.g., f(k)) of the activation function. Therefore, by using the internal memory 160, pre-stored output data can be read instead of directly performing the activation function operation in the DNN operation. This effectively reduces the time spent processing DNN operations.

[0076] The internal memory 160 can store function data for multiple activation functions. For example, the internal memory 160 can simultaneously store function data for the tanh function and function data for the ReLU function. For example, when the Sigmoid function is used in the first neural network operation and the ReLU function is used in the second neural network operation, the internal memory 160 can store function data for the Sigmoid function and function data for the ReLU function.

[0077] In some embodiments, if hardware block 130 is reconfigured as a first hardware core for performing a first neural network operation, internal memory 160 may store function data of activation functions for the operation of the first neural network. Furthermore, if hardware block 130 is reconfigured as a second hardware core for performing a second neural network operation, internal memory 160 may store function data of activation functions for the operation of the second neural network.

[0078] In some embodiments, the internal memory 160 may update the stored function data based on the operations performed on the first processing core 110 and / or hardware block 130. Additionally, when hardware block 130 is reconfigured into a first hardware core or a second hardware core, the internal memory 160 may update the stored function data based on the operations performed in the first hardware core or the second hardware core. For example, if hardware block 130 is configured as a first hardware core, and the operation to be performed corresponds to the second hardware core, then the internal memory 160 may update the stored function data to reconfigure hardware block 130 into a second hardware core.

[0079] The internal memory 160 can delete currently stored function data and store new function data from memory located outside the neural processing unit 104 to update the stored function data. Additionally, the internal memory 160 can update the stored function data while the neural processing unit 104 is performing application tasks.

[0080] In some embodiments, internal memory 160 may be a CAM (content-addressable memory). In some embodiments, internal memory 160 may be a binary CAM or a TCAM (ternary CAM).

[0081] A conventional neural processing unit (NPU) includes one or more hardware structures (e.g., kernels) for performing one or more types of deep neural network (DNN) operations (e.g., CNN, RNN, and / or MAC). Conventional NPUs are constructed to minimize or reduce the amount of hardware structures included therein to reduce cost, resource consumption, and / or NPU size and / or complexity. However, if a conventional NPU receives an application job for processing (which involves a type of DNN operation not constructed for the conventional NPU), that DNN operation may not be executed (e.g., execution errors may occur), and / or it may be executed with high latency. Therefore, there is a trade-off between the higher cost, resource consumption, and / or NPU size and / or complexity resulting from constructing a conventional NPU for performing many types of DNN operations and the high latency and / or execution errors resulting from constructing a conventional NPU for performing fewer types of DNN operations. Furthermore, in cases where the received application workload includes numerous operations of the first type of DNN operation and a small number of operations of the second type of DNN operation, conventional NPUs with more than one hardware architecture for performing more than one type of DNN operation suffer from excessive processing latency and / or resource waste (e.g., inefficiency). For example, the processing time of the second processing core is wasted because while the first processing core, configured to perform the first type of DNN operation, remains overloaded performing numerous operations, the second processing core, configured to perform the second type of DNN operation, becomes idle after performing a small number of operations.

[0082] However, the example embodiment provides an improved NPU that includes reconfigurable hardware blocks (e.g., hardware block 130) capable of being reconfigured to perform different types of DNN operations. Therefore, the improved NPU includes less hardware architecture while enabling it to handle many types of DNN operations, thereby reducing cost, resource consumption, and / or NPU size and / or complexity, while also reducing latency and / or execution errors of conventional NPUs. Additionally, the improved NPU may include a job distributor 120 that distributes application work between the reconfigurable hardware block and one or more processing cores (e.g., first processing core 110), causing the reconfigurable hardware block to perform a portion of the workload of the one or more processing cores to minimize or reduce idle time among the reconfigurable hardware block and the one or more processing cores. Therefore, the improved NPU prevents or reduces excessive processing latency and / or resource waste (e.g., inefficiency) of conventional NPUs.

[0083] In the following text, reference will be made to Figure 5 , Figure 9 and Figure 10 This describes function data stored in internal memory according to some embodiments of this application. Figure 10 It is a diagram used to interpret function data. Figure 10 The case where the activation function is tanh is shown, but this disclosure is not limited thereto. Figure 10 In the diagram, the horizontal axis represents the input domain x of the function, and the vertical axis represents the output domain y of the function.

[0084] In some embodiments, internal memory 160 may store function data including a portion of an activation function. For example, the function data stored in internal memory 160 may include... Figure 10 The data area R shown D The function data. Data area R D This can be a portion of the tanh function. For example, the data region R. D It can be a region with a specific domain (e.g., -1.5 < x < 1.5). By storing function data that includes a portion of the activation function instead of the entire activation function data, the internal memory 160 can save resources (e.g., memory space).

[0085] In the following text, reference will be made to Figures 11 to 13 This application describes neural processing systems according to some embodiments. For ease of explanation, the description will primarily focus on... Figures 1 to 10 The differences described in the text. Figures 11 to 13 This is a block diagram of a neural processing system according to some embodiments.

[0086] Reference Figure 11 The neural processing system 200 according to some embodiments of this application may include a neural processing unit 104 and / or external memory 170. According to some example embodiments, operations described herein as being performed by external memory 170 may be executed by at least one processor (e.g., at least one first processor, at least one second processor, and / or at least one third processor) that executes program code including instructions corresponding to the operations. These instructions may be stored in the memory of the neural processing unit 100 (e.g., internal memory 160), the memory of the first processing core 110, and / or the memory of the hardware block 130.

[0087] External memory 170 may store metadata for reconstructing hardware block 130 of neural processing unit 104. For example, external memory 170 may include CNN reconstruction data for reconstructing hardware block 130 into CNN processing hardware core 132 and / or RNN reconstruction data for reconstructing hardware block 130 into RNN processing hardware core 134. Additionally, external memory 170 may include MAC reconstruction data for reconstructing hardware block 130 into MAC processing hardware core 136. For example, MAC reconstruction data may include reconstruction data for reconstructing hardware block 130 into MAC processing hardware core 136, including a lookup table for storing quantized weighted data.

[0088] External memory 170 may store all function data used in the first processing core 110 and / or hardware block 130. External memory 170 may provide the stored function data to neural processing unit 104 according to the operations performed in the first processing core 110 and / or hardware block 130. Function data provided from external memory 170 may be stored in internal memory 160. Function data stored in internal memory 160 may be updated using function data provided from external memory 170.

[0089] For example, refer to Figure 12 and Figure 13 The external memory 170 includes first metadata (metadata 1) for reconstructing the hardware block 130 into a first hardware core and second metadata (metadata 2) for reconstructing the hardware block 130 into a second hardware core. Additionally, the external memory 170 includes first activation function data f1(x) and second activation function data f2(x).

[0090] exist Figure 12 In this embodiment, external memory 170 can provide first metadata to neural processing unit 104. In this embodiment, the first metadata can be used to reconstruct hardware block 130 into a first hardware core. Additionally, external memory 170 can provide first activation function data to neural processing unit 104. In this embodiment, the first activation function data can be stored in internal memory 160.

[0091] exist Figure 13 In this embodiment, external memory 170 can provide second metadata to neural processing unit 104. In this embodiment, the second metadata can be used to reconstruct hardware block 130 into a second hardware core. Furthermore, external memory 170 can provide second activation function data to neural processing unit 104. In this embodiment, the second activation function data can be stored in internal memory 160.

[0092] External memory 170 can be a normally functioning memory device. For example, external memory 170 can be dynamic random access memory (DRAM) and / or flash memory. Alternatively, external memory can be (but is not limited to) an SSD (solid-state drive) device that includes flash memory.

[0093] In the following text, reference will be made to Figure 14 This application describes some embodiments of the application system. Figure 14 This is a block diagram of an application system according to some embodiments.

[0094] Reference Figure 14 According to some embodiments of this application, the application system 300 includes a software layer 310, a kernel layer 320, and / or a hardware layer 330. For ease of explanation, Figure 14 The application system 300 disclosed herein is shown to include a software layer 310, a kernel layer 320, and a hardware layer 330, but the embodiments are not limited thereto. Furthermore, the software layer 310, kernel layer 320, and / or hardware layer 330 may be conceptual layers used to interpret the operations of the application system 300.

[0095] In some embodiments, an application (APP) 312 is executed on software layer 310. For example, software layer 310 may instruct application operations executed in CPU 331, GPU 334 and / or neural processing unit 106 to be executed, as will be described later.

[0096] In some embodiments, hardware layer 330 may include neural processing unit 106, CPU 331, memory 332, DMA (direct memory access) 333 and / or GPU 334.

[0097] In some embodiments, the neural processing unit 106 may include a first processing core, a working distributor, a hardware block, and / or a bus. As an example, the neural processing unit 106 may be a reference... Figures 1 to 4 The neural processing unit 100 is described. As another example, neural processing unit 106 may be a reference. Figure 8 The neural processing unit 102 is described. As another example, neural processing unit 106 may be a reference... Figure 9 The neural processing unit 104 is described.

[0098] CPU 331 can perform operations to execute application 312. For example, when a user device requests execution of application 312, CPU 331 can perform a series of application operations for executing application 312. Additionally, CPU 331 can distribute a portion of the application operations to neural processing unit 106 and / or GPU 334. In some embodiments, CPU 331 distributes application operations including a first neural network operation to neural processing unit 106. Additionally, CPU 331 can distribute a portion of the application operations to GPU 334.

[0099] In some embodiments, the application may determine on which processor (e.g., CPU 331, GPU 334, and / or Neural Processing Unit 106) the application task is executed on at compile time. The CPU 331 may identify information about the application task from the compiled file and determine whether to distribute a specific application task to the NPU 106 and / or GPU 334.

[0100] In some embodiments, CPU 331 may be a shared central processing unit. For example, CPU 331 may consist of registers, a programming counter, an instruction register, an algorithm logic unit, a control unit, an internal bus, etc. For ease of explanation, a detailed description of CPU 331 will not be provided.

[0101] GPU 334 can execute application tasks distributed from CPU 331. For example, CPU 331 can distribute application tasks, including image processing, to GPU 334. GPU 334 executes the application tasks, including image processing, and can then pass the execution results back to CPU 331.

[0102] In some embodiments, GPU 334 may be a fully functional graphics processing device. For example, GPU 334 may include (but is not limited to) multiple ALUs (Algorithm Logic Units).

[0103] The memory 332 may store data for executing the application 312. For example, the memory 332 may store program code, library code, etc. for executing the application 312. Specifically, for example, when the application 312 is executed in the CPU 331, the memory 332 may retrieve the program code, library code, etc. for executing the application 312 from an external storage device such as an SSD device.

[0104] The program code may be code used to execute application 312. In some embodiments, the program code may be executed in CPU 331, GPU 334, and / or neural processing unit 106. For example, application work for executing application 312 may be stored in memory 332 in the form of program code.

[0105] The library code may include function data for executing application 312. For example, when the application is being executed in GPU 334 and / or neural processing unit 106, GPU 334 and / or neural processing unit 106 may call function data for executing the application from the library code stored in memory 332.

[0106] The memory 332 may store metadata for reconstructing hardware blocks of the neural processing unit 106. For example, the memory 332 may store reconstruction data for reconstructing hardware blocks into hardware cores for performing first neural network operations. Additionally, the memory 332 may store reconstruction data for reconstructing hardware blocks into hardware cores for processing RNN, CNN, and / or MAC operations.

[0107] In some embodiments, memory 332 may be a normally functioning memory device. For example, memory 332 may be (but is not limited to) DRAM.

[0108] DMA 333 can transfer data stored in memory 332 to GPU 334 or neural processing unit 106 without CPU 331 intervention. For example, when GPU 334 or neural processing unit 106 requests function data for performing application tasks, DMA 333 can transfer function data stored in memory 332 to GPU 334 and / or neural processing unit 106 without CPU 331. In some embodiments, DMA 333 can transfer metadata stored in memory 332 to neural processing unit 106.

[0109] DMA 333 can be a fully functional DMA. For example, DMA 333 can be implemented as (but not limited to) hardware including source registers, destination registers, counter registers, CPU control registers, and / or bus controllers.

[0110] Kernel layer 320 may include a refactoring loader 322. In some embodiments, the refactoring loader 322 may be connected to a DMA 333, CPU 331, and / or neural processing unit 106 included in hardware layer 330 to transfer metadata from memory 332 to neural processing unit 106. According to some example embodiments, operations described herein as being performed by the refactoring loader 322 may be executed by at least one processor (e.g., CPU 331 and / or at least one first processor) executing program code including instructions corresponding to a budget. The instructions may be stored in the memory of application system 300 and / or neural processing unit 106 (e.g., memory 332 and / or internal memory 160).

[0111] More specifically, the work distributor of the neural processing unit 106 can allocate a portion of the application work distributed from the CPU 331 to hardware block work. For example, the work distributor of the neural processing unit 106 can allocate a portion of the application work to hardware block work based on a first workload of the first processing core. The neural processing unit 106 can request metadata from the refactoring loader 322 for refactoring the hardware block into a hardware core for performing hardware block work. For example, the neural processing unit 106 can request metadata from the requesting refactoring loader 322 based on a first workload of the first processing core. The refactoring loader 322 can use DMA 333 to transfer the metadata stored in memory 332 to the neural processing unit 106. Furthermore, when DMA 333 cannot be used or is not desired, the refactoring loader 322 can use the CPU 331 to transfer the metadata stored in memory 332 to the neural processing unit 106.

[0112] In the following text, reference will be made to Figures 15 to 16 This application describes the application workflow executed in an application system according to some embodiments of this application. Figure 15 This is a block diagram illustrating an application workflow according to some embodiments. Figure 16 This is a block diagram illustrating data flowing in a memory according to some embodiments.

[0113] Reference Figure 15 The application can execute a first application task through an m-th application task to perform the application. In some embodiments, the first application task through the m-th application task can be executed by a CPU, a GPU, and / or a neural processing unit (NPU). For example, the first, m-2, and m-th application tasks can be executed in the CPU, the third and m-1 application tasks can be executed in the neural processing unit, and the second application task can be executed in the GPU.

[0114] Reference Figure 16 When the application is executed, program code and / or library code can be read from memory 332. In some embodiments, such as Figure 15 As shown, when executing the first to the m-th application tasks, CPU program code, GPU library code, and / or NPU library code can be read from memory 332. For example, when executing the first, m-2, and m-th application tasks in CPU 331, CPU 331 can read CPU program code from memory 332. Additionally, when executing the third and m-1-th application tasks by neural processing unit 106, DMA 333 and CPU 331 can read neural processing unit library code stored in memory 332 and transfer the neural processing unit library code to neural processing unit 106. Furthermore, when executing the second application task by GPU 334, DMA 333 and / or CPU 331 can read GPU library code stored in memory 332 and transfer the GPU library code to GPU 334.

[0115] The various operations described above can be performed by any suitable device capable of performing operations (such as various hardware and / or software implemented in some form of hardware, such as a processor).

[0116] The software may include an ordered list of executable instructions for implementing logical functions and may be implemented in any processor-readable medium for use by, or in conjunction with, an instruction execution system, device, or apparatus (such as a single-core or multi-core processor or a processor-containing system).

[0117] The blocks or operations of methods, algorithms, and functions described in conjunction with the exemplary embodiments disclosed herein can be implemented directly in hardware, in a software module executed by a processor, or a combination of both. If implemented in software, the functions can be stored as one or more instructions or code on a tangible, non-transitory computer-readable medium, or transmitted as one or more instructions or code. The software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0118] As a summary of specific embodiments, those skilled in the art should understand that many changes and modifications can be made to the exemplary embodiments without substantially departing from the principles of this disclosure. Therefore, the exemplary embodiments of this disclosure are used in a general and descriptive sense only and are not intended to be limiting.

Claims

1. A neural processing unit configured to perform application work including a first neural network operation and a second neural network operation different from the first neural network operation, the neural processing unit comprising: A first processing core is configured to perform the first neural network operation; A second processing core is configured to perform the second neural network operation; A hardware block that can be reconfigured into different types of hardware cores, the different types of hardware cores being configured to perform different types of operations, including first neural network operations and second neural network operations; as well as At least one processor is configured to execute computer-readable instructions to distribute a portion of the first neural network operation or a portion of the second neural network operation as hardware block work to a hardware block based on a comparison between a first workload of the first processing core and a second workload of the second processing core, such that the hardware block is reconfigured as a hardware core that performs the hardware block work.

2. The neural processing unit according to claim 1, wherein, The at least one processor is configured to execute computer-readable instructions to distribute a portion of the first neural network operation as hardware block work to the hardware block based on whether the first workload exceeds a certain value.

3. The neural processing unit according to claim 1, wherein, The at least one processor is configured to execute computer-readable instructions to reconstruct the hardware block into a convolutional neural network processing hardware core when the hardware block is operating including convolutional neural network operations.

4. The neural processing unit according to claim 3, wherein, Convolutional neural network operations include image recognition processing.

5. The neural processing unit according to claim 1, wherein, The at least one processor is configured to execute computer-readable instructions to reconstruct the hardware block into a recurrent neural network processing hardware core when the hardware block is operating including recurrent neural network operations.

6. The neural processing unit according to claim 5, wherein, Recurrent neural network operations include speech recognition processing.

7. The neural processing unit according to claim 1, wherein, The first neural network operation includes a first multiply-add operation, and The at least one processor is configured to execute computer-readable instructions to distribute a portion of the first multiply-accumulate operation as hardware block work to the hardware block based on the ratio of the first multiply-accumulate operation to the first workload.

8. The neural processing unit according to claim 1, wherein, The at least one processor is configured to execute computer-readable instructions to reconfigure the hardware block into a multiply-accumulate processing hardware core when the hardware block operates including multiply-accumulate operations, and The hardware block reconfigured into a multiply-accumulate processing hardware core is configured to perform multiply-accumulate operations using a lookup table that includes quantized weighted data.

9. The neural processing unit according to claim 8, wherein, The lookup table includes one or more result values ​​obtained based on the computational input data and the quantized weighted data.

10. The neural processing unit according to claim 1, wherein, The at least one processor is configured to execute computer-readable instructions, to When the amount of the first workload is greater than the amount of the second workload, the portion of the first neural network operation is distributed as the hardware block workload to the hardware block, and When the amount of the first workload is less than the amount of the second workload, the portion of the second neural network operation is distributed to the hardware block as the hardware block workload.

11. A neural processing unit, comprising: The first processing core is configured to perform operations of the first neural network; A second processing core is configured to perform operations of a second neural network that are different from those of the first neural network; The hardware block can be reconfigured as: A first hardware core, configured to perform operations on the first neural network, or The second hardware core is configured to perform operations on the second neural network; Internal memory storing function data for performing operations of the first neural network or the second neural network; and At least one processor is configured to execute computer-readable instructions to perform the following operations: Based on a comparison between the first workload of the first processing core and the second workload of the second processing core, a portion of the computation of the first neural network or a portion of the computation of the second neural network is distributed as hardware block workloads to the hardware block, thereby refactoring the hardware block into a hardware core that executes the hardware block workloads. The internal memory updates the function data stored in the internal memory based on the operations performed on the first or second hardware core.

12. The neural processing unit according to claim 11, further comprising: At least one processor configured to execute computer-readable instructions, to When the hardware block is reconstructed into the first hardware core, the function data used for the operations of the first neural network is stored in internal memory, and When the hardware block is reconstructed into a second hardware core, the function data used for the operations of the second neural network is stored in the internal memory.

13. The neural processing unit according to claim 11, wherein, The at least one processor is configured to execute computer-readable instructions to update the function data in the internal memory by causing the internal memory to delete function data stored in the internal memory and causing the internal memory to store new function data received from memory outside the neural processing unit.

14. The neural processing unit according to claim 11, wherein, The function data includes sigmoid function data, hyperbolic tangent tanh function data, or modified linear unit ReLU function data.

15. A neural processing system, comprising: The neural processing unit includes: The first processing core is configured to perform operations of the first neural network; A second processing core is configured to perform a second neural network operation that is different from that of the first neural network; A hardware block that can be reconfigured into a first hardware core performing operations on the first neural network or a second hardware core performing operations on the second neural network; and At least one processor is configured to execute computer-readable instructions to distribute a portion of the computation of the first neural network or a portion of the computation of the second neural network as hardware block work to the hardware block based on a comparison between a first workload of the first processing core and a second workload of the second processing core, such that the hardware block is reconfigured into a hardware core that performs the hardware block work; and External memory, which stores: The first reconstruction data can be used to reconstruct the hardware block into the first hardware core, or The second reconstruction data can be used to reconstruct the hardware block into the second hardware core.

16. The neural processing system according to claim 15, wherein, The operations of the first neural network include multiplication and addition, and The external memory stores third reconstruction data for reconstructing the hardware block into a multiply-accumulate processing hardware core. The third reconstruction data includes a lookup table that includes quantization weighting information.

17. The neural processing system according to claim 15, wherein, External memory stores function data used by the first processing core or hardware block, and The neural processing unit includes at least one processor configured to execute computer-readable instructions to obtain the function data from external memory based on operations performed in a first processing core or hardware block.

Images