A deep learning acceleration core and deep learning system-on-a-chip based thereon

Abstract

The application discloses a kind of deep learning acceleration core and based on its deep learning system on a chip, deep learning acceleration core is mainly by operation management unit, instruction control unit, memory is constituted;Operation management unit is equipped with data transmission arbitration selection logic unit and various operation units in;Memory will input data or weight data be transmitted to corresponding operation unit after data transmission arbitration selection logic unit, each operation unit will be written into memory after the operation result of data transmission arbitration selection logic unit;Instruction control unit is used to read instruction and complete instruction distribution, memory and operation management unit determine the execution time of current instruction according to the execution time length of previous instruction and the dependency relationship between the instruction and other instructions.Deep learning acceleration core is equipped with various operation units, can be according to the characteristics of specific calculation mode, utilize the multiplicity of data, design internal circuit, reduce the number of repeated import, export, reduce unnecessary energy consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and particularly to a deep learning acceleration core and a deep learning system on chip based on the same. BACKGROUND

[0002] Deep learning is a new artificial intelligence technology emerging in recent years and is also a hot spot of current research and innovation. Nowadays, deep learning technology has achieved great success in the fields of computer vision, speech recognition, natural language processing, etc., and its performance far exceeds that of traditional methods in these fields. However, the parameter quantity and the computation quantity of deep learning are extremely large, and the parameter quantity can reach the order of magnitude of hundreds of megabytes, and the computation quantity can reach the order of magnitude of tens to hundreds of GFLOPS. Such a large parameter quantity and computation quantity cannot be processed in real time on a traditional CPU (real-time processing requires at least 25 FPS or more, i.e., more than 25 inferences per second).

[0003] Edge-side applications not only have high requirements for the processing speed of the processor for deep learning model processing, but also have strong limitations on the energy consumption of the processor. Therefore, it is necessary to develop a new artificial intelligence processor. On the one hand, the new artificial intelligence processor needs to improve the computing power to meet the real-time requirements of deep learning model inference. On the other hand, the new artificial intelligence processor needs to improve the energy efficiency to meet the power consumption requirements of edge-side devices.

[0004] Traditional processors, such as CPUs and DSPs, generally adopt the Von Neumann or Harvard structure, and registers are provided in the processor. Data is transmitted to the registers under the control of instructions, and the data in the registers is read by the operation component under the control of instructions to perform operations. The feature of this type of architecture is flexibility. Each instruction performs a very basic operation (such as addition, subtraction, multiplication, and moving one or several data), and various instructions can be combined to achieve various functions, so that the processor can implement rich software and complete various functions under the control of software.

[0005] However, this type of architecture is not suitable for the field of artificial intelligence chips with high computing power. The reasons are as follows:

[0006] First, an artificial intelligence chip with high computing power must have many operation devices because the computing power is realized by specific operation devices (such as multipliers and multiply-accumulators). When a large number of operation devices are working, a large amount of data is needed. These data need to be moved from the storage data hardware location (registers or memory) to the operation devices in each working period. Traditional CPUs and DSPs use instructions to drive data and use instructions to control the transmission of the required data from the registers (or memory) to the operation devices. Obviously, when the computing power of the processor is improved to a high level, a large amount of data needs to be moved from the storage location to the operation unit, and this processing method is very inefficient.

[0007] Second, in the field of artificial intelligence computing, there is a phenomenon that a piece of data is used multiple times in one calculation, for example, in two-dimensional convolution calculation of multiple input channels and multiple output channels, the weights of the convolution kernel are repeatedly used, and the input feature map is also repeatedly used when calculating the convolution results of different output channels. If the multiplicity of this data is not considered, the data will be repeatedly imported from the storage data location (register or memory) to the operation unit, causing unnecessary energy consumption. CPU and DSP are designed for general-purpose computing, and their instructions and operation units are designed to perform basic computing operations (such as addition, subtraction, multiplication, and moving one or several data), and the data multiplicity of deep learning model training in this specific field is not considered. Therefore, using traditional CPU and DSP for deep learning model training will cause repeated data movement and unnecessary energy consumption. SUMMARY

[0008] In view of the problem that the instruction stream processor represented by the traditional CPU and DSP is not suitable for deep learning application scenarios with very high computing power and energy efficiency requirements, the application provides a deep learning acceleration core and a deep learning system on chip based on the same.

[0009] The application protects a deep learning acceleration core, which is mainly composed of an operation management unit, an instruction control unit and a memory.

[0010] The operation management unit is provided with a data transmission arbitration selection logic unit and various operation units, including but not limited to a matrix operation unit, a maximum pooling operation unit, a softmax operation unit, an activation function operation unit, a vector operation unit, a sparse operation unit and a binary neural network operation unit.

[0011] The memory is bidirectionally connected with the data transmission arbitration selection logic unit, and the data transmission arbitration selection logic unit is bidirectionally connected with each operation unit. The memory transmits input data or weight data to the corresponding operation unit through the data transmission arbitration selection logic unit, and each operation unit writes the operation result to the memory through the data transmission arbitration selection logic unit.

[0012] The instruction control unit is used for reading instructions and completing instruction distribution. The memory and the operation management unit determine the execution time of the current instruction according to the execution time of the previous instruction and the dependency relationship between the current instruction and other instructions. The dependency relationship between the current instruction and other instructions meets the sequential requirements of instruction execution.

[0013] Further, the memory adopts a ping-pang structure, that is, comprises a memory ping and a memory pang; the memory ping and the memory pang each comprise three storage arrays; when performing matrix multiplication, the three storage arrays are respectively used for storing left matrix data, right matrix data and result matrix data; when performing two-dimensional convolution calculation, the three storage arrays are respectively used for storing input feature map data, convolution kernel weight data and output feature map data.

[0014] Further, the dependency relationship between the instructions is realized through dependency relationship flag bits in the instructions and logical relationship therebetween, the flag bits comprise dependency flag bits and notification flag bits, the dependency flag bits comprise a dependency memory ping flag, a dependency memory pang flag and a dependency computing unit flag, the notification flag bits comprise a notification memory ping flag, a notification memory pang flag and a notification computing unit flag; if an instruction can be executed without depending on any instruction, the dependency flag bits of the instruction are all configured as 0; if the execution of an instruction does not affect any other instruction, the notification flag bits of the instruction are all configured as 0.

[0015] Further, if the dependency memory ping flag in the instruction received by the operation management unit is 1, the instruction must be executed after receiving the notification from the memory ping; the memory ping only sends the notification to the operation management unit when the notification computing unit flag in the execution being performed is 1 and the execution has been completed.

[0016] If the dependency memory pang flag in the instruction received by the operation management unit is 1, the instruction must be executed after receiving the notification from the memory pang; the memory pang only sends the notification to the operation management unit when the notification computing unit flag in the execution being performed is 1 and the execution has been completed.

[0017] If the dependency computing unit flag in the instruction received by the memory ping is 1, the instruction must be executed after receiving the notification from the operation management unit; the operation management unit only sends the notification to the memory ping when the notification memory ping flag in the execution being performed is 1 and the execution has been completed.

[0018] If the dependency computing unit flag in the instruction received by the memory pang is 1, the instruction must be executed after receiving the notification from the operation management unit; the operation management unit only sends the notification to the memory pang when the notification memory pang flag in the execution being performed is 1 and the execution has been completed.

[0019] Further, the operation management unit, the memory ping and the memory pang are each provided with an instruction queue and an instruction analysis logic unit; the instruction queue is used to receive instructions transmitted by the instruction control unit and transmit the instructions to the instruction analysis logic unit, and the instruction analysis logic unit is used to analyze the instructions and logically judge and control the dependency relationship between the instructions and other instructions.

[0020] The application also protects a deep learning system on a chip, which comprises a CPU, a storage system and the deep learning acceleration core described above; the CPU analyzes a deep learning model structure and a weight parameter file, and stores input data and weights in a position accessible to the deep learning acceleration core, then generates instructions according to the hardware characteristics and the deep learning model structure, and sends the instructions to the deep learning acceleration core, and then waits for an interruption feedback from the deep learning acceleration core after the end of instruction execution, and then performs subsequent processing.

[0021] The application has the following beneficial effects:

[0022] 1. The deep learning acceleration core is provided with various operation units, each of which is responsible for a common calculation in the field of deep learning model inference, can utilize the data reusability according to the characteristics of a specific calculation mode, design internal circuits, reduce the number of repeated data import and export, reduce unnecessary energy consumption, and improve the energy efficiency;

[0023] 2. The execution of each instruction is performed in the order of data dependency relationship through the inter-instruction dependency mechanism, and the execution of the instruction is temporarily suspended when the data is not ready or the data storage position is temporarily occupied;

[0024] 3. Data is directly transmitted from the memory to the calculation unit, meeting the data demand of a large-calorie artificial intelligence chip;

[0025] 4. The memory adopts a ping-pang structure, improving the parallelism of data transmission and calculation. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 The structural block diagram of the deep learning acceleration core disclosed in Embodiment 1 is shown in the figure;

[0027] Figure 2 The structural block diagram of the deep learning system on a chip disclosed in Embodiment 2 is shown in the figure. DETAILED DESCRIPTION

[0028] The application is further described in detail below with reference to the drawings and specific embodiments. The embodiments of the application are given for illustrative and descriptive purposes only and are not intended to be exhaustive or to limit the application to the forms disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. Embodiments were chosen and described in order to best explain the principles of the application and its practical application, and to enable others skilled in the art to understand the application for various embodiments with various modifications as are suited to the particular use contemplated.

[0029] Embodiment 1

[0030] A deep learning acceleration core mainly consists of an operation management unit, an instruction control unit, a memory ping, and a memory pang, as shown in Figure 1 The dashed line represents a control information transmission line, and the solid line represents a data information transmission line.

[0031] The operation management unit is provided with a data transmission arbitration selection logic unit and various operation units, including but not limited to a matrix operation unit, a max pooling operation unit, a softmax operation unit, an activation function operation unit, a vector operation unit, a sparse operation unit, and a binary neural network operation unit, i.e. each operation unit is responsible for a common calculation in the field of deep learning model inference. The advantage of such a setting is that each operation unit can utilize the data multiplexing according to the characteristics of a specific calculation mode, design internal circuits, reduce the number of repeated data imports and exports, reduce unnecessary energy consumption, and improve energy efficiency. The internal circuit structure of each operation unit belongs to the prior art and is not the core issue of the present application, which will not be described here.

[0032] The memory ping and the memory pang are bidirectionally connected with the data transmission arbitration selection logic unit, the data transmission arbitration selection logic unit is bidirectionally connected with each operation unit, the memory ping and the memory pang transmit input data to the corresponding operation unit through the data transmission arbitration selection logic unit, and each operation unit writes the operation result to the memory ping and the memory pang through the data transmission arbitration selection logic unit.

[0033] The main purpose of the ping-pang structure of the memory is to improve the parallelism of data transmission and calculation. When the memory ping is used to provide data to the operation management unit and receive the operation result of the operation management unit, the memory pang can load new data from a certain position outside the deep learning acceleration or write the calculation result to the external memory; similarly, when the memory pang is used to provide data to the operation management unit and receive the operation result of the operation management unit, the memory ping can load new data from a certain position outside the deep learning acceleration or write the operation result to the external memory.

[0034] The instruction control unit is used to read instructions and complete instruction distribution, and the execution time of the current instruction is determined according to the execution time of the previous instruction and the dependency relationship between the instruction and other instructions, and the dependency relationship between the instruction and other instructions meets the sequential requirement of instruction execution.

[0035] The necessity of setting such a dependency relationship between instructions is that the execution of instructions has a certain sequential requirement, for example, a calculation instruction needs to read data in the memory ping, then the instruction must wait until the execution of the instruction for loading data into the memory ping is completed, and the execution time of the instruction for loading data into the memory ping is not fixed (determined by the size of the loaded data); for another example, a calculation instruction needs to read data in the memory ping, then the memory ping should not execute a new data loading instruction during the execution of the instruction, so as to avoid covering the data used by the calculation unit in the memory ping, and the execution time of the calculation instruction is not fixed (determined by the mode of calculation and the length of data). Therefore, through such a dependency mechanism between instructions, the normal execution of instructions can be ensured.

[0036] In the embodiment, the dependency relationship between the instruction and other instructions is realized through the dependency flag bit in the instruction and the logical relationship therebetween, and the flag bit includes a dependency flag bit and a notification flag bit.

[0037] If an instruction can be executed without depending on any instruction, the dependency flag bit of the instruction is configured as 0; if the execution of an instruction does not affect any other instruction, the notification flag bit of the instruction is configured as 0.

[0038] Specifically, in the embodiment, the dependency flag bit includes a dependency memory ping flag, a dependency memory pang flag and a dependency calculation unit flag, the notification flag bit includes a notification memory ping flag, a notification memory pang flag and a notification calculation unit flag, and the flag bit and the logical relationship therebetween are further limited as follows:

[0039] 1、If the instruction received by the operation management unit has the memory ping flag = 1, the instruction must be executed after receiving the notification from the memory ping, and the memory ping will send a notification to the operation management unit only when the instruction being executed has notified the calculation unit flag = 1 and has been executed.

[0040] 2、If the instruction received by the operation management unit has the memory pang flag = 1, the instruction must be executed after receiving the notification from the memory pang, and the memory pang will send a notification to the operation management unit only when the instruction being executed has notified the calculation unit flag = 1 and has been executed.

[0041] 3、If the instruction received by the memory ping has the calculation unit flag = 1, the instruction must be executed after receiving the notification from the operation management unit, and the operation management unit will send a notification to the memory ping only when the instruction being executed has notified the memory ping flag = 1 and has been executed.

[0042] 4、If the instruction received by the memory pang has the calculation unit flag = 1, the instruction must be executed after receiving the notification from the operation management unit, and the operation management unit will send a notification to the memory pang only when the instruction being executed has notified the memory pang flag = 1 and has been executed.

[0043] Based on the above deep learning acceleration core architecture, the deep learning acceleration core instruction is different from the traditional CPU and DSP instructions in the following two points:

[0044] 1、The deep learning acceleration core proposed in the application does not contain general-purpose registers, so the instruction will not specify the operation on the specific register, but only specify the storage position of the data in the memory.

[0045] 2、The execution timing and execution order of the instruction are not fixed, the instruction control unit reads the instruction and distributes it to other modules, i.e. the operation management unit, the memory ping and the memory pang; the execution time of these modules cannot be directly determined, but is determined by the execution time of the previous instruction and the dependency relationship between the instruction and other instructions.

[0046] The execution time of each instruction is not fixed, but is determined by the operation complexity of the instruction. For example, if the instruction is a matrix multiplication instruction, the execution time of the instruction is proportional to the size of the matrix; if the instruction is a two-dimensional convolution instruction, the execution time of the instruction is proportional to the number of input channels, the number of output channels, the height of the input feature map and the width of the input feature map, which may be 100 cycles or 100,000 cycles.

[0047] The main operations involved in deep learning are matrix multiplication and two-dimensional convolution. Therefore, in this embodiment, both the memory ping and the memory pang contain three storage arrays (physical partitions).

[0048] When performing matrix multiplication, the three storage arrays are used to store the left matrix data, the right matrix data, and the result matrix data, respectively; when performing two-dimensional convolution calculation, the three storage arrays are used to store the input feature map data, the convolution kernel weight data, and the output feature map data, respectively.

[0049] Specifically, in this embodiment, the arithmetic management unit, the memory ping, and the memory pang are all equipped with instruction queues and instruction parsing logic units (not shown in the block diagram of memory ping and memory pang).

[0050] The instruction queue is used to receive instructions transmitted from the instruction control unit and transmit them to the instruction parsing logic unit. The instruction parsing logic unit is used to parse the instructions and perform logical judgment and control on the dependency relationship between the instructions and other instructions, and transmit them to the data transmission arbitration selection logic unit.

[0051] The data transmission arbitration selection logic unit receives data transmitted from the memory ping and memory pang, and, based on the currently executing instruction, arbitrates and selects which unit to transmit the data to to complete the corresponding operation.

[0052] After the arithmetic unit completes the corresponding calculation, it transmits the calculation result to the data transmission arbitration selection logic unit, which then transmits the calculation result to the memory block (i.e., storage location) specified by the current instruction.

[0053] also, Figure 1 DMAC (Direct Memory Access Controller) refers to the direct memory access controller, which is usually used to complete data transfer functions; AXI Master and AHB Slave are bus protocols, but in actual applications, they are not limited to these on-chip bus protocols and other protocols can also be used.

[0054] Example 2

[0055] A deep learning on-chip system, such as Figure 2 As shown, it includes a CPU, a storage system, and a deep learning acceleration core disclosed in Embodiment 1.

[0056] The deep learning acceleration core disclosed in Example 1 cannot work independently and needs to work with a CPU to complete the inference of the deep learning model.

[0057] The CPU parses the deep learning model structure and weight parameter file, and stores the input data and weights in a location accessible to the deep learning acceleration core. Then, according to the hardware characteristics and the deep learning model structure, the CPU generates instructions and sends them to the deep learning acceleration core. After the end of the instruction execution, the CPU waits for the feedback interrupt from the deep learning acceleration core, and then performs subsequent processing.

[0058] Since the instructions can complete the inference of a complete deep learning model, they can also complete the inference of part of the layers. When the inference of part of the layers is completed, the deep learning acceleration core generates an interrupt. Then, the CPU generates instructions for executing the subsequent layers according to the deep learning model structure, sends the instructions to the deep learning acceleration core, and completes the inference of the subsequent layers.

[0059] The deep learning acceleration core obtains the input data and weights from the memory according to the specified location in the instructions, performs operations, generates operation results, and writes the operation results back to the memory.

[0060] The storage system accessible to both the CPU and the deep learning acceleration core can be an off-chip DDR or an on-chip memory. The storage system is an intermediate medium for transmitting data between the CPU and the deep learning acceleration core, and the data is not directly transmitted from the CPU to the deep learning acceleration core.

[0061] Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art and related fields without creative labor should fall within the scope of protection of the present application.

Claims

1. A deep learning acceleration core, comprising: The operation management unit, the instruction control unit and the memory are mainly composed, the memory adopts ping-pang structure, namely contains memory ping and memory pang; The operation management unit is equipped with data transmission arbitration selection logic unit and various operation units, and the various operation units include but are not limited to matrix operation unit, maximum pooling operation unit, softmax operation unit, activation function operation unit, vector operation unit, sparse operation unit and binary neural network operation unit; The memory is bidirectionally connected with the data transmission arbitration selection logic unit, the data transmission arbitration selection logic unit is bidirectionally connected with each operation unit, the memory transmits input data or weight data to corresponding operation unit through the data transmission arbitration selection logic unit, and each operation unit writes operation result into the memory through the data transmission arbitration selection logic unit; The instruction control unit is used for reading instruction and completing instruction distribution, the memory and the operation management unit determine the execution time of the current instruction according to the execution time length of the previous instruction and the dependency relationship between the instruction and other instructions, and the dependency relationship between the instruction and other instructions meets the order requirement of instruction execution, Wherein, the dependency relationship between the instruction and other instructions is realized through the dependency relationship flag bit in the instruction and the logical relationship therebetween, the flag bit includes dependency flag bit and notification flag bit, the dependency flag bit includes dependency memory ping flag, dependency memory pang flag and dependency calculation unit flag, and the notification flag bit includes notification memory ping flag, notification memory pang flag and notification calculation unit flag; If an instruction can be executed without depending on any instruction, the dependency flag bit of the instruction is all configured as 0, and if the execution of an instruction does not affect any other instruction, the notification flag bit of the instruction is all configured as 0.

2. The deep learning acceleration core of claim 1, wherein, The memory ping and the memory pang all contain three storage arrays; When performing matrix multiplication, the three storage arrays are respectively used for storing left matrix data, right matrix data and result matrix data; When performing two-dimensional convolution calculation, the three storage arrays are respectively used for storing input feature map data, convolution kernel weight data and output feature map data.

3. The deep learning acceleration core according to claim 1, wherein If the dependency memory ping flag in the instruction received by the operation management unit is 1, the instruction must be executed after receiving the notification from the memory ping, and the memory ping sends the notification to the operation management unit only when the notification calculation unit flag in the instruction being executed is 1 and the instruction has been executed; If the dependency memory pang flag in the instruction received by the operation management unit is 1, the instruction must be executed after receiving the notification from the memory pang, and the memory pang sends the notification to the operation management unit only when the notification calculation unit flag in the instruction being executed is 1 and the instruction has been executed. If the instruction received by the memory ping has the dependency on the calculation unit flag = 1, the instruction must receive the notification from the operation management unit before execution, and the operation management unit will only send the notification to the memory ping when the instruction it is executing has the memory ping flag = 1 and has been executed; If the instruction received by the memory pang has the dependency on the calculation unit flag = 1, the instruction must receive the notification from the operation management unit before execution, and the operation management unit will only send the notification to the memory pang when the instruction it is executing has the memory pang flag = 1 and has been executed.

4. The deep learning acceleration core of claim 3, wherein, The operation management unit, the memory ping and the memory pang are each provided with an instruction queue and an instruction parsing logic unit; The instruction queue is used to receive the instructions transmitted by the instruction control unit and transmit them to the instruction parsing logic unit, and the instruction parsing logic unit is used for instruction parsing and logical judgment and control of the dependency relationship between the instructions.

5. A deep learning system on chip, comprising: The deep learning acceleration core comprises a CPU, a storage system and any one of the deep learning acceleration cores according to claims 1-4. The CPU parses a deep learning model structure and a weight parameter file, stores input data and weights in a position accessible by the deep learning acceleration core, generates instructions according to hardware characteristics and the deep learning model structure, and sends the instructions to the deep learning acceleration core, then waits for an interrupt feedback from the deep learning acceleration core after the execution of the instructions is completed, and then performs subsequent processing.

Images