General neural network calculation unit

Abstract

The invention belongs to the technical field of neural network acceleration design, and provides a three-dimensional circulation expansion general neural network calculation unit. The computing unit comprises a central control module, a first-stage buffer area, a first-stage buffer area controller, a zero-stage buffer area, a zero-stage buffer area controller and a core computing unit; the first-level buffer area comprises an input data first-level buffer area, a weight data first-level buffer area, a quantized weight data buffer area, a partial sum data buffer area and an output data buffer area; the zero-level buffer area comprises an input zero-level data buffer area and a weight data zero-level buffer area; the zero-level buffer controller comprises an input and weight data zero-level read-write control module; an input and weight data first-level read-write control module of the first-level buffer area read-write controller is respectively connected with an input and weight data zero-level read-write control module through an input and weight data zero-level buffer area and then connected into a core computing unit. And the calculation unit performs parallel calculation on three cycle dimensions at the same time, so that the calculation speed is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of neural network acceleration and processing unit design, and proposes a universal neural network computing unit. Background Art

[0002] In recent years, artificial intelligence technology has developed rapidly, especially in the field of neural networks. Convolutional neural networks (CNN) and Transformer have become core technologies in fields such as computer vision and natural language processing. CNN is popular for its excellent image processing capabilities, while Transformer has attracted much attention for its superior performance in processing sequence data. The rapid iteration and optimization of these technologies enable them to effectively handle increasingly complex tasks.

[0003] As a single network structure is unable to meet the increasingly complex application requirements, hybrid neural networks have emerged. This type of network structure combines a variety of different algorithms and technologies, such as combining the image recognition capabilities of CNN with the sequence processing capabilities of Transformer, in order to achieve better performance. In these hybrid networks, different types of operators (such as convolution operators, matrix multiplication operators, etc.) are integrated into a single network architecture to handle different data types and tasks.

[0004] Although hybrid neural networks can provide excellent performance in theory, their implementation in reality faces multiple technical challenges. Traditional neural network computing units are mostly designed to optimize specific types of operators, such as convolution operators, and are not efficient enough to handle diverse computing tasks. Especially in networks containing Transformer structures, due to their complex computational graphs and high data density, higher requirements are placed on the performance and flexibility of computing units. This limitation leads to low computational efficiency and difficulties in data scheduling.

[0005] In response to the above problems, the present application is dedicated to proposing a neural network computing unit that is adaptable to multiple neural network operators. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of low computing efficiency and difficult data scheduling of convolution-oriented neural network computing units caused by the widespread application of Transformer, and proposes a universal neural network computing unit. The unit introduces a zero-level buffer and a zero-level buffer controller after the first-level read-write control module of input data and the first-level read-write control module of weight data, respectively, to obtain an improved computing unit; the improved computing unit buffers and performs read-write control on the input data and weight data again, respectively, which can not only process different types of operators, but also simplify data paths and control logic, thereby improving the overall computing efficiency and flexibility; the computing unit also solves the mismatch problem between the global buffer and the computing unit in bandwidth and data format, thereby realizing the unification of read-write control logic between different operators.

[0007] In order to achieve the above object, the present invention adopts the following technical solution.

[0008] A general neural network computing unit is connected to an external memory and its controller, and includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller and a core computing unit;

[0009] The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer;

[0010] The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module;

[0011] The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer;

[0012] The zero-level buffer controller includes an input data zero-level read-write control module and a weight data zero-level read-write control module;

[0013] The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller and the output data read-write control module respectively.

[0014] The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

[0015] Part of the first-level buffer controller and the data read-write control module, the quantization weight data read-write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

[0016] The core computing unit is respectively connected to the partial data read-write control module, the quantization weight data read-write control module, the input data zero-level read-write control module, the weight data zero-level read-write control module and the output data read-write control module.

[0017] The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

[0018] The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

[0019] The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data that may be generated during the neural network calculation process, and the output data generated after the neural network calculation; the input data first-level buffer is used to store the input feature map data, the weight data first-level buffer is used to store the weight data, the quantized weight data buffer is used to store the quantized weight data, the partial sum data buffer is used to store the partial sum data, and the output data buffer is used to store the output data.

[0020] The first-level buffer controller receives instructions and adjusts the read-write control logic in real time to schedule computing resources; the first-level read-write control module for input data is responsible for data rearrangement and partial zero-padding of input data; the first-level read-write control module for weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit; the output data read-write control module is responsible for data rearrangement of possible output data; the partial sum data control read-write control module and the quantized weight data read-write module are responsible for direct reading of partial sum data and quantized weight data.

[0021] The zero-level buffer receives instructions and caches input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; reads and writes the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

[0022] Beneficial Effects

[0023] The present invention proposes a universal neural network computing unit, which has the following beneficial effects compared with the existing neural network accelerator:

[0024] 1. The computing unit relies on a multi-level data storage and access mechanism, fully utilizing the data locality and data arrangement characteristics in the neural network computing process, decoupling the data reading logic and rearrangement logic, and solving the problem of inconsistent data bandwidth and data format between the storage subsystem and the computing subsystem when facing different neural network operators;

[0025] 2. The computing unit performs parallel computing in three loop dimensions at the same time, making full use of the data characteristics of the input data in the neural network as high-dimensional tensors, thereby increasing the upper limit of the computing speed;

[0026] 3. The computing unit adopts a unified data flow control and transmission mechanism, designs a specific buffer structure, realizes flexible reading and writing of upstream and downstream data in the buffer, and improves computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the composition and connection relationship of a general neural network computing unit of the present invention;

[0028] Figure 2 The composition and structural block diagram of the core computing unit in a general neural network computing unit of the present invention;

[0029] Figure 3 The internal structure of a zero-level buffer in a universal neural network computing unit of the present invention. DETAILED DESCRIPTION

[0030] The deployment, practical application process and advantages and beneficial effects of a general neural network computing unit of the present invention are described in detail below in conjunction with the accompanying drawings and embodiments.

[0031] Example 1

[0032] This embodiment describes the specific implementation of the general neural network computing unit.

[0033] The general neural network computing unit is connected to the external memory and its controller, and includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller and a core computing unit;

[0034] The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer;

[0035] The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module;

[0036] The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer;

[0037] The zero-level buffer controller includes an input data zero-level read-write control module and a weight data zero-level read-write control module;

[0038] The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller and the output data read-write control module respectively.

[0039] The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

[0040] Part of the first-level buffer controller and the data read-write control module, the quantization weight data read-write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

[0041] The core computing unit is respectively connected to the partial data read-write control module, the quantization weight data read-write control module, the input data zero-level read-write control module, the weight data zero-level read-write control module and the output data read-write control module.

[0042] The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

[0043] The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

[0044] The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data that may be generated during the neural network calculation process, and the output data generated after the neural network calculation; the input data first-level buffer is used to store the input feature map data, the weight data first-level buffer is used to store the weight data, the quantized weight data buffer is used to store the quantized weight data, the partial sum data buffer is used to store the partial sum data, and the output data buffer is used to store the output data.

[0045] The first-level buffer controller receives instructions and adjusts the read-write control logic in real time to schedule computing resources; the first-level read-write control module for input data is responsible for data rearrangement and partial zero-padding of input data; the first-level read-write control module for weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit; the output data read-write control module is responsible for data rearrangement of possible output data; the partial sum data control read-write control module and the quantized weight data read-write module are responsible for direct reading of partial sum data and quantized weight data.

[0046] The zero-level buffer receives instructions and caches input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; reads and writes the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

[0047] In specific implementation, in traditional neural network accelerators, when input data is transmitted between the computing subsystem and the storage subsystem, the following two solutions are often adopted:

[0048] Solution 1. Read input data, weight data, quantized weight data, and partial sum data directly from the global buffer and send them to the computing unit, and write the output results to the output data buffer. Because the input data buffer, weight data buffer, quantized weight data buffer, partial sum data buffer, and output data buffer are all directly connected to the core computing unit, this method can minimize the complexity of the system structure.

[0049] However, this connection method has two major disadvantages: on the one hand, due to the lack of multi-level buffering, the format of the buffer data must be exactly the same as the format in the core computing unit, that is, the rearrangement of the data needs to be implemented outside the computing unit; on the other hand, due to the lack of a flow control mechanism, the bandwidth of the input data must be exactly the same as the core computing, which seriously limits the flexibility of the system.

[0050] Solution 2. Set a zero-level buffer between the input data and the weight data, but the data format in the zero-level buffer is required to be consistent with the format required by the core computing unit. At this time, the core computing unit's behavior from the quantized weight data buffer, partial sum data buffer, and output data buffer remains unchanged, but the input data and weight data will be read out by the first-level buffer controller, written into the zero-level buffer, and then sent to the core computing unit by the zero-level buffer.

[0051] Because of the introduction of the zero-level buffer, this method no longer requires the data format and read-write bandwidth in the first-level buffer to match the data format and read-write bandwidth in the core computing unit, which reduces the complexity of the data reading logic to a certain extent, but the zero-level buffer still has the same restrictions and requirements. The hardware structure corresponding to this zero-level buffer is often a register stack or FIFO with a specific structure, which often leads to a decrease in the versatility of the computing unit.

[0052] Combining the above two structures, the computing unit described in the present invention introduces a zero-level buffer and a zero-level buffer read-write controller. Similar to Solution 2, the introduced zero-level buffer can reduce the complexity of the data reading logic, but has the following two advantages: it does not require the data read-write bandwidth of the input zero-level buffer to be consistent with the read-write bandwidth of the core computing unit, nor does it require the data format of the input zero-level buffer to be consistent with the data format of the core computing unit. Instead, the zero-level buffer read-write controller reads the data from the zero-level buffer, further completes the data rearrangement, and then sends it to the core computing unit.

[0053] Therefore, this structure only requires that the output data format and data bandwidth of the zero-level buffer controller match those of the core computing unit. This leaves a large margin for the data reading logic and reordering logic, so that data can be read and reordered step by step in blocks during the flow of data from the first-level buffer to the zero-level buffer and then to the core computing unit.

[0054] The computing unit proposed in the present invention relies on a multi-level data storage and access mechanism, fully utilizes the data locality and data arrangement characteristics in the neural network calculation process, decouples data reading logic and re-arrangement logic, and solves the problem of inconsistent data bandwidth and data format between the storage subsystem and the computing subsystem when facing different neural network operators.

[0055] Taking the convolution operation as an example, assuming that the convolution kernel size is 3×3, the input feature map is 8×60×60, the output feature map is 1×60×60, the unit is Byte, and the computing unit parallelism is 8×8×8. When using solution 1, the input data buffer size is required to be 259.2KB; when using solution 2, the first-level input buffer size is required to be 86.4KB; when using the solution in the computing unit, the input data buffer size is required to be 28.8KB. It can be seen that the computing unit greatly reduces the required size of the on-chip buffer.

[0056] In specific implementation, the input data level 1 buffer read-write control module reads data from the input data level 1 buffer, and after zero padding and transposition, it becomes 8×64×60, with one row of zero padding on the top and three rows of zero padding on the bottom, and is sent to the zero-level buffer; the input data level 0 buffer read-write control module reads data from the zero-level buffer, rearranges the data to 8×64×62, and writes the rearranged data into the core computing unit. Therefore, the zero padding requirements of the convolution calculation and the zero padding requirements caused by loop unrolling are uniformly divided into two-stage data rearrangement logic, and the reading logic and rearrangement logic are split into two equal parts.

[0057] Through the two-stage data rearrangement and zero-filling operation, the original complex zero-filling logic is decoupled, the combinational logic level required for zero-filling operation is reduced by 50%, and the upper limit of the system's operating frequency is increased. Through the above structure, a total of 9 times the on-chip buffer size can be saved, and the number of times data is read from the global buffer can be reduced by 9 times, reaching the theoretical upper limit.

[0058] With regard to beneficial effect 2, the calculation unit proposed in the present invention has the following advantages:

[0059] When the computing unit accelerates the convolution operation, it is realized by three-dimensional loop expansion; that is, parallel calculations are performed on three loop dimensions at the same time, which fully utilizes the data characteristics of the input data in the neural network as a high-dimensional tensor, and improves the upper limit of the calculation speed; when the computing unit is specifically implemented, it can be compatible with convolution calculations and Transformer matrix multiplication calculations at the same time, unifying the calculation logic of the neural network accelerator, and is configurable while implementing high-performance computing, and can be flexibly deployed in any application scenario with a given resource amount and size;

[0060] In order to simplify the control signal and data path as much as possible, traditional neural network accelerators often use two-dimensional loop expansion. However, it is difficult for neural network accelerators corresponding to two-dimensional loop expansion to maintain relatively simple data reuse while expanding the scale.

[0061] The following introduces the specific implementation of the general neural network computing unit proposed in this application by relying on three common neural networks:

[0062] In a convolutional neural network accelerator, the number of dimensions of loop expansion directly determines the scale of the convolutional neural network accelerator. The calculation formula of the known convolution is as follows:

[0063]

[0064] In the above formula, the convolution operation is divided into four dimensions: convolution kernel (LOOP1), input channel (LOOP2), output feature map (LOOP3) and output channel (LOOP4), with a total of six loops.

[0065] Unrolling different loops will correspond to different core computing unit structures. The following introduces several common two-dimensional loop unrolling calculation methods and their corresponding core computing unit structures.

[0066] Neural network 1: Parallel calculations are performed on the convolution kernel (LOOP1) and the input channel (LOOP2). This loop expansion method has the simplest data path. The use of row buffering combined with sliding window calculation can simplify the control logic to the greatest extent, and has very strong data reuse. Although this method has a relatively simple control path and data path, due to the parallel calculations performed on the convolution kernel dimension, on the one hand, the computing unit can only calculate convolutions with fixed convolution kernel sizes, and it is difficult to adapt to convolutions with different convolution kernel sizes and different step lengths; on the other hand, it also limits the number of multipliers in the convolution kernel dimension to match the scale of the convolution. Its common parallelism ranges from 9×16 to 9×64 or 9×256, with a scale ranging from tens to hundreds.

[0067] Neural network 2: Parallel calculations are performed on the input channel (LOOP2) and the output channel (LOOP4). Because the coupling between the input channel and the output channel is low, this expansion method can obtain relatively simple control logic while being compatible with convolutions of different kernel sizes and different step lengths. In general, the input channel of the neural network can achieve a computing parallelism from 16×32 to 128×256 or even greater without wasting the input channel. Although this expansion method has a higher upper limit on the computing speed, when facing a smaller end-side neural network model, its input channel and output channel may be small, making it difficult to achieve high hardware utilization.

[0068] Neural network 3: Parallel calculations are performed on the output feature map (LOOP3) and the output channel (LOOP4). Therefore, the output feature map is often the largest in terms of value compared to the other three dimensions. Even with a small feature map of 28×28, it is possible to ensure that hundreds of points on the output channel can be calculated in parallel without worrying about low hardware utilization. Since the coupling between the output feature map and the output channel is also low, if the double loops in the output feature map are calculated in parallel at the same time, it is very easy to achieve a very large scale of 32×32×256 while maintaining high hardware utilization. Although this expansion method has very ideal hardware utilization and computing speed, because the loop is expanded on the output feature map (LOOP3), the accelerator often has extremely complex data paths and control logic, making it difficult to adapt to convolutions of different kernel sizes and different step lengths while ensuring data reusability.

[0069] The computing unit chooses to perform parallel computing in three dimensions: input channel (LOOP2), output channel (LOOP4) and output feature map (LOOP3). Through this expansion method and the corresponding hardware structure, it is easy to obtain parallelism from 8×8×8 to 16×16×16 or even 32×32×32. Compared with the several more common convolutional neural networks mentioned above, it has a higher performance ceiling than the first two, and compared with the third, although it has the same performance ceiling, it has a simpler data path and better adaptability.

[0070] In addition to convolutional neural networks, the core computing unit can also support matrix multiplication calculations in Transformer. Matrix multiplication in Transformer neural networks can be roughly divided into two categories: one is the projection layer of the form [B,M,N] and [1,N,K]; the other is the activation layer of the form [B,M,N] and [B,N,M]. The first type of matrix multiplication can be regarded as broadcasting [1,N,K] in the second type of matrix multiplication to [B,N,M]. At this time, the matrix multiplication in Transformer can be uniformly referred to as batch matrix multiplication. The calculation formula of the known convolution is as follows:

[0071]

[0072] Referring to the calculation order of the convolutional neural network, the matrix multiplication can be very well mapped to the entire computing unit, that is, the three dimensions of M, N, and K are expanded at the same time. At this time, the calculation logic is exactly the same as the convolutional neural network, which achieves compatibility with convolution calculations and Transformer matrix multiplication and unifies the calculation logic.

[0073] In order to improve the configurability of the computing unit and flexibly deploy it in any application scenario with a given amount of resources and size, the computing unit provides any parallelism from 2×2×2 to 8×8×8. When targeting lightweight neural networks and embedded platforms with low real-time requirements, the small version with a total parallelism of 8 can be used; when targeting lightweight neural networks and embedded platforms with high real-time requirements, the medium version with a total parallelism of 64 can be used; when targeting neural networks with large amounts of computation and resources, the large version with a parallelism of 512 can be used. The hardware structure of the core computing unit corresponding to different degrees of parallelism is introduced in detail below.

[0074] The core computing units in traditional neural network accelerators have different structures depending on their computing modes, but the overall architecture is relatively fixed, that is, they include three core components: multipliers, adder trees and accumulators. If a systolic array is used, it can be simplified to a multiplier array, an accumulator array and a quantization array. The core computing unit adds a broadcast control module based on the traditional method, and controls the input data to be broadcast in different forms according to different computing modes through instructions. In this way, the input data re-arrangement of multiple neural network computing units can be achieved.

[0075] On this basis, the core computing unit does not choose to use the systolic array to implement matrix operations. The idea is to regard multipliers and adders as different types of computing resources for unified scheduling, which is conducive to more flexible scheduling and planning of the overall computing resources in the future, and thus to achieve more flexible grouping and reduction operations. The specific workflow is illustrated by an example below.

[0076] like Figure 2 The core computing unit shown includes a broadcast control module, a multiplier array, an adder array, an accumulator array, and a quantization module, taking a parallelism of 4×4×4 as an example. At this time, the broadcast control module is responsible for broadcasting the input 4×4 input data and weight data to 4×4×4. At this time, the number of multipliers is 64. The broadcasted input data and weight data are sent to the multiplier array and multiplied two by two. The output results are sent to the adder tree array. The adder number array contains 4×4 groups of 4-2-1 adder trees. The 64 multiplication results are grouped and reduced to 16 output values, which are sent to the 16 accumulator arrays for accumulation. The accumulator array outputs 16 accumulator results every time a number of clock cycles are accumulated according to the configuration information, and sent to the quantization module to complete the quantization operation, and finally output 16 data.

[0077] When different parallelism versions are used, different sizes of multiplier arrays, adder arrays, accumulator arrays and quantization modules will be generated, but due to the expansion method of the computing unit, no matter which version is used, higher hardware utilization and flexibility can be achieved.

[0078] Regarding beneficial effect 3, in traditional neural network accelerators, when input data is transmitted between the computing subsystem and the storage subsystem, an intermediate buffer is sometimes introduced. The specific implementation of the buffer often adopts the following two solutions:

[0079] Solution A: Use a ping-pong buffer RAM to implement data buffering. In this case, there will be two RAMs, the upper and lower RAMs. The upstream writes data to the two RAMs in turn, and the core computing unit reads the data from the full RAM. The structural advantage of this buffer is that when the downstream speed is greater than the upstream speed, half of the computing delay can be hidden in the reading delay, but the disadvantage is that the core computing unit must wait until one of the RAMs is full before starting the calculation.

[0080] Solution B: Use FIFO structure to implement data buffering. This structure can start reading and writing data immediately after writing data into the buffer. However, since it cannot be accessed by address, data needs to be repeatedly read from the global buffer and written into the FIFO, which greatly increases power consumption.

[0081] The calculation unit adopts Figure 3 The buffer structure shown in Figure 1 uses a flow control mechanism.

[0082] The flow control mechanism consists of three parts: the upstream control module, the buffer, and the downstream control module. The upstream control module is connected to the write data interface, writes data to the buffer through the write control logic, and sends a write enable and a write pointer to the buffer. The downstream control module is connected to the read data interface, reads data from the buffer through the read control logic, and sends a read enable and a read pointer to the buffer. The read pointer and the write pointer will make empty and full judgments in the buffer according to the configuration information of the instruction. The specific workflow is shown in the figure below.

[0083] Take the convolution kernel size of 3*3 and the input channel (LOOP2) in the innermost layer as an example. Assume that the input channel and output channel are 32 and 64 respectively. The workflows of the input data zero-level buffer and the weight data zero-level buffer are introduced respectively.

[0084] For the input data zero-level buffer, the upstream control module is the input data first-level buffer controller, and the downstream control module is the input data zero-level buffer controller. When the buffer is not full, the upstream writes the data of one address in each clock cycle and increases the write pointer by 1; when the buffer is not empty, the downstream reads the data of one address in each clock cycle, and increases the read pointer by 3 after an interval of 3 clock cycles. When the input channel moves outward, the downstream increases 3× the number of rows after an interval of 3× the number of rows of clock cycles.

[0085] For the weight data buffer, the upstream control module is the weight data first-level buffer controller, and the downstream control module is the weight data zero-level buffer controller. When the buffer is not full, the upstream will write data of one address every clock cycle and increase the write pointer by 1; when the buffer is not empty, the downstream will read data of one address every clock cycle. The workflow is similar to the input data zero-level buffer. After writing enough weights required for one calculation, the upstream will increase the write pointer by 3×3×(32 / 4) at one time.

[0086] =72. At this time, the downstream sees that the buffer has enough data required to calculate a LOOP4 cycle, so it starts to repeatedly read the required data from the buffer and gradually increase the read pointer.

[0087] In this case, the upstream will use the full signal to determine when to write data to the buffer, and the downstream will use the empty signal to read data from the buffer. The judgment conditions of the empty and full signals are the same as those of the traditional FIFO, and the judgment conditions of the empty and full signals are determined by the configuration information in the instruction.

[0088] After adjusting the calculation order of the convolution loop expansion, the mechanism can automatically receive instruction information and adjust the behavior logic to start calculation immediately without too much waiting. The traditional ping-pong structure has to wait for the data to fill at least one buffer before it can start, reaching the speed limit of data reading in the process.

[0089] In summary, the computing unit described in the present application adopts a unified data flow control and transmission mechanism, designs a specific buffer structure, realizes flexible reading and writing of upstream and downstream data in the buffer, and improves computing efficiency. The above is only a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the examples and drawings. Any equivalent or modification that is completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A general neural network computing unit connected to an external memory and its controller, characterized in that: It includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller and a core computing unit; The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer; The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module; The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer; The zero-level buffer controller includes an input data zero-level read-write control module and a weight data zero-level read-write control module; The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller, the partial data read and write control module, the quantization weight data read and write control module and the output data read and write control module respectively.

2. A general neural network computing unit according to claim 1, characterized in that: The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

3. A general neural network computing unit according to claim 2, characterized in that: Part of the first-level buffer controller and the data read-write control module, the quantization weight data read-write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

4. A general neural network computing unit according to claim 3, characterized in that: The core computing unit is respectively connected to the partial data read-write control module, the quantization weight data read-write control module, the input data zero-level read-write control module, the weight data zero-level read-write control module and the output data read-write control module.

5. A general neural network computing unit according to claim 4, characterized in that: The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

6. A general neural network computing unit according to claim 5, characterized in that: The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

7. A general neural network computing unit according to claim 6, characterized in that: The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data that may be generated during the neural network calculation, and the output data generated after the neural network calculation; The input data first-level buffer is used to store input feature map data, the weight data first-level buffer is used to store weight data, the quantized weight data buffer is used to store quantized weight data, the partial sum data buffer is used to store partial sum data, and the output data buffer is used to store output data.

8. A general neural network computing unit according to claim 7, characterized in that: The first-level buffer controller receives instructions and adjusts the read-write control logic in real time to schedule computing resources; the first-level read-write control module for input data is responsible for data rearrangement and partial zero padding of input data; the first-level read-write control module for weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit.

9. A general neural network computing unit according to claim 8, characterized in that: The output data read-write control module is responsible for realizing the data rearrangement of possible output data; the partial sum data control read-write control module and the quantization weight data read-write module are responsible for realizing the direct reading of partial sum data and quantization weight data.

10. A general neural network computing unit according to claim 9, characterized in that: The zero-level buffer receives instructions and caches input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; reads and writes the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

Images