A universal neural network computing unit

Abstract

The present invention belongs to the technical field of neural network acceleration design and proposes a universal neural network computing unit with three-dimensional loop expansion. The computing unit includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller, and a core computing unit; the first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer, and an output data buffer; the zero-level buffer includes an input zero-level data buffer and a weight data zero-level buffer; the zero-level buffer controller includes an input and weight data zero-level read-write control module; the input and weight data first-level read-write control module of the first-level buffer read-write controller is connected to the input and weight data zero-level read-write control module through the input and weight data zero-level buffer, and then connected to the core computing unit. The computing unit simultaneously performs parallel computing in three loop dimensions, thereby improving computing speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network acceleration and processing unit design, and proposes a universal neural network computing unit. Background Art

[0002] In recent years, artificial intelligence technology has developed rapidly, particularly in the field of neural networks. Convolutional neural networks (CNNs) and Transformers have become core technologies in fields such as computer vision and natural language processing. CNNs are widely popular for their outstanding image processing capabilities, while Transformers have attracted attention for their superior performance in processing sequential data. The rapid iteration and optimization of these technologies have enabled them to effectively handle increasingly complex tasks.

[0003] As single network structures become increasingly inadequate for complex applications, hybrid neural networks have emerged. These networks combine multiple algorithms and technologies, for example, combining the image recognition capabilities of CNNs with the sequence processing capabilities of Transformers to achieve superior performance. In these hybrid networks, different types of operators (such as convolution and matrix multiplication) are integrated into a single network architecture to handle diverse data types and tasks.

[0004] While hybrid neural networks offer superior performance in theory, their implementation faces numerous technical challenges. Traditional neural network computational units are mostly designed to optimize specific operators, such as convolution, and are insufficient for efficiently handling diverse computational tasks. Transformer-based networks, in particular, place higher demands on the performance and flexibility of computational units due to their complex computational graphs and high data density. This limitation leads to low computational efficiency and difficulties in data scheduling.

[0005] In response to the above problems, the present application is dedicated to proposing a neural network computing unit that is adaptable to multiple neural network operators. Summary of the Invention

[0006] The purpose of the present invention is to solve the problems of low computational efficiency and difficult data scheduling in convolution-oriented neural network computing units caused by the widespread application of Transformer. A universal neural network computing unit is proposed. The unit introduces a zero-level buffer and a zero-level buffer controller after the first-level read-write control module of input data and the first-level read-write control module of weight data, respectively, to obtain an improved computing unit; the improved computing unit buffers and performs read-write control on the input data and weight data again, which can not only process different types of operators, but also simplify data paths and control logic, thereby improving the overall computing efficiency and flexibility; this computing unit also solves the mismatch problem between the global buffer and the computing unit in bandwidth and data format, thereby realizing the unification of read-write control logic between different operators.

[0007] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions.

[0008] A general neural network computing unit is connected to an external memory and its controller, and includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller, and a core computing unit;

[0009] The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer;

[0010] The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module;

[0011] The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer;

[0012] The zero-level buffer controller includes an input data zero-level read and write control module and a weight data zero-level read and write control module;

[0013] The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller and the output data read and write control module respectively.

[0014] The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

[0015] Part of the first-level buffer controller and the data read-write control module, the quantization weight data read-write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

[0016] The core computing unit is respectively connected to the partial data read and write control module, the quantization weight data read and write control module, the input data zero-level read and write control module, the weight data zero-level read and write control module and the output data read and write control module.

[0017] The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

[0018] The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

[0019] The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data that may be generated during the neural network calculation process, and the output data generated after the neural network calculation; the input data first-level buffer is used to store the input feature map data, the weight data first-level buffer is used to store the weight data, the quantized weight data buffer is used to store the quantized weight data, the partial sum data buffer is used to store the partial sum data, and the output data buffer is used to store the output data.

[0020] The first-level buffer controller adjusts the read-write control logic in real time by receiving instructions to schedule computing resources; the first-level read-write control module for input data is responsible for data rearrangement and partial zero padding of input data; the first-level read-write control module for weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit; the output data read-write control module is responsible for data rearrangement of possible output data; the partial sum data control read-write control module and the quantized weight data read-write module are responsible for direct reading of partial sum data and quantized weight data.

[0021] The zero-level buffer receives instructions and is used to cache input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; read and write control is performed on the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

[0022] Beneficial effects

[0023] The present invention proposes a universal neural network computing unit, which has the following advantages over existing neural network accelerators:

[0024] 1. The computing unit relies on a multi-level data storage and access mechanism, fully utilizing the data locality and data arrangement characteristics of the neural network calculation process, decoupling the data reading logic and reordering logic, and solving the data bandwidth and data format inconsistency issues between the storage subsystem and the computing subsystem when facing different neural network operators;

[0025] 2. The computing unit performs parallel computations in three loop dimensions simultaneously, fully utilizing the high-dimensional tensor nature of the neural network input data, thereby increasing the upper limit of computational speed.

[0026] 3. The computing unit adopts a unified data flow control and transmission mechanism and designs a specific buffer structure to achieve flexible reading and writing of upstream and downstream data in the buffer, thereby improving computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the composition and connection relationship of a universal neural network computing unit of the present invention;

[0028] Figure 2 This is a block diagram of the composition and structure of a core computing unit in a general neural network computing unit of the present invention;

[0029] Figure 3 This is the internal structure of a zero-level buffer in a universal neural network computing unit of the present invention. DETAILED DESCRIPTION

[0030] The following describes in detail the deployment, practical application process, and advantages and beneficial effects of a general neural network computing unit of the present invention in conjunction with the accompanying drawings and embodiments.

[0031] Example 1

[0032] This embodiment describes the specific implementation of the general neural network computing unit.

[0033] The universal neural network computing unit is connected to the external memory and its controller, and includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller and a core computing unit;

[0034] The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer;

[0035] The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module;

[0036] The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer;

[0037] The zero-level buffer controller includes an input data zero-level read and write control module and a weight data zero-level read and write control module;

[0038] The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller and the output data read and write control module respectively.

[0039] The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

[0040] Part of the first-level buffer controller and the data read-write control module, the quantization weight data read-write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

[0041] The core computing unit is respectively connected to the partial data read and write control module, the quantization weight data read and write control module, the input data zero-level read and write control module, the weight data zero-level read and write control module and the output data read and write control module.

[0042] The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

[0043] The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

[0044] The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data that may be generated during the neural network calculation process, and the output data generated after the neural network calculation; the input data first-level buffer is used to store the input feature map data, the weight data first-level buffer is used to store the weight data, the quantized weight data buffer is used to store the quantized weight data, the partial sum data buffer is used to store the partial sum data, and the output data buffer is used to store the output data.

[0045] The first-level buffer controller adjusts the read-write control logic in real time by receiving instructions to schedule computing resources; the first-level read-write control module for input data is responsible for data rearrangement and partial zero padding of input data; the first-level read-write control module for weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit; the output data read-write control module is responsible for data rearrangement of possible output data; the partial sum data control read-write control module and the quantized weight data read-write module are responsible for direct reading of partial sum data and quantized weight data.

[0046] The zero-level buffer receives instructions and is used to cache input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; read and write control is performed on the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

[0047] In a traditional neural network accelerator, when input data is transmitted between the computing subsystem and the storage subsystem, two solutions are often adopted:

[0048] Solution 1: Read input data, weight data, quantized weight data, and partial sum data directly from the global buffer and feed it into the computation unit, writing the output result into the output data buffer. Because the input data buffer, weight data buffer, quantized weight data buffer, partial sum data buffer, and output data buffer are all directly connected to the core computation unit, this approach minimizes the complexity of the system architecture.

[0049] However, this connection method has two major disadvantages: on the one hand, due to the lack of multi-level buffering, the format of the buffer data must be completely consistent with the format in the core computing unit, that is, the reordering of data needs to be implemented outside the computing unit; on the other hand, due to the lack of a flow control mechanism, the bandwidth of the input data must be completely consistent with the core computing, which seriously limits the flexibility of the system.

[0050] Solution 2: Set up a level-0 buffer between the input data and weight data, but ensure that the data format in the level-0 buffer matches the format required by the core computing unit. In this case, the core computing unit's behavior from the quantized weight data buffer, partial sum data buffer, and output data buffer remains unchanged, but the input data and weight data are read from the level-1 buffer controller, written to the level-0 buffer, and then sent to the core computing unit from the level-0 buffer.

[0051] Because of the introduction of the zero-level buffer, this method no longer requires the data format and read-write bandwidth in the first-level buffer to match the data format and read-write bandwidth in the core computing unit, which reduces the complexity of the data reading logic to a certain extent. However, the zero-level buffer still has the same restrictions and requirements. The hardware structure corresponding to this zero-level buffer is often a register stack or FIFO with a specific structure, which often leads to a decrease in the versatility of the computing unit.

[0052] Combining the above two structures, the computing unit described in the present invention introduces a zero-level buffer and a zero-level buffer read-write controller. Similar to Solution 2, the introduced zero-level buffer can reduce the complexity of the data reading logic, but has the following two advantages: it does not require the data read and write bandwidth of the input zero-level buffer to be consistent with the read and write bandwidth of the core computing unit, nor does it require the data format in the input zero-level buffer to be consistent with the data format of the core computing unit. Instead, the zero-level buffer read-write controller reads the data from the zero-level buffer, further completes the data rearrangement, and then sends it to the core computing unit.

[0053] Therefore, this structure only requires that the output data format and data bandwidth of the zero-level buffer controller match those of the core computing unit. This leaves a large margin for the data reading logic and reordering logic, so that the data can be read and reordered step by step in blocks during the flow from the first-level buffer to the zero-level buffer and then to the core computing unit.

[0054] The computing unit proposed in the present invention relies on a multi-level data storage and access mechanism, fully utilizing the data locality and data arrangement characteristics in the neural network calculation process, decoupling the data reading logic and re-arrangement logic, and solving the problem of inconsistent data bandwidth and data format between the storage subsystem and the computing subsystem when facing different neural network operators.

[0055] Taking convolution as an example, assuming a convolution kernel size of 3×3, an input feature map of 8×60×60, an output feature map of 1×60×60, and a computing unit parallelism of 8×8×8, then using solution 1 requires an input data buffer size of 259.2KB; using solution 2 requires a first-level input buffer size of 86.4KB; and using the solution in the computing unit, requires an input data buffer size of 28.8KB. This shows that the computing unit significantly reduces the required on-chip buffer size.

[0056] In specific implementations, the input data level 1 buffer read / write control module reads data from the input data level 1 buffer, pads it with zeros, and transposes it to 8×64×60, with one row of zeros on the top and three rows of zeros on the bottom, before sending it to the level 0 buffer. The input data level 0 buffer read / write control module reads data from the level 0 buffer, rearranges it to 8×64×62, and then writes the rearranged data to the core computing unit. The zero-padding requirements for convolutional computation and the zero-padding requirements due to loop unrolling are thus unified into a two-stage data rearrangement logic, with both the read logic and the rearrangement logic being split equally.

[0057] By implementing a two-stage data rearrangement and zero-padding operation, the complex zero-padding logic is decoupled, reducing the number of combinational logic levels required for zero-padding by 50% and raising the upper limit of the system's operating frequency. This structure reduces the on-chip buffer size by a factor of nine and reduces the number of data reads from the global buffer by a factor of nine, reaching the theoretical upper limit.

[0058] With respect to beneficial effect 2, the calculation unit proposed in the present invention has the following advantages:

[0059] The computing unit accelerates convolution operations by performing three-dimensional loop expansion, that is, performing parallel computations on three loop dimensions simultaneously, fully utilizing the data characteristics of the neural network input data as high-dimensional tensors, and raising the upper limit of computational speed. When implemented, the computing unit is simultaneously compatible with convolution computations and Transformer matrix multiplication computations, unifying the computational logic of the neural network accelerator. While implementing high-performance computing, it is also configurable and can be flexibly deployed in any application scenario with a given resource amount and size.

[0060] Traditional neural network accelerators often use two-dimensional loop unrolling to simplify control signals and data paths. However, neural network accelerators based on two-dimensional loop unrolling have difficulty scaling up while maintaining relatively simple data reuse.

[0061] The following describes the specific implementation of the general neural network computing unit proposed in this application based on three common neural networks:

[0062] In a convolutional neural network accelerator, the number of dimensions of loop expansion directly determines the scale of the convolutional neural network accelerator. The calculation formula for convolution is as follows:

[0063]

[0064] In the above formula, the convolution operation is divided into four dimensions: convolution kernel (LOOP1), input channel (LOOP2), output feature map (LOOP3) and output channel (LOOP4), with a total of six loops.

[0065] Unrolling different loops will correspond to different core computing unit structures. The following introduces several common two-dimensional loop unrolling calculation methods and their corresponding core computing unit structures.

[0066] Neural Network 1: Parallel computation is performed on the convolution kernel (LOOP1) and input channels (LOOP2). This loop unrolling method has the simplest data path. Using row buffering combined with sliding window calculations can greatly simplify the control logic while also offering very strong data reuse. Although this method has relatively simple control and data paths, the parallel computation performed on the convolution kernel dimension means that, on the one hand, the computing unit can only compute convolutions with a fixed kernel size, making it difficult to adapt to convolutions with different kernel sizes and different step lengths. On the other hand, it also limits the number of multipliers in the convolution kernel dimension to match the scale of the convolution. Common parallelism ranges from 9×16 to 9×64 or 9×256, with scales ranging from tens to a few hundred.

[0067] Neural Network 2: Parallel computation is performed on the input channel (LOOP2) and the output channel (LOOP4). Because the input and output channels are loosely coupled, this expansion method allows for simpler control logic while being compatible with convolutions of varying kernel sizes and step lengths. Typically, neural network input channels can be scaled from 16×32 to 128×256, or even higher, without wasting computational parallelism. Although this expansion method has a high computational speed ceiling, it can be difficult to achieve high hardware utilization for smaller on-device neural network models, where both input and output channels may be smaller.

[0068] Neural Network 3: Parallel computation is performed on the output feature map (LOOP3) and output channels (LOOP4). Therefore, the output feature map is often numerically the largest compared to the other three dimensions. Even with a small 28×28 feature map, hundreds of points on the output channels can be computed in parallel without worrying about low hardware utilization. Because the coupling between the output feature map and the output channels is also low, if the two loops within the output feature map are computed in parallel, it is very easy to achieve an extremely large scale of 32×32×256 while maintaining high hardware utilization. While this expansion method offers very ideal hardware utilization and computational speed, the loop expansion on the output feature map (LOOP3) often requires extremely complex data paths and control logic in the accelerator, making it difficult to adapt to convolutions of different kernel sizes and different step lengths while ensuring data reusability.

[0069] The computing unit chooses to perform parallel computing in three dimensions: input channel (LOOP2), output channel (LOOP4) and output feature map (LOOP3). Through this expansion method and the corresponding hardware structure, it is easy to obtain parallelism from 8×8×8 to 16×16×16 or even 32×32×32. Compared with the several more common convolutional neural networks mentioned above, it has a higher performance ceiling than the first two. Compared with the third one, although it has the same performance ceiling, it has a simpler data path and better adaptability.

[0070] In addition to convolutional neural networks, the core computing unit can also support matrix multiplication calculations in Transformer. Matrix multiplication in Transformer neural networks can be roughly divided into two categories: one is the projection layer of the form [B,M,N] and [1,N,K]; the other is the activation layer of the form [B,M,N] and [B,N,M]. The first type of matrix multiplication can be regarded as broadcasting [1,N,K] in the second type of matrix multiplication to [B,N,M]. In this case, the matrix multiplication in Transformer can be uniformly referred to as batch matrix multiplication. The calculation formula for the known convolution is as follows:

[0071]

[0072] Referring to the calculation order of the convolutional neural network, the matrix multiplication can be very well mapped to the entire computing unit, that is, the three dimensions M, N, and K are expanded at the same time. At this time, the calculation logic is completely consistent with the convolutional neural network, achieving compatibility with convolution calculation and Transformer matrix multiplication, and unifying the calculation logic.

[0073] In order to improve the configurability of the computing unit and flexibly deploy it in any application scenario with a given resource amount and size, the computing unit provides any parallelism from 2×2×2 to 8×8×8. When targeting lightweight neural networks and embedded platforms with low real-time requirements, the small version with a total parallelism of 8 can be used; when targeting lightweight neural networks and embedded platforms with high real-time requirements, the medium version with a total parallelism of 64 can be used; when targeting neural networks with large computational and resource amounts, the large version with a parallelism of 512 can be used. The following details the hardware structure of the core computing units corresponding to different parallelisms.

[0074] The core computing units in traditional neural network accelerators have different structures depending on their computing modes, but the overall architecture is relatively fixed, that is, it includes three core components: multipliers, adder trees and accumulators. If a systolic array is used, it can be simplified to a multiplier array, an accumulator array and a quantization array. The core computing unit adds a broadcast control module on the basis of the traditional method, and controls the input data to be broadcast in different forms according to different computing modes through instructions. In this way, the input data rearrangement of multiple neural network computing units can be realized.

[0075] On this basis, the core computing unit does not choose to use a systolic array to implement matrix operations. Its idea is to treat multipliers and adders as different types of computing resources and perform unified scheduling. This is conducive to more flexible scheduling and planning of the overall computing resources in the future, and thus realize more flexible grouping and reduction operations. The specific workflow is illustrated below with an example.

[0076] like Figure 2 The core computing unit shown includes a broadcast control module, a multiplier array, an adder array, an accumulator array, and a quantization module. Taking a parallelism of 4×4×4 as an example, the broadcast control module is responsible for broadcasting the 4×4 input data and weight data to 4×4×4. In this case, there are 64 multipliers. The broadcasted input data and weight data are fed into the multiplier array and multiplied two by two. The output results are fed into the adder tree array, which consists of 4×4 groups of 4-2-1 adder trees. The 64 multiplication results are grouped and reduced to 16 output values. These are then fed into the 16 accumulator arrays for accumulation. Based on the configuration information, the accumulator array outputs 16 accumulator results every several clock cycles. These are fed into the quantization module for quantization, ultimately outputting 16 data points.

[0077] When different parallelism versions are used, different sizes of multiplier arrays, adder arrays, accumulator arrays and quantization modules will be generated, but due to the way the computing unit is expanded, higher hardware utilization and flexibility can be achieved regardless of which version.

[0078] Regarding beneficial effect 3, in traditional neural network accelerators, when input data is transmitted between the computing subsystem and the storage subsystem, an intermediate buffer is sometimes introduced. The specific implementation of the buffer often adopts the following two solutions:

[0079] Solution A: Use RAM with a ping-pong buffer to implement data buffering. In this case, there are two RAMs, the upper and lower RAMs. The upstream writes data to the two RAMs in turn, and the core computing unit reads the data from the full RAM. The advantage of this buffer structure is that when the downstream speed is greater than the upstream speed, half of the calculation delay can be hidden in the read delay. However, the disadvantage is that the core computing unit must wait until one of the RAMs is full before starting calculations.

[0080] Solution B: Use FIFO structure to implement data buffering. This structure can start reading and writing immediately after the data is written to the buffer. However, since it cannot be accessed by address, the data needs to be repeatedly read from the global buffer and written to the FIFO, which greatly increases power consumption.

[0081] The calculation unit adopts Figure 3 The buffer structure shown in Figure 1 uses a flow control mechanism.

[0082] This flow control mechanism consists of three parts: an upstream control module, a buffer, and a downstream control module. The upstream control module connects to the write data interface and writes data to the buffer through write control logic, sending a write enable and write pointer to the buffer. The downstream control module connects to the read data interface and reads data from the buffer through read control logic, sending a read enable and read pointer to the buffer. The read and write pointers determine whether the buffer is empty or full based on the instruction configuration information. The specific workflow is shown in the figure below.

[0083] As an example, consider a convolution kernel with a size of 3*3 and an input channel (loop 2) in the innermost layer. Assume that the input and output channels are 32 and 64, respectively. The workflows for the input data zero-level buffer and the weight data zero-level buffer are described separately.

[0084] For the input data level 0 buffer, the upstream control module acts as the input data level 1 buffer controller, while the downstream control module acts as the input data level 0 buffer controller. When the buffer is not full, the upstream module writes data from one address per clock cycle and increments the write pointer by 1. When the buffer is not empty, the downstream module reads data from one address per clock cycle and increments the read pointer by 3 after three clock cycles. As the input channel moves outward, the downstream module increments the read pointer by 3 × the number of rows after 3 × the number of rows per clock cycle.

[0085] For the weight data buffer, the upstream control module is the weight data first-level buffer controller, and the downstream control module is the weight data zero-level buffer controller. When the upstream buffer is not full, it will write data of one address every clock cycle and increase the write pointer by 1; when the downstream buffer is not empty, it will read data of one address every clock cycle. The workflow is similar to the input data zero-level buffer. After writing enough weights required for one calculation, the upstream will increase the write pointer by 3×3×(32 / 4) at one time.

[0086] =72. At this time, the downstream sees that the buffer has enough data required to calculate a LOOP4 cycle, so it starts to repeatedly read the required data from the buffer and gradually increase the read pointer.

[0087] In this case, the upstream will use the full signal to determine when to write data to the buffer, and the downstream will read data from the buffer by using the empty signal. The judgment conditions of the empty and full signals are the same as those of the traditional FIFO, and the judgment conditions of the empty and full signals are determined by the configuration information in the instruction.

[0088] After adjusting the calculation order of the convolution loop expansion, the mechanism can automatically receive instruction information and adjust the behavior logic to start calculation immediately without too much waiting. The traditional ping-pong structure has to wait for the data to fill at least one buffer before it can start, reaching the speed limit of data reading in this process.

[0089] In summary, the computing unit described in this application adopts a unified data flow control and transmission mechanism, designs a specific buffer structure, realizes flexible reading and writing of upstream and downstream data in the buffer, and improves computing efficiency. The above is only a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the embodiment and the accompanying drawings. Any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A general neural network computing unit connected to an external memory and its controller, characterized in that: It includes a central control module, a first-level buffer, a first-level buffer controller, a zero-level buffer, a zero-level buffer controller, and a core computing unit; The first-level buffer includes an input data first-level buffer, a weight data first-level buffer, a quantized weight data buffer, a partial sum data buffer and an output data buffer; The first-level buffer controller includes an input data first-level read-write control module, a weight data first-level read-write control module, an output data read-write control module, a partial sum data read-write control module and a quantized weight data read-write control module; The zero-level buffer includes an input data zero-level buffer and a weight data zero-level buffer; The zero-level buffer controller includes an input data zero-level read and write control module and a weight data zero-level read and write control module; The first-level buffer is connected to the external controller and the first-level buffer controller; the zero-level buffer is connected to the first-level buffer controller and the zero-level buffer controller respectively; the core computing unit is connected to the zero-level buffer controller, the partial data read and write control module, the quantization weight data read and write control module and the output data read and write control module respectively; The core computing unit relies on a multi-level data storage and access mechanism, and utilizes the local characteristics and data arrangement characteristics of data in the general neural network calculation process to decouple data reading logic and reordering logic.

2. A general neural network computing unit according to claim 1, characterized in that: The central control module is connected to the first-level buffer controller, the zero-level buffer controller and the core computing unit respectively.

3. A general neural network computing unit according to claim 2, characterized in that: Part of the first-level buffer controller and the data read and write control module, the quantization weight data read and write control module and the zero-level buffer controller are respectively connected to the broadcast control module of the core computing unit.

4. A general neural network computing unit according to claim 3, characterized in that: The core computing unit is respectively connected to the partial data read and write control module, the quantization weight data read and write control module, the input data zero-level read and write control module, the weight data zero-level read and write control module and the output data read and write control module.

5. A general neural network computing unit according to claim 4, characterized in that: The core computing unit includes a broadcast control module, a multiplier array, an adder array, an accumulator array and a quantization array.

6. A general neural network computing unit according to claim 5, characterized in that: The broadcast control module, the multiplier array, the adder tree array, the accumulator array and the quantization module are connected in sequence.

7. A general neural network computing unit according to claim 6, characterized in that: The first-level buffer is responsible for storing the input feature map data, weight data and quantized weight data required before the neural network calculation, the partial sum data generated during the neural network calculation, and the output data generated after the neural network calculation; The input data first-level buffer is used to store input feature map data, the weight data first-level buffer is used to store weight data, the quantized weight data buffer is used to store quantized weight data, the partial sum data buffer is used to store partial sum data, and the output data buffer is used to store output data.

8. A general neural network computing unit according to claim 7, characterized in that: The first-level buffer controller adjusts the read-write control logic in real time by receiving instructions to schedule computing resources; the first-level read-write control module of input data is responsible for data rearrangement and partial zero padding of input data; the first-level read-write control module of weight data is responsible for data rearrangement of weight data to adapt to the computing requirements of the core computing unit.

9. A general neural network computing unit according to claim 8, characterized in that: The output data read-write control module is responsible for realizing the data rearrangement of the existing output data; the partial sum data control read-write control module and the quantization weight data read-write module are responsible for realizing the direct reading of partial sum data and quantization weight data.

10. A universal neural network computing unit according to claim 9, characterized in that: The zero-level buffer receives instructions and is used to cache input data and weight data after data rearrangement through the zero-level input data buffer and the zero-level weight data buffer; read and write control is performed on the zero-level buffer to achieve real-time regulation of data reading and calculation speed.

Images