Universal neural network-oriented multi-core CNN acceleration processor

Abstract

The invention discloses a universal neural network-oriented multi-core CNN acceleration processor, which comprises a plurality of acceleration cores, and is characterized in that each acceleration core is integrated with an instruction component, a scalar execution component, a vector matrix execution component, a multi-level cache component, an interrupt control component and a clock control component; the method is based on an RISC style instruction set, supports concurrent execution of a scalar instruction, a vector instruction and a matrix instruction, is used for accelerating reasoning calculation of a neural network, and is suitable for end-side equipment and an embedded scene; wherein the vector matrix execution component supports single-instruction multi-cycle execution, and the matrix operation unit processes 512 multiplication operations and 512 addition operations in each cycle so as to complete matrix multiply-accumulate operation. Through the multi-core architecture design and the efficient storage system, while high universality is kept, the neural network reasoning calculation performance is improved, the problem that flexibility and efficiency of a traditional accelerator are difficult to consider at the same time is solved, and an ideal hardware acceleration solution is provided for AI application in an edge calculation scene.

Description

Technical Field

[0001] This invention relates to the fields of neural network and integrated circuit design technology, and specifically to a multi-core CNN acceleration processor for general-purpose neural networks. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, convolutional neural networks, as its core component, have demonstrated outstanding performance in fields such as computer vision, autonomous driving, and medical image analysis, becoming a significant force driving the development of related technologies. The industry generally refers to all processing units or processors that accelerate deep neural network models as NPUs. In the terminal field, edge intelligent chips employing NPUs for real-time image acquisition and recognition have become a current application hotspot. Typically, a system-on-a-chip (SoC) architecture is used to integrate the NPU and other functional processor cores on a single chip, with different processor cores performing different stages of the recognition task.

[0003] Early convolutional neural network (CNN) accelerators primarily relied on central processing units (CPUs) and graphics processing units (GPUs) for acceleration. However, CPUs proved inefficient for handling large-scale algorithms, while GPUs, while offering speed improvements, introduced high power consumption. Therefore, to address these issues, CNN accelerator research typically employs fully custom application-specific integrated circuit (ASIC) accelerators or semi-custom field-programmable gate arrays (FPGAs) to accelerate the forward inference process of convolutional networks. However, these existing technologies have significant limitations: traditional CPUs are slow at processing neural network input data because general-purpose processors lack native matrix operation support; existing solutions require fully or semi-custom designs for specific types of neural networks, lacking flexibility; and while neural network algorithms achieve speed improvements, they often incur high power consumption, with a lack of effective power control mechanisms.

[0004] The root causes of these technical shortcomings are as follows: First, while all modern general-purpose processors support scalar and vector operations, matrix operations are primarily implemented in software, with only a very few high-end processors offering native hardware support. The core design philosophy of general-purpose processors is "generality" rather than "dedicated acceleration." For example, RISC-V architecture processors have good support for scalar operations, but vector operations require support through the "V" extension, and matrix operations currently lack native instruction set support, requiring indirect implementation through other methods, which severely impacts processing efficiency. Second, the current neural network inference process can be abstracted into a series of combined operations on scalar, vector, and matrix operations. Each operation has different processing requirements, necessitating customized designs for specific types in pursuit of ultimate performance and energy efficiency, which limits the processor's versatility and flexibility. Third, neural network inference operations have real-time requirements; the faster the processing speed, the better. However, the hardware architecture lacks a mechanism to dynamically adjust power consumption based on real-time data volume, inevitably leading to high power consumption while pursuing high performance. These deep-seated technical contradictions limit the widespread application and development of neural network accelerators in edge computing scenarios. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a multi-core CNN acceleration processor for general neural networks, which can improve the performance of neural network inference while maintaining high versatility, thus overcoming the limitations of traditional accelerators in terms of flexibility and efficiency.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention includes: The processor comprises multiple acceleration cores, each integrating an instruction unit, a scalar execution unit, a vector-matrix execution unit, a multi-level cache unit, an interrupt control unit, and a clock control unit. Based on a RISC-style instruction set, the processor supports concurrent execution of scalar, vector, and matrix instructions to accelerate neural network inference computation and is suitable for edge devices and embedded scenarios. The vector-matrix execution unit supports single-instruction multi-cycle execution, with the matrix operation unit processing 512 multiplication operations and 512 addition operations per cycle to complete matrix multiplication and accumulation operations.

[0007] As a further improvement of this invention, the multiple acceleration cores are interconnected via a network on-chip (NoC); the network on-chip (NoC) supports data communication and load balancing between multiple cores to adapt to neural network models of different sizes.

[0008] As a further improvement of this invention, the instruction unit adopts an out-of-order execution mechanism, including a reorder buffer (ROB) with a depth of 64, which is used to tolerate long-latency operations and ensure accurate exception handling; the out-of-order execution mechanism supports a dual-issue structure, improves instruction-level parallelism, and reduces pipeline congestion.

[0009] As a further improvement of this invention, the multi-level caching component includes a level 1 instruction cache, a level 1 data cache, and a level 2 unified cache; the level 1 data cache uses a victim cache to optimize data access performance, and the level 1 instruction cache is equipped with a stream buffer hardware prefetching strategy to reduce average memory access latency by utilizing the spatial and temporal locality of the program.

[0010] As a further improvement of this invention, the vector matrix execution unit includes an extensible operation cluster structure, supporting matrix multiplication and accumulation operations, vector multiplication and accumulation matrix operations, and dot product and accumulation operations; the vector matrix execution unit executes in a fixed order, reducing register temporary storage overhead, and supports byte integer operations, making it suitable for batch data processing in convolutional neural networks.

[0011] As a further improvement of this invention, the accelerated processor is configured to support general neural network types, including convolutional neural networks (CNNs) and special operators implemented through extended vector matrix instructions, including converting convolution operations into matrix multiplication and accumulation, converting activation functions into SIMD vector instructions, and abstracting data format conversion into matrix transformations.

[0012] As a further improvement of this invention, it also includes a power consumption control unit, which adopts a parallel structure, operand isolation, clock gating and dynamic power management; the power consumption control unit dynamically adjusts the number of activated acceleration cores according to the real-time data computation volume, and combines operand isolation to input a fixed value into useless data paths to reduce power consumption.

[0013] As a further improvement to this invention, the accelerated processor's storage system adopts a three-level structure: the register level includes scalar registers and vector registers; the on-chip storage level includes a level 2 cache and shared SRAM; and the memory level is connected to external DDR memory through a DDR controller. The storage system supports block transfer of vector matrix data and address mapping optimization.

[0014] As a further improvement to this invention, the external interface of the accelerated processor includes a connection to the host processor, a high-capacity external storage (DDR) interface, a data acquisition interface, and a JTAG debugging interface; the processor integrates an independent DDR controller to ensure high-bandwidth data access, wherein the data acquisition interface supports LVDS or ISP protocols for receiving image data and triggering inference calculations.

[0015] As a further improvement of this invention, the data acquisition interface works in conjunction with the image signal processor (ISP) to support image preprocessing operations, including geometric transformation, grayscale transformation and image enhancement; the preprocessed image data is stored in the DDR memory via DMA and read by the acceleration core according to the address for inference calculation.

[0016] Compared with the prior art, the advantages of the present invention are as follows: This invention, through its multi-core architecture design and efficient storage system, improves neural network inference computing performance while maintaining high versatility. It overcomes the technical bottleneck of traditional accelerators, which struggle to balance flexibility and efficiency, and provides an ideal hardware acceleration solution for AI applications in edge computing scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the acceleration core described in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the instruction flow of the acceleration core described in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the storage system of the accelerated processor described in this embodiment of the invention.

[0020] Figure 4 This is a connection diagram of a multi-core CNN acceleration processor for general neural networks described in this embodiment of the invention within a neural network system. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0022] This embodiment employs a multi-core CNN acceleration processor for general-purpose neural networks, comprising: Multiple acceleration cores, such as Figure 1As shown, each acceleration core integrates an instruction unit, a scalar execution unit, a vector-matrix execution unit, a multi-level cache unit, an interrupt control unit, and a clock control unit. The processor is based on a RISC-style instruction set and supports concurrent execution of scalar instructions, vector instructions, and matrix instructions to accelerate neural network inference computation. It is suitable for edge devices and embedded scenarios. Among them, the vector-matrix execution unit supports single-instruction multi-cycle execution. The matrix operation unit processes 512 multiplication operations and 512 addition operations per cycle to complete matrix multiplication and accumulation operations.

[0023] In specific application scenarios, the multiple acceleration cores are interconnected via a network on-chip (NoC); the network on-chip (NoC) supports data communication and load balancing between multiple cores to accommodate neural network models of different sizes.

[0024] In specific application examples, the instruction unit adopts an out-of-order execution mechanism, including a reorder buffer (ROB) with a depth of 64, which is used to tolerate long-latency operations and ensure accurate exception handling; the out-of-order execution mechanism supports a dual-issue structure, improves instruction-level parallelism, and reduces pipeline congestion.

[0025] In specific application examples, the multi-level caching components include a level 1 instruction cache, a level 1 data cache, and a level 2 unified cache. The level 1 data cache uses a victim cache to optimize data access performance, and the level 1 instruction cache has a stream buffer hardware prefetching strategy to reduce average memory access latency by leveraging the spatial and temporal locality of the program.

[0026] In specific application examples, the vector matrix execution unit includes an extensible operation cluster structure, supporting matrix multiplication and accumulation operations, vector multiplication and accumulation matrix operations, and dot product and accumulation operations; the vector matrix execution unit executes in a fixed order, reducing register temporary storage overhead, and supports byte integer operations, making it suitable for batch data processing in convolutional neural networks.

[0027] In specific application examples, the accelerated processor is configured to support general neural network types through RISC instruction set extensions, including convolutional neural networks (CNNs) and special operators implemented through extended vector matrix instructions, including converting convolution operations into matrix multiplication and accumulation, converting activation functions into SIMD vector instructions, and abstracting data format conversion into matrix transformations.

[0028] In specific application examples, a power consumption control unit is also included, which adopts a parallel structure, operand isolation, clock gating, and dynamic power management. The power consumption control unit dynamically adjusts the number of activated acceleration cores according to the real-time data computation volume, and combines operand isolation to input a fixed value into useless data paths to reduce power consumption.

[0029] In specific application examples, such as Figure 3 As shown, the accelerated processor's memory architecture adopts a three-level structure: the register level includes scalar registers and vector registers; the on-chip memory level includes a level 2 cache and shared SRAM; and the memory level is connected to external DDR memory through a DDR controller. The memory architecture supports block transfer of vector matrix data and address mapping optimization.

[0030] In specific application examples, the external interfaces of the acceleration processor include a connection to the host processor, a high-capacity external storage (DDR) interface, a data acquisition interface, and a JTAG debugging interface; the processor integrates an independent DDR controller to ensure high-bandwidth data access, wherein the data acquisition interface supports LVDS or ISP protocols to receive image data and trigger inference calculations.

[0031] In specific application scenarios, the data acquisition interface works in conjunction with the image signal processor (ISP) to support image preprocessing operations, including geometric transformation, grayscale transformation, and image enhancement. The preprocessed image data is stored in the DDR memory via DMA and then read by the acceleration core according to the address for inference calculation.

[0032] In a specific example, the process of the multi-core CNN acceleration processor for general neural networks processing instructions is as follows: Figure 2 As shown, the kernel fetches instructions through the instruction fetch unit and sends the fetched instruction packets to the instruction dispatch unit. Instruction packets with a maximum length of four instructions are sent to the scalar and vector processing units via the instruction dispatch unit. The scalar processing unit performs serially executed tasks, including scalar data memory access, operations, control of the vector processing unit, scalar processing branching, and interrupt operations. The vector matrix processing unit is a scalar operation cluster architecture suitable for performing computationally intensive parallel tasks. Local storage provides data for vector, scalar, and matrix operations and can perform data transfer operations. L1D, L1P, and L2 can access specific memory data, thereby providing instructions and data to the scalar and vector matrix processing units.

[0033] The multi-core CNN acceleration processor for general neural networks, such as Figure 4 As shown, when connected to the neural network, the host stores the NPU's code and data segments in a specified location in the DDR, stores the neural network's parameters in a specified location in the DDR, and configures the NPU's initial PC and operating mode through the configuration register interface. Then, it configures the flag for the NPU to start working. Based on the information in the configuration register, the NPU reads the code and data segments from the specified location in the DDR and begins working.

[0034] To improve the accuracy of NPU recognition, preprocessing of real-time acquired images is necessary. This typically includes geometric transformations (translation, transpose, mirroring, rotation, scaling, etc.), grayscale transformations (linear or nonlinear transformations of multi-channel pixel values ​​to improve image clarity), and image enhancement (using spatial or frequency domain methods to highlight the most relevant information). The ISP module stores the preprocessed image data in DDR, using a buffer to accelerate storage. After storing an image, it writes the completed image information and the image's starting address to the NPU's registers. Once the NPU detects that image storage is complete, it performs neural network inference calculations on the current image and writes the results to a specified location in DDR. The NPU sends an interrupt request to the host via its registers. The host clears the interrupt by configuring its registers and reads the output feature map from the specified area in DDR for subsequent computation. This CNN multi-core accelerator platform has implemented numerous image preprocessing algorithms to meet different image types and image recognition needs, and these algorithms can be widely applied in a wide range of image processing scenarios.

[0035] Compared with the prior art, the advantages of this embodiment are: By employing a multi-core architecture and an efficient storage system, this system improves neural network inference performance while maintaining high versatility. It overcomes the technical bottleneck of traditional accelerators, which struggle to balance flexibility and efficiency, and provides an ideal hardware acceleration solution for AI applications in edge computing scenarios.

Claims

1. A multi-core CNN acceleration processor for general-purpose neural networks, characterized in that, include: Multiple acceleration cores, each integrating an instruction unit, a scalar execution unit, a vector matrix execution unit, a multi-level cache unit, an interrupt control unit, and a clock control unit; The processor is based on a RISC-style instruction set and supports concurrent execution of scalar instructions, vector instructions, and matrix instructions to accelerate neural network inference computation. It is suitable for edge devices and embedded scenarios. The vector matrix execution unit supports single-instruction multi-cycle execution, and the matrix operation unit processes 512 multiplication operations and 512 addition operations per cycle to complete matrix multiplication and accumulation operations.

2. The multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The multiple acceleration cores are interconnected via an on-chip network; the on-chip network supports data communication and load balancing between multiple cores to accommodate neural network models of different sizes.

3. The multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The instruction unit employs an out-of-order execution mechanism, including a reordering buffer with a depth of 64, to tolerate long-latency operations and ensure accurate exception handling. The out-of-order execution mechanism supports a dual-issue structure, improving instruction-level parallelism and reducing pipeline congestion.

4. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The multi-level caching component includes a level 1 instruction cache, a level 1 data cache, and a level 2 unified cache. The level 1 data cache uses a victim cache to optimize data access performance, and the level 1 instruction cache has a stream buffer hardware prefetch strategy to reduce average memory access latency by utilizing the spatial and temporal locality of the program.

5. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The vector matrix execution unit includes an extensible operation cluster structure, supporting matrix multiplication and accumulation operations, vector multiplication and accumulation matrix operations, and dot multiplication and accumulation operations. The vector matrix execution unit executes in a fixed order, reducing register storage overhead, and supports byte integer operations, making it suitable for batch data processing in convolutional neural networks.

6. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The accelerated processor is configured to support general neural network types via RISC instruction set extensions, including convolutional neural networks and special operators implemented through extended vector matrix instructions, including converting convolution operations into matrix multiplication and accumulation, converting activation functions into SIMD vector instructions, and abstracting data format conversion into matrix transformations.

7. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, It also includes a power consumption control unit, which adopts a parallel structure, operand isolation, clock gating and dynamic power management; the power consumption control unit dynamically adjusts the number of activated acceleration cores according to the real-time data computation volume, and combines operand isolation to input a fixed value into useless data paths to reduce power consumption.

8. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The accelerated processor's memory architecture adopts a three-level structure: the register level includes scalar registers and vector registers; the on-chip memory level includes L2 cache and shared SRAM; and the memory level is connected to external DDR memory through a DDR controller. The memory architecture supports block transfer of vector matrix data and address mapping optimization.

9. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 1, characterized in that, The external interfaces of the accelerated processor include a connection to the host processor, a large-capacity external storage interface, a data acquisition interface, and a JTAG debugging interface; the processor integrates an independent DDR controller to ensure high-bandwidth data access, wherein the data acquisition interface supports LVDS or ISP protocols for receiving image data and triggering inference calculations.

10. A multi-core CNN acceleration processor for general-purpose neural networks according to claim 9, characterized in that, The data acquisition interface works in conjunction with the image signal processor to support image preprocessing operations, including geometric transformation, grayscale transformation, and image enhancement. The preprocessed image data is stored in the DDR memory via DMA, and then read by the acceleration core according to the address for inference calculation.

Images