CNN accelerator based on hybrid NoC architecture

Abstract

The invention discloses a CNN accelerator based on a hybrid NoC architecture, and relates to the technical field of convolutional neural networks of artificial intelligence. Comprising a hybrid NoC link, a network interface, a router, a processing unit PE, a global controller and an on-chip global cache module. Wherein the hybrid NoC link comprises a bus link used for transmitting configuration information and input data, and an NoC link used for transmitting intermediate data in a calculation process; the network interface is used for data format conversion; the router is used for coordinating the transmission of the data packets in the NoC link; and the processing unit PE is used for completing the CNN calculation task. According to the invention, the communication delay of the accelerator is obviously reduced through the hybrid NoC link, and the calculation parallelism is improved through the configurable PE module, so that the operation performance of the accelerator is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of convolutional neural network of artificial intelligence, and particularly relates to a CNN accelerator based on a hybrid NoC architecture. BACKGROUND

[0002] A convolutional neural network (CNN) is a deep learning model that efficiently extracts spatial characteristics of data through weight sharing, local connection, and pooling operations, and is particularly good at processing image and sequence data. In recent years, with the optimization of algorithms and the improvement of hardware computing power, convolutional neural networks have shown excellent performance in tasks such as target classification, target detection, and scene analysis, and have been widely used in the field of artificial intelligence. However, as the prediction accuracy of CNN models continues to improve and the algorithm complexity rapidly increases, the huge computational overhead poses a severe performance requirement on the hardware platform, which limits the deployment of CNN on embedded platforms and thus affects its widespread application in more fields. In addition, with the failure of Moore's law, the performance of general-purpose processors is increasing at a slower rate, and traditional central processing units (CPUs) and graphics processing units (GPUs) are often difficult to balance high performance and low power consumption when facing high energy efficiency requirements and resource constraints. Therefore, the design of special CNN hardware accelerators has important application value.

[0003] Traditional CNN accelerators usually use a shared bus architecture to implement the interconnection of processing elements (PEs), relying on shared buses for data communication. This architecture can provide relatively simple design and implementation in a small-scale CNN. However, as the size and complexity of CNN increase, the shared bus feature leads to a sharp decline in data exchange efficiency between PEs as the scale expands, which restricts system performance. In addition, as the number of PEs increases, the data transmission delay grows exponentially, which seriously affects the scalability and communication performance of the accelerator. In contrast, a network on chip (NoC) architecture overcomes the bandwidth bottleneck of the bus architecture through parallel communication and communication path optimization, and can simultaneously meet the data interaction needs between large-scale PEs. Its high throughput, low latency, and high scalability make the NoC architecture exhibit significant advantages in the design of large-scale CNN accelerators. Existing technologies have deficiencies in communication efficiency, parallel computing capability, and scalability. SUMMARY

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a CNN accelerator based on a hybrid NoC architecture. By using the bus links and NoC links in the hybrid NoC interconnection link and the corresponding communication protocols, configuration information and intermediate data during the calculation process are distributed. A configurable processing unit PE is formed by multiple sets of systolic arrays and multiply-accumulate trees to realize efficient parallel computing of CNN tasks.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] According to the present invention, a CNN accelerator based on a hybrid NoC architecture includes:

[0007] The hybrid NoC interconnect link comprises multiple network interfaces, multiple routers, multiple processing units (PE), a global controller, and an on-chip global cache module. The hybrid NoC interconnect link includes bus links and NoC links.

[0008] Each PE (Preinstallation Environment) is connected to its respective router via a network interface. The routers are interconnected via NoC (No-Chip) links, forming the communication path between the PEs. Simultaneously, each PE is connected to the global controller and the on-chip global cache module via bus links.

[0009] The network interface is responsible for converting shared data from the PE into a format conforming to the NoC communication protocol and encapsulating it into data packets for transmission in the NoC link, as well as decapsulating data packets received from the NoC link and converting them back into their original format;

[0010] A router is responsible for selecting an output port from which to send a data packet based on the packet's destination address and the routing algorithm.

[0011] The global controller is used to generate control signals and transmit input data read from the on-chip global cache module to each PE via the bus link;

[0012] The processing unit (PE) is used to perform CNN computation tasks based on input data received from the bus link and shared data received from the NoC link, obtain CNN inference result data, and output it to the on-chip global cache module for storage.

[0013] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the hybrid NoC interconnect communication protocol includes a bus communication protocol for bus links and a NoC interconnect communication protocol for NoC links, wherein,

[0014] The bus communication protocol consists of two parts: data frame format and transmission mode; among them,

[0015] The data frame format specifies the uniform format used for data transmitted within the bus link. This format includes a 1-bit valid field, a 1-bit flag field, a 2-bit type field, a Q-bit destination address field, and an H-bit payload field.

[0016] The transmission mode supports both one-to-one and one-to-many. The specific implementation method is as follows: each PE listens to the bus information, compares the destination address of the bus information with its own address, and if the addresses match, it receives and decodes the information.

[0017] NoC interconnection communication protocols include flow control mechanisms, routing protocols, arbitration mechanisms, and packet formats.

[0018] The flow control mechanism is used to dynamically manage the data flow between routers. It adopts a credit-based flow control mechanism. Each router's output port is equipped with a counter to record the free capacity of the input buffer in the corresponding direction of the downstream router. During communication, adjacent routers update the counter status through the credit signal. If the counter value decreases to 0, it means that the downstream buffer is full. At this time, the upstream router will pause transmission until it receives the credit signal from the downstream to release space.

[0019] The routing protocol is used to plan the path of data packets in the NoC link; it adopts a deadlock-free XY multicast routing algorithm, which first transmits data along the X-axis direction. When the destination node is detected in the Y-axis direction, the data packet is copied, with one part transmitted along the Y-axis direction and the other part continuing to be transmitted along the X-axis direction.

[0020] The arbitration mechanism handles concurrent requests from multiple input ports to the same output port in a router, dynamically allocating resources to prevent packet loss due to collisions. The arbitration mechanism is a fixed-priority two-step arbitration process. When arbitration occurs, a preliminary arbitration is performed according to the fixed priority of the first step, approving the request with the highest priority. Subsequent unapproved requests enter a second round of arbitration, where the compatibility with approved requests is further analyzed according to the priority order of the second step, and transmission permissions are allocated. The specific priorities and arbitration rules are as follows:

[0021] The first step is to conduct an initial arbitration for all competing requests, prioritizing them from highest to lowest input port (East, West, Local, North, South), and approve the successful arbitration request.

[0022] The second step is to conduct a second round of arbitration among the remaining unapproved requests, based on the input port priority from high to low: South, North, Local, West, East, and approve requests that do not conflict with the requests approved in the first round.

[0023] The data packet format is used to specify the uniform format of data transmitted in the network. Data packets are transmitted in units of chips in NoC. Each data packet includes a header chip with a destination address and a feature map data ID, several body chips carrying feature map data, and a tail chip that marks the end of the data packet.

[0024] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the network interface includes a protocol conversion module, a data encapsulation / decapsulation module, and an input / output buffer; wherein,

[0025] The protocol conversion module is used to convert local data from PE into a data packet format that conforms to the NoC communication protocol, or to convert NoC data packets into PE data format;

[0026] The data encapsulation / decapsulation module is used to encapsulate data into micro-pieces according to the NoC protocol format, add header and tail micro-pieces to the data micro-piece sequence, encapsulate into data packets, and decapsulate the received data packets, stripping the header information and restoring the original data;

[0027] Input / output buffers are used for cross-clock domain data exchange and data bit width matching between the PE and the network interface.

[0028] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the router includes input ports, an input FIFO buffer, a routing calculation module, a credit calculation module, an arbitration module, a crossbar switch, and output ports. The input ports have five ports: East, West, South, North, and Local. Each port corresponds to a dedicated input FIFO buffer for temporarily storing data fragments to be routed. The routing calculation module calculates the set of output ports corresponding to the fragments from each input port based on the routing algorithm, the destination address information of the data packets, and the current router address information, and passes the result to the arbitration module. The credit calculation module calculates the current router's credit value based on the current buffer occupancy, sends the current router's idle capacity to downstream routers and the local port in real time, and receives their credit values. The arbitration module determines whether multiple input ports are competing for the same output port. If there is no competition, it directly approves the request; if there is competition, it allocates transmission rights according to a two-step arbitration mechanism with fixed priority. After arbitration, the approved data fragments are forwarded from the input port to the corresponding output port via the crossbar switch.

[0029] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the processing unit PE includes a PE buffer module, a receive decoding module, a PE controller, input / output buffers, and a computation module, wherein...

[0030] Input data includes input feature map data, weight data, bias data, and configuration information. Shared data consists of intermediate data. CNN inference result data is a summary of output feature map data generated by multiple PEs.

[0031] The PE cache module is responsible for storing the input feature map data, weight data, bias data, intermediate data, and output feature map data generated by the PE calculation. The PE cache module is connected to the calculation module through a data bus for data exchange.

[0032] The receiving and decoding module is responsible for receiving configuration information, input feature map data, weight data, and bias data from the bus link. The configuration information is decoded and sent to the PE controller. The input feature map data, weight data, and bias data are decoded and stored in the PE cache module.

[0033] After receiving the decoded configuration information, the PE controller generates the required control signals for each module in the PE, and controls each module to collaboratively execute the CNN computation task.

[0034] Input / output buffers are used for cross-clock domain data exchange and data bit width matching between the PE buffer module and the network interface when receiving and transmitting intermediate data;

[0035] The computation module is used to perform convolution, pooling, and fully connected computations in the CNN based on the input feature map data, weight data, bias data, configuration information, and intermediate data, and obtain the output feature map data.

[0036] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the configuration information includes the type of computation task, whether convolutional pooling is fused, the size of the input and output feature maps, the size of the convolutional kernel, the number of sub-maps into which the output feature map is divided, the amount of data in each sub-map, and the destination address of each sub-map; the number of network interfaces, routers, and processing units (PE) is the same.

[0037] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the computation module includes a convolution computation submodule, a pooling computation submodule, and a fully connected computation submodule. The convolution computation submodule includes N multi-channel systolic arrays and an output addition tree. A multi-channel systolic array contains D vertically cascaded single-channel computation arrays. A single-channel computation array consists of P×P MAC units, which are horizontally cascaded in rows. A multi-channel systolic array includes a total of P×P×D MAC units. The N multi-channel systolic arrays are compatible with convolution computations with different numbers of input channels through the output addition tree structure, where N, P, and D are all integers greater than 0.

[0038] The pooling calculation submodule includes two levels of comparison logic and a cache unit with a width consistent with the bit width of the feature map data and a depth of M, which is responsible for performing max pooling calculation, where the value of M is consistent with the maximum size of the input feature map.

[0039] As a further optimization scheme for a CNN accelerator based on a hybrid NoC architecture as described in this invention, the fully connected computation submodule consists of G groups of parallel multiply-accumulate trees. Each time the fully connected computation submodule is reused, G groups of weights and input data are read from the cache unit at once, G groups of parallel multiplication operations are performed, and the product results are accumulated level by level. The fully connected computation submodule will be reused multiple times until all input data is calculated, thereby completing the fully connected layer computation.

[0040] As a further optimization scheme for the CNN accelerator based on a hybrid NoC architecture described in this invention, the PE is reused multiple times to complete the computation task. The process of the PE performing one computation task is as follows:

[0041] Step 1, Parameter Configuration: The configuration information is received via the bus link, decoded by the receiving and decoding module, and then sent to the PE controller. After receiving the decoded configuration information, the PE controller generates an enable signal to start the computing module.

[0042] Step 2, Data Loading: Receive the weights, biases, and input feature map data required for calculation via the bus link, receive intermediate data via the NoC link, decode the data by the receiving and decoding module, and store it in the corresponding partition of the PE cache module. After the data loading is complete, the receiving and decoding module sends a pulse to the PE controller to indicate that the data is ready.

[0043] Step 3, Start Calculation: The PE controller sends a start signal and reads weight and bias data from the PE cache module; if it is the first reuse, it reads input feature map data from the PE cache module; otherwise, it reads intermediate data and sends the read data to the calculation module at the speed of the matching calculation pipeline to start the calculation.

[0044] Step 4, Calculation result caching: When the calculation module starts generating output data, the PE controller generates an address and enable signal, and writes the calculation result into the feature map cache area of ​​the PE cache module;

[0045] Step 5, Output of Calculation Results: Once the calculation is complete, all calculation results have been written to the feature map buffer. The PE controller generates an output control signal according to the configuration information, retrieves the calculation result data that needs to be transmitted through the NoC link from the feature map buffer, and outputs it to the network interface through the output FIFO buffer.

[0046] As a further optimization scheme for the CNN accelerator based on a hybrid NoC architecture described in this invention, the computation module in PE includes a convolution computation submodule, and the computation method of the convolution computation submodule is as follows:

[0047] Step A, Task Allocation and Configuration: The PE controller allocates the number of input channels C to N multi-channel systolic arrays of size P×P×D based on the received convolution calculation configuration information. When C≤D, each multi-channel systolic array independently completes the convolution operation of one convolution kernel, achieving parallel calculation of N convolution kernels, with a corresponding output parallelism of N. When D<C≤2D, two multi-channel systolic arrays collaboratively calculate one convolution kernel, with each multi-channel systolic array responsible for the convolution calculation of C / 2 channels. The calculation results of the two arrays are added together through an addition tree to obtain the complete convolution output, with a corresponding output parallelism of N / 2. This structure supports a maximum of D×N channels of convolution calculation in one operation. For the case where C>D×N, the multi-channel systolic arrays are reused multiple times and the results are accumulated to complete the calculation.

[0048] Step B, Parallel Computation and Accumulation: The convolutional computation submodule is driven by the enable signal generated by the PE controller to start parallel computation of N multi-channel systolic arrays. The single-channel computation arrays within each multi-channel systolic array group maintain a delay interval of 4 cycles. Starting from the first channel, the required weights, biases, and input data are received sequentially from the cache module. The input data is transmitted from left to right along the horizontal cascade structure inside the single-channel computation array. The part and result generated by each single-channel computation array are accumulated sequentially along the vertical cascade structure to form pipeline accumulation between channels.

[0049] Step C, Result Merging and Output: The addition tree accumulates the outputs of the N multi-channel systolic arrays according to the configuration information and produces the final convolution output result.

[0050] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0051] (1) This invention proposes a hybrid NoC architecture CNN accelerator. This architecture transmits configuration information and intermediate data during the computation process on the bus link and the NoC link respectively through a hybrid NoC interconnect link, resulting in a smaller communication latency when performing CNN tasks.

[0052] (2) The configurable processing unit PE proposed in this invention has an internal cascaded structure of a multi-channel pulsating array that enables pipelined accumulation between channels, minimizing the consumption of register resources; the addition tree structure makes the PE compatible with convolution calculations with different numbers of input channels, and can achieve high computational parallelism. Attached Figure Description

[0053] Figure 1This is a diagram of the overall architecture of a CNN accelerator based on a hybrid NoC architecture;

[0054] Figure 2 This is a flowchart of the workflow of a CNN accelerator based on a hybrid NoC architecture;

[0055] Figure 3 This is a diagram of the data frame format of the bus communication protocol;

[0056] Figure 4 This is a diagram of the data packet format in the NoC interconnect communication protocol;

[0057] Figure 5 This is a schematic diagram of the flow control mechanism based on Credit; where (a) is an example of NoC packet transmission, (b) is the packet transmission situation of the south input buffer, and (c) is the packet transmission situation of the west input buffer.

[0058] Figure 6 This is a router structure diagram;

[0059] Figure 7 This is a structural diagram of the routing calculation module;

[0060] Figure 8 This is a structural diagram of the arbitration module;

[0061] Figure 9 This is a structural diagram of the PE module;

[0062] Figure 10 This is a diagram of a single-channel computing array structure;

[0063] Figure 11 This is a diagram of a multi-channel pulsating array structure. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] A CNN accelerator based on a hybrid NoC architecture includes:

[0066] The hybrid NoC interconnect link comprises multiple network interfaces, multiple routers, multiple processing units (PE), a global controller, and an on-chip global cache module. The hybrid NoC interconnect link includes bus links and NoC links.

[0067] Each PE (Preinstallation Environment) is connected to its respective router via a network interface. The routers are interconnected via NoC (No-Chip) links, forming the communication path between the PEs. Simultaneously, each PE is connected to the global controller and the on-chip global cache module via bus links.

[0068] The network interface is responsible for converting shared data from the PE into a format conforming to the NoC communication protocol and encapsulating it into data packets for transmission in the NoC link, as well as decapsulating data packets received from the NoC link and converting them back into their original format;

[0069] A router is responsible for selecting an output port from which to send a data packet based on the packet's destination address and the routing algorithm.

[0070] The global controller is used to generate control signals and transmit input data read from the on-chip global cache module to each PE via the bus link;

[0071] The processing unit (PE) is used to perform CNN computation tasks based on input data received from the bus link and shared data received from the NoC link, obtain CNN inference result data, and output it to the on-chip global cache module for storage.

[0072] Hybrid NoC interconnect communication protocols include bus communication protocols for bus links and NoC interconnect communication protocols for NoC links, wherein,

[0073] The bus communication protocol consists of two parts: data frame format and transmission mode; among them,

[0074] The data frame format specifies the uniform format used for data transmitted within the bus link. This format includes a 1-bit valid field, a 1-bit flag field, a 2-bit type field, a Q-bit destination address field, and an H-bit payload field.

[0075] The transmission mode supports both one-to-one and one-to-many. The specific implementation method is as follows: each PE listens to the bus information, compares the destination address of the bus information with its own address, and if the addresses match, it receives and decodes the information.

[0076] NoC interconnection communication protocols include flow control mechanisms, routing protocols, arbitration mechanisms, and packet formats.

[0077] The flow control mechanism is used to dynamically manage the data flow between routers. It adopts a credit-based flow control mechanism. Each router's output port is equipped with a counter to record the free capacity of the input buffer in the corresponding direction of the downstream router. During communication, adjacent routers update the counter status through the credit signal. If the counter value decreases to 0, it means that the downstream buffer is full. At this time, the upstream router will pause transmission until it receives the credit signal from the downstream to release space.

[0078] The routing protocol is used to plan the path of data packets in the NoC link; it adopts a deadlock-free XY multicast routing algorithm, which first transmits data along the X-axis direction. When the destination node is detected in the Y-axis direction, the data packet is copied, with one part transmitted along the Y-axis direction and the other part continuing to be transmitted along the X-axis direction.

[0079] The arbitration mechanism handles concurrent requests from multiple input ports to the same output port in a router, dynamically allocating resources to prevent packet loss due to collisions. The arbitration mechanism is a fixed-priority two-step arbitration process. When arbitration occurs, a preliminary arbitration is performed according to the fixed priority of the first step, approving the request with the highest priority. Subsequent unapproved requests enter a second round of arbitration, where the compatibility with approved requests is further analyzed according to the priority order of the second step, and transmission permissions are allocated. The specific priorities and arbitration rules are as follows:

[0080] The first step is to conduct an initial arbitration for all competing requests, prioritizing them from highest to lowest input port (East, West, Local, North, South), and approve the successful arbitration request.

[0081] The second step is to conduct a second round of arbitration among the remaining unapproved requests, based on the input port priority from high to low: South, North, Local, West, East, and approve requests that do not conflict with the requests approved in the first round.

[0082] The data packet format is used to specify the uniform format of data transmitted in the network. Data packets are transmitted in units of chips in NoC. Each data packet includes a header chip with a destination address and a feature map data ID, several body chips carrying feature map data, and a tail chip that marks the end of the data packet.

[0083] The network interface includes a protocol conversion module, a data encapsulation / decapsulation module, and input / output buffers; among which,

[0084] The protocol conversion module is used to convert local data from PE into a data packet format that conforms to the NoC communication protocol, or to convert NoC data packets into PE data format;

[0085] The data encapsulation / decapsulation module is used to encapsulate data into micro-pieces according to the NoC protocol format, add header and tail micro-pieces to the data micro-piece sequence, encapsulate into data packets, and decapsulate the received data packets, stripping the header information and restoring the original data;

[0086] Input / output buffers are used for cross-clock domain data exchange and data bit width matching between the PE and the network interface.

[0087] The router includes input ports, an input FIFO buffer, a routing calculation module, a credit calculation module, an arbitration module, a crossbar switch, and output ports. The input ports are designated as East, West, South, North, and Local. Each port has a dedicated input FIFO buffer for temporarily storing data fragments to be routed. The routing calculation module calculates the set of output ports corresponding to the fragments from each input port based on the routing algorithm, the destination address information of the data packets, and the current router address information, and passes the result to the arbitration module. The credit calculation module calculates the current router's credit value based on the current buffer occupancy, sends the current router's available capacity to downstream routers and the local port in real time, and receives their credit values. The arbitration module determines whether multiple input ports are competing for the same output port. If there is no competition, it directly approves the request; if there is competition, it allocates transmission rights according to a two-step arbitration mechanism with fixed priority. After arbitration, the approved data fragments are forwarded from the input port to the corresponding output port via the crossbar switch.

[0088] The processing unit (PE) includes a PE buffer module, a receive decoding module, a PE controller, input / output buffers, and a calculation module.

[0089] Input data includes input feature map data, weight data, bias data, and configuration information. Shared data consists of intermediate data. CNN inference result data is a summary of output feature map data generated by multiple PEs.

[0090] The PE cache module is responsible for storing the input feature map data, weight data, bias data, intermediate data, and output feature map data generated by the PE calculation. The PE cache module is connected to the calculation module through a data bus for data exchange. The PE cache module includes a feature map cache area, a weight cache area, a bias cache area, and intermediate data is stored in the feature map cache area.

[0091] The receiving and decoding module is responsible for receiving configuration information, input feature map data, weight data, and bias data from the bus link. The configuration information is decoded and sent to the PE controller. The input feature map data, weight data, and bias data are decoded and stored in the PE cache module.

[0092] After receiving the decoded configuration information, the PE controller generates the required control signals for each module in the PE, and controls each module to collaboratively execute the CNN computation task.

[0093] Input / output buffers are used for cross-clock domain data exchange and data bit width matching between the PE buffer module and the network interface when receiving and transmitting intermediate data;

[0094] The computation module is used to perform convolution, pooling, and fully connected computations in the CNN based on the input feature map data, weight data, bias data, configuration information, and intermediate data, and obtain the output feature map data.

[0095] The configuration information includes the type of computation task, whether convolutional pooling is fused, the size of the input and output feature maps, the size of the convolutional kernel, the number of sub-maps into which the output feature map is divided, the amount of data in each sub-map, and the destination address of each sub-map; the number of network interfaces, routers, and processing units (PEs) is the same.

[0096] The computation module includes a convolution computation submodule, a pooling computation submodule, and a fully connected computation submodule. The convolution computation submodule includes N multi-channel systolic arrays and an output addition tree. Each multi-channel systolic array contains D vertically cascaded single-channel computation arrays. Each single-channel computation array consists of P×P MAC units, which are horizontally cascaded in rows. Each multi-channel systolic array contains a total of P×P×D MAC units. The N multi-channel systolic arrays are compatible with convolution computations with different numbers of input channels through the output addition tree structure, where N, P, and D are all integers greater than 0.

[0097] The pooling calculation submodule includes two levels of comparison logic and a cache unit with a width consistent with the bit width of the feature map data and a depth of M, which is responsible for performing max pooling calculation, where the value of M is consistent with the maximum size of the input feature map.

[0098] The fully connected computation submodule consists of G groups of parallel multiply-accumulate trees. Each time the fully connected computation submodule is reused, the G groups of weights and input data are read from the cache unit at once, the G groups of parallel multiplication operations are performed, and the product results are accumulated level by level. The fully connected computation submodule will be reused multiple times until all the input data has been calculated, thereby completing the fully connected layer computation.

[0099] The PE is reused multiple times to complete the computation task. The process of PE executing a computation task is as follows:

[0100] Step 1, Parameter Configuration: The configuration information is received via the bus link, decoded by the receiving and decoding module, and then sent to the PE controller. After receiving the decoded configuration information, the PE controller generates an enable signal to start the computing module.

[0101] Step 2, Data Loading: Receive the weights, biases, and input feature map data required for calculation via the bus link, receive intermediate data via the NoC link, decode the data by the receiving and decoding module, and store it in the corresponding partition of the PE cache module. After the data loading is complete, the receiving and decoding module sends a pulse to the PE controller to indicate that the data is ready.

[0102] Step 3, Start Calculation: The PE controller sends a start signal and reads weight and bias data from the PE cache module; if it is the first reuse, it reads input feature map data from the PE cache module; otherwise, it reads intermediate data and sends the read data to the calculation module at the speed of the matching calculation pipeline to start the calculation.

[0103] Step 4, Calculation result caching: When the calculation module starts generating output data, the PE controller generates an address and enable signal, and writes the calculation result into the feature map cache area of ​​the PE cache module;

[0104] Step 5, Calculation Result Output: When the calculation is completed, all calculation results have been written to the feature map buffer. The PE controller generates an output control signal according to the configuration information, retrieves the calculation result data that needs to be transmitted through the NoC link from the feature map buffer, and outputs it to the network interface through the output FIFO buffer.

[0105] Finally, the output feature maps generated by multiple PEs are aggregated into one PE through a NoC link to form the CNN inference result data.

[0106] The computation module in PE includes a convolution computation submodule, and the computation method of the convolution computation submodule is as follows:

[0107] Step A, Task Allocation and Configuration: The PE controller allocates the number of input channels C to N multi-channel systolic arrays of size P×P×D based on the received convolution calculation configuration information. When C≤D, each multi-channel systolic array independently completes the convolution operation of one convolution kernel, achieving parallel calculation of N convolution kernels, with a corresponding output parallelism of N. When D<C≤2D, two multi-channel systolic arrays collaboratively calculate one convolution kernel, with each multi-channel systolic array responsible for the convolution calculation of C / 2 channels. The calculation results of the two arrays are added together through an addition tree to obtain the complete convolution output, with a corresponding output parallelism of N / 2. This structure supports a maximum of D×N channels of convolution calculation in one operation. For the case where C>D×N, the multi-channel systolic arrays are reused multiple times and the results are accumulated to complete the calculation.

[0108] Step B, Parallel Computation and Accumulation: The convolutional computation submodule is driven by the enable signal generated by the PE controller to start parallel computation of N multi-channel systolic arrays. The single-channel computation arrays within each multi-channel systolic array group maintain a delay interval of 4 cycles. Starting from the first channel, the required weights, biases, and input data are received sequentially from the cache module. The input data is transmitted from left to right along the horizontal cascade structure inside the single-channel computation array. The part and result generated by each single-channel computation array are accumulated sequentially along the vertical cascade structure to form pipeline accumulation between channels.

[0109] Step C, Result Merging and Output: The addition tree accumulates the outputs of the N multi-channel systolic arrays according to the configuration information and produces the final convolution output result.

[0110] like Figure 1 As shown, a CNN accelerator based on a hybrid NoC architecture (taking a 4×4 scale as an example) includes a hybrid NoC link, a router, a network interface, a processing unit (PE), a global controller module, and an on-chip global cache module; the hybrid NoC link includes a bus link and a NoC link.

[0111] After the weight data and input feature map data are transferred from the off-chip DDR to the on-chip global cache module, the CNN accelerator begins to work, and the workflow is as follows: Figure 2 As shown.

[0112] Step 1, Convolutional Layer Configuration Stage: Based on the task allocation and mapping method, the convolutional layer computation tasks in the CNN computation task are mapped to the PE. The global controller transmits convolutional configuration information to the PE through the bus link, including the computation task type, whether convolution and pooling are fused, the input and output feature map sizes, the convolutional kernel size, the number of sub-maps into which the output feature map is split, the data volume of each sub-map, and the destination address of each sub-map. Among them, whether convolution and pooling are fused is used to indicate the computation mode of the convolutional layer and the pooling layer. When this parameter is 0, it means that the current task is a separate convolutional layer computation. When this parameter is 1, it means that the convolutional layer and the following pooling layer are fused into a composite layer for computation.

[0113] Step 2, Convolutional Layer Data Transmission Stage: The global controller generates control signals to read the input data required by each PE from the on-chip global cache module and transmits it to each PE through the bus. The PE listens to the information on the bus, receives and decodes data whose destination address is consistent with its own address.

[0114] Step 3, Convolution Calculation and Result Output Stage: The PE that receives the input data begins to execute the convolution calculation task. After the calculation module inside the PE completes the calculation, it stores the result in the PE cache module, and then outputs it to the network interface through the output buffer for protocol conversion and data packet encapsulation, and transmits it to the destination convolution PE via the NoC link. After receiving all the required data, the destination convolution PE continues to perform a series of operations including calculation, storage, and forwarding.

[0115] Step 4, Input Sharing Configuration Phase: After all PEs have completed one computation, the accelerator enters the input sharing configuration phase, notifying some PEs to transfer some of their held feature map data to other PEs, so as to avoid reading input feature map data from external storage during the next mapping.

[0116] Step 5, Input Sharing Execution Phase: Each PE transmits a portion of its data to other PEs via the NoC link according to the configuration in the previous step, completing all computation and data transmission tasks for one convolutional layer mapping; repeat the above steps until all convolutional layers have been computed.

[0117] Step 6, Fully Connected Layer Configuration Stage: After all convolutional layers have been computed, the fully connected layer computation stage begins. The fully connected layer computation tasks are mapped to PEs according to the task allocation and mapping method. The global controller transmits the fully connected configuration information, including the input and output dimensions of the fully connected layer and the output destination address, to the corresponding PEs via the bus.

[0118] Step 7, Fully Connected Layer Data Transmission Stage: The global controller generates control signals, reads the input data required by each PE from the on-chip global cache module, and transmits it to each PE through the bus. The PE listens to the information on the bus, receives and decodes data whose destination address is consistent with its own address.

[0119] Step 8, Fully Connected Computation Phase: The PE that receives the input data begins to execute the fully connected layer computation task. After the computation module inside the PE completes the computation, it stores the result in the PE cache module, and then outputs it to the network interface through the output buffer. After protocol conversion and data packet encapsulation, it is transmitted to the destination PE through the NoC link. After the destination PE obtains the required data, it continues to perform a series of operations such as computation, storage and forwarding. The above steps are repeated until all fully connected layer computation tasks are completed. After the task is completed, each PE transmits the output feature map data results it holds to the PE located in the lower right corner of the network through the NoC link, and the results are aggregated to obtain the CNN inference results.

[0120] Step 9, Result Output Stage: The PE holding the final inference result transmits the data to the on-chip global cache module via the bus, and then transmits it back to the off-chip DDR.

[0121] Figure 3 The specific design of the data frame format in the bus communication protocol is given, in which,

[0122] Valid field (1 bit): Indicates whether the information is valid, "1" represents valid, and "0" represents invalid;

[0123] 1-bit flag field: distinguishes between configuration parameters and data information; "1" represents data information and "0" represents configuration parameters.

[0124] The 2-bit Type field identifies the specific configuration or data type. In the configuration information, there are four configuration types: convolution configuration with pooling (corresponding to "00"), convolution configuration without pooling (corresponding to "01"), input-shared configuration (corresponding to "10"), and fully connected layer configuration (corresponding to "11"). In the data information, there are three types: weight parameters (corresponding to "00"), bias parameters (corresponding to "01"), and first-layer input image data (corresponding to "10").

[0125] Q-bit Destination field: Indicates the destination address of the information. Each bit in this field represents a PE node. The bit corresponding to the destination node is set to "1", and the bit corresponding to a non-destination node is "0".

[0126] H-position payload field: Contains specific configuration information or data content.

[0127] Figure 4 This is a schematic diagram of the data packet format in the NoC interconnection communication protocol. A NoC data packet consists of a header micro-piece, several body micro-pieces carrying data, and a tail micro-piece indicating the end of the packet. The header micro-piece includes a 2-bit micro-piece type field, where "00" indicates an invalid micro-piece, "01" indicates a header micro-piece, "11" indicates a body micro-piece, and "10" indicates a tail micro-piece. Next is the Q-bit destination address field, where each bit represents the location of a node in the NoC network. The bit corresponding to the destination node is set to "1", and the bit for non-destination nodes is "0", thus flexibly supporting unicast, multicast, and broadcast communication requirements. Following this is a 6-bit data ID field, used to distinguish different feature map data streams in the network. Finally, there is a J-bit payload field, used to carry the feature map data. Both the body micro-piece and the tail micro-piece contain a 2-bit micro-piece type field and a payload field.

[0128] Figure 5 This is a diagram illustrating a credit-based flow control mechanism. Router B's west input buffer is initially empty, and Router A's east output port count is 8, corresponding to the capacity of Router B's west input buffer. Router A has two data packets that need to be transmitted to Router B through its east output port. The data packet from the south input buffer is transmitted first. The specific transmission steps are as follows:

[0129] Step A: Query the Credit value of the east output port of router A. If it is greater than 0, start transmitting the data packets in the south input buffer.

[0130] Step B: After the data packet from the south input buffer of router A is transmitted, the Credit value of the east output port is 2. Router A determines that it can continue to transmit, and the data packet from the west input buffer begins to be transmitted.

[0131] Step C: After router A transmits two micro-slices of the data packet in the west input buffer, the Credit value of the east output port is 0, router A's transmission is paused, and it waits for the downstream buffer to release space.

[0132] Step D: When router A detects that the Credit value of the east output port is greater than 0, it continues to transmit.

[0133] Figure 6 This is a router structure diagram. Data slices enter the router through the input ports and are stored in the corresponding input FIFO buffer. After being read from the FIFO, the data slices enter the routing calculation module. Based on the destination address information and the current location of the router, the routing algorithm calculates the set of output ports corresponding to each input port slice and passes the result to the arbitration module. The arbitration module allocates transmission permissions according to a two-step arbitration mechanism with fixed priority based on the routing calculation results and the credit values ​​of each output port. Requests approved through arbitration forward the data slices from the input ports to the corresponding output ports via a crossbar switch.

[0134] Figure 7 The diagram shows the structure of the routing calculation module, which mainly consists of two sub-modules: a chip type determiner and a routing determiner. When chips from the five input directions are retrieved from the input buffer, the chip type field FT_x[1:0] (where x represents a direction) is extracted and sent to the chip type determiner to identify the current chip type. The chip type determiner outputs the FT_head[4:0] signal, where each bit corresponds to a direction; "1" indicates the chip in that direction is a head chip, otherwise it is "0". The FT_head[4:0] signal is input to the routing determiner. Only when the chip is a head chip is routing determined for the destination address field signal Dest_x[15:0] of that direction. In the Dest_x[15:0] signal, one bit corresponds to a resource node; the bit corresponding to the destination node is set to "1", and the bit for non-destination nodes is "0". The routing determiner performs routing determination based on the destination address field of each input direction combined with its own address, and outputs the chip's output port set Port_set[4:0][4:0]. The signal is a two-dimensional array. Each dimension of the array, from the most significant bit to the least significant bit, represents the five directions: north, south, east, west, and local. The first dimension represents the input direction, and the second dimension represents the output direction. If a certain input direction requests a certain output direction, the corresponding position is "1".

[0135] Figure 8This is a schematic diagram of the arbitration module, which mainly consists of three parts: a contention determiner, a priority determiner, and a delay unit. The result of the routing calculation module, the Port_set[4:0][4:0] signal, and the downstream router's buffer free capacity, the Credit[19:0] signal, are input to the contention determiner. The contention determiner judges the requests from the five input ports and outputs the Comp[4:0] signal, indicating which input ports' requests are competing. Priority determiner 1 spends one clock cycle performing the first round of arbitration, arbitrating according to the fixed priority from the first step, approving successfully arbitrated requests, and outputting the Arbit1[4:0] signal. Simultaneously, both the Port_set[4:0][4:0] and Comp[4:0] signals pass through the delay unit, outputting the Port_set1[4:0][4:0] and Comp1[4:0] signals, which are then sent to priority determiner 2. The second round of arbitration is performed according to the priority from the second step, approving requests that do not conflict with those approved in the first round, and finally outputting the Arbit2[4:0] signal to control the non-blocking crossbar switch to transmit all approved requests simultaneously.

[0136] Figure 9 This is a schematic diagram of the Processing Unit (PE), which consists of a PE buffer module, a receive / decode module, a PE controller, input / output buffers, and a computation module. The PE buffer module stores input / output feature maps, weights, biases, and some temporary data used in computation. The receive / decode module receives configuration and data information from the bus; the decoded configuration information is sent to the PE controller, which then performs operations such as enabling signal generation, pointer traversal, and jumps. The input / output buffers are used for cross-clock domain data exchange and data bit width matching between the PE and the network interface. The computation module handles the specific computational tasks of CNN forward inference, including convolution, pooling, and fully connected layer computations.

[0137] Figure 10 The diagram shows a single-channel computation array structure. P×P MAC units are horizontally cascaded row by row. Convolutional weights are stored fixedly within each unit, while the P rows of input feature map data are sequentially fed into the leftmost multiply-accumulate unit of the array each clock cycle and passed horizontally with each clock cycle. This matches the sliding window feature of convolutional data, enabling input data reuse between MAC units. Each MAC unit is responsible for calculating O = A×B + C, where A represents the input feature map data, B represents the weight data, C represents the partial sum from the previous channel, and O represents the output data.

[0138] Figure 11This diagram illustrates a multi-channel systolic array structure. To achieve multi-channel convolution calculations, D single-channel computation arrays are cascaded vertically to form a multi-channel systolic array. The C signal line of the next channel is connected to the O signal line of the previous channel. After the computation array of the first channel completes its calculation, the result is passed to the computation array of the second channel. The MAC unit of the second channel multiplies the input data and corresponding weights, adds the result passed from the previous channel, and passes the calculation result to the third channel, and so on. This cascaded transmission method allows for continuous computation, but it also dictates that the second channel must have a certain delay compared to the first channel when the input data is sent to the computation array. This design uses a pipeline in the multiplication operation of the MAC unit, using 4 clock cycles to complete the calculation function of A×B+C, meaning that the input data of the next channel must undergo a delay of 4 cycles compared to the input data of the previous channel. After the array of the Dth channel completes its calculation, an addition tree is used to add the calculation results of the same window to obtain the final convolution output. The convolution computation submodule of this design contains N multi-channel systolic arrays consisting of P×P×D MAC units, where P represents the kernel size dimension and D represents the channel dimension.

[0139] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A CNN accelerator based on hybrid NoC architecture, characterized in that, The hybrid NoC interconnection communication protocol comprises a bus communication protocol of the bus link and a NoC interconnection communication protocol of the NoC link, wherein, the bus communication protocol comprises a data frame format and a transmission mode; wherein, the data frame format specifies a format uniformly adopted by data transmitted in the bus link, and the format comprises a 1-bit valid field, a 1-bit flag field, a 2-bit type field, a Q-bit destination address field, and a H-bit payload field; the transmission mode supports one-to-one and one-to-many, and the specific implementation method is that each PE listens to bus information, compares the destination address of the bus information with the address of the PE, and if the addresses match, the PE receives and decodes; the NoC interconnection communication protocol comprises a flow control mechanism, a routing protocol, an arbitration mechanism, and a data packet format, wherein, the flow control mechanism is used for dynamically managing the data flow between the routers; a credit-based flow control mechanism is adopted, each output port of the router is provided with a counter for recording the free capacity of the input buffer of the downstream router in the corresponding direction, in the communication process, the counter state is updated between adjacent routers through a credit signal, and if the value of the counter decreases to 0, it indicates that the downstream buffer is full, at this time, the upstream router will pause transmission until a credit signal of the released space of the downstream is received; the routing protocol is used for planning the path of the data packet in the NoC link; a non-deadlock XY multicast routing algorithm is adopted, the data is first transmitted along the X-axis direction, and when the destination node is detected in the Y-axis direction, the data packet is copied, part of the data packet is transmitted along the Y-axis direction, and the other part continues to be transmitted along the X-axis direction; and 2.The CNN accelerator based on a hybrid NoC architecture according to claim 1, wherein, the arbitration mechanism is used for arbitrating the transmission right of the data packet in the NoC link; a round-robin arbitration mechanism is adopted, and the arbitration mechanism is responsible for selecting a data packet from the input buffer of the router to be transmitted through the output port of the router. the global controller is used for generating a control signal and transmitting the input data read from the on-chip global cache module to each PE through the bus link; the processing unit PE is used for executing the calculation task of the CNN according to the input data received from the bus link and the shared data received from the NoC link, obtaining the CNN inference result data, and outputting the CNN inference result data to the on-chip global cache module for storage. ​ ​ ​ ​ The arbitration mechanism is used to process the concurrent requests of multiple input ports to the same output port in the router, dynamically allocate resources to avoid the loss of data packets after the conflict occurs. The arbitration mechanism is a fixed priority two-step arbitration mechanism. When the arbitration situation occurs, the first step of the fixed priority is used for preliminary arbitration, and the request with the highest priority is approved. Then, the unapproved requests enter the second round of arbitration, and the compatibility of the unapproved requests is further analyzed according to the priority order of the second step. The transmission permission is allocated, and the specific priority and arbitration rule are as follows: In the first step, all the competing requests are initially arbitrated according to the priority of the input port from high to low, and the approved request is approved. In the second step, the remaining unapproved requests are arbitrated again according to the priority of the input port from high to low, and the request that does not conflict with the first round of approved requests is approved. The data packet format is used to specify the format of the data uniformly used in the network. The data packet is transmitted in the network in the form of flits. Each data packet includes a header flit with a destination address and a feature map data ID, a plurality of body flits carrying feature map data, and a tail flit indicating the end of the data packet.

3. The CNN accelerator based on hybrid NoC architecture according to claim 1, wherein, The network interface includes a protocol conversion module, a data encapsulation / decapsulation module, and an input / output buffer. Among them, The protocol conversion module is used to convert the local data from the PE into the data packet format conforming to the NoC communication protocol, or convert the NoC data packet into the PE data format. The data encapsulation / decapsulation module is used to encapsulate the data into flits according to the NoC protocol format, add a header flit and a tail flit to the data flit sequence, encapsulate it into a data packet, and decapsulate the received data packet, strip the header information and restore the original data. The input / output buffer is used for cross-clock domain data exchange and data bit width matching between the PE and the network interface.

4. The CNN accelerator based on hybrid NoC architecture according to claim 1, wherein, The router includes an input port, an input FIFO buffer, a routing calculation module, a Credit calculation module, an arbitration module, a crossbar switch, and an output port. The input port has five ports, east, west, south, north, and local. Each port corresponds to a dedicated input FIFO buffer for temporarily storing data flits to be routed. The routing calculation module calculates the output port set corresponding to the flits from each input port according to the routing algorithm, the data packet destination address information, and the current router address information, and transmits the result to the arbitration module. The Credit calculation module calculates the Credit value of the current router according to the occupation status of the current buffer, and sends the idle capacity of the current router to the downstream routers and the local port in real time, and receives their Credit values. The arbitration module determines whether there is a situation of multiple input ports competing for the same output port. If there is no competition, it is directly approved. If there is competition, the transmission permission is allocated according to the fixed priority two-step arbitration mechanism. After the arbitration is completed, the approved data flits are forwarded from the input port to the corresponding output port through the crossbar switch.

5. The CNN accelerator based on hybrid NoC architecture according to claim 1, wherein, The processing unit PE comprises a PE cache module, a receiving and decoding module, a PE controller, an input and output buffer, and a calculation module, wherein, The input data comprises input feature map data, weight data, bias data, and configuration information, the shared data is intermediate data, and the CNN inference result data is output feature map data aggregated by the plurality of PEs; The PE cache module is configured to store the input feature map data, the weight data, the bias data, the intermediate data, and the output feature map data generated by the PE calculation. The receiving and decoding module is configured to receive the configuration information, the input feature map data, the weight data, and the bias data from the bus link. The PE controller is configured to generate the required control signals of the modules in the PE after receiving the decoded configuration information, and control the modules to cooperatively perform the CNN calculation task. The input and output buffer is configured to receive and transmit the intermediate data, and match the cross-clock domain data exchange and the data bit width of the PE cache module and the network interface. The calculation module is configured to complete the convolution calculation, the pooling calculation, and the full connection calculation in the CNN according to the input feature map data, the weight data, the bias data, the configuration information, and the intermediate data, and obtain the output feature map data.

6. The CNN accelerator based on hybrid NoC architecture according to claim 5, characterized in that, The configuration information comprises a calculation task type, whether the convolution and the pooling are fused, input and output feature map sizes, a convolution kernel size, a number of subgraphs into which the output feature map is divided, a data amount of each subgraph, and a destination address of each subgraph.

7. The CNN accelerator based on hybrid NoC architecture according to claim 5, characterized in that, The calculation module comprises a convolution calculation submodule, a pooling calculation submodule, and a full connection calculation submodule. The convolution calculation submodule comprises N multi-channel systolic arrays and an output addition tree. Each multi-channel systolic array comprises D single-channel calculation arrays arranged in a vertical cascade manner. Each single-channel calculation array comprises P*P MAC units arranged in a horizontal cascade manner.

8. The CNN accelerator based on hybrid NoC architecture according to claim 5, characterized in that, Each multi-channel systolic array comprises P*P*D MAC units.

9. The CNN accelerator based on hybrid NoC architecture according to claim 5, characterized in that, The pooling calculation submodule comprises two-level comparison logic and a cache unit with a depth of M and a bit width consistent with that of the feature map data. The full connection calculation submodule comprises G groups of parallel multiply-add trees. The PE is reused multiple times to complete the calculation task. The process of performing one calculation task by the PE is as follows. Step 1, parameter configuration: receive configuration information through the bus link, decode through the receiving decoding module and send to the PE controller, the PE controller receives the decoded configuration information, generates an enable signal to start the calculation module; Step 2, data loading: receive the weight, bias and input feature map data required for calculation through the bus link, receive the intermediate data through the NoC link, store in the corresponding partition of the PE cache module after decoding through the receiving decoding module, and send a pulse to the PE controller after data loading to indicate that the data is ready; Step 3, start calculation: the PE controller sends a start signal to read the weight and bias data from the PE cache module; If it is the first reuse, read the input feature map data from the PE cache module, otherwise read the intermediate data, and let the read data enter the calculation module at the speed of the matching calculation pipeline, and start calculation; Step 4, calculation result caching: when the calculation module starts to produce output data, the PE controller generates an address and an enable signal to write the calculation result to the feature map cache area of the PE cache module; Step 5, calculation result output: when the calculation is completed and all the calculation results have been written into the feature map cache area, the PE controller generates an output control signal according to the requirements of the configuration information to take out the calculation result data that needs to be transmitted through the NoC link from the feature map cache area, and outputs it to the network interface through the output FIFO buffer area.

10. The CNN accelerator based on hybrid NoC architecture according to claim 9, characterized in that, The calculation module in the PE includes a convolution calculation submodule, and the calculation method of the convolution calculation submodule is as follows: Step A, task allocation and configuration: the PE controller allocates the input channel number C to N multi-channel systolic arrays with a size of P×P×D according to the received convolution calculation configuration information, when C≤D, each multi-channel systolic array independently completes the convolution operation of one convolution kernel, realizing N convolution kernel parallel calculation, and the corresponding output parallel degree is N; when D<C≤2D, two multi-channel systolic arrays cooperatively calculate one convolution kernel, and each multi-channel systolic array is responsible for the convolution calculation of C / 2 channels, and the calculation results of the two are added through an addition tree to obtain the complete convolution output, and the corresponding output parallel degree is N / 2; this structure supports at most one-time completion of D×N channel convolution calculation; for the case of C>D×N, the multi-channel systolic array is reused multiple times and the results are accumulated to complete the calculation; Step B, parallel calculation and accumulation: the convolution calculation submodule is driven by the enable signal generated by the PE controller, and N multi-channel systolic arrays are started for parallel calculation, a delay interval of 4 cycles is maintained between the single-channel calculation arrays in each multi-channel systolic array group, and the required weight, bias and input data are received from the cache module from the first channel in turn, the input data is transmitted from left to right along the horizontal cascading structure inside the single-channel calculation array, and the partial sum results generated by each single-channel calculation array are sequentially accumulated along the vertical cascading structure, forming inter-channel pipeline accumulation; Step C, result merging and output: the addition tree accumulates the outputs of the N multi-channel systolic arrays according to the configuration information and generates the final convolution output result.

Images