Optimization method and system for improving bandwidth of neural network accelerator

Abstract

The invention discloses an optimization method and system for improving the bandwidth of a neural network accelerator, and relates to the technical field of neural network accelerators, and the optimization method for improving the bandwidth of the neural network accelerator comprises the following steps: carrying out the convolution processing of input data through a convolution pipeline; performing activation function processing on the data subjected to convolution processing by adopting a single data processor; an on-chip buffer is adopted to replace a traditional on-chip memory for caching, and pooling processing and local response normalization processing are carried out on the data after activation function processing through a plane data processor and a channel data processor; and performing data format conversion on the data subjected to the local response normalization processing through a data recombination engine. According to the invention, by optimizing the storage read-write process of the data when the accelerator performs acceleration operation, the utilization rate of memory bandwidth is effectively improved, the time consumed by single acceleration operation is reduced, and the performance of the accelerator is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network accelerators, and in particular to a method and system for optimizing the bandwidth of a neural network accelerator. Background Art

[0002] Convolutional neural networks are widely used in various deep learning fields, especially in edge computing applications. Mobile devices are often limited by available computing and storage resources, necessitating more efficient hardware to achieve smaller storage, faster inference, and lower power consumption. However, the increasing complexity of convolutional neural network (CNN) algorithms will significantly increase computing and storage requirements, posing a significant challenge to the design of deep neural network (DNN) hardware accelerators. In fields such as artificial intelligence and visual inspection, a large number of matrix operations are required. To process signals as quickly as possible, hardware accelerators are often required to improve the efficiency of parallel computing and enable rapid algorithm inference.

[0003] With the introduction of various neural network algorithms, various neural network accelerators have emerged. Early neural network accelerators were designed for specific networks. To improve the versatility and flexibility of neural network accelerators, general-purpose neural network accelerators were designed.

[0004] Currently, mainstream hardware accelerators have the following defects: since each step of calculation after convolution processing must be cached in the memory, it is limited by the bandwidth bottleneck of the memory. The memory bandwidth utilization rate of the hardware accelerator for single operation acceleration is too low, so the overall operation rate cannot be fully released. The overall accelerated operation performance still has a lot of room for improvement.

[0005] The current solution to the problem of low memory bandwidth utilization in hardware accelerators is mainly to improve storage performance by using high-performance external storage units, such as HBM. However, this results in higher overall power consumption and overall cost for the hardware accelerator. At the same time, bandwidth utilization is still limited by memory, and it is impossible to break through the bottleneck.

[0006] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0007] In response to the problems in the related art, the present invention proposes a method and system for improving the bandwidth optimization of a neural network accelerator to overcome the above-mentioned technical problems existing in the existing related art.

[0008] To this end, the specific technical solutions adopted in the present invention are as follows:

[0009] According to one aspect of the present invention, a method for optimizing the bandwidth of a neural network accelerator is provided. The method for optimizing the bandwidth of a neural network accelerator comprises the following steps:

[0010] S1, convolution processing is performed on the input data through the convolution pipeline;

[0011] S2, using a single data processor to perform activation function processing on the data after convolution processing;

[0012] S3, using on-chip cache instead of traditional on-chip storage for caching, and performing pooling and local response normalization on the data processed by the activation function through the plane data processor and channel data processor;

[0013] S4. The data format of the data after local response normalization processing is converted through the data reorganization engine, and data transfer between the dynamic random access memory and the high-performance storage interface is realized.

[0014] Furthermore, convolution processing of the input data through the convolution pipeline includes the following steps:

[0015] S11, using code division multiple access to batch read the image to be convolved, feature values ​​and weights, and store them in the convolution buffer in the order of data required for convolution;

[0016] S12. Transfer data from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence.

[0017] Furthermore, transferring data from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence includes the following steps:

[0018] S121, transmitting the data and weights processed by the convolution sequence controller to the multiplier and perform multiplication and accumulation calculation;

[0019] S122. The multiplication and accumulation calculation results are transmitted to the accumulator, and the partial sums are accumulated. The effective number of digits of the calculation results is retained as the input of the single data processor.

[0020] Furthermore, using a single data processor to perform activation function processing on the convolution-processed data includes the following steps:

[0021] S21. In the convolutional layer, bias addition is performed using a linear function.

[0022] S22, operating the activation layer using a nonlinear function;

[0023] S23, in the batch normalization stage, performs batch normalization operations on the parameters of each layer and provides standard deviation parameters;

[0024] S24. Using an element-by-element operation method, perform element-by-element multiplication on two sets of data matrices having the same size to obtain two sets of input data.

[0025] S25. Convert the formats of the two sets of input data obtained by a single data processor and adapt them to different hardware layers;

[0026] S26. Extract the two sets of input data after format conversion and perform numerical comparison.

[0027] Furthermore, an on-chip buffer is used to replace the traditional on-chip storage for caching, and the data processed by the activation function is pooled and local response normalized by the plane data processor and the channel data processor, including the following steps:

[0028] S31, using a plane data processor to perform maximum and minimum operations and a summation operation of partial sums on the two sets of input data after format conversion;

[0029] S32, performing a pooling operation on the two sets of input data after performing the maximum and minimum operations and the sum of the partial sums in the plane;

[0030] S33. After the pooling operation is completed, if the pooling type is maximum pooling, the two sets of input data in the shared row cache are used as the final result; if the pooling type is average pooling, the two sets of input data are extracted from the shared row cache and divided to calculate the average value;

[0031] S34 , using the channel data processor to receive the two groups of input data after the pooling processing from the on-chip buffer, and performing local response normalization processing on the two groups of input data.

[0032] Furthermore, using a channel data processor to receive the two sets of input data after pooling from the on-chip buffer and performing local response normalization processing on the two sets of input data includes the following steps:

[0033] S341, processing the two sets of input data using local response normalization, and determining whether to use a channel data processor by configuring corresponding registers;

[0034] S342: After the processing is completed, the channel data processor writes the two sets of input data that have undergone local response normalization processing into the cache.

[0035] Furthermore, performing data format conversion on the data after local response normalization processing by the data reorganization engine and implementing data transfer between the dynamic random access memory and the high-performance storage interface includes the following steps:

[0036] S41. Using a data reorganization engine, the mapping format of the two sets of input data that have undergone local response normalization processing is changed;

[0037] S42. Move two sets of input data between the dynamic random access memory and the high-performance storage interface through the data copy engine.

[0038] Furthermore, the working modes of the data reorganization engine include: 3D data splicing, 3D data conversion to 2D data, and merging 2D data into 3D data.

[0039] Furthermore, methods of changing the mapping format of data include: splitting, slicing, merging, shrinking, and reshaping and transposing.

[0040] According to another aspect of the present invention, a system for optimizing the bandwidth of a neural network accelerator is provided. The system for optimizing the bandwidth of a neural network accelerator includes:

[0041] Convolution pipeline module, used to perform convolution processing on input data through convolution pipeline;

[0042] A single data processor module is used to perform activation function processing on the data after convolution processing using a single data processor;

[0043] An on-chip cache module, used for using an on-chip cache to replace traditional on-chip storage for caching;

[0044] A plane data processor module is used to perform pooling processing on the data processed by the activation function through the plane data processor;

[0045] A channel data processor module is used to perform local response normalization processing on the pooled data through the channel data processor;

[0046] A data reorganization engine module is used to convert the data format of the data after the local response normalization processing through the data reorganization engine;

[0047] The data copy engine module is used to realize data transfer between dynamic random access memory and high-performance storage interface.

[0048] The beneficial effects of the present invention are:

[0049] 1. The present invention optimizes the data storage, reading, and writing processes of the hardware accelerator during accelerated operations, reducing the two reading and writing processes between a single data processor, a flat data processor, and external storage, and replacing them with on-chip caches. Without affecting data caching and read and write rates, this effectively improves memory bandwidth utilization, reduces the time consumed by a single accelerated operation, and improves the performance of the hardware accelerator.

[0050] 2. The present invention utilizes an on-chip buffer to replace the static random access memory to act as a cache when data is transmitted between a single data processor, a plane data processor, and a channel data processor, thereby reducing the occupancy of the static random access memory and the double-rate bandwidth by a single operation, thereby improving the memory bandwidth utilization during parallel operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 is a flow chart of a method for improving bandwidth optimization of a neural network accelerator according to an embodiment of the present invention;

[0053] Figure 2 is a principle block diagram of a system for improving the bandwidth optimization of a neural network accelerator according to an embodiment of the present invention;

[0054] Figure 3 This is a pre-optimization design diagram of a method for improving the bandwidth of a neural network accelerator according to an embodiment of the present invention;

[0055] Figure 4 is an optimized design diagram of a method for improving the bandwidth of a neural network accelerator according to an embodiment of the present invention;

[0056] Figure 5 is a schematic structural diagram of an on-chip buffer according to an embodiment of the present invention;

[0057] Figure 6 It is a timing flow chart of the present invention;

[0058] Figure 7 It is a traditional sequential flow chart.

[0059] In the picture:

[0060] 1. Convolution pipeline module; 2. Single data processor module; 3. On-chip cache module; 4. Plane data processor module; 5. Channel data processor module; 6. Data reassembly engine module; 7. Data copy engine module. DETAILED DESCRIPTION

[0061] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0062] According to an embodiment of the present invention, a method and system for improving the bandwidth optimization of a neural network accelerator are provided.

[0063] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an embodiment of the present invention, the method for optimizing the bandwidth of a neural network accelerator includes the following steps:

[0064] S1, convolution processing is performed on the input data through the convolution pipeline;

[0065] It's important to explain that convolution is performed on the input data through the convolution pipeline. The convolution pipeline is one of the core logic components of the hardware accelerator, accelerating the convolution algorithm and supporting programmable convolution size control. Features such as Winograd and Multi-Batch are implemented in the convolution pipeline to improve multiply-accumulator efficiency and computational performance. In this step, the image, feature values, weights, and other data to be convolved are first read in batches via code division multiple access (CDMA) and stored in the convolution buffer (CBUF) in the order required for convolution to meet the computational needs of the convolution engine. The data is then transferred from the CBUF to the convolution sequence controller (CSC), which calculates and controls the decomposition of the convolution sequence and executes the logic for loading feature, pixel, and weight sequences. The data and weights processed by the CSC are then transferred to the CMAC, which utilizes its 16 integrated CMAC units to perform operations such as INT16 and FP16 calculations and post-POA addition. The accumulator (CACC) is the last module of the convolution pipeline. It is mainly responsible for receiving the data output from the multiplier-accumulator (CMAC), accumulating the partial sums, retaining the valid number of bits of the result, and serving as the input of the single data processor (SDP).

[0066] S2, using a single data processor (SDP) to perform activation function processing on the convolution-processed data;

[0067] It should be explained that the single data processor (SDP) performs activation function processing on the single data after convolution calculation, and applies linear and nonlinear functions to the target data. First, using linear functions, the convolutional layer performs simple bias addition or scaling. Second, using nonlinear functions, the SDP performs operations on the activation layer. Third, batch normalization is performed, supporting batch normalization of parameters per layer or per channel, and providing mean / standard deviation parameters. Next, element-wise operations are performed, multiplying or performing other calculations on corresponding elements of two data matrices of the same size. In online mode, one set of data comes from the convolution pipeline, and the other set of input data comes from memory. In offline mode, the SDP obtains both sets of input data from memory. Furthermore, the SDP provides a Pre-ReLU mode, in which the multiplier ignores positive values ​​and only scales negative values. Finally, the SDP converts the input data format to support INT8, INT16, and FP16 precision to accommodate different hardware layers, allowing the SDP to perform precision conversion and other operations simultaneously. Finally, a numerical comparison is performed, extracting two sets of input data from storage and then comparing them. To save bandwidth, no output is written to external memory in comparison mode.

[0068] S3, uses an on-chip buffer (FIFO) instead of traditional on-chip storage for caching, and uses a plane data processor (PDP) and a channel data processor (CDP) to perform pooling and local response normalization (LRN) on the data processed by the activation function;

[0069] It should be noted that data processed by the SDP is directly transferred to the Plane Data Processor (PDP) for pooling via an on-chip first-in-first-out (FIFO) buffer, without passing through external memory. The PDP performs planar operations along the width × height axis, which are configurable at runtime and support max, min, and average pooling. The calculation method is to send several adjacent inputs within the plane to a nonlinear function to calculate the local result. Pooling is performed within a plane, and there is no interference between different planes. The main logical processing of the PDP is in the pre-processing and post-processing units. These two units are responsible for major size comparison operations or average calculations: pre-processing, which mainly performs max and min operations and partial sums; post-processing, which mainly performs relevant post-processing operations for average pooling. In the case of max / min pooling, the shared line buffer is the final result; in the case of average pooling, the post-processing unit extracts data from the shared line buffer for division.

[0070] It should be noted that the Channel Data Processor (CDP) receives data from the on-chip buffer (FIFO) buffer cached by the PDP transmission buffer for local response normalization (LRN) processing. Finally, the processed data is written to the buffer for subsequent processing by the Data Reassembly Engine (RUBIK) and the Data Copy Engine (BDMA). The CDP primarily performs the LRN (Local Response Normalization) function, primarily for lateral inhibition. The CDP can be enabled or skipped by configuring the corresponding registers. In addition to the LRN function, the CDP can also accelerate per-channel quantization calculations.

[0071] It needs to be explained that, Figure 5 As shown in the figure, the on-chip buffer (FIFO) is a first-in, first-out data buffer with no external read or write address lines, and only reads and writes data sequentially. The FIFO has a built-in clock, pointer, and full / empty flag to support sequential read and write operations. After the FIFO is reset, the read and write pointers are equal and both are empty. At this time, the FIFO is empty, and read operations are blocked, while only write operations are allowed. When there is no space to write, the pointer position is SIZE-1. A write operation at this time will roll the pointer back to 0, and the read and write pointers will be equal again. When the FIFO is full or nearly full, the full flag is sent to prevent further write operations from causing the FIFO to overflow. When the FIFO is empty or nearly empty, the empty flag is sent to prevent further read operations from the FIFO and reading invalid data.

[0072] S4. The data format of the data after local response normalization (LRN) is converted through the data reconstruction engine (RUBIK), and data transfer between the dynamic random access memory (DRAM) and the high-performance storage interface is realized.

[0073] It's important to note that the Data Restructuring Engine (RUBIK) receives processed data from the Channel Data Processor (CDP) and converts the data format. During inference within a convolutional network, data in memory often needs to be reconfigured or reshaped. The Data Restructuring Engine (RUBIK) doesn't require any data computation. It operates in three modes: 3D data splicing, 3D data conversion to 2D data, and merging 2D data with 3D data. It's responsible for modifying the data mapping format, including splitting, slicing, merging, shrinking, and reshaping and transposing.

[0074] It should be explained that the data copy engine (BDMA) realizes the data transfer between the dynamic random access memory (DRAM) and the high-performance storage interface. This is an acceleration channel that directly connects the dynamic random access memory (DRAM) and the high-performance storage interface. In the present invention, both the input image and the final processed result need to be stored in the external dynamic random access memory (DRAM), but the bandwidth and delay of the external dynamic random access memory (DRAM) are bottlenecked, so the static random access memory (SRAM) is required as an auxiliary memory. In order to ensure the synchronization of the data inside the static random access memory (SRAM) and the dynamic random access memory (DRAM), the BDMA plays the role of moving data between the dynamic random access memory (DRAM) and the static random access memory (SRAM).

[0075] Preferably, performing convolution processing on the input data through the convolution pipeline includes the following steps:

[0076] S11, using code division multiple access to batch read the image to be convolved, feature values ​​and weights, and store them in the convolution buffer in the order of data required for convolution;

[0077] S12. Transfer data from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence.

[0078] Preferably, transferring data from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence includes the following steps:

[0079] S121, transmitting the data and weights processed by the convolution sequence controller to the multiplier and perform multiplication and accumulation calculation;

[0080] S122. The multiplication and accumulation calculation results are transmitted to the accumulator, and the partial sums are accumulated. The effective number of digits of the calculation results is retained as the input of the single data processor.

[0081] Preferably, using a single data processor to perform activation function processing on the convolution-processed data includes the following steps:

[0082] S21. In the convolutional layer, bias addition is performed using a linear function.

[0083] S22, operating the activation layer using a nonlinear function;

[0084] S23, in the batch normalization stage, performs batch normalization operations on the parameters of each layer and provides standard deviation parameters;

[0085] S24. Using an element-by-element operation method, perform element-by-element multiplication on two sets of data matrices having the same size to obtain two sets of input data.

[0086] S25. Convert the formats of the two sets of input data obtained by a single data processor and adapt them to different hardware layers;

[0087] S26. Extract the two sets of input data after format conversion and perform numerical comparison.

[0088] Preferably, using an on-chip buffer to replace traditional on-chip storage for caching, and performing pooling processing and local response normalization processing on the data processed by the activation function through a plane data processor and a channel data processor includes the following steps:

[0089] S31, using a plane data processor to perform maximum and minimum operations and a summation operation of partial sums on the two sets of input data after format conversion;

[0090] S32, performing a pooling operation on the two sets of input data after performing the maximum and minimum operations and the sum of the partial sums in the plane;

[0091] S33. After the pooling operation is completed, if the pooling type is maximum pooling, the two sets of input data in the shared row cache are used as the final result; if the pooling type is average pooling, the two sets of input data are extracted from the shared row cache and divided to calculate the average value;

[0092] S34 , using the channel data processor to receive the two groups of input data after the pooling processing from the on-chip buffer, and performing local response normalization processing on the two groups of input data.

[0093] Preferably, receiving the two sets of input data after pooling processing from the on-chip buffer using the channel data processor and performing local response normalization processing on the two sets of input data includes the following steps:

[0094] S341, processing the two sets of input data using local response normalization, and determining whether to use a channel data processor by configuring corresponding registers;

[0095] S342: After the processing is completed, the channel data processor writes the two sets of input data that have undergone local response normalization processing into the cache.

[0096] Preferably, performing data format conversion on the data after local response normalization processing by the data reorganization engine and realizing data transfer between the dynamic random access memory and the high-performance storage interface includes the following steps:

[0097] S41. Using a data reorganization engine, the mapping format of the two sets of input data that have undergone local response normalization processing is changed;

[0098] S42. Move two sets of input data between the dynamic random access memory and the high-performance storage interface through the data copy engine.

[0099] Preferably, the working modes of the data reassembly engine include: three-dimensional data splicing, three-dimensional data conversion to two-dimensional data, and two-dimensional data merging to three-dimensional data.

[0100] Preferably, the manner of changing the mapping format of the data includes: splitting, slicing, merging, shrinking and reorganizing and transposing.

[0101] According to another embodiment of the present invention, Figure 2 As shown, a system for improving the bandwidth optimization of a neural network accelerator is also provided, and the system for improving the bandwidth optimization of a neural network accelerator includes:

[0102] Convolution pipeline module 1, used to perform convolution processing on input data through the convolution pipeline;

[0103] A single data processor module 2 is used to perform activation function processing on the data after convolution processing using a single data processor;

[0104] On-chip cache module 3, used for using on-chip cache to replace traditional on-chip storage for caching;

[0105] A plane data processor module 4 is used to perform pooling processing on the data processed by the activation function through the plane data processor;

[0106] A channel data processor module 5 is configured to perform local response normalization processing on the pooled data through a channel data processor;

[0107] A data reorganization engine module 6 is used to convert the data format of the data after the local response normalization processing through the data reorganization engine;

[0108] The data copy engine module 7 is used to realize data transfer between the dynamic random access memory and the high-performance storage interface.

[0109] To sum up, with the help of the above technical solution of the present invention, the present invention uses an on-chip cache to replace the static random access memory to play a caching role when data is transmitted between a single data processor, a plane data processor, and a channel data processor, thereby reducing the occupancy of the static random access memory and the double-rate bandwidth by a single operation, thereby improving the memory bandwidth utilization during parallel operations.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for improving the bandwidth optimization of a neural network accelerator, characterized in that: The method for improving the bandwidth optimization of a neural network accelerator comprises the following steps: S1, convolution processing is performed on the input data through the convolution pipeline; S2, using a single data processor to perform activation function processing on the data after convolution processing; S3, using on-chip cache instead of traditional on-chip storage for caching, and performing pooling and local response normalization on the data processed by the activation function through the plane data processor and channel data processor; S4. The data format of the data after local response normalization processing is converted through the data reorganization engine, and data transfer between the dynamic random access memory and the high-performance storage interface is realized.

2. The method for optimizing the bandwidth of a neural network accelerator according to claim 1, wherein: The convolution processing of the input data by the convolution pipeline comprises the following steps: S11, using code division multiple access to batch read the image to be convolved, feature values ​​and weights, and store them in the convolution buffer in the order of data required for convolution; S12. Transfer data from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence.

3. The method for optimizing the bandwidth of a neural network accelerator according to claim 2, wherein: The data is transferred from the convolution buffer to the convolution sequence controller to calculate and control the convolution sequence decomposition and execute the logic of the feature, pixel, and weight loading sequence, including the following steps: S121, transmitting the data and weights processed by the convolution sequence controller to the multiplier and perform multiplication and accumulation calculation; S122. The multiplication and accumulation calculation results are transmitted to the accumulator, and the partial sums are accumulated. The effective number of digits of the calculation results is retained as the input of the single data processor.

4. The method for optimizing the bandwidth of a neural network accelerator according to claim 1, wherein: The method of using a single data processor to perform activation function processing on the convolution-processed data includes the following steps: S21. In the convolutional layer, bias addition is performed using a linear function. S22, operating the activation layer using a nonlinear function; S23, in the batch normalization stage, performs batch normalization operations on the parameters of each layer and provides standard deviation parameters; S24. Using an element-by-element operation method, perform element-by-element multiplication on two sets of data matrices having the same size to obtain two sets of input data. S25. Convert the formats of the two sets of input data obtained by a single data processor and adapt them to different hardware layers; S26. Extract the two sets of input data after format conversion and perform numerical comparison.

5. The method for optimizing the bandwidth of a neural network accelerator according to claim 1, wherein: The method of using an on-chip buffer to replace a traditional on-chip storage for caching, and performing pooling processing and local response normalization processing on the data processed by the activation function through a plane data processor and a channel data processor includes the following steps: S31, using a plane data processor to perform maximum and minimum operations and a summation operation of partial sums on the two sets of input data after format conversion; S32, performing a pooling operation on the two sets of input data after performing the maximum and minimum operations and the sum of the partial sums in the plane; S33. After the pooling operation is completed, if the pooling type is maximum pooling, the two sets of input data in the shared row cache are used as the final result; if the pooling type is average pooling, the two sets of input data are extracted from the shared row cache and divided to calculate the average value; S34 , using the channel data processor to receive the two groups of input data after the pooling processing from the on-chip buffer, and performing local response normalization processing on the two groups of input data.

6. The method for optimizing the bandwidth of a neural network accelerator according to claim 5, wherein: The method of receiving two sets of input data after pooling processing from the on-chip buffer by using the channel data processor and performing local response normalization processing on the two sets of input data includes the following steps: S341, processing the two sets of input data using local response normalization, and determining whether to use a channel data processor by configuring corresponding registers; S342: After the processing is completed, the channel data processor writes the two sets of input data that have undergone local response normalization processing into the cache.

7. The method for optimizing the bandwidth of a neural network accelerator according to claim 1, wherein: The data reorganization engine performs data format conversion on the data after the local response normalization processing and realizes data transfer between the dynamic random access memory and the high-performance storage interface, which includes the following steps: S41. Using a data reorganization engine, the mapping format of the two sets of input data that have undergone local response normalization processing is changed; S42. Move two sets of input data between the dynamic random access memory and the high-performance storage interface through the data copy engine.

8. The method for optimizing the bandwidth of a neural network accelerator according to claim 7, wherein: The working modes of the data reorganization engine include: three-dimensional data splicing, three-dimensional data conversion to two-dimensional data, and two-dimensional data merging to three-dimensional data.

9. The method for optimizing the bandwidth of a neural network accelerator according to claim 7, wherein: The methods of changing the mapping format of data include: splitting, slicing, merging, shrinking and reshaping transposition.

10. A system for optimizing bandwidth of a neural network accelerator, for implementing the method for optimizing bandwidth of a neural network accelerator according to any one of claims 1 to 9, characterized in that: The neural network accelerator bandwidth optimization system includes: Convolution pipeline module, used to perform convolution processing on input data through convolution pipeline; A single data processor module is used to perform activation function processing on the data after convolution processing using a single data processor; An on-chip cache module, used for using an on-chip cache to replace traditional on-chip storage for caching; A plane data processor module is used to perform pooling processing on the data processed by the activation function through the plane data processor; A channel data processor module is used to perform local response normalization processing on the pooled data through the channel data processor; A data reorganization engine module is used to convert the data format of the data after the local response normalization processing through the data reorganization engine; The data copy engine module is used to realize data transfer between dynamic random access memory and high-performance storage interface.

Images