A neural network accelerator based on a multi-channel systolic array

Abstract

A neural network accelerator based on a multi-channel systolic array includes a novel systolic array composed of a configurable number of basic computational units. This systolic array includes a configurable number and direction of weight data stream channels and feature data stream channels. The multi-channel systolic array can arbitrate the allocation of data stream channels and basic computational units according to the scale of the neural network, achieving parallel four-thread acceleration for up to four types of neural networks. The multi-channel systolic array feeds a row of feature data in each computation cycle, using a data stream composed of entire rows of data to improve upon the discrete data stream, significantly reducing acceleration processing time. This invention achieves accelerated neural network computation based on a multi-channel systolic array, which significantly optimizes acceleration processing time and hardware resource utilization compared to traditional systolic arrays; it can achieve more efficient accelerated computation while saving hardware overhead.

Description

Technical Field

[0001] This invention belongs to the field of computer artificial intelligence technology, and in particular relates to a neural network accelerator device based on a multi-channel pulsating array. Background Technology

[0002] With the rapid development and application of artificial intelligence, neural networks have undergone architectural iterations from perceptrons and multilayer perceptrons to the widely used convolutional neural networks. Devices used to accelerate neural networks have also evolved from general-purpose computing platforms like CPUs and GPUs to dedicated neural network accelerator platforms. Current dedicated neural network accelerators typically use systolic arrays to perform matrix operations, enabling convolution and fully connected layer operations within the neural network structure. As the types, depths, and breadths of neural networks increase, the increased data volume and computational complexity place higher demands on the throughput and utilization of systolic arrays in neural network accelerators.

[0003] Traditional systolic arrays typically have only two data stream channels: a feature data stream channel and a weight data stream channel, with the two channels perpendicular to each other. During computation, each clock cycle, the two data stream channels sequentially receive a set of feature data and a set of weight data, respectively. In the Clock0 calculation cycle, the feature data stream channel sends a single feature data point from the corresponding feature matrix F(0,0) to PE(0,0), and the weight data stream channel sends a single weight data point from the corresponding weight matrix W(0,0) to PE(0,0). The PE(0,0) calculation unit performs the multiplication of the feature data and weight data and then sends them to the next adjacent PE unit in the data flow direction. In the Clock1 calculation cycle, the feature data stream channel sends two feature data points from the corresponding feature matrices F(1,0) and F(0,1) to PE(0,0) and PE(0,1) respectively, and the weight data stream channel sends two weight data points from the corresponding weight matrices W(0,1) and W(1,0) to PE(0,0) and PE(1,0) respectively. The corresponding PE unit performs a multiplication operation on the newly sent data and adds it to the previous result. This process continues, with feature data and weight data being sequentially pulsed into the pulse array in each new calculation cycle, ultimately completing the data stream input and calculation.

[0004] While traditional systolic arrays can achieve parallel processing of multiple PE computing units within each computation cycle, they have certain limitations when dealing with diverse and complex neural networks. When faced with neural networks of varying sizes and types, traditional systolic arrays can only perform matrix multiplication operations in a single thread with a fixed number of PE computing units and a fixed data flow. Typically, acceleration can only be achieved for one type of neural network per computation, resulting in low computational efficiency, wasted PE computing unit resources, and a lack of reusability and scalability. Furthermore, traditional systolic arrays suffer from excessively long data processing and result loading times during acceleration, requiring more processing time for the same PE computing unit capacity, leading to reduced throughput and computational speed. Summary of the Invention

[0005] The purpose of this invention is to provide a neural network accelerator device based on a multi-channel pulsating array, so as to solve the problems of excessive processing time, low utilization rate of PE basic units, lack of reusability and scalability of traditional neural network accelerators based on pulsating arrays.

[0006] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0007] A neural network accelerator device based on a multi-channel systolic array includes a multi-channel systolic array composed of a configurable number (n×n) of basic computational units (PEs). The multi-channel systolic array includes multiple feature data stream channels (four channels in each direction) and multiple weight data stream channels (four channels in each direction). The feature data stream channels are used to feed feature data into the systolic array; the weight data stream channels are used to feed weight data into the systolic array. The direction and number of the feature data stream channels and weight data stream channels can be dynamically configured according to requirements in specific applications. If the matrix operation scale is large, the size of the data stream channels can be reduced to support large-scale matrix operations; if the matrix operation scale is small, the size of the data stream channels can be expanded to support multiple small-scale matrix operations, enabling parallel operation of up to four matrices simultaneously. The data stream channels achieve parallel operation by increasing the data flow direction and data flow scale, accelerating multiple neural networks in one computation cycle. This avoids resource waste of PE basic units during small-scale operations, achieving efficient utilization and reuse of PE basic units and saving hardware costs.

[0008] Furthermore, the multi-channel systolic array has made a new improvement to the data flow input mode of the feature data stream channel and the weight data stream channel. Traditional systolic arrays feed single or multiple separate data points in two directions gradually each calculation cycle. The multi-channel systolic array, however, feeds entire rows of weight data in multiple directions each calculation cycle to replace the separate data. During computation, the systolic array must wait for all feature data and weight data to completely flow through each PE basic unit before completing the operation. Traditional systolic arrays, because they feed single data points each calculation cycle, require a longer waiting time for data to flow through each PE basic unit. Assuming the feature matrix size is (m×p) and the weight matrix size is (m×n), a comparison is made under the condition that the PE basic unit operation speed is the same. For traditional systolic arrays, when all feature data and weight data have flowed through the PE basic operation unit, it is considered that one processing cycle has been completed. Let T be defined. P For the required processing time, for a traditional pulsating array, there is T. P = m + p + n - 2; Since traditional systolic arrays input data in discrete form, in addition to the processing time, a period of time is needed to load all the computation results. This loading time is equal to the step size of the PE basic unit used in the systolic array from the upper left corner to the lower right corner, defined as T. L For the load time, for a traditional pulsating array, there is T. L = p + n - 1; Since the data sent in the early stage will be loaded out first during the processing of the data sent in the later stage, there is an overlap between the overall systolic array processing time and the loading time. After subtracting the overlap, the computation time of the traditional systolic array is T = 2p + m + n - 2; The multi-channel systolic array improves the excessively high computation time of the traditional systolic array. In each calculation cycle, the entire row of feature data is sent into a row of PE basic units of the systolic array, while the other feature data remains stationary. The weight data is sent into the row of PE basic units one by one in a discrete manner. The processing time is equal to the number of time steps required to move the length of feature data on the systolic array. Therefore, for the multi-channel systolic array, T = p + n - 1; P =p+n-1; The multi-channel systolic array can start the result loading process when the first result is obtained, thereby greatly increasing the overlap ratio of loading time and processing time. The total computation time of the multi-channel systolic array is T = p+n+logm-1; Under ideal conditions, the multi-channel systolic array can process four different network matrices in parallel, thereby achieving a maximum overall computation time of three cycles and a single-cycle optimization time compared to the traditional single-threaded systolic array. When processing four parallel neural networks with acceleration, the computation time can be reduced by up to ΔT = (7p+4m+3n-logm-7), thereby greatly improving computational efficiency.

[0009] Furthermore, the multi-channel systolic array is based on a configurable array of (n×n) PE basic operation units. The PE basic unit interface includes an uplink PE feature data input / output interface, a downlink PE feature data input / output interface, a left-column PE weight input / output interface, a right-column PE weight input / output interface, and an operation result output interface. The internal components of the PE basic unit include four data selectors, four data temporary registers, a multi-channel result register group, a channel selector, a multiplier, and an adder. The PE basic unit includes weight input interfaces and feature input interfaces in two directions to meet the multi-directional, multi-channel data flow characteristics of the multi-channel systolic array. The PE basic unit can simultaneously receive data from four input / output interfaces, and channel selection and data storage are achieved through controller arbitration. In each computation cycle, the PE basic unit prioritizes one PE feature input interface and one PE weight input interface in one direction through arbitration to receive the data to be processed in the current computation cycle. For the data from the other input interfaces, if the data does not require processing, it is directly sent to the secondary PE unit through a data selector; if the data requires processing, it is temporarily stored in a data temporary storage register for processing in the next computation cycle. For data that needs processing, the channel selector selects an appropriate channel based on the arbitration result, and sends the data to be processed sequentially to the multiplier and adder to complete the matrix multiplication operation, and temporarily stores the result in a multi-channel result register group. The multi-channel register group stores the temporary operation results of the input data in different data flow directions according to different data flow input direction combinations, and sends the temporary data to the adder to accumulate the results when the next data of that data flow combination arrives. The PE basic unit can realize arbitration operation for four data flow combinations, realizing the reuse of the PE basic unit in multiple data flow directions and multiple neural network situations, greatly saving design overhead and improving the utilization rate of the PE basic unit.

[0010] The present invention also provides a series of components to realize the function of a neural network accelerator based on a multi-channel systolic array. The components include a controller unit, an interface unit, a memory unit, and a pooling activation unit.

[0011] The controller unit is used to execute algorithms to adapt to different types of neural networks, and to configure the number of channels and flow direction of arbitration control data flow according to the computational requirements and algorithms, and to coordinate the work of various internal hardware components.

[0012] The interface unit includes four parallel raw data input interfaces and result output interfaces, an external interface for debugging, and a configuration information interface for configuration information transmission. The four parallel input / output interfaces are implemented based on the AXI protocol bus and are used to accept unprocessed raw data and output the final processed results. Arbitration is used to achieve parallel acceleration of four different neural networks. The debugging interface is implemented based on the APB protocol and is used to view and debug the processing results of each internal hardware unit without affecting the data flow input and output.

[0013] The memory unit is used to initialize the original data, convert the original data and data in the process into appropriate feature matrix data according to the operation requirements and algorithm configuration, cache the intermediate result data of the pulsation array, and send the cached data into the pulsation calculation array after processing according to the configuration.

[0014] The pooling activation unit is used to perform activation and pooling operations on intermediate result data. The activation operation uses the LeakyRelu activation function; the pooling operation uses max pooling.

[0015] A neural network accelerator device based on a multi-channel pulsating array, the operation of which is as follows:

[0016] S1 obtains configuration information through the configuration interface and the raw input interface, including the type of each neural network, the number of network layers, the size of the convolution kernel, stride, etc. Arbitration is performed based on the obtained information, and the number of networks that can be processed in parallel is determined based on the scale of the multi-channel systolic array.

[0017] S2, based on the arbitration result of the controller unit, obtains basic information such as the weight data of the neural network to be processed from the interface unit; accepts the raw data of the parallel neural network from multiple input interfaces, and temporarily stores the data from the remaining interfaces.

[0018] S3 initiates neural network computation. The memory unit organizes the raw data to form a feature data matrix and a weight data matrix to be processed.

[0019] S4 arbitrates based on the size of the feature data matrix and weight data matrix of multiple networks, and allocates the network data processed in parallel to the appropriate data flow direction and data flow interface.

[0020] S5, according to the arbitration result, the feature data and weight data are sent into the multi-channel pulsation array through the data stream in the corresponding calculation cycle, the matrix multiplication and addition operations are completed in the PE basic calculation unit of the pulsation array, and the data in the register group are integrated into the complete result data after the pulsation ends;

[0021] S6, the data that has been processed by the multi-channel pulse array is sent to the activation and pooling unit for further processing;

[0022] S7, the controller unit performs arbitration processing on the processed data according to the type of neural network defined by the algorithm, and performs further data integration by the memory unit for the operation of the lower-level network or outputs the result;

[0023] The advantages and beneficial effects of this invention are as follows:

[0024] This invention discloses a neural network accelerator device based on a multi-channel systolic array. It improves the computational efficiency and utilization of the systolic array portion of a traditional systolic array-based neural network accelerator. Because the improved multi-channel systolic array has more data flow channels and data flow directions, it can simultaneously utilize multiple data flow channels to process data in parallel using idle PE (Preinstallation Environment) basic unit resources, thus improving the utilization rate of the systolic array. Furthermore, since the improved multi-channel systolic array uses whole-line feature data transmission instead of discrete feature data transmission in the data flow transmission, it can significantly reduce the result loading delay in processing time, improving the computational efficiency of the systolic array. Combined with the improved PE basic unit, the device can achieve parallel acceleration processing of multiple neural networks with a smaller hardware specification, while also having higher computational processing speed and higher systolic array utilization. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the multi-channel pulsating array structure and data structure in this invention;

[0026] Figure 2 This is a schematic diagram of the structure of the PE basic operation unit in this invention;

[0027] Figure 3 This is a schematic diagram comparing the present invention with acceleration computation based on traditional pulsating array neural networks;

[0028] Figure 4 A schematic diagram illustrating the workflow for accelerating computation in this invention;

[0029] Figure 5 A schematic diagram of the real-time data flow during accelerated computation in this invention;

[0030] The markings in the diagram are as follows: 1. Data of feature data matrix A sent to the systolic array in whole rows every calculation cycle; 2. Data of weight data matrix A sent to the systolic array every calculation cycle; 3. PE basic operation unit resources actually used in matrix calculation A; 4. PE basic operation unit resources actually used in matrix calculation B; 5. Composition of PE basic operation units in the systolic array; 6. A systolic array of configurable size consisting of (6×6) PE basic units; 7. Weight data stream channel; 8. PE basic operation unit resources actually used in matrix calculation C; 9. Feature data stream channel; 10. PE basic operation unit resources actually used in matrix calculation D. Detailed Implementation

[0031] To better understand the purpose, structure, and function of this invention, a neural network accelerator device based on a multi-channel pulsating array will be described in further detail below with reference to the accompanying drawings.

[0032] To address the problems of long processing time, low computational efficiency, and low utilization of basic PE units in traditional neural network accelerators based on systolic arrays, this application discloses a neural network accelerator device based on a multi-channel systolic array through the following examples.

[0033] like Figure 1The diagram illustrates the parallel acceleration of four neural networks using a multi-channel systolic array, as shown in Figure 9. For ease of description, it is assumed that the multi-channel systolic array consists of 36 basic PE (Programmable Execution) units (6×6) as indicated by label 6. In practical applications, the scale of the systolic array will far exceed this assumption and can be configured for different applications. The four accelerated neural networks are represented by the multiplication of four feature data matrices with weight data matrices, as shown in the figure. The scales of the four matrix operations are: A: (4×3)*(3×4), B: (2×2)*(2×2), C: (3×3)*(3×3), and D: (5×3)*(5×3). The data flow channels and directions of different feature data are represented by the four black arrows shown in Figure 9, and the data flow channels and directions of different weight data are represented by the four white arrows shown in Figure 7. The figure illustrates the data flow during parallel acceleration of the four neural networks when all four data flow channels are open. In each calculation cycle, the multi-channel systolic array sends feature data in vector form. Taking matrix operation A as an example, label 1 in the diagram represents the entire row of feature data sent to the systolic array during clock cycle 0. Simultaneously, the three sets of weight data (label 2) are sequentially sent to the systolic array. After WA(0,0) and FA(0,0) are multiplied, the weight data (label 2) is pulsed one bit to the right, and WA(0,0) is multiplied by FA(0,1), and WA(0,1) is multiplied by FA(0,0). This process continues until the multiplication and addition operations of the data in labels 1 and 2 are completed. Then, the second calculation cycle (Clock 1) begins, and the next entire row of feature data and weight data is pulsed again for multiplication and addition. The data input method for the other matrix multiplication operations in the diagram is similar to that of matrix operation A. Different matrix operations require different PE (Programmable Execution Unit) sizes. The actual PE size used in matrix operations A, B, C, and D during the operation is represented by labels 3, 4, 8, and 10 in the diagram. Because the data flow directions of different matrix operations are relative during parallel processing, the same PE operation units can be avoided in the same computation cycle when their occupied PE unit sizes do not overlap. However, when their occupied PE unit sizes overlap, data caching and sequential processing must be handled through arbitration by the control unit. In the parallel 4-thread operation shown in the figure, the utilization rate of the PE operation units of the systolic array reaches 97.23%, far exceeding the 11.11%-41.67% utilization rate of traditional systolic array serial single-thread operation. Simultaneously, the parallel 4-thread operation shown in the figure achieves a significant reduction in computation time and a substantial increase in computation speed.

[0034] like Figure 2 The diagram shown is a schematic diagram of the circuit structure of the PE basic operation unit used to implement matrix multiplication and addition operations according to the present invention. Figure 1Figure 5 shows a schematic diagram of the PE basic operation unit in the pulse array. As shown in the figure, the PE basic unit includes the following interfaces: uplink PE feature data input / output interface, downlink PE feature data input / output interface, left column PE weight input / output interface, right column PE weight input / output interface, and operation result output interface; the PE basic unit includes the following internal components: four data selectors, four data temporary registers, two channel selectors, a multi-channel result register group, a multiplier, and an adder.

[0035] Among them, the uplink and downlink characteristic data input / output interfaces and the left and right column input / output interfaces are used to realize the multi-directional and multi-channel data flow characteristics of the pulsating array. All input / output interfaces can receive data simultaneously, and channel priority selection and parallel operation are realized through the arbitrator.

[0036] Four data selectors are connected to the input / output interfaces to perform data selection and arbitration; the data temporary storage register is used to store the data to be processed. When the data in the data stream does not require processing, the data selector sends the data directly from the input interface to the output interface; when the data in the data stream is data to be processed, the data selector temporarily stores the data in the data temporary storage register.

[0037] The channel selector is used to select the data stream channel. When data from two data stream channels connected to the channel selector needs to be processed, the channel selector selects the corresponding data stream channel in the corresponding calculation cycle based on the arbitration result, thereby achieving accurate data processing.

[0038] The multiplier is connected to a two-channel selector. In each calculation cycle, it receives the feature data and weight data selected by the two-channel selector, multiplies the received data, and sends it to the adder.

[0039] The adder is connected to the multiplier and the multi-channel result register. It is used to accumulate the result data output by the multiplier with the data before the corresponding data stream channel is combined in each calculation cycle.

[0040] The multi-channel register is connected to the adder and is used to store intermediate result data for different combinations of data flow channels. Since the PE unit has four-way data flow channels, there are a maximum of four combinations of data flow channels: uplink feature input & left column weight input, uplink feature input & right column weight input, downlink feature input & left column weight input, and downlink feature input & right column weight input. Therefore, two registers are allocated to each combination to implement temporary storage of intermediate data and temporary storage of accumulated results, while sufficient registers are reserved to prevent data congestion under large data scales.

[0041] The implementation is described using one working scenario. Assume that in this scenario, the PE basic arithmetic unit needs to process matrix multiplication and addition operations from the upstream and downstream feature data and the left column weight data. Simultaneously, weight data from the right column passes through the PE basic arithmetic unit in each calculation cycle. In the first calculation cycle, since the left column weight data, upstream feature data, and downstream feature data all need to be processed, data selectors 1, 2, and 4 all choose to store the data in the data temporary storage register. Since the right column weight data does not need processing, data selector 3 directly outputs the data through the left column PE weight data output interface. Data in both data stream channels belonging to channel selector 1 needs to be processed. At this time, channel selector 1 arbitrates and first selects the upstream feature data; channel selector 2 does not need arbitration and selects the left column weight data. The two channel selectors send the data to the multiplier and adder to complete the matrix multiplication and addition operations, and store the data in the corresponding data stream channel combination. After completing the data processing of the upstream feature data, channel selector 1 selects the downstream feature data to complete the matrix multiplication and addition operation. This completes the matrix multiplication and addition of the uplink and downlink feature data and the left column weight data, thus completing the calculation cycle.

[0042] The single PE basic unit disclosed in this invention can achieve the functionality equivalent to four traditional PE basic units under limited hardware resources, performing arbitration operations on four data flow channel combinations. This enables the parallelism and multiplexing of the PE basic unit under multiple data flow directions and multiple neural network scenarios, significantly reducing design overhead and improving the efficiency and utilization rate of the PE basic unit.

[0043] like Figure 3 The diagram shows a comparison between data processed by a traditional pulsating array and data processed by a multi-channel pulsating array.

[0044] A schematic diagram of the operation of a traditional pulse array is shown below. Figure 3 As shown in the diagram above, a traditional systolic array typically has only two data stream channels: a feature data stream channel and a weight data stream channel, with the directions of the two data stream channels perpendicular to each other. During the calculation process, each clock cycle, the two data stream channels sequentially feed in a set of feature data and a set of weight data, respectively.

[0045] During the Clock0 calculation cycle, the feature data stream channel sends a single feature data of the corresponding feature matrix F(0,0) to PE(0,0), and the weight data stream channel sends a single weight data of the corresponding weight matrix W(0,0) to PE(0,0). After the PE(0,0) calculation unit completes the multiplication of the feature data and the weight data, it sends the feature data and the weight data to the subsequent adjacent PE unit in the data flow direction.

[0046] During the Clock1 calculation cycle, the feature data stream channel sends the two feature data of the corresponding feature matrix F(1,0) to PE(0,0) and PE(0,1) respectively. The weight data stream channel sends the two weight data of the corresponding weight matrix W(0,1) and W(1,0) to PE(0,0) and PE(1,0) respectively. The corresponding PE unit performs a multiplication operation on the newly sent data and adds it to the previous result.

[0047] Similarly, in each new calculation cycle, feature data and weight data will be sequentially pulsated into the pulsation array to complete the pulsation processing of all feature data and weight data.

[0048] like Figure 3 The figure below shows a schematic diagram of the multi-channel systolic array operation disclosed in the example method of this invention. The advantages of the improved systolic array in processing time are analyzed by comparing the two figures above and below. It is assumed that the two systolic arrays will process matrix multiplication and addition operations with a feature matrix of size (m×p) and a weight matrix of size (m×n), and the comparison is made under the condition that the operation speed of the PE basic unit is the same.

[0049] For traditional systolic arrays, a processing cycle is considered complete only when all feature data and weight data have flowed through the PE basic operation unit. Let T be defined. P For the required processing time, for a traditional pulsating array, there is T. P =m+p+n-2; The input time for feature data in the same row of traditional pulsed processing varies significantly. Therefore, in addition to the processing time, a considerable amount of time is needed to load all the computation results. This loading time is equal to the step size of the PE basic unit used, from the top left corner to the bottom right corner. Define T. L For the load time, for a traditional pulsating array, there is T. L =p+n-1; Since the data sent in the early stage will be loaded out first during the processing of the data sent in the later stage, there is an overlap between the overall systolic array processing time and the loading time. After subtracting the overlapping part, the computation time of the traditional systolic array can be obtained as T = 2p+m+n-2.

[0050] In each computation cycle, a multi-channel systolic array feeds a whole row of feature data into a row of PE basic cells of the systolic array, while the remaining feature data remains stationary. Weight data is fed into the PE basic cells of that row discretely, one by one. The processing time is equal to the number of time steps required to move the length of feature data across the systolic array. Therefore, for a multi-channel systolic array, T... P =p+n-1; The multi-channel systolic array can load the process when it obtains the first result, which greatly increases the overlap between loading time and processing time, and the operation time of the multi-channel systolic array is greatly reduced, so T = p+n+logm-1.

[0051] Under ideal conditions, a multi-channel systolic array can process four different network matrices in parallel. Compared with a single-threaded traditional systolic array, it can achieve a reduction in overall computation time for up to three cycles and optimization time for a single cycle, with a maximum computation time reduction of ΔT = (7p + 4m + 3n - logm - 7), thereby significantly improving computation speed and efficiency.

[0052] like Figure 4 The diagram illustrates the acceleration process of a neural network accelerator device based on a multi-channel pulsating array, including the following steps:

[0053] Step S1: Obtain configuration information through the configuration interface and the original input interface, including the type of each neural network, the number of network layers, the size of the convolution kernel, stride, etc. Arbitrate based on the obtained information, and determine the number of networks that can be processed in parallel based on the scale of the multi-channel systolic array.

[0054] Step S2: Based on the arbitration result of the controller unit, obtain basic information such as the weight data of the neural network to be processed from the interface unit; accept the raw data of the parallel neural network from multiple input interfaces, and temporarily store the data from the remaining interfaces.

[0055] Step S3: Start the neural network operation. The memory unit organizes the original data to form the feature data matrix and weight data matrix to be processed.

[0056] Step S4: Arbitrate based on the size of the feature data matrix and weight data matrix of multiple networks, and allocate the network data processed in parallel to the appropriate data flow direction and data flow interface.

[0057] Step S5: According to the arbitration result, the feature data and weight data are sent into the multi-channel pulsation array through the data stream during the corresponding calculation cycle. The matrix multiplication and addition operations are completed in the PE basic calculation unit of the pulsation array. After the pulsation ends, the data in the register group is integrated into the complete result data.

[0058] Step S6: The data that has been processed by the multi-channel pulse array is sent to the activation and pooling unit for further processing.

[0059] Step S7: The controller unit performs arbitration processing on the processed data according to the type of neural network defined by the algorithm, and the data is further integrated by the memory unit to perform calculations on the lower-level network or to output the results.

[0060] like Figure 5 The diagram shows the data flow of a multi-channel pulsating array during accelerated computation. Figure 1The implementation method is explained using the operation of feature data matrix B and weight data B as an example. The actual PE (Processing Unit) size used in the accelerated computation of matrix operation B is as follows: Figure 5 The diagram shows four basic PE (Programming Point) operation units (2×2). Data transmission begins at the start of the Clock 0 computation cycle; in the Clock 1 computation cycle, the first row of feature data is loaded into the systolic array and multiplied / added with the weight data; in the Clock 2 computation cycle, the second row of feature data is loaded into the systolic array, at which point the first row of feature data is multiplied / added with the newly loaded weight data, and the second row of feature data is multiplied / added with the previously loaded weight data; in the Clock 3 computation cycle, the first row of feature data, having completed all computations, is loaded; and in the Clock 4 computation cycle, all result data is loaded, and the computation ends. Figure 5 The data flow diagram shown illustrates that the processing cycle required by the multi-channel pulse array is significantly shortened, resulting in a substantial improvement in processing efficiency.

[0061] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A neural network accelerator based on a multi-channel pulsating array, characterized in that, The neural network accelerator is used to achieve parallel acceleration of various types of neural networks, including simple neural networks based on fully connected layers and various multilayer neural networks based on convolution operations. The neural network accelerator includes an interface unit, a controller unit, a memory unit, a pooling activation unit, and a computing array unit based on a multi-channel systolic array. The interface unit includes a data input interface, a result output interface, a configuration interface, and an external debugging interface for debugging. The data input interface and the result output interface are implemented based on the AXI protocol bus and are used to accept unprocessed raw data and output the final processed results, enabling the input and output of parallel four-channel neural network data. The debugging interface is implemented based on the APB protocol bus and is used to debug the configuration of each internal hardware unit and intermediate results without affecting the accelerated processing. The controller unit is used to execute algorithms to adapt to different types of neural networks, control the flow of data according to algorithm requirements, and coordinate the work of internal hardware. The memory unit is used to initialize the original data, converting the original data and data in the process into appropriate feature matrix data according to the computational requirements; it is also used to temporarily store matrix array data and intermediate results, and send the data into the computing array unit at the correct time. The pooling activation unit is used to perform activation and pooling operations on the output data. The activation operation uses the LeakyRelu activation function; the pooling operation uses max pooling. The computing array unit based on the multi-channel pulsating array is used to implement matrix multiplication and addition operations. The size of the computing unit is allocated according to the size of the neural network, and parallel matrix multiplication and addition operations of multiple feature matrices and weight matrices are implemented in parallel. The operation results are sent to the memory unit for temporary data storage or sent to the result output interface for result output.

2. The neural network accelerator based on a multi-channel pulsating array according to claim 1, characterized in that, The aforementioned computing array based on a multi-channel pulsating array includes multiple configurable-size PE basic computing units, multiple configurable-number and directional weight data stream channels, and multiple configurable-number and directional feature data stream channels, wherein... The PE basic computing unit is used to implement multiplication and addition operations. The size of the pulsating array can be configured according to the size of the neural network. The hardware size of the PE unit can theoretically range from a minimum (2×2) size to a maximum (n×n) size with no upper limit. The weight data stream channel is used to send in the processed weight data to be calculated. The number of weight data channels and the direction of data flow can be configured according to the scale of the neural network, and can be configured from a minimum of 1 channel and 1 direction to a maximum of 4 channels and 2 directions. The feature data stream channel is used to send in the processed feature data to be calculated. The number of feature data channels and the direction of data flow can be configured according to the scale of the neural network, and can achieve a scale from a minimum of 1 channel and 1 direction to a maximum of 4 channels and 2 directions.

3. A neural network accelerator based on a multi-channel pulsating array according to claim 2, characterized in that, The multi-channel pulsating array consists of (n×n) basic PE computing units.

4. A neural network accelerator based on a multi-channel pulsating array according to claim 2, characterized in that, The PE basic computing unit includes a multiplier for performing multiplication operations; the PE basic computing unit includes an adder for accumulating results; the PE basic computing unit includes a multi-channel result register group for storing parallel multi-channel data; the PE basic computing unit supports four-way four-channel data stream input processing, can achieve data exchange through up to eight input / output interfaces, and achieves data stream arbitration through data selectors and channel selectors; Before the operation, the PE basic computing unit obtains the feature data and weight data to be processed from the previous PE basic computing unit, selects the data flow channel through arbitration, temporarily stores the operation result after completing the multiplication and addition operation, and sends the feature data and weight data to be processed to the secondary PE basic computing unit.

5. A neural network accelerator based on a multi-channel pulsating array according to claim 2, characterized in that, The feature data stream channel sequentially feeds feature data into the pulse array according to the calculation cycle. The feature data stream channel feeds the first row of feature data (F) in the initial calculation cycle Clock 0. A (0,0),……,F A (0,n)), in the next calculation cycle Clock1, the second row of feature data (F) is fed in. A (1,0),……,F A (1,n)), and so on, until all feature data rows are sent in the Clock n cycle; the feature data stream channel exchanges data with the memory unit and obtains the feature data to be processed from the memory unit; the feature data stream channel exchanges data with the PE basic computing unit and sends the feature data to be processed into the corresponding PE basic computing unit according to the different computing cycles.

6. A neural network accelerator based on a multi-channel pulsating array according to claim 2, characterized in that, The weighted data stream channel sequentially feeds weighted data into a pulsating array composed of PE basic computing units according to the clock cycle. In each computing cycle, the weighted data stream channel sequentially feeds weighted data from different rows. Within each computing cycle, the weighted data stream channel sequentially feeds in single weighted data. After the weighted data is multiplied and added with the current feature data, the next weighted data is pulsatingly fed in. The weighted data stream channel also exchanges data with the memory unit to obtain the weighted data to be processed. Furthermore, the weighted data stream channel exchanges data with the PE basic computing units, sending the weighted data to be processed to the corresponding PE basic computing unit.

7. A neural network accelerator based on a multi-channel pulsating array according to claim 5, characterized in that, The feature data stream channels can be configured in terms of number and direction according to the size of the feature data matrix. The multi-channel pulsation array can configure the PE basic calculation unit into 1-4 sub-pulsation arrays according to the size of the feature matrix and weight matrix. The feature data stream channels can be adjusted according to the number of sub-pulsation matrix arrays to achieve a configuration from 1 channel in 1 direction to 4 channels in 2 directions.

8. A neural network accelerator based on a multi-channel pulsating array according to claim 6, characterized in that, The weighted data stream channels can be configured in terms of number and direction according to the size of the weighted data matrix. The multi-channel pulsation array can configure the PE basic calculation unit into 1-4 sub-pulsation arrays according to the size of the weight matrix and feature matrix. The weighted data stream channels can be adjusted according to the number of sub-pulsation matrix arrays to achieve a configuration from 1 channel in 1 direction to 4 channels in 2 directions.

Images