MAC unit supporting input activation and weight double-end sparsity

Abstract

The invention discloses an MAC unit supporting input activation and weight double-end sparsity, and relates to the technical field of neural network accelerator hardware design. Aiming at the technical defects existing in a sparse matrix acceleration scheme, the scheme is adopted, an activation value and non-zero elements of a weight matrix are stored through a special compressed data register, corresponding zero value distribution is recorded through a matched sparse bitmap register, and a joint sparse bitmap marking a double-non-zero effective calculation position is generated through dual-port joint operation; and the control and address generation subunit scans the joint sparse bitmap and outputs an address and an enable signal, the MAC calculation subunit is driven to read non-zero data to complete multiply-accumulate operation, and finally, an operation result is compressed and coded through an output end sparse encoder, so that efficient transmission of interlayer sparse data streams is realized. According to the method, input activation and weight double-end sparsity can be utilized at the same time, end-to-end sparse data flow is supported, zero value calculation can be dynamically skipped, and the energy efficiency ratio and the calculation throughput of neural network reasoning are improved.

Description

A MAC unit that supports input activation and weighted two-sided sparsity Technical Field

[0001] This invention relates to the field of neural network accelerator hardware design technology, specifically a MAC unit that supports input activation and weight sparsity. Background Technology

[0002] With the widespread application of deep learning models in fields such as image recognition and natural language processing, the demand for computing resources and memory bandwidth has increased dramatically. Especially in edge computing devices, computing power, storage space, and power consumption are severely limited, making the efficient execution of large-scale matrix multiplication and accumulation operations in neural networks a critical issue.

[0003] In typical convolutional neural networks or fully connected networks, the multiply-accumulate (MAC) operation is a core computational operation. Traditional MAC units exhibit significant computational redundancy when processing sparse data. Since activation values ​​and weights in neural networks often have high sparsity (i.e., a large number of zero values), processing them in a dense manner would result in a large number of invalid multiply-accumulate operations and memory accesses, wasting computational resources and power.

[0004] Existing technologies offer several acceleration solutions for sparse matrices, such as utilizing sparsity only for activations or weights, or skipping zero-value computations through preprocessing. However, these methods often only leverage one-sided sparsity (e.g., activation sparsity or weight sparsity only), failing to fully exploit the computational optimization potential of two-sided sparsity. Furthermore, traditional sparse acceleration architectures still suffer from redundant movement of zero-value data during data loading, storage, and transmission, leading to a waste of on-chip bandwidth and storage resources. Summary of the Invention

[0005] This invention addresses the needs and shortcomings of current technological development by providing a MAC unit that supports both input activation and weight sparsity, thereby improving the energy efficiency and computational throughput of neural network inference.

[0006] The present invention provides a MAC unit that supports input activation and weighted two-sided sparsity, and the technical solution adopted to solve the above-mentioned technical problems is as follows:

[0007] A MAC unit supporting dual-end sparsity of input activation and weights is deployed in the computational core region of a neural network accelerator. It stores activation values ​​and non-zero elements of the weight matrix through a dedicated compressed data register, and a corresponding sparse bitmap register records the distribution of zero values. A joint sparse bitmap marking the valid computation positions of both non-zero values ​​is generated through dual-port joint operations. A control and address generation subunit scans the joint sparse bitmap and outputs an address and enable signal, driving the MAC computation subunit to read non-zero data and complete multiplication-accumulation operations. Finally, the output sparse encoder compresses and encodes the computation result, achieving efficient transmission of sparse data streams between layers.

[0008] Optionally, the MAC units involved specifically include:

[0009] The IF compressed data register is used to store the sequence of non-zero elements of the activation value matrix in a zero-value compressed format, while retaining the valid values;

[0010] The IF sparse bitmap register is used to store the sparse bitmap corresponding to the activation value matrix. This bitmap represents whether each position in the activation value matrix is ​​zero ("1" indicates non-zero, "0" indicates zero) and provides the coordinate position information of the non-zero elements.

[0011] The FL compressed data register is used to store the non-zero elements of the weight matrix in a zero-value compressed format while retaining the valid values;

[0012] The FL sparse bitmap register is used to store the sparse bitmap corresponding to the weight matrix. This bitmap represents whether each position in the weight matrix is ​​zero ("1" indicates non-zero and "0" indicates zero) and provides the coordinate position information of the non-zero elements.

[0013] The dual-port sparse bitmap joint AND operation subunit is used to receive the sparse bitmaps from the IF sparse bitmap register and the FL sparse bitmap register, and perform bitwise AND operations through preset hardware logic to generate a joint sparse bitmap. Each "1" bit in the joint sparse bitmap represents that the input activation and weight at the corresponding position are both non-zero data.

[0014] The joint sparse bitmap register is used to cache the joint sparse bitmap generated by the dual-port sparse bitmap joint AND operation subunit, which is then sequentially scanned and called by the subsequent control and address generation subunit.

[0015] The control and address generation subunit is used to scan the joint sparse bitmap, identify the location of "double non-zero" data, and then dynamically calculate the storage address of the corresponding non-zero data in the IF compressed data register and the FL compressed data register, while outputting an enable signal.

[0016] The MAC calculation subunit is used to read a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register after receiving the enable signal and the storage address of non-zero data from the control and address generation subunit, perform multiplication and accumulation operations, and transmit the operation result to the output sparse encoder.

[0017] The output sparse encoder is used to perform zero-value detection and compression on the calculation results of the MAC calculation subunit, generate new sparse bitmaps and compressed output data, and realize the continuous transmission of sparse data streams between layers.

[0018] Alternatively, the IF compressed data register and FL compressed data register involved may be implemented using multi-port SRAM or register file, respectively;

[0019] The IF compressed data register stores the sequence of non-zero elements of the activation matrix in a zero-value compressed format, and the FL compressed data register stores the sequence of non-zero elements of the weight matrix in a zero-value compressed format. The data in both types of registers are stored consecutively in row-major order. Furthermore, the bit width of the IF compressed data register and the FL compressed data register are matched with the precision of the corresponding stored data (such as adapting to 8-bit, 16-bit integer, and other data precisions).

[0020] Optionally, the IF sparse bitmap register uses on-chip memory (such as SRAM) to store the sparse bitmap, and each bit in the register corresponds one-to-one with an element in the activation matrix; there is a preset mapping relationship between the sparse bitmap stored in the IF sparse bitmap register and the activation matrix stored in the IF compressed data register, specifically: the non-zero data corresponding to the activation matrix stored in the IF compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap;

[0021] The FL sparse bitmap register uses on-chip memory (such as SRAM) to store the sparse bitmap. Each bit in the register corresponds one-to-one with an element in the activation matrix. There is a preset mapping relationship between the sparse bitmap stored in the FL sparse bitmap register and the weight matrix stored in the FL compressed data register. Specifically, the non-zero data corresponding to the weight matrix stored in the FL compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap.

[0022] Optionally, the dual-port sparse bitmap union and operation subunit involved is a pure combinational logic circuit for sparse data processing scenarios. It is configured with two independent parallel input ports, which are directly connected to the output ports of the IF sparse bitmap register and the FL sparse bitmap register, respectively. The bit widths of the two input bitmaps are perfectly matched, and each bit corresponds one-to-one to represent the bit state of the corresponding position in the IF sparse bitmap and the FL sparse bitmap under the same data dimension.

[0023] The dual-port sparse bitmap joint AND operation subunit performs a fully parallel bitwise AND operation on the two input bitmaps through a preset hardware logic circuit, and outputs a joint sparse bitmap with the same bit width as the input. This joint sparse bitmap is used to filter the target data valid bits that simultaneously satisfy the corresponding bit values ​​of the IF sparse bitmap and the FL sparse bitmap as "1", and transmits them to the joint sparse bitmap register for storage, so that the subsequent control and address generation subunit can sequentially scan and call them.

[0024] Alternatively, the control and address generation subunit involved is a controller integrated inside the MAC unit, the core of which is a bitmap scanner;

[0025] The bitmap scanner traverses the valid bits in the joint sparse bitmap register and calculates the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register based on the coordinate position of the valid bits in the joint sparse bitmap using a preset address mapping algorithm. At the same time, it generates a read enable signal and an address signal and sends them to the IF compressed data register, FL compressed data register and MAC calculation subunit respectively to control the reading and operation triggering of non-zero data.

[0026] The MAC calculation subunit is activated only when both the IF compressed data register and the FL compressed data register have valid data pairs consisting of non-zero data corresponding to the same valid bit.

[0027] Alternatively, the MAC calculation subunit may include a multiplier, an accumulator, and a matching pipelined register.

[0028] The MAC calculation subunit and the control and address generation subunit are synchronized via handshake signals to ensure that each operation corresponds to a set of valid "double non-zero" data pairs. The specific workflow is as follows:

[0029] First, receive the enable signal from the control and address generation subunit, and based on the storage of non-zero data, synchronously read valid data pairs from the IF compressed data register and the FL compressed data register;

[0030] Subsequently, the multiplier performs multiplication on the valid data pairs, and the result is fed into the accumulator and added to the current accumulated value. During the cycle of a single multiplication-accumulation operation, the accumulator remains in its current state until the multiplication-accumulation operation is completed, and then the final accumulated result is transmitted to the output sparse encoder.

[0031] Optionally, the output sparse encoder receives the accumulated result from the MAC calculation subunit, performs zero-value detection and compression operations using a built-in zero-value detection algorithm, and then generates a new sparse bitmap based on the detection result. At the same time, the non-zero output values ​​are stored sequentially into the output compression buffer according to the bit mapping relationship with the new sparse bitmap. Finally, the compressed output data and the new sparse bitmap are output to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

[0032] Alternatively, the workflow of the MAC unit involved is as follows:

[0033] S1. Load the compressed IF / FL data and sparse bitmap into the corresponding IF compressed data register, IF sparse bitmap register, FL compressed data register and FL sparse bitmap register inside the MAC unit.

[0034] S2, the dual-port sparse bitmap joint AND operation subunit performs a bitwise AND operation on the sparse bitmaps in the IF sparse bitmap register and the FL sparse bitmap register through preset hardware logic, generates a joint sparse bitmap, and stores it in the joint sparse bitmap register.

[0035] S3, the control and address generation subunit scans the joint sparse bitmap to identify the location of "double non-zero" data, and then dynamically calculates the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register, while outputting an enable signal.

[0036] According to the enable signal, the S4 and MAC calculation subunits read a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register, and perform multiply-accumulate operations.

[0037] S5. Repeat steps S3-S4 until the current joint sparse bitmap scan is completed, and transmit the cumulative operation result of the MAC calculation subunit to the output sparse encoder.

[0038] S6. The output sparse encoder performs zero-value detection and compression on the accumulated operation result, generates a new sparse bitmap and compressed output data, and outputs it to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

[0039] The MAC unit of the present invention, which supports input activation and weighted two-sided sparsity, has the following advantages compared with the prior art:

[0040] 1. This invention can simultaneously utilize the sparsity of input activation and weights, support end-to-end sparse data flow, and dynamically skip zero-value calculations, thereby improving the energy efficiency and computational throughput of neural network inference.

[0041] 2. This invention designs dedicated IF sparse bitmap registers, FL sparse bitmap registers, a two-port sparse bitmap joint AND operation subunit, and a joint sparse bitmap register. It natively supports and identifies the sparsity of input data (activation values) and weight data at the hardware level. The two-port sparse bitmap joint AND operation subunit performs a bitwise AND operation on the two sparse bitmaps to generate a joint sparse bitmap. This joint sparse bitmap precisely identifies the positions where both input activations and weights are simultaneously non-zero, ensuring that the MAC calculation subunit is only triggered to perform calculations at these "double non-zero" positions, fundamentally addressing the issue. Zero-value calculations are eliminated at any end; by dynamically scanning the joint sparse bitmap through the control and address generation subunit, and intelligently generating the read address of non-zero data in the compressed register, efficient scheduling of "on-demand data retrieval" and "precise calculation" is achieved; at the input end, data is loaded in a compressed format (non-zero value sequence + sparse bitmap); at the output end, the calculation result is zero-value detected and compressed again through the output sparse encoder to generate new sparse data for use by the next layer; this design realizes the compression state of data in the entire link of storage, transmission and calculation, and avoids the propagation of zero values ​​between layers. Attached Figure Description

[0042] Figure 1 is a block diagram of the architecture of the present invention;

[0043] Figure 2 is a flowchart of the process of this invention. Detailed Implementation

[0044] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0045] The following explains the English words used in this embodiment.

[0046] MAC: Multiply-Accumulate, a core computational operation in neural networks.

[0047] IF: Input Feature, refers to the input data of each layer in a neural network.

[0048] FL: Filter, refers to the weight parameters of a convolutional kernel or fully connected layer.

[0049] Sparse Bitmap: A binary mask used to indicate whether each element in a matrix is ​​zero (1 for non-zero, 0 for zero).

[0050] SRAM: Static Random-Access Memory, commonly used as on-chip cache.

[0051] End-to-End Sparse Data Flow refers to data that remains in a compressed and sparse format throughout the computation process, with zero values ​​not participating in storage or computation.

[0052] Example 1:

[0053] This embodiment proposes a MAC unit that supports the sparsity of both input activation and weights. The MAC unit is deployed in the computational core area of ​​a neural network accelerator. It stores the activation values ​​and non-zero elements of the weight matrix through a dedicated compressed data register, and records the corresponding zero value distribution through a sparse bitmap register. A joint sparse bitmap is generated by dual-port joint operation to mark the effective computation positions of the two non-zero values. The control and address generation subunit scans the joint sparse bitmap and outputs the address and enable signal to drive the MAC computation subunit to read the non-zero data and complete the multiplication and accumulation operation. Finally, the operation result is compressed and encoded by the output sparse encoder to achieve efficient transmission of sparse data streams between layers.

[0054] Specifically, referring to Figure 1, the MAC unit of this embodiment includes the following structure:

[0055] The IF compressed data register is used to store the sequence of non-zero elements of the activation value matrix in a zero-value compressed format, while retaining the valid values;

[0056] The IF sparse bitmap register is used to store the sparse bitmap corresponding to the activation value matrix. This bitmap represents whether each position in the activation value matrix is ​​zero ("1" indicates non-zero, "0" indicates zero) and provides the coordinate position information of the non-zero elements.

[0057] The FL compressed data register is used to store the non-zero elements of the weight matrix in a zero-value compressed format while retaining the valid values;

[0058] The FL sparse bitmap register is used to store the sparse bitmap corresponding to the weight matrix. This bitmap represents whether each position in the weight matrix is ​​zero ("1" indicates non-zero and "0" indicates zero) and provides the coordinate position information of the non-zero elements.

[0059] The dual-port sparse bitmap joint AND operation subunit is used to receive the sparse bitmaps from the IF sparse bitmap register and the FL sparse bitmap register, and perform bitwise AND operations through preset hardware logic to generate a joint sparse bitmap. Each "1" bit in the joint sparse bitmap represents that the input activation and weight of the corresponding position are both non-zero data, that is, marking the "double non-zero" position where valid MAC calculation needs to be performed.

[0060] The joint sparse bitmap register is used to cache the joint sparse bitmap generated by the dual-port sparse bitmap joint AND operation subunit, which is then sequentially scanned and called by the subsequent control and address generation subunit.

[0061] The control and address generation subunit is used to scan the joint sparse bitmap, identify the location of "double non-zero" data, and then dynamically calculate the storage address of the corresponding non-zero data in the IF compressed data register and the FL compressed data register, while outputting an enable signal.

[0062] The MAC calculation subunit is used to read a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register after receiving the enable signal and the storage address of non-zero data from the control and address generation subunit, perform multiplication and accumulation operations, and transmit the operation result to the output sparse encoder.

[0063] The output sparse encoder is used to perform zero-value detection and compression on the calculation results of the MAC calculation subunit, generate new sparse bitmaps and compressed output data, and realize the continuous transmission of sparse data streams between layers.

[0064] In this embodiment, the IF compressed data register and the FL compressed data register are implemented using multi-port SRAM or register file, respectively. The IF compressed data register stores the sequence of non-zero elements of the activation matrix in zero-value compressed format, and the FL compressed data register stores the sequence of non-zero elements of the weight matrix in zero-value compressed format. The data in both types of registers are stored consecutively in row-major order; and the bit width of the IF compressed data register and the FL compressed data register are matched with the precision of the corresponding stored data (e.g., adapting to 8-bit, 16-bit integer, etc. data precision).

[0065] The IF sparse bitmap register uses on-chip memory (such as SRAM) to store the sparse bitmap. Each bit in the register corresponds one-to-one with an element in the activation matrix. There is a preset mapping relationship between the sparse bitmap stored in the IF sparse bitmap register and the activation matrix stored in the IF compressed data register. Specifically, the non-zero data corresponding to the activation matrix stored in the IF compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap.

[0066] The FL sparse bitmap register uses on-chip memory (such as SRAM) to store the sparse bitmap. Each bit in the register corresponds one-to-one with an element in the activation matrix. There is a preset mapping relationship between the sparse bitmap stored in the FL sparse bitmap register and the weight matrix stored in the FL compressed data register. Specifically, the non-zero data corresponding to the weight matrix stored in the FL compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap.

[0067] In this embodiment, the dual-port sparse bitmap joint operation subunit is a pure combinational logic circuit for sparse data processing scenarios. It is configured with two independent parallel input ports, which are directly connected to the output ports of the IF sparse bitmap register and the FL sparse bitmap register, respectively. The bit widths of the two input bitmaps are completely matched, and each bit corresponds to the bit state of the corresponding position of the IF sparse bitmap and the FL sparse bitmap under the same data dimension.

[0068] The dual-port sparse bitmap joint AND operation subunit performs a fully parallel bitwise AND operation on the two input bitmaps through a preset hardware logic circuit, and outputs a joint sparse bitmap with the same bit width as the input. This joint sparse bitmap is used to filter the target data valid bits that simultaneously satisfy the corresponding bit values ​​of the IF sparse bitmap and the FL sparse bitmap as "1", and transmits them to the joint sparse bitmap register for storage, so that the subsequent control and address generation subunit can sequentially scan and call them.

[0069] For example, for the (i, j)th position in the joint sparse bitmap, the corresponding bit in the joint sparse bitmap is "1" only if both the IF sparse bitmap and the FL sparse bitmap are "1" at that position.

[0070] The joint sparse bitmap register uses on-chip memory (such as SRAM) of the same size as the IF sparse bitmap register and FL sparse bitmap register to store the joint sparse bitmap.

[0071] In this embodiment, the control and address generation subunit is a controller integrated within the MAC unit, and its core is a bitmap scanner. This bitmap scanner traverses the valid bits in the joint sparse bitmap register and, based on the coordinate positions (row and column numbers) of the valid bits in the joint sparse bitmap, calculates the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register using a preset address mapping algorithm. Simultaneously, it generates a read enable signal and an address signal, which are sent to the IF compressed data register, FL compressed data register, and MAC calculation subunit, respectively, thereby controlling the reading and operation triggering of non-zero data.

[0072] The MAC calculation subunit is activated only when both the IF compressed data register and the FL compressed data register have valid data pairs consisting of non-zero data corresponding to the same valid bit.

[0073] In this embodiment, the MAC calculation subunit includes a multiplier, an accumulator, and a corresponding pipelined register. The MAC calculation subunit and the control and address generation subunit are synchronized via a handshake signal to ensure that each operation corresponds to a set of valid "double non-zero" data pairs. The specific workflow is as follows:

[0074] First, receive the enable signal from the control and address generation subunit, and based on the storage of non-zero data, synchronously read valid data pairs (such as int8 activation value and int8 weight value) from the IF compressed data register and FL compressed data register.

[0075] Subsequently, the multiplier performs multiplication on valid data pairs (such as int8 activation value and int8 weight value), and the result is fed into the accumulator and added to the current accumulated value;

[0076] During a single multiply-accumulate operation cycle, the accumulator remains in its current state until the multiply-accumulate operation is completed, and then the final accumulation result is transmitted to the output sparse encoder.

[0077] In this embodiment, the output sparse encoder receives the accumulated result output by the MAC calculation subunit, performs zero-value detection and compression operations through the built-in zero-value detection algorithm, and then generates a new sparse bitmap based on the detection result. At the same time, the non-zero output values ​​are stored sequentially into the output compression buffer according to the bit mapping relationship with the new sparse bitmap. Finally, the compressed output data and the new sparse bitmap are output to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

[0078] It should be added that the above registers and sub-units are connected through dedicated data paths: the sparse bitmap path goes from the bitmap register to the control and address generation sub-unit via joint operation; the data path goes from the compressed data register to the MAC calculation sub-unit via address selection. Control signals are generated uniformly by the control and address generation sub-unit, scheduling data reading, calculation enable, and output encoding initiation.

[0079] Referring to Figure 2, the MAC unit based on this embodiment operates as follows:

[0080] S1. Load the compressed IF / FL data and sparse bitmap into the corresponding IF compressed data register, IF sparse bitmap register, FL compressed data register and FL sparse bitmap register inside the MAC unit.

[0081] S2, the dual-port sparse bitmap joint AND operation subunit performs a bitwise AND operation on the sparse bitmaps in the IF sparse bitmap register and the FL sparse bitmap register through preset hardware logic, generates a joint sparse bitmap, and stores it in the joint sparse bitmap register.

[0082] S3, the control and address generation subunit scans the joint sparse bitmap to identify the location of "double non-zero" data, and then dynamically calculates the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register, while outputting an enable signal.

[0083] According to the enable signal, the S4 and MAC calculation subunits read a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register, and perform multiply-accumulate operations.

[0084] S5. Repeat steps S3-S4 until the current joint sparse bitmap scan is completed, and transmit the cumulative operation result of the MAC calculation subunit to the output sparse encoder.

[0085] S6. The output sparse encoder performs zero-value detection and compression on the accumulated operation result, generates a new sparse bitmap and compressed output data, and outputs it to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

[0086] In summary, the MAC unit of the present invention, which supports both input activation and weight sparsity, can simultaneously utilize the sparsity of input activation and weight, support end-to-end sparse data flow, and dynamically skip zero-value calculations, thereby improving the energy efficiency and computational throughput of neural network inference.

[0087] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A MAC unit supporting input activation and weighted two-sided sparsity, characterized in that, The MAC unit is deployed in the core computing area of ​​the neural network accelerator. It stores the activation values ​​and non-zero elements of the weight matrix through a dedicated compressed data register, and records the corresponding zero value distribution through a sparse bitmap register. A joint sparse bitmap is generated by dual-port joint operation to mark the effective computing positions of the two non-zero values. The control and address generation subunit scans the joint sparse bitmap and outputs the address and enable signals, driving the MAC calculation subunit to read non-zero data to complete the multiplication and accumulation operations. Finally, the output sparse encoder compresses and encodes the operation results to achieve efficient transmission of sparse data streams between layers.

2. The MAC unit supporting input activation and weighted two-sided sparsity according to claim 1, characterized in that, The MAC unit specifically includes: an IF compressed data register, used to store the sequence of non-zero elements of the activation value matrix in zero-value compressed format and retain valid values; an IF sparse bitmap register, used to store a sparse bitmap corresponding to the activation value matrix, which represents whether each position in the activation value matrix is ​​zero and provides the coordinate position information of non-zero elements; an FL compressed data register, used to store the non-zero elements of the weight matrix in zero-value compressed format and retain valid values; an FL sparse bitmap register, used to store a sparse bitmap corresponding to the weight matrix, which represents whether each position in the weight matrix is ​​zero and provides the coordinate position information of non-zero elements; and a dual-port sparse bitmap joint AND operation subunit, used to receive the sparse bitmaps from the IF sparse bitmap register and the FL sparse bitmap register, perform a bitwise AND operation through preset hardware logic to generate a joint sparse bitmap, where each "1" bit in the joint sparse bitmap represents the input activation and weight at the corresponding position. All data are non-zero. The joint sparse bitmap register is used to cache the joint sparse bitmap generated by the dual-port sparse bitmap joint AND operation subunit for subsequent sequential scanning and calling by the control and address generation subunit. The control and address generation subunit is used to scan the joint sparse bitmap, identify the location of "double non-zero" data, and then dynamically calculate the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register, while outputting an enable signal. The MAC calculation subunit is used to read a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register after receiving the enable signal and the storage address of the non-zero data from the control and address generation subunit, perform multiplication and accumulation operations, and transmit the calculation result to the output sparse encoder. The output sparse encoder is used to perform zero value detection and compression on the calculation result of the MAC calculation subunit, generate a new sparse bitmap and compressed output data, and realize the continuous transmission of inter-layer sparse data stream.

3. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 2, characterized in that, The IF compressed data register and the FL compressed data register are implemented using multi-port SRAM or register file, respectively. The IF compressed data register stores the sequence of non-zero elements of the activation matrix in zero-value compressed format, and the FL compressed data register stores the sequence of non-zero elements of the weight matrix in zero-value compressed format. The data in both types of registers are stored continuously in row-major order. The bit width of the IF compressed data register and the FL compressed data register are matched with the precision of the corresponding stored data.

4. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 3, characterized in that, The IF sparse bitmap register uses on-chip memory to store the sparse bitmap, and each bit in the register corresponds one-to-one with an element in the activation matrix. There is a preset mapping relationship between the sparse bitmap stored in the IF sparse bitmap register and the activation matrix stored in the IF compressed data register. Specifically, the non-zero data corresponding to the activation matrix stored in the IF compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap. The FL sparse bitmap register uses on-chip memory to store the sparse bitmap, and each bit in the register corresponds one-to-one with an element in the activation matrix. There is a preset mapping relationship between the sparse bitmap stored in the FL sparse bitmap register and the weight matrix stored in the FL compressed data register. Specifically, the non-zero data corresponding to the weight matrix stored in the FL compressed data register can be directly indexed by the position sequence of the bits with a value of "1" in the sparse bitmap.

5. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 4, characterized in that, The dual-port sparse bitmap joint AND operation subunit is a pure combinational logic circuit designed for sparse data processing scenarios. It is configured with two independent parallel input ports, which are directly connected to the output ports of the IF sparse bitmap register and the FL sparse bitmap register, respectively. The bit widths of the two input bitmaps are perfectly matched, and each bit corresponds one-to-one with the bit state of the corresponding position in the IF sparse bitmap and the FL sparse bitmap under the same data dimension. The dual-port sparse bitmap joint AND operation subunit performs a fully parallel bitwise AND operation on the two input bitmaps through a preset hardware logic circuit, and outputs a joint sparse bitmap with the same bit width as the inputs. This joint sparse bitmap is used to filter the target data valid bits that simultaneously satisfy the condition that the corresponding bit value in both the IF sparse bitmap and the FL sparse bitmap is "1", and transmits them to the joint sparse bitmap register for storage, so that the subsequent control and address generation subunit can sequentially scan and call them.

6. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 5, characterized in that, The control and address generation subunit is a controller integrated within the MAC unit, the core of which is a bitmap scanner. This bitmap scanner traverses the valid bits in the joint sparse bitmap register and, based on the coordinate position of the valid bits in the joint sparse bitmap, calculates the storage address of the corresponding non-zero data in the IF compressed data register and FL compressed data register through a preset address mapping algorithm. At the same time, it generates a read enable signal and an address signal, and sends them to the IF compressed data register, FL compressed data register, and MAC calculation subunit, respectively, thereby controlling the reading and operation triggering of non-zero data. The MAC calculation subunit is only activated when the valid data pairs consisting of non-zero data corresponding to the same valid bit in the IF compressed data register and FL compressed data register are all ready.

7. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 6, characterized in that, The MAC calculation subunit includes a multiplier, an accumulator, and a matching pipelined register. The MAC calculation subunit is synchronized with the control and address generation subunit via a handshake signal to ensure that each operation corresponds to a set of "double non-zero" valid data pairs. The specific workflow is as follows: First, the enable signal of the control and address generation subunit is received. Based on the storage of non-zero data, valid data pairs are read synchronously from the IF compressed data register and the FL compressed data register. Then, the multiplier performs a multiplication operation on the data pairs, and the result is fed into the accumulator and added to the current accumulated value. During the cycle of a single multiply-accumulate operation, the accumulator remains in its current state until the multiply-accumulate operation is completed, and then the final accumulated result is transmitted to the output sparse encoder.

8. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 7, characterized in that, The output sparse encoder receives the accumulated result from the MAC calculation subunit, performs zero-value detection and compression operations through the built-in zero-value detection algorithm, and then generates a new sparse bitmap based on the detection result. At the same time, the non-zero output values ​​are stored in the output compression buffer according to the bit mapping relationship with the new sparse bitmap. Finally, the compressed output data and the new sparse bitmap are output to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

9. A MAC unit supporting input activation and weighted two-sided sparsity according to claim 8, characterized in that, The workflow of the MAC unit is as follows: S1, Load the compressed IF / FL data and sparse bitmap into the corresponding IF compressed data register, IF sparse bitmap register, FL compressed data register and FL sparse bitmap register inside the MAC unit. S2. The dual-port sparse bitmap joint AND operation subunit performs a bitwise AND operation on the sparse bitmaps in the IF sparse bitmap register and the FL sparse bitmap register through preset hardware logic to generate a joint sparse bitmap and store it in the joint sparse bitmap register; S3. The control and address generation subunit scans the joint sparse bitmap, identifies the location of "double non-zero" data, and then dynamically calculates the storage address of the corresponding non-zero data in the IF compressed data register and the FL compressed data register, while outputting an enable signal; S4. The MAC calculation subunit reads a pair of non-zero activation values ​​and weight values ​​from the corresponding IF compressed data register and FL compressed data register according to the enable signal, and performs a multiply-accumulate operation; S5. Repeat steps S3-S4 until the current joint sparse bitmap scan is completed, and transmit the accumulation operation result of the MAC calculation subunit to the output sparse encoder. S6. The output sparse encoder performs zero-value detection and compression on the accumulated operation result, generates a new sparse bitmap and compressed output data, and outputs it to the next level memory or MAC unit through the write-back interface to ensure the continuous transmission of sparse data streams between layers.

Images