In-memory calculation device supporting sparse matrix calculation

Abstract

The invention relates to a storage calculation device supporting sparse matrix calculation, which comprises a storage core particle, a logic core particle, a multiply-accumulate circuit, a characteristic value distribution circuit and a sparse matrix decomposition circuit, and is characterized in that the storage core particle is used for storing a sparse weight matrix, and the logic core particle is used for data interaction, caching and control; the multiply-accumulate circuit executes multiply-accumulate operation of characteristic values and weights, the sparse matrix decomposition circuit screens out position codes of front # MAC non-zero weights in each operation period and outputs control signals, and the characteristic value distribution circuit dynamically maps input characteristic values to corresponding non-zero weights according to the control signals, so that calculation of an arbitrarily distributed sparse matrix is realized. The method has the advantages that multiply-accumulate operation is performed on the non-zero weight, the zero weight is skipped, power consumption is reduced, calculation efficiency is improved, and efficient processing of the sparse matrix is achieved through a storage and calculation integrated framework.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of integrated circuits and artificial intelligence accelerators, and in particular to an in-memory computing device supporting sparse matrix computation. BACKGROUND

[0002] The memory-compute integrated technology embeds computing units in the memory array to realize arithmetic operations and logical operations, including multiply-accumulate operations, bitwise AND operations, XOR operations, etc. As a new computing architecture, the memory-compute integrated technology aims to break through the "memory wall" bottleneck caused by the separation of "computing units" and "storage units" in the traditional von Neumann architecture, i.e., to avoid the delay and power consumption overhead caused by the frequent transfer of data between storage units and computing units. Therefore, the memory-compute integrated technology has advantages such as high parallel computing, large capacity storage, and high bandwidth transmission, and is naturally suitable for accelerating artificial intelligence technologies such as neural networks.

[0003] Matrix multiplication between weight matrices and feature value matrices is the most important computation in neural networks, Figure 1 The chip in the related art is a typical architecture of a three-dimensional stacked memory-compute integrated chip for accelerating matrix computation. The chip includes a plurality of stacked memory dies and a logic die. The memory dies are used to store parameters such as weight matrices in a neural network, and the logic die includes an interface for interacting with data outside the chip, a bus, a cache, and control logic. The two are interconnected by through-silicon vias (TSVs) for transmission of data signals and control signals. Each memory die includes a plurality of memory banks. Each memory bank includes a memory cell array, an I0 data read / write circuit, and is used to store and read / write corresponding data. Matrix multiplication operations are realized by integrating multiply-accumulate circuits between the I0 data read / write circuit and the global data path. The memory cell array is used to store weight matrices of a neural network. Feature matrices are transmitted from the logic die to the corresponding memory bank of the memory die via the global data path, and matrix multiplication is performed with the weight matrices in the multiply-accumulate circuit. The calculation results are written back to the logic die again via the global data path.

[0004] According to the related art in the above, the inventors believe that the existing memory-compute integrated technology supporting sparse matrix computation has the following defects: Pruning and compression of neural networks can effectively reduce the storage size of weight matrices, and thus reduce the corresponding computation size, thereby improving the overall latency and power consumption performance of the system. During this process, part of the elements of the weight matrix are zeroed out and thus constitute a sparse weight matrix. Limited by the memory access mode of the memory-compute integrated circuit rule, it generally does not support irregular sparse matrix computation. Some works integrate feature value distribution circuits between the multiply-accumulate circuit and the global data path, which supports sparse matrix computation to a limited extent. Figure 2The schematic diagram of the memory-compute integrated circuit supporting sparse matrix calculation is given in the specification. The memory-compute cell array stores the compressed weight matrix, i.e., only stores the non-zero weights W5, W4, W1, W0, and no longer stores the weight elements after pruning and zeroing. The compressed weight matrix records the relative positions of the remaining non-zero weight elements in the original weight matrix through position coding, where 1 represents the remaining non-zero weight, and 0 represents the weight after pruning and zeroing. After the feature matrix is transmitted to the corresponding memory bank of the memory core through the global data channel, and before entering the multiply-accumulate circuit, the feature value distribution circuit is additionally used. The feature value distribution network dynamically controls the data channel according to the position coding of the sparse weight matrix, and transmits the corresponding feature value to the remaining weight, so as to correctly complete the matrix multiplication and accumulation calculation. The structure of the feature value distribution circuit can be any common data distribution circuit, such as cross array routing network, Barnes routing network and butterfly routing network. The technical solution uses the butterfly routing network as the implementation scheme of the feature value distribution circuit.

[0005] Limited by the fixed circuit size of the feature value distribution circuit and the subsequent multiply-accumulate circuit, the memory-compute integrated circuit can only distribute a fixed number of feature values and complete a fixed number of multiply-accumulate calculations per cycle. Referring to the memory-compute integrated circuit shown in the specification Figure 2 The memory-compute integrated circuit can only distribute 4 feature values and complete 4 multiply-accumulate calculations at the same time. When the number of remaining weights in the sparse weight matrix is greater than 4, it means that the number of effective feature values to be distributed is also greater than 4, which will cause the feature value distribution circuit and the multiply-accumulate circuit to be blocked, and the multiply-accumulate operation between the feature values A6, A7 and the weights W6, W7 cannot be correctly completed. Therefore, Figure 2 The memory-compute integrated circuit shown in the specification can only support N:M sparse weight matrix calculation, i.e., N non-zero weights are retained after pruning of every M consecutive weight elements. In the specification Figure 2 In the specification, M is equal to 8, and N is equal to 4. This limits the distribution of non-zero weight elements in the pruned sparse weight matrix. In order to meet the N:M sparse weight distribution, the weight that needs to be retained may be incorrectly pruned, and the weight that should be pruned may be incorrectly retained, thereby causing the accuracy of the neural network to decrease. SUMMARY

[0006] In order to improve the technical problem of causing the accuracy of the neural network to decrease, the application provides a memory-compute integrated device supporting sparse matrix calculation.

[0007] The memory-compute integrated device supporting sparse matrix calculation provided by the application adopts the following technical scheme: A memory-compute integrated device supporting sparse matrix calculation, comprising: Memory core: the memory core is used for storing a sparse weight matrix; Logic chip: The logic chip is used for data interaction, caching, and control; multiply-accumulate circuit, used to perform multiply-accumulate operations on feature values ​​and weights; Eigenvalue allocation circuit: The eigenvalue allocation circuit is located between the global data path of the logic chip and the multiply-accumulate circuit, and is used to dynamically allocate the input eigenvalues ​​to the corresponding non-zero weights according to the position encoding of the sparse weight matrix. Sparse matrix decomposition circuit: The sparse matrix decomposition circuit is set between the global data path and the eigenvalue allocation circuit. It is used to filter out the position codes of the first #MAC non-zero weights in each operation cycle, form a control signal and transmit it to the eigenvalue allocation circuit, so as to avoid the multiply-accumulate circuit blocking and support the calculation of sparse weight matrix with arbitrary non-zero weight distribution.

[0008] By adopting the above technical solution, the in-memory computing device effectively combines storage and computing resources by storing a sparse weight matrix in the storage core and integrating data interaction, caching, and control functions in the logic core. This reduces frequent data movement between storage and computation, improving computational efficiency. The multiply-accumulate circuit can directly perform multiply-accumulate operations on the input eigenvalues ​​and non-zero weights in the sparse matrix, achieving efficient computation of the sparse matrix. Furthermore, by setting an eigenvalue allocation circuit between the global data path in the logic core and the multiply-accumulate circuit, the input eigenvalues ​​can be dynamically and accurately allocated to the corresponding non-zero weights based on the sparse structure and positional encoding of the sparse matrix, thereby avoiding invalid computation and improving performance. In terms of efficiency, the sparse matrix factorization circuit can quickly select the position codes of the first #MAC non-zero weights in each operation cycle, generate corresponding control signals to drive the eigenvalue allocation circuit for data scheduling, effectively avoid the blocking phenomenon caused by the multiply-accumulate circuit waiting for sparse matrix data, ensure the continuity and high parallelism of the calculation process, and the architecture supports arbitrarily distributed non-zero weight sparse matrix calculations. It has both flexibility and versatility, maximizes the use of sparsity features at the hardware level, reduces power consumption, shortens operation latency, and improves the overall system throughput and energy efficiency. It is suitable for application scenarios with strong demand for sparse matrix operations, such as deep learning and artificial intelligence inference.

[0009] Optionally, the sparse matrix decomposition circuit includes four modules: a bit-by-bit accumulation circuit, a comparison circuit, a bit-by-bit AND circuit, and a bit-by-bit XOR circuit.

[0010] By adopting the above technical solution, the sparse matrix decomposition circuit achieves efficient counting and accumulation of non-zero weight position codes through a bit-by-bit accumulation circuit, and completes rapid screening of thresholds by combining a comparison circuit. It can accurately extract the target number of non-zero weights within the operation cycle. At the same time, the bit-by-bit AND circuit and the bit-by-bit XOR circuit are used together to realize logical screening and difference judgment of position codes, ensuring the accuracy and uniqueness of control signal generation. It can quickly complete the decomposition and scheduling control of sparse matrices at the hardware level, improve allocation efficiency and reduce computational latency.

[0011] Optionally, the bit-by-bit accumulation circuit is used to accumulate the position encoding of the sparse weight matrix bit by bit to generate an accumulation result of the number of non-zero weights at each position.

[0012] By adopting the above technical solution, the bit-by-bit accumulation circuit can perform bit-by-bit accumulation operations on the position encoding of the sparse weight matrix, count the number of non-zero weights at each position in real time, and generate the corresponding accumulation results. This provides accurate basic data for subsequent comparison and screening, avoids complex counting operations at the software level, and improves processing speed. The circuit completes the accumulation statistics at the hardware level, which can reduce the delay and power consumption of the control logic, ensure the rapid location and extraction of non-zero weights, and thus improve the overall efficiency and parallelism of sparse matrix calculation.

[0013] Optionally, the comparison circuit is used to compare the accumulation result of the bit-by-bit accumulation circuit with the circuit size #MAC to obtain the position flag corresponding to the accumulation result that is less than #MAC.

[0014] By adopting the above technical solution, the comparison circuit compares the accumulation result generated by the bit-by-bit accumulation circuit with the circuit size #MAC in real time, thereby filtering out valid results smaller than #MAC and generating corresponding position flags. It accurately limits the number of non-zero weights that need to participate in the calculation within the operation cycle, avoiding resource conflicts or blocking caused by exceeding the parallel computing capacity of the circuit. The hardware-level comparison enables fast flag generation, which greatly improves the efficiency and response speed of the sparse matrix decomposition process and ensures efficient connection and continuous operation of eigenvalue allocation and multiply-accumulate calculation.

[0015] Optionally, the bitwise AND circuit is used to perform a bitwise logical AND operation between the output of the comparison circuit and the position code of the sparse weight matrix to obtain the position codes of the first #MAC non-zero weights.

[0016] By adopting the above technical solution, the bitwise AND circuit can perform a bitwise AND operation between the output of the comparison circuit and the position code of the sparse weight matrix. At the hardware level, it can quickly extract the position codes corresponding to the first #MAC non-zero weights, improve the matching between the screening results and the scale of parallel computing of the circuit, avoid invalid or redundant codes from entering the subsequent allocation stage, complete the extraction of effective codes within one operation cycle, and improve the execution efficiency of sparse matrix calculation and the overall throughput of the system.

[0017] Optionally, the bitwise XOR circuit is used to XOR the bitwise AND result with the position code of the sparse weight matrix bit by bit to form the input position code for the next cycle.

[0018] By adopting the above technical solution, the bitwise XOR circuit performs a bitwise XOR operation on the output of the bitwise AND circuit and the position code of the sparse weight matrix, efficiently removing the selected non-zero weight position codes and forming a new position code to be input into the next operation cycle. This realizes the dynamic updating and iterative filtering of unprocessed non-zero weights, ensuring that the sparse matrix decomposition process is completed continuously and without repetition in multiple cycles, achieving rapid updates, reducing the complexity of control logic, and improving the processing continuity and overall efficiency of sparse matrix calculation.

[0019] Optionally, the eigenvalue allocation circuit is a butterfly routing network, used to dynamically map input eigenvalues ​​to corresponding non-zero weights according to the control signal output by the sparse matrix decomposition circuit.

[0020] By adopting the above technical solution, the eigenvalue allocation circuit adopts a butterfly routing network structure, which can realize dynamic mapping and efficient routing of eigenvalues ​​to non-zero weights at the hardware level according to the control signal output by the sparse matrix decomposition circuit. It can flexibly adapt to arbitrarily distributed sparse matrix structures, avoid invalid data transmission, maintain low-latency data allocation performance under multi-path parallel conditions, realize high-bandwidth, low-conflict eigenvalue scheduling, and improve the overall efficiency and energy efficiency ratio of sparse matrix multiplication and accumulation operations.

[0021] Optionally, the memory chip is interconnected with the logic chip via a through-silicon via (TSV) to enable high-speed transmission of data and control signals.

[0022] By adopting the above technical solutions, high-speed, low-latency transmission of data signals and control signals is achieved, shortening the signal path, reducing transmission delay and power consumption, increasing bandwidth and interconnection density, and effectively alleviating the "memory wall" problem in the storage-compute separation architecture.

[0023] Optionally, the sparse matrix decomposition circuit can support the calculation of sparse weight matrices with arbitrary distributions.

[0024] By adopting the above technical solutions, the device can efficiently extract and schedule non-zero weights when processing different models, different layer structures, or dynamically changing sparse matrices, avoiding the degradation of computing performance caused by irregular distribution. Through hardware-level adaptability, it maintains high parallelism while ensuring the continuity and accuracy of the computing process, thereby improving the applicability and robustness of the overall system.

[0025] Optionally, the device performs multiplication and accumulation operations only on the non-zero weights in the sparse weight matrix and skips the operations corresponding to zero-value weights, thereby reducing power consumption and improving computational efficiency. The sparse matrix decomposition circuit and the eigenvalue allocation circuit work together to gradually complete the full calculation of the sparse matrix over multiple cycles until all non-zero weight elements have been processed. The logic core includes cache and control logic, which are used to temporarily store feature matrix data and coordinate the operation process of sparse matrix decomposition circuit and multiply-accumulate circuit.

[0026] By adopting the above technical solution, the non-zero weights in the sparse weight matrix perform multiplication and accumulation operations, effectively avoiding the ineffective energy consumption caused by zero-value calculation, significantly reducing power consumption and improving computational efficiency. The sparse matrix decomposition circuit and the eigenvalue allocation circuit work together to gradually complete the full calculation of the sparse matrix in multiple cycles, ensuring that any distributed non-zero weights can be processed efficiently. The cache and control logic in the logic core further realize the temporary storage and operation scheduling of data, ensuring efficient collaboration between circuits and the continuity of the calculation process.

[0027] In summary, this application includes at least one of the following beneficial technical effects: 1. By storing a sparse weight matrix in the storage core and integrating caching and control logic in the logic core, a tight integration of computation and storage is achieved, reducing data movement and improving overall computational efficiency; 2. Multiplication and accumulation operations are performed only on non-zero weights, skipping operations corresponding to zero-value weights, which significantly reduces ineffective power consumption and improves energy efficiency ratio; 3. The eigenvalue allocation circuit adopts a butterfly routing network, which dynamically maps the input eigenvalues ​​according to the control signal output by the sparse matrix decomposition circuit, thereby achieving high-bandwidth and low-latency eigenvalue scheduling. 4. The sparse matrix factorization circuit supports the calculation of sparse weights with arbitrary distribution, and is not restricted by the arrangement rules of non-zero weights, thus ensuring the versatility and robustness of the system. 5. By hierarchically coordinating bit-by-bit accumulation circuits, comparison circuits, bit-by-bit AND circuits, and bit-by-bit XOR circuits, the decomposition, filtering, and iterative update of sparse matrices can be quickly achieved at the hardware level, reducing latency and avoiding computational blockage.

[0028] 6. Storage chips and logic chips are interconnected at high speed and low latency through TSV, which improves data transmission bandwidth and alleviates the "memory wall" problem in the storage-compute separation architecture; 7. The sparse matrix decomposition circuit and the eigenvalue allocation circuit work together to gradually complete the calculation of all non-zero weights over multiple cycles, ensuring high parallelism and continuous operation characteristics; 8. The device can significantly improve throughput and energy efficiency in scenarios with high demand for sparse matrix operations, such as deep learning and artificial intelligence inference. Attached Figure Description

[0029] Figure 1 This is a typical architecture diagram of a three-dimensional stacked in-memory computing chip used to accelerate matrix calculations.

[0030] Figure 2 This is a schematic diagram showing the limited support of in-memory computing circuits for N:M sparse weight matrix calculations.

[0031] Figure 3 This is a schematic diagram of an in-memory computing device that supports sparse matrix computation according to an embodiment of this application. Detailed Implementation

[0032] The following is in conjunction with the appendix Figure 3 This application will be described in further detail.

[0033] This application discloses an in-memory computing device that supports sparse matrix computation. (Refer to...) Figure 3 This includes storage cores, which are used to store sparse weight matrices. Combined with the data interaction, caching, and control functions of logic cores, this effectively reduces the frequent movement of data between storage and computing units, thereby reducing data transmission latency and power consumption. The logic core is used for data interaction, caching, and control. The multiply-accumulate circuit is used to perform multiply-accumulate operations on feature values ​​and weights. The multiply-accumulate circuit only performs operations on input feature values ​​and non-zero weights, avoiding the invalid overhead caused by zero-value calculations and achieving a higher energy efficiency ratio. The feature value allocation circuit is set between the global data path of the logic core and the multiply-accumulate circuit. It is used to dynamically allocate the input feature values ​​to the corresponding non-zero weights according to the position encoding of the sparse weight matrix. A sparse matrix factorization circuit is positioned between the global data path and the eigenvalue allocation circuit. In each computation cycle, it filters out the position codes of the first #MAC non-zero weights, forming a control signal that is transmitted to the eigenvalue allocation circuit. This avoids blocking of the multiply-accumulate circuit and supports the calculation of sparse weight matrices with arbitrary non-zero weight distributions. It rapidly filters out the first #MAC non-zero weight position codes in each computation cycle and converts them into control signals to drive the eigenvalue allocation circuit, effectively preventing the multiply-accumulate circuit from being blocked due to data waiting, ensuring the continuity and high parallelism of the computation process. Furthermore, this factorization circuit can support the calculation of sparse weight matrices with arbitrary distributions, unrestricted by the arrangement of non-zero weights, enhancing the system's versatility and robustness.

[0034] Reference Figure 3 An eigenvalue allocation circuit and a sparse matrix factorization circuit are integrated simultaneously between the global data path and the multiply-accumulate circuit. The sparse matrix factorization circuit is the key to this technical solution. Within each cycle, the sparse matrix factorization circuit filters out the relative positions of the first #MAC non-zero weights in the sparse weight matrix, thus avoiding blockage of the eigenvalue allocation circuit and the multiply-accumulate circuit. Here, #MAC represents the scale of the eigenvalue allocation circuit and the multiply-accumulate circuit. Figure 3 The sparse matrix decomposition circuit has a size of 4, meaning that 4 valid eigenvalues ​​are allocated and 4 multiplication-accumulation operations are performed in each cycle. The circuit consists of 4 main steps / 4 levels: a bitwise accumulation circuit, a comparison circuit, a bitwise AND circuit, and a bitwise XOR circuit. First, the bitwise accumulation circuit reads the position code of the sparse weight matrix and accumulates it bit by bit in order from the least significant bit to the most significant bit. That is, the result at this position is obtained by adding the accumulation result of the previous bit and the position code of this position. The accumulation result represents the number of non-zero weights at this position and the positions before it. Then, the comparison circuit compares the accumulation result of each position with the circuit size #MAC. An accumulation result smaller than #MAC means that the number of non-zero weights at this position and the positions before it is less than the circuit size, and the corresponding eigenvalues ​​need to be allocated by the eigenvalue allocation network. Second, the comparison result at each position is bitwise ANDed with the position code of the sparse weight matrix, thereby truncating the position codes of the first #MAC non-zero weights. The bitwise AND result serves as the control signal for the eigenvalue allocation circuit of this cycle, controlling the formation of the corresponding data path and allocating the eigenvalues ​​corresponding to the first #MAC non-zero weights.

[0035] like Figure 3As shown, the eigenvalue allocation network only allocates eigenvalues ​​A0 to A4 and performs multiplication and accumulation operations with non-zero weights W0, W1, W2, and W4. Since the position codes of non-zero weights W5 to WN are truncated to zero in this period, the eigenvalue allocation network will not allocate the corresponding eigenvalues ​​A5 to AN. Finally, the bitwise AND result of each position is XORed with the position code of the sparse weight matrix. The XOR result sets the position code of the first #MAC non-zero weights to zero, while retaining the position codes of the remaining positions. The XOR result is used as the new position code as the input position code of the sparse matrix decomposition circuit in the next period and is used in the eigenvalue allocation circuit. The above four-level operation logic is repeated until all non-zero weights in the sparse weight matrix are completely traversed. At this time, all valid calculations corresponding to the sparse weight matrix are completed.

[0036] In specific implementation, such as Figure 3 As shown, an in-memory computing device supporting sparse matrix computation includes: Storage core: Used to store sparse weight matrix. By directly storing the weight data in the sparse matrix in the storage core, storage and computation are tightly coupled, avoiding frequent data transfer between external storage units. Logic Chip: The logic chip is used to perform data interaction, caching, and control operations. It integrates cache units and control logic, enabling it to temporarily store input feature matrix data and schedule and coordinate the operations of the sparse matrix decomposition circuit and the multiply-accumulate circuit to ensure the continuity and efficiency of the overall computation process. Multiply-accumulate circuit (MAC): Used to perform multiply-accumulate operations on the input feature values ​​and the non-zero weights in the sparse matrix to achieve efficient computation of the sparse matrix; In this embodiment, the multiply-accumulate circuit only performs operations on the non-zero weights and automatically skips the calculations corresponding to the zero-value weights, thereby effectively reducing ineffective energy consumption and improving energy efficiency ratio; Eigenvalue allocation circuit: Located between the global data path and the multiply-accumulate circuit of the logic core, it is used to dynamically allocate the input eigenvalues ​​to the corresponding non-zero weights according to the position encoding of the sparse weight matrix. Preferably, the eigenvalue allocation circuit adopts a butterfly routing network structure, which can achieve low-latency eigenvalue scheduling under multi-path parallel conditions, avoid invalid data transmission, and improve computational efficiency.

[0037] Sparse matrix factorization circuit: Located between the global data path and the eigenvalue allocation circuit, it is used to filter out the position codes of the first #MAC non-zero weights in each operation cycle, form a control signal, and transmit it to the eigenvalue allocation circuit to drive the dynamic allocation of eigenvalues; In this embodiment, the sparse matrix factorization circuit includes four-level modules: bit-by-bit accumulation circuit, comparison circuit, bitwise AND circuit, and bitwise XOR circuit. 1. Bit-by-bit accumulation circuit: The positional encoding of the sparse weight matrix is ​​accumulated bit by bit to generate an accumulation result with a non-zero weight number; 2. Comparison circuit: Compare the accumulated result with the circuit size #MAC to obtain the position flag corresponding to the valid result that is less than #MAC; 3. Bitwise AND circuit: Perform a bitwise logical AND operation between the comparison result and the position code of the sparse weight matrix to obtain the position codes of the first #MAC non-zero weights; 4. Bitwise XOR circuit: The output of the bitwise AND circuit is XORed bit by bit with the position code of the sparse matrix to remove the processed weight code and form a new input code to be passed to the next cycle, realizing the dynamic update and iterative filtering of unprocessed weights.

[0038] During operation, the in-memory computing device in this embodiment can extract a limited number of non-zero weights in each operation cycle, and the eigenvalue allocation circuit dynamically allocates eigenvalues ​​to the corresponding multiply-accumulate units to ensure the continuity and high parallelism of the calculation process. After multiple cycles, all non-zero weights are processed, realizing the complete calculation of the sparse matrix.

[0039] In addition, high-speed interconnection between memory chips and logic chips is achieved through through silicon vias (TSVs), ensuring low-latency transmission of data and control signals and further alleviating the "memory wall" problem in traditional architectures.

[0040] In this specific embodiment, the provided in-memory computing device fully utilizes the sparsity characteristics of sparse matrices at the hardware level, performing multiplication and accumulation operations only on non-zero weights to avoid invalid calculations, significantly reducing power consumption and improving computing efficiency. At the same time, the device supports the calculation of arbitrarily distributed sparse weight matrices in terms of structure, and has good versatility and flexibility, and can be widely used in application scenarios with high requirements for sparse matrix operations, such as deep learning and artificial intelligence inference.

[0041] The technical solution of this application has the following beneficial effects: 1. This technical solution integrates sparse matrix decomposition and eigenvalue allocation circuits into a memory computing circuit, enabling typical memory computing circuits to support sparse weight matrix calculations. Simultaneously, it avoids the restriction on the distribution of non-zero weights in the sparse weight matrix, thus supporting sparse weight matrix calculations with arbitrary non-zero weight distributions. Compared to memory computing circuits that only partially support N:M distribution sparse weight calculations, this effectively guarantees the accuracy of the pruned neural network.

[0042] 2. This technical solution only needs to store the non-zero weights retained after clipping, without having to store the weights after setting them to zero. This effectively improves the storage efficiency and memory access efficiency of the in-memory computing circuit, thereby reducing the corresponding memory access latency and power consumption.

[0043] 3. The sparse matrix decomposition circuit in this technical solution, in conjunction with the eigenvalue allocation circuit, only performs the multiplication and accumulation calculations corresponding to the retained non-zero weights, and skips the multiplication and accumulation calculations corresponding to the weights after they are set to zero, thereby improving the computational efficiency of the in-memory computing circuit and reducing the corresponding computational delay and power consumption.

[0044] Compared with existing technical solutions, this technical solution integrates a sparse matrix decomposition circuit and an eigenvalue allocation circuit in the in-memory computing circuit to support sparse weight matrix calculation, thereby improving the storage and computing efficiency of the in-memory computing circuit. At the same time, this technical solution avoids the restriction on the distribution of non-zero weights in the sparse weight matrix, effectively ensuring the accuracy of the neural network.

[0045] This technical solution proposes a unique sparse matrix decomposition circuit, which includes four main modules: a bit-by-bit accumulation circuit, a comparison circuit, a bit-by-bit AND circuit, and a bit-by-bit XOR circuit. This circuit is used to decompose a sparse weight matrix with arbitrary non-zero weight distribution into multiple N:M sparse weight vectors with fixed distributions. The sparse weight matrix calculation is then completed in the eigenvalue allocation circuit and the multiply-accumulate circuit. The sparse matrix decomposition circuit is the main innovation and key technology of this technical solution.

[0046] The implementation principle of an in-memory computing device supporting sparse matrix computation according to an embodiment of this application is as follows: By storing the sparse weight matrix in the storage core and integrating data interaction, caching, and control logic in the logic core, the computation process is carried out as close as possible to the data storage end, thereby reducing the frequent data movement between the storage and computing units and reducing bandwidth pressure. In each operation cycle, the sparse matrix decomposition circuit performs bit-by-bit accumulation and comparison on the position codes of the input sparse weight matrix, quickly selecting the position codes corresponding to the first #MAC non-zero weights, generating a control signal output to the eigenvalue allocation circuit, which then... This control signal dynamically routes the input feature values ​​to the corresponding non-zero weights, achieving precise matching between data and weights. The multiply-accumulate circuit directly performs multiply-accumulate operations on the feature values ​​and non-zero weights, thereby completing part of the matrix calculation. The bitwise AND circuit and the bitwise XOR circuit alternately update the position encoding during the cycle, ensuring that all non-zero weights can be fully processed within multiple cycles. Through the above chain operation of decomposition-allocation-computation, it efficiently adapts to arbitrarily distributed sparse weight matrices at the hardware level, achieving low-power, high-throughput matrix operations, which is suitable for computational scenarios with significant sparsity in deep learning and artificial intelligence inference.

[0047] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An in-memory computing device supporting sparse matrix computation, characterized in that: include: Storage cores: Storage cores are used to store sparse weight matrices; Logic chip: The logic chip is used for data interaction, caching, and control; The multiply-accumulate circuit is used to perform multiply-accumulate operations on the feature values ​​and weights; Eigenvalue allocation circuit: The eigenvalue allocation circuit is located between the global data path of the logic chip and the multiply-accumulate circuit, and is used to dynamically allocate the input eigenvalues ​​to the corresponding non-zero weights according to the position encoding of the sparse weight matrix. Sparse matrix decomposition circuit: The sparse matrix decomposition circuit is set between the global data path and the eigenvalue allocation circuit. It is used to filter out the position codes of the first #MAC non-zero weights in each operation cycle, form a control signal and transmit it to the eigenvalue allocation circuit, so as to avoid the multiply-accumulate circuit blocking and support the calculation of sparse weight matrix with arbitrary non-zero weight distribution.

2. The in-memory computing device according to claim 1, characterized in that: The sparse matrix decomposition circuit includes four modules: bit-by-bit accumulation circuit, comparison circuit, bit-by-bit AND circuit, and bit-by-bit XOR circuit.

3. The in-memory computing device according to claim 2, characterized in that: The bit-by-bit accumulation circuit is used to accumulate the position encoding of the sparse weight matrix bit by bit to generate the accumulation result of the number of non-zero weights at each position.

4. The in-memory computing device according to claim 2, characterized in that: The comparison circuit is used to compare the accumulation result of the bit-by-bit accumulation circuit with the circuit size #MAC to obtain the position flag corresponding to the accumulation result that is less than #MAC.

5. The in-memory computing device according to claim 2, characterized in that: The bitwise AND circuit is used to perform a bitwise logical AND operation between the output of the comparison circuit and the position code of the sparse weight matrix to obtain the position codes of the first #MAC non-zero weights.

6. The in-memory computing device according to claim 2, characterized in that: The bitwise XOR circuit is used to XOR the bitwise AND result with the position code of the sparse weight matrix bit by bit to form the input position code for the next cycle.

7. The in-memory computing device according to claim 1, characterized in that: The eigenvalue allocation circuit is a butterfly routing network used to dynamically map input eigenvalues ​​to corresponding non-zero weights based on the control signal output by the sparse matrix decomposition circuit.

8. The in-memory computing device according to claim 1, characterized in that: The memory chip is interconnected with the logic chip via through-silicon vias (TSVs) to enable high-speed transmission of data and control signals.

9. The in-memory computing device according to claim 1, characterized in that: The sparse matrix decomposition circuit can support the calculation of sparse weight matrices with arbitrary distributions, and is not limited to N:M sparse distribution.

10. The in-memory computing device according to claim 1, characterized in that: The device performs multiplication and accumulation operations only on the non-zero weights in the sparse weight matrix and skips the operations corresponding to zero-value weights, thereby reducing power consumption and improving computational efficiency. The sparse matrix decomposition circuit and the eigenvalue allocation circuit work together to gradually complete the full calculation of the sparse matrix over multiple cycles until all non-zero weight elements have been processed. The logic core includes cache and control logic, which are used to temporarily store feature matrix data and coordinate the operation process of sparse matrix decomposition circuit and multiply-accumulate circuit.

Images