Storage and calculation integrated acceleration macro, chip and equipment supporting multi-mode convolution

Abstract

The invention discloses a storage and calculation integrated acceleration macro supporting multimode convolution, a chip and equipment, the storage and calculation integrated acceleration macro comprises an input cache (1), a column address decoder (2) and 64 multiply-accumulate operation arrays (3), each multiply-accumulate operation array (3) comprises a row address decoder (31), a multimode adaptability controller (32), an SRAM array (33), a multiplier (34) and an adder tree network (35), and the 64 multiply-accumulate operation arrays (3) share one common column address decoder (2) and are connected to the same input cache (1). According to the method, the inherent flexibility and expandability of the digital CIM are utilized, efficient control logic is designed under the condition that the original CIM storage density is not sacrificed, various kinds of convolution are supported with the minimum extra area cost, and high calculation efficiency is kept.

Description

Technical Field

[0001] The present invention belongs to the field of digital SRAM in-memory computing technology, and specifically relates to a storage-computation integrated acceleration macro, chip and device supporting multi-mode convolution. Background Art

[0002] With the increase of data movement between CPU and memory units, SRAM Computing-in-Memory (SRAM-CIM) deep neural network accelerators (SRAM-CIM macros) based on SRAM have two orders of magnitude improvement in energy efficiency (63TOPS / W vs 2.827TOPS / W) compared to traditional von Neumann architectures (such as H100). Digital SRAM-CIM has high noise resilience and precise computing capabilities, and is regarded as a promising alternative to traditional von Neumann DNN accelerators. The currently proposed digital SRAM-CIM macros use weight stationary (WS) in-situ computing, which is a computing paradigm with less weight data movement. However, these SRAM-CIM macros still face several challenges: 1) WS does not reduce off-chip and on-chip data communication, and additional data movement will put pressure on bandwidth. 2) WS cannot ensure high utilization of on-chip computing units in some convolutional layers.

[0003] Figure 1 Comparison of the amount of activation data in different networks using the CIM (Computing-in-Memory) macro computed in WS in situ, and the computational resource utilization diagram for each layer of MobileNet-V2, Figure 1 (a) in the figure is a comparison of the amount of activation data in different networks. Figure 1 The vertical axis of (a) is the total amount of activated data, and the vertical axis is the network type, including VGG16, ResNet18, ResNet50 and MobileNet-V2. The diagonal bar graph represents the actual amount of data that needs to be moved from outside the chip to the CIM macro. The square bar graph represents the total amount of activated data of the network. The data on the left side of the arrow represents the multiple difference between the bar graph data. Figure 1 (b) in the figure is the computing resource utilization diagram for each layer of MobileNet-V2. Figure 1 The vertical axis of (b) is the computing resource utilization of CIM, and the horizontal axis is the network layer splitting of MobileNet-V2. Figure 1As shown in (a) of Figure 1, the 64×64 CIM macro exhibits data redundancy in different networks. This redundancy is particularly evident when executing VGG16. Therefore, for any CIM macro that only supports WS data flow, the problem of data redundancy is inevitable due to the unified functionality of the CIM macro. In addition, we map the MobileNet-V2 network to the 64×64 CIM macro, as shown in Figure 1 As shown in (b) in Figure 2. Due to the mismatch between the matrix and CIM sizes, less than 25% of the CIM is used during the execution of the first convolution layer. In addition, when performing operations other than traditional convolutions (such as depthwise convolutions in MobileNet-V2), where depthwise convolutions are DepthWidth (DW) convolutions. Due to the inflexibility of the adder tree, the utilization of the CIM macro drops to less than 5%.

[0004] Figure 2 DW convolution in the prior art and its mapping in the traditional 64×64 digital SRAM-CIM macro, standard convolution and its mapping in the traditional 64×64 digital SRAM-CIM macro, as well as the digital SRAM-CIM macro structure and the area ratio of each module. Figure 2 (a) in the figure shows the DW convolution and its mapping in the traditional 64×64 digital SRAM-CIM macro. Figure 2 (b) in the figure shows the standard convolution and its mapping in the traditional 64×64 digital SRAM-CIM macro. Figure 2 (c) in the figure is the digital SRAM-CIM macro structure and the area ratio of each module, where C represents the number of channels of the input image, I represents the side length of the input image, N represents the number of output channels, k×k represents the convolution kernel size, and im2col represents the original convolution form converted into matrix multiplication form for calculation through the im2col method. Figure 2 (c) in Figure 3 details the computational flow of the digital SRAM-CIM macro. In this setup, the input vector is broadcast across the entire SRAM array and then multiplied with each data column in the array. The results are then aggregated vertically to form the final output. To alleviate the overhead associated with data exchange in the SRAM array, multiple SRAM arrays typically use a set of shared adder trees. Therefore, once the processing of one matrix is ​​completed, only the adjacent SRAM arrays need to be activated for continuous computation. Specifically, for all SRAM-CIM macros that adopt WS data flow, the weight data is stored in the SRAM cells in a fixed order, while the activation data is converted to matrix form via im2col. In a typical neural network, such as Figure 2As shown in (b) in the figure, there are C activations and N weights. Each weight is convolved with C activation functions to produce the final result. However, in some special networks, such as the DW convolution of MobileNet, each activation requires a corresponding weight. Figure 2 As shown in (a) in the figure, when we try to apply these types of convolutions to traditional CIM macros, we observe that for traditional neural network convolutions, if N is not enough to match the scale of the CIM macro, the weights cannot occupy all the storage units within the CIM macro. This problem is even more important for DW convolutions. Moreover, the amount of activation data after im2col conversion is amplified to varying degrees, resulting in repeated movement of activation data. The activation redundancy R can be used to characterize the repeated movement of activation data. The calculation function expression of the activation redundancy R is as follows: , In the above formula, W K , H K , W I and H I Here, we substitute the parameters of the first layer of the VGG16 model, where S is equal to 1 and W is equal to 1. I and H I All are 226, W K and H K All are 3, and C equals 3. In this case, a total of 1.2 million activation data chips from off-chip memory need to be copied, resulting in 8 times more data movement and bandwidth than the original requirements. Therefore, how to design efficient control logic without sacrificing the original CIM storage density has become a key technical issue that needs to be solved urgently. Summary of the invention

[0005] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a storage-computation integrated acceleration macro, chip and device supporting multi-mode convolution are provided. The present invention aims to utilize the inherent flexibility and scalability of digital CIM to design efficient control logic without sacrificing the original CIM storage density, support various convolutions with minimal additional area cost and maintain high computing efficiency.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: A storage-and-computation-in-one acceleration macro supporting multi-mode convolution comprises an input cache, a column address decoder and 64 multiplication-and-accumulation operation arrays, wherein the multiplication-and-accumulation operation array comprises: a row address decoder for selecting a row of an SRAM array into which data is written so as to cooperate with the column address decoder to complete the accurate writing of data into the SRAM array; a multi-mode adaptive controller for controlling the multiplication-and-accumulation operation array according to external input signals Xin, Yin and Zin to perform convolution or matrix multiplication; an SRAM array for storing weights WS or activation data IS according to the data flow mode of the storage-and-computation-in-one acceleration macro; a multiplier for performing a bitwise multiplication operation of output data of the row SRAM array with output data of the input cache; an adder tree network for grouping and accumulating data output by all multipliers according to control signals and outputting the data to the outside of the storage-and-computation-in-one acceleration macro; the 64 multiplication-and-accumulation operation arrays share a common column address decoder and are connected to the same input cache.

[0007] Optionally, the SRAM array is composed of an SRAM cell, and the SRAM cell includes a first transmission gate, a second transmission gate, a third transmission gate, a first inverter, a second inverter and a third inverter. The input signal WBLB of the SRAM cell is connected to a first input end of the first transmission gate, and the output end of the first transmission gate is connected to the input end of the third transmission gate after passing through the first inverter and the third inverter in sequence. The output end of the third transmission gate is used to output the output signal RBLB of the SRAM cell. The second input end of the first transmission gate is connected to the write signal WWL, and the control end is connected to the write signal WWLB. The first input end of the second transmission gate is connected to the middle node of the first inverter and the third inverter through the second inverter arranged in reverse. The second transmission gate is connected to the control end of the control end of the control end of the control end of the control end of the control end. The second input end of the input gate is connected to the write signal WWLB, the control end is connected to the write signal WWL, and the output end is connected to the output end of the first transmission gate. One input end of the third transmission gate is connected to the output end of the third inverter, the other input end is connected to the read signal RWL, and the control end is connected to the read signal RWLB. When the write signal WWL and the write signal WWLB are set to "1" and "0" respectively to activate the first transmission gate, the data of the input WBLB is written into the SRAM cell, and the data is maintained inside the SRAM cell through the first inverter and the third inverter; when the read signal RWL and the read signal RWLB are set to "1" and "0" respectively to activate the data stored inside the SRAM cell, the SRAM cell is output through the third transmission gate as the output signal RBLB.

[0008] Optionally, the SRAM array is composed of 64 memory banks, each memory bank contains 4×4 SRAM units, the storage space size of the SRAM array is 1kb, each memory bank is connected to the multiplier through four read data lines, and each operation selects one column of each memory bank for output. The selection process is controlled by a multi-mode adaptive controller; in each clock cycle, the multiplier uses a 4×1 multiplexer to select 4-bit data from the input cache and perform bit multiplication with the data stored in the SRAM array, and then outputs it to the addition tree network. The addition tree network summarizes the 4-bit×1-bit results according to the requirements of the control logic, and these summarized results are then passed to the internal shift accumulator. The shift accumulator uses a shift-and-add method to obtain the final required 4-bit×4-bit or 4-bit×8-bit data and outputs it through the output line NOUT of the addition tree network.

[0009] Optionally, the adder tree network is composed of a 6-layer adder tree and an integration module, wherein the first layer of the adder tree includes 32 2-input adders and an interrupt module IE arranged at the front end of the 32 2-input adders, the 2-input adders are composed of multiple full adders FA, and the first layer of the adder tree is used to receive the 64×4-bit multiplication result of each SRAM unit output data and the input cache input C-CIM data, and the multiplication result is sorted in the order of the serial numbers of the memory bank from 0 to 63 as the addend, and the addends with corresponding serial numbers are added in pairs by the 2-input adder, and if the two adjacent addends do not need to be accumulated, the corresponding interrupt module IE is enabled to interrupt the addition and then output directly, wherein one The first addend is output to the second-layer adder tree, and the other addend is output to the third or fourth-layer adder tree; the second-layer adder tree includes 16 2-input adders, which are used to add the 32 2-input adders of the first-layer adder tree in pairs. If there is an addition interruption in a 2-input adder of the 2-input adder of the first-layer adder tree, only the addend of the two addends of the 2-input adder that is closest to the first-layer adder tree is used as the input of the corresponding 2-input adder in the second-layer adder tree; the third-layer adder tree includes 6 2-input adders and 2 N-2 addition selectors. The 2-input adders of the third-layer adder tree are used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree, and the 2 N-2 addition selectors of the third-layer adder tree are used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree. The selectors are used to add the outputs of the 12 2-input adders of the 2nd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 4th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are connected to the shift accumulator RA, the 2 2-input adders of the 4th layer of the addition tree are used to accumulate the outputs of the 4 2-input adders of the 3rd layer of the addition tree, the 2 N-2 addition selectors of the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 5th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; The tree includes 2 N-2 addition selectors, and the output ends of the 2 N-2 addition selectors in the 5th layer of the addition tree are connected to the shift accumulator RA, and the 2 N-2 addition selectors in the 5th layer of the addition tree are respectively used to accumulate the outputs of a 2-input adder and an N-2 addition selector in the 4th layer of the addition tree; the 6th layer of the addition tree includes 1 N-2 addition selector, and the output ends of the N-2 addition selectors in the 6th layer of the addition tree are connected to the shift accumulator RA, and the N-2 addition selectors in the 6th layer of the addition tree are respectively used to accumulate the outputs of the 2 N-2 addition selectors in the 5th layer of the addition tree; the outputs of the shift accumulators RA of each layer of the addition tree are connected to the input end of the integration module.

[0010] Optionally, the input of the full adder FA includes three addends, namely, addend A, addend B and addend Cin, and the output includes a carry CO and an addition result S. Only when the input addends A, B and Cin are all set to "1" or two of them are set to "1", the output carry CO is "1", otherwise the output carry CO is always "0"; when two of the input addends A, B and Cin are set to "1" or all are set to "0" or all three addends are set to "0", the output addition result S is set to "0", otherwise the output addition result S is always "1".

[0011] Optionally, the interrupt module IE includes multiple one-bit data interrupt modules IE, and the one-bit data interrupt module IE is composed of a MOS tube and a transmission gate. The input one-bit data Ii is connected to the input end of the transmission gate, and the output end of the transmission gate outputs Ii' and is connected to an input end of a 2-input adder. The control end of the transmission gate is respectively connected to control signals Ctr and CtrB, and the output end of the transmission gate is grounded through the MOS tube, and the control end of the MOS tube is connected to the control signal Ctr. When the control signals Ctr and CtrB are set to "1" and "0", the transmission gate is interrupted, and the output Ii' is grounded and set to zero, thereby achieving the purpose of interrupt transmission.

[0012] Optionally, the shift accumulator RA includes: a first N-1 selector, used to select an output from input data c_in and 0 based on a control signal ctrl; a three-input adder, used to perform accumulation calculation on the output of the first N-1 selector, the result of the left shift operation of the shift accumulator RA output, and the input data P_sum; a second N-1 selector, used to select an output from the output of the three-input adder and the input data P_sum based on the control signal ctrl; a register, used to temporarily store the output data of the second N-1 selector and then output it; a one-bit shifter, which performs a left shift operation on the partial sum output by the register by one bit and inputs it into the three-input adder as the result of the left shift operation of the shift accumulator RA output by one bit.

[0013] Optionally, the multi-mode adaptive controller includes multiple controller modules and multiple C-NOT gates, the controller module includes multiple data channels, each data channel includes three sub-control modules and a switch connected in sequence, each controller module receives the same input Xin and its own input Yin and inputs the internal sub-control module and the switch respectively, the sub-control module is composed of a switch connected to a C-NOT gate, the switch is composed of a transmission gate T0, a MOS tube T2 and an inverter T1, the control ends of the transmission gate T0 and the MOS tube T2 are connected to the input Xin, the transmission gate T0 determines whether to open the input Yin to the output RWL of the transmission gate T0 according to the input Xin; if the input Xin is set to "1", the transmission gate T0 is opened, the MOS tube T2 is closed, the input Yin is passed to the output RWL, and the inverter T1 performs inversion processing according to the input Yin to generate a reverse output RWLB; if the input Xin is set to "0", the transmission gate T0 is closed, the MOS tube T2 is opened, the output RWL is grounded and the output is "0"; the C-NOT gate It is composed of a transmission gate T4 and an inverter T3. If the input Xin is set to "1", the inverter T3 works in the inverter mode, the transmission gate T4 is closed, and the output Yout is the inversion of the input Yin. If the input Xin is set to "0", the inverter T3 works in the transmission gate mode, and the transmission gate T4 is opened to enhance the Yin input. At this time, the output Yout is equal to the input Yin. The sub-control module outputs Yout and the output RWL and the reverse output RWLB based on the preset truth table through the input Xin and the input Yin. The output Yout is transmitted from left to right to the next-level sub-control module until it is transmitted to the switch. The output RWL and the reverse output RWLB signals of each level of sub-control module and the switch are integrated to finally output the multi-mode adaptive controller. Each output RWL and the reverse output RWLB control the data reading of three columns of the SRAM array. The C-NOT gate combines the output RWL and the reverse output RWLB and the input Z to obtain the data selection signals of the other five columns of the SRAM array for controlling the data reading of the other five columns of the SRAM array.

[0014] In addition, the present invention also provides a chip, including a chip body and a storage-computation-in-one acceleration macro arranged in the chip body, wherein the storage-computation-in-one acceleration macro is the aforementioned storage-computation-in-one acceleration macro supporting multi-mode convolution.

[0015] In addition, the present invention also provides a computer device, which includes a microprocessor and a memory connected to each other, and the microprocessor includes the aforementioned storage and computing-in-one acceleration macro that supports multi-mode convolution.

[0016] Compared with the prior art, the present invention mainly has the following advantages: the storage-computation-in-one acceleration macro supporting multi-mode convolution of the present invention includes an input cache, a column address decoder and 64 multiplication-accumulation operation arrays. The storage-computation-in-one acceleration macro supporting multi-mode convolution of the present invention can flexibly switch data flow modes to maintain high utilization of C-CIM under any workload; the present invention can support multiple convolution modes and can flexibly support convolutions of different convolution kernel sizes; the present invention greatly reduces the data transmission between off-chip and on-chip by supporting on-chip convolution, saving power consumption and time overhead caused by transmitting data from off-chip to C-CIM. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure shows the comparison of the amount of activated data in different networks of CIM using WS in-situ calculation in the existing technology, as well as the computing resource utilization diagram of each layer for MobileNet-V2.

[0018] Figure 2 The DW convolution in the prior art and its mapping in the traditional 64×64 digital SRAM-CIM macro, the standard convolution and its mapping in the traditional 64×64 digital SRAM-CIM macro, the digital SRAM-CIM macro structure, and the area ratio of each module are shown.

[0019] Figure 3 Schematic diagram of the structure of the storage-computation-in-one acceleration macro in an embodiment of the present invention.

[0020] Figure 4 Schematic diagram of the structure of the SRAM array in an embodiment of the present invention.

[0021] Figure 5 Schematic diagram of the structure of the addition tree network in an embodiment of the present invention.

[0022] Figure 6 Schematic diagram of the local structure (first two layers) of the addition tree network in an embodiment of the present invention.

[0023] Figure 7 FIG. 4 is a schematic diagram of the structure of the shift accumulator RA in an embodiment of the present invention.

[0024] Figure 8 Schematic diagram of the structure of a multi-mode adaptive controller in an embodiment of the present invention.

[0025] Fig. 9 Schematic diagram of the structure of the sub-control module in the embodiment of the present invention.

[0026] Fig.10 Schematic diagram of the control principle of the SRAM array in an embodiment of the present invention.

[0027] Fig.11The figure shows the calculation process of C-CIM in the 3×3 convolution mode in an embodiment of the present invention.

[0028] Fig.12 The figure shows the calculation process of C-CIM in the matrix multiplication mode in an embodiment of the present invention.

[0029] Fig.13 This is the calculation process of C-CIM in the 5×5 convolution mode in an embodiment of the present invention.

[0030] Legend: 1. Input buffer; 2. Column address decoder; 3. Multiply-accumulate operation array; 31. Row address decoder; 32. Multi-mode adaptive controller; 321. Controller module; 322. C-NOT gate; 323. Sub-control module; 324. Switch; 33. SRAM array; 331. First transmission gate; 332. Second transmission gate; 333. Third transmission gate; 334. First inverter; 335. Second inverter; 336. Third inverter; 34. Multiplier; 35. Adder tree network; 351. First N-1 selector; 352. Three-input adder; 353. Second N-1 selector; 354. Register; 355. One-bit shifter. DETAILED DESCRIPTION

[0031] like Figure 3 As shown, the present embodiment provides a storage-computation-in-one acceleration macro (C-CIM) supporting multi-mode convolution, including an input cache 1, a column address decoder 2 and 64 multiplication-accumulation operation arrays (MAC arrays) 3, the multiplication-accumulation operation array 3 including: a row address decoder 31, used to select the row of the SRAM array 33 into which data is written so as to cooperate with the column address decoder 2 to complete the accurate writing of data to the SRAM array 33; a multi-mode adaptive controller 32, used to control the multiplication-accumulation operation array 3 according to external input signals Xin, Yin and Zin to perform convolution or matrix multiplication; an SRAM array 33, used to store weights WS or activation data IS according to the data flow mode of the storage-computation-in-one acceleration macro; a multiplier 34, used to perform a bitwise multiplication operation of the output data of the row SRAM array 33 with the output data of the input cache 1; an adder tree network 35, used to group and accumulate the data output by all the multipliers 34 according to the control signal and output it to the outside of the storage-computation-in-one acceleration macro; the 64 multiplication-accumulation operation arrays 3 share a common column address decoder 2 and are connected to the same input cache 1.

[0032] In this embodiment, each multiplication-accumulation operation array 3 includes a 1kb SRAM bit unit and a bit multiplication unit constructed using a NOR gate and an addition tree network. The SRAM array consists of 64 memory banks, each of which contains 4×4 12TSRAM bit units. The 64 banks are divided into 8 blocks, with the first seven blocks each divided into 9 banks and the last block divided into 1 bank. In order to realize the simultaneous bit multiplication and addition operations of 64 groups of 4-bit data, each bank is connected to the multiplier through four read data lines, and each operation selects one column of each bank for output, and the selection process is controlled by the controller. In each clock cycle, the multiplier 34 uses a 4×1 multiplexer to select 4-bit data from the input cache 1 to perform bit multiplication with the data stored in the SRAM array 33 and then outputs it to the addition tree network 35. The addition tree network 35 summarizes the 4-bit×1-bit results according to the requirements of the control logic. These summarized results are then passed to the internal shift accumulator. The shift accumulator uses a shift-and-accumulate method to obtain the final required 4-bit×4-bit or 4-bit×8-bit data and outputs it through the output line NOUT of the addition tree network 35.

[0033] like Figure 4As shown, the SRAM array 33 is composed of SRAM cells, and the SRAM cells include a first transmission gate 331, a second transmission gate 332, a third transmission gate 333, a first inverter 334, a second inverter 335 and a third inverter 336. The input signal WBLB of the SRAM cell is connected to the first input end of the first transmission gate 331, and the output end of the first transmission gate 331 is connected to the input end of the third transmission gate 333 after passing through the first inverter 334 and the third inverter 336 in sequence. The output end of the third transmission gate 333 is used to output the output signal RBLB of the SRAM cell. The second input end of the first transmission gate 331 is connected to the write signal WWL, and the control end is connected to the write signal WWLB. The first input end of the second transmission gate 332 is connected to the first inverter 334 and the third inverter 336 through the second inverter 335 arranged in reverse. The middle nodes are connected, the second input end of the second transmission gate 332 is connected to the write signal WWLB, the control end is connected to the write signal WWL, and the output end is connected to the output end of the first transmission gate 331, one input end of the third transmission gate 333 is connected to the output end of the third inverter 336, the other input end is connected to the read signal RWL, and the control end is connected to the read signal RWLB. When the write signal WWL and the write signal WWLB are set to "1" and "0" respectively, the first transmission gate 331 is activated, the data input WBLB is written into the SRAM cell, and the data is maintained inside the SRAM cell through the first inverter 334 and the third inverter 336; when the read signal RWL and the read signal RWLB are set to "1" and "0" respectively, the data stored in the SRAM cell is activated and output from the SRAM cell through the third transmission gate 333 as the output signal RBLB. The input WBLB indicates the data that needs to be written into the SRAM cell; the write signal WWL / WWLB, when set to "1 / 0" to activate the transmission gate, the data input by WBLB is written into the SRAM memory, and the data is maintained inside the memory through two inverters; the read signal RWL / RWLB, when set to "1 / 0" to activate, the data stored in the SRAM memory passes through the transmission gate and then outputs the memory via RBLB.

[0034] In this embodiment, the SRAM array 33 is composed of 64 memory banks, each of which contains 4×4 SRAM cells. The storage space size of the SRAM array 33 is 1kb. Each memory bank is connected to the multiplier 34 via four read data lines. Each operation selects one column of each memory bank for output. The selection process is controlled by the multi-mode adaptive controller 32. In each clock cycle, the multiplier 34 uses a 4×1 multiplexer to select 4-bit data from the input cache 1 and performs bitwise multiplication with the data stored in the SRAM array 33, and then outputs it to the addition tree network 35. The addition tree network 35 summarizes the 4-bit×1-bit results according to the requirements of the control logic. These summarized results are then passed to the internal shift accumulator. The shift accumulator uses a shift-and-add method to obtain the final required 4-bit×4-bit or 4-bit×8-bit data and outputs it through the output line NOUT of the addition tree network 35.

[0035] Based on the fact that different sizes of convolution kernels correspond to different accumulation groups, an adder tree network (ATN) is designed in this embodiment to enhance the interconnection between the adders. This enables the adder tree to be automatically split according to the corresponding convolution mode. Figure 5As shown, the adder tree network 35 is composed of 6 layers of adder trees and integration modules, wherein the first layer of the adder tree includes 32 2-input adders and an interrupt module IE arranged at the front end of the 32 2-input adders, the 2-input adders are composed of multiple full adders FA, and the first layer of the adder tree is used to receive the 64×4-bit multiplication result of each SRAM unit output data and the input cache 1 input C-CIM data, and the multiplication result is sorted in the order of the memory bank from 0 to 63 as the addend, and the addends with corresponding serial numbers are added by the 2-input adder for odd and even addition, and if two adjacent addends do not need to be accumulated, the corresponding interrupt module IE is enabled to interrupt the addition and then output directly, and one of the addends is output. The first adder is output to the second-layer adder tree, and the other adder is output to the third or fourth-layer adder tree; the second-layer adder tree includes 16 2-input adders, which are used to add the 32 2-input adders of the first-layer adder tree in pairs. If there is an addition interruption in a 2-input adder of the 2-input adder of the first-layer adder tree, only the addend of the two addends of the 2-input adder that is closest to the first-layer adder tree is used as the input of the corresponding 2-input adder in the second-layer adder tree; the third-layer adder tree includes 6 2-input adders and 2 N-2 addition selectors. The 2-input adders of the third-layer adder tree are used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree, and the 2 N-2 addition selectors of the third-layer adder tree are used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree. The selectors are used to add the outputs of the 12 2-input adders of the 2nd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 4th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are connected to the shift accumulator RA, the 2 2-input adders of the 4th layer of the addition tree are used to accumulate the outputs of the 4 2-input adders of the 3rd layer of the addition tree, the 2 N-2 addition selectors of the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 5th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; The tree includes 2 N-2 addition selectors, and the output ends of the 2 N-2 addition selectors in the 5th layer of the addition tree are connected to the shift accumulator RA, and the 2 N-2 addition selectors in the 5th layer of the addition tree are respectively used to accumulate the outputs of a 2-input adder and an N-2 addition selector in the 4th layer of the addition tree; the 6th layer of the addition tree includes 1 N-2 addition selector, and the output ends of the N-2 addition selectors in the 6th layer of the addition tree are connected to the shift accumulator RA, and the N-2 addition selectors in the 6th layer of the addition tree are respectively used to accumulate the outputs of the 2 N-2 addition selectors in the 5th layer of the addition tree; the outputs of the shift accumulators RA of each layer of the addition tree are connected to the input end of the integration module. Figure 5The addition tree accumulation process of the 3×3 convolution mode is described in FIG. Since the SRAM cells of the C-CIM are placed in units of 4, the 64 SRAM cells are also divided into 16 groups. Since the on-chip convolution with a convolution kernel size of 3×3 makes each operation involve the accumulation of only 9 data, data 8 is not accumulated with data 9. At this time, the interrupt module IE storing data 8 and data 9 is enabled by the control signal Ctr signal to separate the accumulation path of data 8 and data 9-11. At this time, data 8 is passed to the 4th level addition tree alone to be accumulated with the accumulation results of the first 8 numbers, while data 9-11 is added with the accumulation results of data 12-17, and finally 7 accumulation results are obtained and output. Similarly, when the 5×5 convolution mode is selected, the interrupt module IE interrupts the addition tree network 35 from the center. For matrix multiplication and 7×7 convolution mode, the CTR signal is disabled, and all data are accumulated with adjacent data. The final result is obtained through a 6-level addition tree.

[0036] Figure 6 The diagram is a partial structure (first two layers) of the addition tree network in the MAC array. Full adder FA: It is composed of 20 MOS tubes and 4 inverters. It adds the three addends A, B and Cin to obtain the carry CO and the addition result S. Interrupt module IE: It is used to interrupt the accumulated signal according to the input Ctr / CtrB signal group. When Ctr / CtrB is set to "1 / 0", the transmission gate is interrupted, and the output A' is grounded and set to zero, so as to achieve the purpose of interrupt transmission. When the data is interrupted, the original input data will be output from the interrupt module IE to the integration module. Integration module: By integrating the output of the interrupt module IE and the full adder FA, the output is divided into left and right parts. The corresponding enabled interrupt module IE accumulates the data of the left and right parts Out_L and Out_R respectively.

[0037] The full adder FA is an existing circuit structure, which is composed of 20 MOS tubes and 4 inverters. The carry CO and the addition result S are obtained by adding the three addends A, B and Cin. Figure 6 As shown, the input of the full adder FA includes three addends, namely, addend A, addend B and addend Cin, and the output includes a carry CO and an addition result S. Only when the input addends A, B and Cin are all set to "1" or two of them are set to "1", the output carry CO is "1", otherwise the output carry CO is always "0"; when two of the input addends A, B and Cin are set to "1" or all are set to "0" or all three addends are set to "0", the output addition result S is set to "0", otherwise the output addition result S is always "1".

[0038] like Figure 6As shown, the interrupt module IE includes multiple one-bit data interrupt modules IE, and the one-bit data interrupt module IE is composed of a MOS tube and a transmission gate. The input one-bit data Ii is connected to the input end of the transmission gate, and the output end of the transmission gate outputs Ii' and is connected to an input end of the 2-input adder. The control end of the transmission gate is respectively connected to the control signals Ctr and CtrB, and the output end of the transmission gate is grounded through the MOS tube, and the control end of the MOS tube is connected to the control signal Ctr. When the control signals Ctr and CtrB are set to "1" and "0", the transmission gate is interrupted, and the output Ii' is grounded and set to zero, thereby achieving the purpose of interrupt transmission.

[0039] like Figure 7 As shown, the shift accumulator RA in this embodiment includes: a first N-1 selector 351, used to select an output from input data c_in and 0 based on a control signal ctrl; a three-input adder 352, used to perform accumulation calculation on the output of the first N-1 selector 351, the result of the left shift operation of the shift accumulator RA output, and the input data P_sum; a second N-1 selector 353, used to select an output from the output of the three-input adder 352 and the input data P_sum based on the control signal ctrl; a register 354, used to temporarily store the output data of the second N-1 selector 353 and then output it; a one-bit shifter 355, which performs a left shift operation on the partial sum output by the register 354 by one bit and inputs it into the three-input adder 352 as the result of the left shift operation of the shift accumulator RA output by one bit.

[0040] The addition tree network 35 is a 6-layer addition tree. The first layer of the addition tree receives the 64×4-bit multiplication result of each SRAM unit output data and the input cache input C-CIM data, sorts the multiplication results in the order of 0-63 by bank, and performs pairwise addition of the data with corresponding sequence numbers, corresponding to Figure 6 The inputs of the two dashed boxes on the left and right are shown. The 2-input adder consists of different numbers of FAs according to the bit width of the input data, such as Figure 6The bit width of the first-layer input addend of the addition tree is 4, so the 2-input adder is composed of 4 FAs. If two adjacent addends do not need to be accumulated, the corresponding interrupt module IE is enabled to interrupt the addition, and the output of the 2-input adder is divided into two parts and output to different layers. For example, the four-bit inputs of result 0 and result 1 correspond to A0-A3 and B0-B3 respectively, A0-A3 input interrupt module IE, B0-B3 directly input FA, the four-bit inputs of result 2 and result 3 correspond to C0-C3 and D0-D3 respectively, D0-D3 input interrupt module IE, C0-C3 directly input the first-layer FA, input Ctr0 / CtrB0 controls the accumulation of result 0 and result 1, input Ctr2 / CtrB2 controls the accumulation of result 2 and result 3, if accumulation is not required, result 0 or result 3 is output to the integration module through IE to obtain Out_L output, and from A' or D' output by the interrupt module IE is set to 0 (that is, data B and data C are added to 0 respectively); the 2-input adder of the second-layer addition tree receives the output results of the 2-input adder of the first layer for pairwise accumulation (if there is an interruption of the addition tree at the first layer, only the output of the first layer closest to it is received. For example, the accumulation path of data 8 and data 9 is split at the first layer, so the 2-input adder corresponding to these two numbers does not accumulate, but directly outputs both, and data 9 is directly sent to the 2-input adder of the second layer), and the accumulation result is output to the N-2 addition selector or 2-input adder of the third layer, and its partial implementation corresponds to Figure 5The second-layer addition tree is composed of 5 FAs, and the interrupt module IE on the left side of FA is the interrupt module IE of the second-layer addition tree; the 2-input adder of the third-layer addition tree receives the output results of the 2-input adder of the second layer for pairwise accumulation, and the N-2 addition selector selects the accumulated data according to the interruption of the first-layer addition tree. If an interruption occurs, for example, due to the interruption of the accumulation path of data 26 and 27 in the first layer, it means that the data before data 26 will not be accumulated with the data after data 27. At this time, the N-2 addition selector only selects the output of the 2-input adder at the first-layer interrupt module and the output of the 2-input adder of the second layer, and directly outputs it to the N-2 addition selector of the sixth layer. Otherwise, the output results of two adjacent 2-input adders are selected for accumulation and output to the 2-input adder of the 4th layer; the 2-input adder of the 4th layer adder tree receives the output results of the 3rd layer 2-input adder for pairwise accumulation, and the N-2 addition selector selects the accumulation path according to the interruption of the 1st layer adder tree. If an interruption of the adder tree occurs, for example, due to the interruption of the accumulation path of data 8 and 9 in the 1st layer, it means that the data before data 8 will not be accumulated with the data after data 9. At this time, the N-2 addition selector only selects the output of the 2-input adder at the 1st layer interrupt module and the output of the 2-input adder of the 3rd layer. Similarly, since the addition tree after the interruption of this layer has been added, the output result of the adder of this layer is directly output to the RA module for shift accumulation operation and output to the outside of the chip. If no interruption occurs, the N-2 selector only selects the outputs of the two nearest 3rd-layer 2-input adders for accumulation, and the output result is transmitted to the 5th layer; in the 5th-layer addition tree, if the previous accumulation path is not split by the interrupt module IE, the N-2 addition selector directly receives the accumulation results of the N-2 addition selector and the 2-input adder of the 4th layer for accumulation, and the output result is output to the 6th layer, otherwise the N-2 addition selector receives the partial accumulation sum of the nearest 2nd layer and 3rd layer for accumulation and outputs it to the shift accumulator RA; in the 6th-layer addition tree, if the previous accumulation path is not split by the interrupt module IE, the output results of the two N-2 addition selectors of the 5th layer are directly received for accumulation, otherwise only the outputs of the two nearest 3rd layer 2-input adders and the N-2 addition selector are received and accumulated. In both cases, the final result will be output to the shift accumulator RA.

[0041] like Figure 8As shown, the multi-mode adaptive controller 32 in this embodiment includes multiple controller modules 321 and multiple C-NOT gates 322. The controller module 321 includes multiple data channels, each of which includes three sub-control modules 323 and a switch 324 connected in sequence. Each controller module 321 receives the same input Xin and its own input Yin, which are input to the internal sub-control module 323 and the switch 324. The multi-mode adaptive controller 32 obtains the RWL / RWLB read signal required by each SRAM unit according to the three input signals Xin, Yin, and Zin, and is processed by the internal sub-control module 323, the C-NOT gate 322, and the switch 324, and outputs it to the SRAM unit, thereby controlling the SRAM unit.

[0042] like Fig. 9 As shown, the sub-control module 323 in this embodiment is composed of a switch 324 connected to a C-NOT gate 322. The switch 324 is composed of a transmission gate T0, a MOS tube T2 (specifically a PMOSFET tube in this embodiment) and an inverter T1. The control ends of the transmission gate T0 and the MOS tube T2 are connected to the input Xin. The transmission gate T0 determines whether to open and transmit the input Yin to the output RWL of the transmission gate T0 according to the input Xin; if the input Xin is set to "1", the transmission gate T0 is opened, the MOS tube T2 is closed, the input Yin is transmitted to the output RWL, and the inverter T1 performs inversion processing according to the input Yin to generate a reverse output RWLB; if the input Xin is set to "0", the transmission gate T0 is closed, the MOS tube T2 is opened, the output RWL is grounded and the output is "0".

[0043] like Fig. 9As shown, the C-NOT gate 322 in this embodiment is composed of a transmission gate T4 and an inverter T3. If the input Xin is set to "1", the inverter T3 works in the form of an inverter, the transmission gate T4 is closed, and the output Yout is the inversion of the input Yin. If the input Xin is set to "0", the inverter T3 works in the form of a transmission gate, and the transmission gate T4 is opened to enhance the Yin input. At this time, the output Yout is equal to the input Yin. The sub-control module 323 outputs Yout, outputs RWL and reverse output RWLB based on the preset truth table through the input Xin and the input Yin. Yout is transmitted from left to right to the next-level sub-control module 323 until it is transmitted to the switch 324. The output RWL and reverse output RWLB signals of each level of sub-control module 323 and switch 324 are integrated to finally output the multi-mode adaptive controller 32. Each output RWL and reverse output RWLB control the data readout of three columns of the SRAM array 33. The C-NOT gate 322 combines the output RWL and reverse output RWLB and the input Z to obtain the data selection signals of the other five columns of the SRAM array 33 for controlling the data readout of the other five columns of the SRAM array 33. In this embodiment, when the sub-control module 323 outputs Yout and outputs RWL and reverse output RWLB based on the preset truth table through the input Xin and the input Yin, the truth table used is shown in Table 1.

[0044] Table 1: Truth table

[0045] The computing-acceleration-in-one macro (C-CIM) supporting multi-mode convolution in this embodiment provides four computing modes: matrix multiplication (equivalent to 1×1 convolution), 3×3 convolution, 5×5 convolution, and 7×7 convolution. Fig.10 : This is a control principle diagram of the SRAM array in this embodiment. In order to facilitate the understanding of the subsequent calculation mode, in this embodiment, the internal modules in the multi-mode adaptive controller 32 are dispersed into the SRAM unit, and each sub-control module 323 is intuitively displayed to control the data reading of the three SRAM unit storage bodies, the input signal Xin[0]-Xin[3] is the external input signal Xin, the input signal Yin[0][0]-Yin[0][2] is the external input signal Yin, and the input signal Zin[0]-Zin[2] is the external input signal Zin.

[0046] Fig.11This is the calculation process of the storage-computation-in-one acceleration macro (C-CIM) that supports multi-mode convolution in the 3×3 convolution mode of this embodiment. Among them, AI in the 3×3 weight is the weight data, 0-17 in the input is the activation data, and the activation data is stored in the SRAM unit according to the serial number. The convolution kernel slides from left to right in sequence. The black solid line short arrow represents that the line is set to "1" to activate. The data in the dashed box area is multiplied by the weight on the left side of the array. The SRAM unit in the dashed box indicates that the internal data is replaced. Xin and Yin are shared in all control modules. When the step size of the convolution is set to 1 and 2 respectively, the input Yin[2:0] will switch in a specific order, and the state will alternate between {111,110,100} and {111,100,110}. At the same time, Xin[3:0] is switched according to the column serial number of the required convolution data, and the control module regulates the data readout of the SRAM unit according to the input Xin and Yin. For the first convolution, the first column of data is selected. Therefore, Xin[3:0] is set to "0001". Subsequently, for the second convolution, the first two columns of data are used, and the result is that Xin[3:0] is set to "0011". At the same time, during the second convolution process, the data "n" is written in parallel to the original data position 0 to alleviate the bandwidth pressure of data replacement. In order to support this data replacement, the present embodiment designs a write selector module WRSE, which selects the first address of the bank according to the input 6-bit address data, and then controls the interval and amount of data to be written through the control signals Winterval[3:0] and Wrows[1:0]. In this case, the present embodiment sets Winterval[3:0] and Wrows[1:0] to 9 and 2, respectively. Therefore, the data is written to the first SRAM column of Bank0 and Bank9.

[0047] Fig.12 This is the calculation process of the storage-computation-in-one acceleration macro (C-CIM) that supports multi-mode convolution in the matrix multiplication mode in this embodiment. The meaning of the signal in the figure is the same as Fig.11 Same. In this case, the storage-computation-in-one acceleration macro (C-CIM) uses the WS data stream. The convolution kernel data is stored in SRAM in sequence. In this mode, only one position of Xin[3:0] is allowed to be "1", and Yin[2:0] is uniformly set to "1". Only one column of data is selected for each calculation, and the calculation of the next column of data and the replacement of the previous column of data are started only when the convolution kernel data of the current column has been used up.

[0048] Fig.13The calculation process of the storage-computation-in-one acceleration macro (C-CIM) supporting multi-mode convolution in the 5×5 convolution mode in this embodiment. When the 5×5 convolution mode is selected, the multiplication-accumulation operation array 3 (MAC array) can only use two 5×5 convolution kernels for sliding operation at the same time. Therefore, this embodiment divides the available SRAM array into two groups, the first group includes blocks 0-block 2 and the first three banks of block 3, and the second group includes the last three banks of block 3 and blocks 4-block 6. At present, these four control modules only share the input Xin. This embodiment divides the SRAM array according to the original 3×3 convolution so that every six banks store five data, and the last bank is set to zero. Every two Yins control the sliding of the convolution kernel. For example, the input Yin[0][0] and Yin[0][1] simultaneously control the reading of the first five data in the SRAM array. The data mapping and replacement strategy follow 3, and the convolution is performed according to the state shown in Table 2 (with 5 convolution operations as a cycle).

[0049] Table 2: State switching table of 5×5 convolution

[0050] When the 7×7 convolution mode is selected, this embodiment uses only the first 7 blocks (block 0 to block 6), each block stores the data of the first 7 banks, and the remaining banks are filled with zeros. In this case, this embodiment introduces an additional input Z[2:0] to control interrupts. The data mapping follows the same pattern as the 3×3 convolution, and the convolution is performed according to the states shown in Table 3 (cycled with 7 convolution operations).

[0051] Table 3: State switching table of 7×7 convolution

[0052] In summary, the storage-computation-in-one acceleration macro (C-CIM) supporting multi-mode convolution in this embodiment aims to leverage the inherent flexibility and scalability of digital CIM to enhance the traditional digital SRAM-CIM architecture by incorporating control logic to regulate data flow. It is critical to design efficient control logic without sacrificing the original CIM storage density. The mapping mechanism between control logic and data is also important because it can reduce repeated movement of data. Secondly, since a single functional adder tree cannot support multiple accumulation forms, it is critical to design a flexible and adaptable adder tree network to address this limitation. The goal of the storage-computation-in-one acceleration macro (C-CIM) supporting multi-mode convolution in this embodiment is to support various convolutions with minimal additional area cost. This embodiment also provides a novel adder tree network to maintain high computational efficiency.

[0053] In addition, this embodiment also provides a chip, including a chip body and a storage-computation-in-one acceleration macro arranged in the chip body, and the storage-computation-in-one acceleration macro is the aforementioned storage-computation-in-one acceleration macro that supports multi-mode convolution.

[0054] In addition, this embodiment also provides a computer device, which includes a microprocessor and a memory connected to each other, and the microprocessor includes the aforementioned storage and computing-in-one acceleration macro that supports multi-mode convolution.

[0055] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A storage-computation-in-one acceleration macro supporting multi-mode convolution, characterized in that: The invention comprises an input buffer (1), a column address decoder (2) and 64 multiplication-accumulation operation arrays (3), wherein the multiplication-accumulation operation arrays (3) comprise: a row address decoder (31), used for selecting a row for writing data into an SRAM array (33) so as to cooperate with the column address decoder (2) to complete the accurate writing of data into the SRAM array (33); a multi-mode adaptive controller (32), used for controlling the multiplication-accumulation operation array (3) according to external input signals Xin, Yin and Zin to perform convolution or matrix multiplication; an SRAM array (33), used for storing weights WS or activation data IS according to a data flow mode of a storage-computation-in-one acceleration macro; a multiplier (34), used for performing a bitwise multiplication operation of output data of the row SRAM array (33) and output data of the input buffer (1); an adder tree network (35), used for grouping and accumulating data outputted by all multipliers (34) according to a control signal and outputting the data to the outside of the storage-computation-in-one acceleration macro; the 64 multiplication-accumulation operation arrays (3) share a common column address decoder (2) and are connected to the same input buffer (1).

2. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 1, characterized in that: The SRAM array (33) is composed of an SRAM cell, wherein the SRAM cell comprises a first transmission gate (331), a second transmission gate (332), a third transmission gate (333), a first inverter (334), a second inverter (335) and a third inverter (336); an input signal WBLB of the SRAM cell is connected to a first input end of the first transmission gate (331); an output end of the first transmission gate (331) is connected to an input end of the third transmission gate (333) after passing through the first inverter (334) and the third inverter (336) in sequence; an output end of the third transmission gate (333) is used to output an output signal RBLB of the SRAM cell; a second input end of the first transmission gate (331) is connected to a write signal WWL, and a control end is connected to the write signal WWLB; a first input end of the second transmission gate (332) is connected to the first inverter (334), the third inverter (335) and the control end through a second inverter (335) arranged in reverse. The middle nodes of the three inverters (336) are connected, the second input end of the second transmission gate (332) is connected to the write signal WWLB, the control end is connected to the write signal WWL, and the output end is connected to the output end of the first transmission gate (331); one input end of the third transmission gate (333) is connected to the output end of the third inverter (336), the other input end is connected to the read signal RWL, and the control end is connected to the read signal RWLB; when the write signal WWL and the write signal WWLB are respectively set to "1" and "0", the first transmission gate (331) is activated, the data input to WBLB is written into the SRAM unit, and the data is maintained inside the SRAM unit through the first inverter (334) and the third inverter (336); when the read signal RWL and the read signal RWLB are respectively set to "1" and "0", the data stored inside the SRAM unit is activated and output from the SRAM unit through the third transmission gate (333) as the output signal RBLB.

3. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 2, characterized in that: The SRAM array (33) is composed of 64 memory banks, each memory bank contains 4×4 SRAM cells, the storage space size of the SRAM array (33) is 1kb, each memory bank is connected to the multiplier (34) via four read data lines, and each operation selects one column of each memory bank for output. The selection process is controlled by a multi-mode adaptive controller (32); in each clock cycle, the multiplier (34) uses a 4×1 multiplexer to select 4-bit data from the input cache (1) and performs bit multiplication with the data stored in the SRAM array (33), and then outputs it to the addition tree network (35). The addition tree network (35) summarizes the 4-bit×1-bit results according to the requirements of the control logic, and then passes these summarized results to the internal shift accumulator. The shift accumulator uses a shift accumulation method to obtain the final required 4-bit×4-bit or 4-bit×8-bit data and outputs it through the output line NOUT of the addition tree network (35).

4. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 1, characterized in that: The adder tree network (35) is composed of a 6-layer adder tree and an integration module, wherein the first layer of the adder tree includes 32 2-input adders and an interrupt module IE arranged at the front end of the 32 2-input adders, the 2-input adders are composed of multiple full adders FA, the first layer of the adder tree is used to receive the multiplication result of each SRAM unit output data and the 64×4-bit input C-CIM data of the input cache (1), the multiplication result is sorted in the order of the memory bank numbers from 0 to 63 and then used as addends, the addends with corresponding numbers are added in pairs by the 2-input adders, and if two adjacent addends do not need to be accumulated, the corresponding interrupt module IE is enabled to interrupt the addition and then directly output, and the One addend is output to the second-layer adder tree, and the other addend is output to the third or fourth-layer adder tree; the second-layer adder tree includes 16 2-input adders, which are used to perform pairwise accumulation of the 32 2-input adders of the first-layer adder tree. If an addition interrupt occurs in a 2-input adder of the 2-input adder of the first-layer adder tree, only the addend of the two addends of the 2-input adder that is closest to the addend is used as the input of the corresponding 2-input adder in the second-layer adder tree; the third-layer adder tree includes 6 2-input adders and 2 N-2 addition selectors. The 2-input adders of the third-layer adder tree are respectively used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree, and the 2 N-2 addition selectors of the third-layer adder tree are respectively used to accumulate the outputs of the 12 2-input adders of the second-layer adder tree. The selectors are used to add the outputs of the 12 2-input adders of the 2nd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 4th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are connected to the shift accumulator RA, the 2 2-input adders of the 4th layer of the addition tree are used to accumulate the outputs of the 4 2-input adders of the 3rd layer of the addition tree, the 2 N-2 addition selectors of the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; the 5th layer of the addition tree includes 2 2-input adders and 2 N-2 addition selectors, the output ends of the 2 2-input adders and the 2 N-2 addition selectors in the 4th layer of the addition tree are used to add the outputs of the 4 2-input adders of the 3rd layer of the addition tree and the addends interrupted by the interrupt module IE of the 1st layer of the addition tree; The addition tree includes 2 N-2 addition selectors, and the output ends of the 2 N-2 addition selectors in the 5th layer of the addition tree are connected to the shift accumulator RA, and the 2 N-2 addition selectors in the 5th layer of the addition tree are respectively used to accumulate the outputs of a 2-input adder and an N-2 addition selector in the 4th layer of the addition tree; the 6th layer of the addition tree includes 1 N-2 addition selector, and the output ends of the N-2 addition selectors in the 6th layer of the addition tree are connected to the shift accumulator RA, and the N-2 addition selectors in the 6th layer of the addition tree are respectively used to accumulate the outputs of the 2 N-2 addition selectors in the 5th layer of the addition tree; the outputs of the shift accumulators RA of each layer of the addition tree are connected to the input end of the integration module.

5. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 4, characterized in that: The input of the full adder FA includes three addends, namely, addend A, addend B and addend Cin, and the output includes a carry CO and an addition result S. Only when the input addends A, B and Cin are all set to "1" or two of them are set to "1", the output carry CO is "1", otherwise the output carry CO is always "0"; when two of the input addends A, B and Cin are set to "1" or all are set to "0" or all three addends are set to "0", the output addition result S is set to "0", otherwise the output addition result S is always "1".

6. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 4, characterized in that: The interrupt module IE includes a plurality of one-bit data interrupt modules IE, each of which is composed of a MOS tube and a transmission gate. i Connected to the input of the transmission gate, the output of the transmission gate outputs I i ' and connected to one input end of the 2-input adder, the control end of the transmission gate is connected to the control signals Ctr and CtrB respectively, the output end of the transmission gate is grounded through the MOS tube, and the control end of the MOS tube is connected to the control signal Ctr, when the control signals Ctr and CtrB are set to "1" and "0", the transmission gate is interrupted, and the output I i 'Ground and set to zero to achieve the purpose of interrupt transmission.

7. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 4, characterized in that: The shift accumulator RA comprises: a first N-1 selector (351), used to select an output from input data c_in and 0 based on a control signal ctrl; a three-input adder (352), used to perform accumulation calculation on the output of the first N-1 selector (351), the result of a one-bit left shift operation on the output of the shift accumulator RA, and the input data P_sum; a second N-1 selector (353), used to select an output from the output of the three-input adder (352) and the input data P_sum based on the control signal ctrl; a register (354), used to temporarily store the output data of the second N-1 selector (353) and then output it; and a one-bit shifter (355), used to perform a one-bit left shift operation on the partial sum output by the register (354), and input the result as the one-bit left shift operation on the output of the shift accumulator RA to the three-input adder (352).

8. The storage-computation-in-one acceleration macro supporting multi-mode convolution according to claim 2, characterized in that: The multi-mode adaptive controller (32) includes a plurality of controller modules (321) and a plurality of C-NOT gates (322). The controller module (321) includes a plurality of data channels. Each data channel includes three sub-control modules (323) and a switch (324) connected in sequence. Each controller module (321) receives the same input Xin and respective input Yin, which are input into the internal sub-control module (323) and the switch (324). The sub-control module (323) is connected by a switch (324). and a C-NOT gate (322), the switch (324) being composed of a transmission gate T0, a MOS transistor T2 and an inverter T1, the control ends of the transmission gate T0 and the MOS transistor T2 being connected to the input Xin, the transmission gate T0 determining whether to open and transmit the input Yin to the output RWL of the transmission gate T0 according to the input Xin; if the input Xin is set to "1", the transmission gate T0 is opened, the MOS transistor T2 is closed, the input Yin is transmitted to the output RWL, and the inverter T1 performs an inversion process according to the input Yin to generate a reverse output RWLB; If the input Xin is set to "0", the transmission gate T0 is closed, the MOS tube T2 is opened, the output RWL is grounded and the output is "0"; the C-NOT gate (322) is composed of a transmission gate T4 and an inverter T3; if the input Xin is set to "1", the inverter T3 works in the form of an inverter, the transmission gate T4 is closed, and the output Yout is the inversion of the input Yin; if the input Xin is set to "0", the inverter T3 works in the form of a transmission gate, and the transmission gate T4 is opened to enhance the Yin input, at which time the output Yout is equal to the input Yin; the sub-control module (323) outputs Yout and outputs RWL and The reverse output RWLB and the output Yout are transmitted from left to right to the next-level sub-control module (323) until they are transmitted to the switch (324). The output RWL and the reverse output RWLB signals of each level of the sub-control module (323) and the switch (324) are integrated through the final output multi-mode adaptive controller (32). Each output RWL and the reverse output RWLB control the data reading of three columns of the SRAM array (33). The C-NOT gate (322) combines the output RWL and the reverse output RWLB and the input Z to obtain the data selection signals of the other five columns of the SRAM array (33) for controlling the data reading of the other five columns of the SRAM array (33).

9. A chip, comprising a chip body and a storage-computation-in-one acceleration macro disposed in the chip body, characterized in that: The storage-computation-in-one acceleration macro is a storage-computation-in-one acceleration macro supporting multi-mode convolution as described in any one of claims 1 to 8.

10. A computer device comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor includes the storage-computation-in-one acceleration macro supporting multi-mode convolution as described in any one of claims 1 to 8.

Images