Computing-in-memory module based on input data sparsity, and chip and electronic device

Abstract

A computing-in-memory module based on input data sparsity, and a chip and an electronic device, which relate to the field of artificial intelligence. The computing-in-memory module comprises: a weight parameter storage array (10), which is used for storing weight parameters; a bit multiplier (12), which is used for receiving a weight read enable signal and input feature data, selecting, when the weight read enable signal is enabled, a weight parameter from the weight parameter storage array (10) on the basis of the weight read enable signal, and multiplying the bit of the input feature data by the selected weight parameter; a storage readout circuit (21), which is used for reading out products and sending same to a logic operation unit (22) when the bit of the input feature data is 1, and not executing a readout operation when the bit of the input feature data is 0; and the logic operation unit (22), which is used for accumulating the products read out by the storage readout circuit (21) when the bit of the input feature data is 1, so as to implement a multiply-accumulate operation between the input feature data and the weight parameter. The present application reduces the power consumption of massive concurrent data reading, and also reduces the data output volume and the requirements for the data transmission bandwidth.

Description

Memory-computing integrated module, chip and electronic device based on input data sparsity

[0001] The present application claims priority to the Chinese patent application No. 202411382656.4, filed on September 30, 2024, and entitled "Memory-computing integrated module, chip and electronic device based on input data sparsity", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, in particular to a memory-computing integrated module, chip and electronic device based on input data sparsity. BACKGROUND

[0003] With the development of the artificial intelligence industry, the computing power and storage requirements of AI applications for chips are continuously increasing, mainly in terms of high computing power, high storage capacity and high data transmission bandwidth. Memory-computing integrated technology helps to solve the "memory wall" and "power wall" problems under the traditional von Neumann architecture. The von Neumann architecture requires data to be constantly read and written between memory units and processing units, which consumes a lot of transmission power.

[0004] The "memory wall" refers to the fact that the performance of the memory cannot keep up with the performance of the CPU (Central Processing Unit), resulting in the CPU spending a lot of time waiting for the memory to complete read and write operations, thereby reducing the overall performance of the system. The "memory wall" has become a major obstacle to data computing applications. In particular, the biggest challenge of deep learning acceleration is the frequent movement of data between computing units and storage units.

[0005] The advantage of memory-computing integration is to break through the memory wall, perform calculations within the storage unit, eliminate unnecessary data movement delays and power consumption, use storage units to improve computing power, and improve computing efficiency by hundreds or thousands of times, reducing costs.

[0006] Sparsity affects the computing efficiency of machine learning, because sparse matrices contain a large number of zero values, resulting in resource waste, unnecessary computation, and low storage space utilization. The most critical issue is resource waste, because in the training process of machine learning, algorithms often need to traverse the entire data set. If the data set is sparse, i.e., contains a large number of 0 or meaningless information, this results in the algorithm needing to process a large amount of data that does not affect the result. This not only makes the processing time longer, but also consumes more computing resources, resulting in low efficiency and high power consumption. SUMMARY

[0007] The application aims to provide a memory-compute integrated module, a chip and an electronic device based on input data sparsity, which can reduce power consumption of a large number of concurrent data reading and reduce the requirement of data output quantity and data transmission bandwidth.

[0008] To achieve the above-mentioned purpose, the application provides the following solutions.

[0009] In a first aspect, the application provides a memory-compute integrated module based on input data sparsity, which comprises a weight parameter storage array, a bit multiplier, a storage readout circuit and a logic operation unit.

[0010] The weight parameter storage array is used for storing weight parameters.

[0011] The bit multiplier is used for receiving a weight read enable signal and input feature data, and when the weight read enable signal is enabled, selecting a weight parameter in the weight parameter storage array according to the weight read enable signal and multiplying a bit of the input feature data with the selected weight parameter.

[0012] The storage readout circuit is used for reading out the multiplied product to the logic operation unit when the bit of the input feature data is 1, and not performing a readout operation when the bit of the input feature data is 0.

[0013] The logic operation unit is used for accumulating the product read out by the storage readout circuit when the bit of the input feature data is 1, so as to realize multiplication and accumulation of the input feature data and the weight parameter.

[0014] In an embodiment, the storage readout circuit comprises readout branches respectively arranged on K rows of bit lines; the readout branches are used for reading out the multiplied product to the logic operation unit when the weight read enable signal is enabled and the bit of the input feature data is 1; each readout branch comprises a switch unit and a readout unit electrically connected with the switch unit; the switch unit is used for outputting a high-level signal to the bit line where the switch unit is located when both the weight read enable signal and the bit of the input feature data are 1; the readout unit is used for reading an electrical signal representing a corresponding product through the bit line where the readout unit is located when the bit line provides a high-level signal; and the switch unit outputs a low-level signal to the bit line where the switch unit is located when at least one of the weight read enable signal and the bit of the input feature data is 0, so as to make the storage readout circuit not perform a readout operation.

[0015] In an embodiment, each of the switch units comprises a PMOS transistor and an NMOS transistor; a gate of the PMOS transistor is connected to a first control signal, a source and a drain of the PMOS transistor are connected between bit lines of a corresponding row, and the drain of the PMOS transistor is further connected to a drain of the NMOS transistor; a gate of the NMOS transistor is connected to the first control signal, and a source of the NMOS transistor is connected to ground; and the first control signal is determined according to the weight read enable signal and the input feature data.

[0016] In an embodiment, the storage readout circuit further comprises a first AND gate; one input of the first AND gate is connected to the input feature data, another input of the first AND gate is connected to the weight read enable signal, and an output of the first AND gate is used to output the first control signal.

[0017] In an embodiment, the weight parameter storage array comprises K rows of N columns of parameter storage units; K rows of parameter storage units are arranged on K rows of bit lines; and the weight read enable signal is used to enable one column of parameter storage units in the N columns of parameter storage units, so that the parameter storage units output electrical signals representing weight parameters to corresponding bit lines.

[0018] In an embodiment, the parameter storage unit comprises a parameter storage region and a switch transistor; the parameter storage region is used to store electrical signals representing the weight parameters; the switch transistor is used to be turned on when a second control signal is 1, so as to output the electrical signals representing the weight parameters to the corresponding bit line, and the switch transistor is turned off when the second control signal is 0; and the second control signal is determined according to the weight read enable signal and the input feature data.

[0019] In an embodiment, the bit multiplier comprises N second AND gates respectively connected to the weight parameter storage array; one input of each of the second AND gates is connected to the input feature data, another input of each of the second AND gates is connected to the weight read enable signal, and an output of each of the second AND gates is used to output the second control signal.

[0020] In an embodiment, the logic operation unit is further used to access the input feature data and identify whether a bit of the input feature data is 0, and keep the accumulated result unchanged when the bit of the input feature data is 0.

[0021] In a second aspect, the present application further provides a chip comprising the storage-computation integrated module based on input data sparsity according to the first aspect.

[0022] In a third aspect, the present application further provides an electronic device comprising the storage-computation integrated module based on input data sparsity according to the first aspect or the chip according to the second aspect.

[0023] According to the embodiments provided in the present application, the following technical effects are disclosed:

[0024] In the storage and readout circuit in the storage and calculation integrated module, the chip and the electronic device based on the input data sparsity in the application, when the bit of the input feature data is 1, the weight parameter is read out to the logic operation unit, and when the bit of the input feature data is 0, the readout operation is not performed, so that the storage and readout circuit does not generate power consumption when the bit of the input feature data is 0, thereby reducing the power consumption of the storage and readout circuit; the output data of the logic operation unit is the accumulation result, and there is no need to transmit the intermediate data in the convolution operation process, the data bit width of the output data is reduced, and if higher accumulation operation is implemented in the logic operation unit, the output data bit width can be further compressed. It can be seen that the application can reduce the power consumption of a large amount of concurrent data reading, and reduce the requirement of data output quantity and data transmission bandwidth. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0026] Fig. 1 is a schematic diagram of the structure of the storage and calculation integrated module according to an embodiment of the application;

[0027] Fig. 2 is a schematic diagram of the structure of the storage and calculation integrated module according to another embodiment of the application;

[0028] Fig. 3 is a schematic diagram of the structure of the storage and calculation integrated module according to another embodiment of the application;

[0029] Fig. 4 is a schematic diagram of the structure of the weight parameter storage array according to an embodiment of the application;

[0030] Fig. 5 is a schematic diagram of the structure of the weight parameter storage array according to another embodiment of the application;

[0031] Fig. 6 is a schematic diagram of the structure of the storage and readout circuit according to an embodiment of the application;

[0032] Fig. 7 is a schematic diagram of the structure of the storage and readout circuit according to another embodiment of the application;

[0033] Fig. 8 is a schematic diagram of the structure of the storage and calculation integrated module according to another embodiment of the application;

[0034] Fig. 9 is a schematic diagram of the structure of the storage and calculation integrated module according to another embodiment of the application. DETAILED DESCRIPTION

[0035] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0036] In view of the problem that more computing resources are consumed for sparse data sets, resulting in low operation efficiency, some algorithms are mainly based on the sparsity of the weight matrix data to optimize and reduce unnecessary calculations. However, it is difficult to optimize the sparsity of the input data, mainly because of the uncertainty of the input data, which needs to be processed in real time inside the chip. The data sparsity varies greatly at different granularities. The smaller the granularity, the higher the sparsity. For example, the probability of an N-bit input vector being all 0 is much lower than the probability of some bits of an N-dimensional input vector being 0. The bit-level sparsity of the data is very high. However, the smaller the granularity of the data sparsity, the more difficult it is to use. Mainly reflected in the need to perform real-time operation judgment on each bit of the input data.

[0037] The purpose of the present application is to provide an input data sparsity-based memory-computing integrated module, chip and electronic device, which can reduce the power consumption of a large number of concurrent data reading and reduce the requirement of data output quantity and data transmission bandwidth.

[0038] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0039] In an exemplary embodiment, as shown in FIG. 1, an input data sparsity-based memory-computing integrated module is provided, which comprises a weight parameter storage array 10, a bit multiplier 12, a storage readout circuit 21 and a logic operation unit 22. One input end of the bit multiplier 12 is connected to a weight reading enable signal, and the other input end is connected to input feature data. The weight parameter acquisition end is connected to the weight parameter storage array 10; the input end of the storage readout circuit 21 is connected to the weight parameter storage array 10, and the output end is connected to the logic operation unit 22; the output end of the logic operation unit 22 is used to output the convolution operation result.

[0040] The weight parameter storage array 10 is used to store weight parameters. As shown in FIG. 2, the input data sparsity-based memory-computing integrated module can further comprise a storage write-in circuit 11; the storage write-in circuit 11 is used to write weight parameters into the weight parameter storage array 10, for example, as shown in FIG. 2, the storage write-in circuit 11 can be connected to write-in data address and weight write-in data signals, and the weight write-in data is written into the weight parameter storage array 10 according to the write-in data address.

[0041] The bit multiplier 12 is configured to receive a weight read enable signal and input feature data (generally required to process the input feature map in a convolutional neural network), and when the weight read enable signal is enabled, select a weight parameter in the weight parameter storage array according to the weight read enable signal, and multiply the bit of the input feature data with the selected weight parameter. The input feature data is used to represent the data output by the previous layer in the convolution operation process, and can include at least one bit, wherein the bit value is 1 or 0. When the bit of the input feature data is 1, it represents that the input feature data is 1, and when the bit of the input feature data is 0, it represents that the input feature data is 0.

[0042] Specifically, when the bit of the input feature data is 1, the product of the bit of the input feature data and the corresponding weight parameter is the corresponding weight parameter, and at this time, the storage readout circuit 21 reads out the corresponding weight parameter from the weight parameter storage array 10 to realize the reading out of the corresponding product; when the bit of the input feature data is 0, the storage readout circuit 21 does not perform the reading out operation at this time, and thus the storage readout circuit 21 can directly perform the reading out operation to the weight parameter storage array 10 to realize the reading out of the corresponding product.

[0043] The storage readout circuit 21 is configured to multiply the bit of the input feature data with the selected weight parameter when the bit of the input feature data is 1, and read out the product to the logic operation unit 22, and when the bit of the input feature data is 0, not to perform the reading out operation, so that the storage readout circuit 21 does not generate power consumption when the bit of the input feature data is 0, thereby reducing the power consumption of the storage readout circuit 21. The storage readout circuit 21 can read out the corresponding weight parameter to the logic operation unit 22 when the bit of the input feature data is 1. The storage medium of the storage readout circuit 21 includes any one of SRAM, DRAM and RRAM, and the storage medium can be a volatile storage medium or a non-volatile storage medium.

[0044] The logic operation unit 22 is configured to accumulate the product read out by the storage readout circuit 21 when the bit of the input feature data is 1, to realize the multiplication and accumulation of the input feature data and the weight parameter. The logic operation unit 22 only needs to output the final convolution operation result, and does not need to transmit the intermediate data in the convolution operation process, thereby reducing the requirement of data output quantity and data transmission bandwidth.

[0045] In an example, the weight parameter storage array 10 includes K rows and N columns of parameter storage units. Each parameter storage unit is configured to store a weight parameter or to store an electrical signal representing a corresponding weight parameter. As shown in FIG. 3, the parameter storage unit in the hth row and jth column stores the weight parameter W(h, j), where 1≤h≤K and 1≤j≤N, K is the number of rows of weight parameters, and N is the number of columns of weight parameters. The K rows of parameter storage units are arranged on K rows of bit lines; the hth row of bit lines is denoted as bitlineh.

[0046] The weight read enable signal is used to enable one column of parameter storage units in the N columns of parameter storage units, i.e., one bit of the weight read enable signal corresponds to one column of parameter storage units. For example, the weight read enable signal corresponding to the jth column of parameter storage units is denoted as wordlinej. When the weight read enable signal wordlinej corresponding to the jth column of parameter storage units is 1, the jth column of parameter storage units outputs an electrical signal representing a weight parameter to the corresponding bit line, so that the storage readout circuit 21 can read the corresponding weight signal through each row of bit lines.

[0047] In an example, as shown in FIG. 4 and FIG. 5, each parameter storage unit includes a parameter storage region 102 and a switch tube 101. The first end of the switch tube 101 is connected to the corresponding bit line, the second end is connected to the corresponding storage region 102, and the control end is configured to access a corresponding control signal. The parameter storage region 102 is configured to store an electrical signal representing a corresponding weight parameter. The switch tube 101 is configured to be turned on when the accessed control signal (e.g., the second control signal) is 1, so as to output the electrical signal representing the corresponding weight parameter to the corresponding bit line. The storage readout circuit 21 can obtain the electrical signal representing the weight parameter on the bit line through the bit line, thereby obtaining the corresponding weight parameter. The switch tube 101 is also configured to be turned off when the accessed control signal is 0, so that the electrical signal stored in the parameter storage region 102 is not transmitted to the corresponding bit line. The control signal accessed by the switch tube 101, such as the second control signal, is determined according to the weight read enable signal and the input feature data.

[0048] The switch tube 101 is implemented by a transistor such as an NMOS tube, and the parameter storage region 102 is implemented by a component such as a capacitor. For example, as shown in FIG. 4 and FIG. 5, the switch tube 101 includes an NMOS tube, and the parameter storage region 102 includes a capacitor. The gate of the switch tube 101 is used as the control end of the switch tube 101 and is configured to access the control signal. The source is grounded through the capacitor, and the drain is connected to the corresponding bit line.

[0049] Specifically, the second control signal to which the switch tube 101 in the parameter storage unit is connected can also be determined according to the data reading mode of the storage readout circuit 21. In some examples, the storage readout circuit 21 reads the parameter storage units on the odd row bit line and the even row bit line in the same way, at which time the control signals to which the parameter storage units on the odd row bit line and the even row bit line are connected are the same, as shown in FIG. 4, the second control signal can perform AND operation on the weight read enable signal and the input feature data to determine, that is, taking 1 when the weight read enable signal and the input feature data are both 1, and taking 0 when at least one of the weight read enable signal and the input feature data is 0. In other examples, the storage readout circuit 21 differentially reads the parameter storage units on the odd row bit line and the even row bit line, at which time the control signals to which the parameter storage units on the odd row bit line and the even row bit line are connected are different (the control signals to which the parameter storage units on the odd row bit line and the even row bit line are connected can be opposite), as shown in FIG. 5, the second control signal to which the parameter storage units on the odd row bit line are connected can perform AND operation on the weight read enable signal and the input feature data, and the control signal to which the parameter storage units on the even row bit line are connected is opposite to the second control signal to which the parameter storage units on the odd row bit line are connected.

[0050] Specifically, the bit multiplier 12 includes N second AND gates 121 respectively connected with the weight parameter storage array, as shown in FIG. 4 and FIG. 5, only the second AND gate 121 in the Nth column is labeled in FIG. 4 and FIG. 5, and each column of parameter storage units corresponds to one second AND gate 121. One input end of the second AND gate 121 is used to access the input feature data, the other input end is used to access the weight read enable signal, and the output end is connected to the control end of the corresponding switch tube 101 in the column, used to output the second control signal; taking the second AND gate 121 in the jth column as an example for specific description, on the jth column, one input end of the second AND gate 121 is used to access the input feature data fmi,j in the jth column, the other input end is used to access the weight read enable signal wordline,j in the jth column, and the output end is connected to the control end of the corresponding switch tube 101 in the jth column.

[0051] In an example, as shown in FIG. 2, the storage readout circuit 21 can also access the input feature data, so as to read out the corresponding weight parameter to the logic operation unit 22 when the input feature data is 1, and not to perform the readout operation when the input feature data is 0.

[0052] Specifically, as shown in FIG. 6, the storage readout circuit 21 includes readout branches respectively arranged on K rows of bit lines; that is, the readout branch h is arranged on the hth row of bit lines, and the readout branch h includes a control terminal for accessing a corresponding control signal determined according to the weight read enable signal wordline,j and the input feature data fmi,j. Each readout branch is used to access the weight read enable signal and the input feature data, and when the weight read enable signal is enabled and the bit of the input feature data is 1, the product of the multiplication is read out to the logic operation unit 22. Each readout branch can read out the weight parameters of a column enabled by the weight read enable signal from the weight parameter storage array 10 when the weight read enable signal is enabled and the bit of the input feature data is 1. Taking the access of the jth column of the weight read enable signal wordline,j and the input feature data fmi,j by each readout branch as an example, the working process of the readout branch is described. If both the weight read enable signal wordline,j and the input feature data fmi,j are 1, each readout branch reads out the corresponding weight parameters from the parameter storage unit on the jth column of the weight parameter storage array; if at least one of the weight read enable signal wordline,j and the input feature data fmi,j is 0, each readout branch does not perform the readout operation at present.

[0053] In an example, as shown in FIG. 7, each readout branch includes a switch unit 211 and a readout unit 212. The switch unit 211 includes a control terminal for accessing a corresponding control signal (such as a first control signal). When the accessed control signal is 1, the switch unit 211 outputs a high-level signal to the corresponding bit line; when the accessed control signal is 0, the switch unit 211 outputs a low-level signal to the corresponding bit line; wherein the control signal is determined according to the weight read enable signal and the input feature data.

[0054] Specifically, the switch unit 211 is used to output a high-level signal to the bit line where it is located when both the weight read enable signal and the bit of the input feature data are 1; and the readout unit 212 is used to obtain an electrical signal representing the corresponding product through the bit line where it is located when the bit line provides a high-level signal, so as to obtain the corresponding product. The switch unit 211 outputs a low-level signal to the bit line where it is located when at least one of the weight read enable signal and the bit of the input feature data is 0, so that the storage readout circuit 21 does not perform the readout operation.

[0055] The working process of the switch unit 211 and the readout unit 212 is described by taking the weight parameter at the jth column parameter storage unit as an example. If the control end of the switch unit 211 is connected to the weight read enable signal wordline,j and the input feature data fmi,j, when the weight read enable signal wordline,j and the input feature data fmi,j are both 1, the switch tube 101 in the jth column parameter storage unit is turned on, and the electrical signal stored in the parameter storage area 102 is transmitted to the corresponding bit line. The switch unit 211 outputs a high-level signal to the bit line. At this time, the readout unit 212 can obtain the electrical signal representing the weight parameter at the jth column parameter storage unit through the bit line when a high-level signal is provided on the bit line, so as to obtain the weight parameter of the jth column. When at least one of the weight read enable signal wordline,j and the input feature data fmi,j is 0, the switch unit 211 outputs a low-level signal to the bit line. At this time, the readout unit 212 does not perform read / write operation. The weight read enable signal wordline,j enables the weight parameter read operation of the jth column parameter storage unit. A plurality of weight read enable signals can enable the weight parameter read operation of a plurality of parameter storage units. For example, the 1st to Nth weight read enable signals wordline,1-N can also be called chip select signals CSL.

[0056] The low-level signal output by the switch unit 211 is directly output to the logic operation unit 22 through the corresponding bit line. When the logic operation unit 22 reads the low-level signal transmitted by the corresponding bit line, it can accumulate 0.

[0057] Each readout unit 212 can also be connected to a write enable signal WE and other related control signals (such as write_data1, write_data2, etc. as shown in FIG. 7). The write enable signal WE can be used to enable the readout operation of the readout unit 212.

[0058] In an example, the storage readout circuit 21 comprises a DRAM storage array, and the structure of the switch unit 211 can refer to that shown in FIG. 7, comprising a PMOS tube and an NMOS tube; wherein the PMOS tube of the hth row is denoted as P,h, and the NMOS tube of the hth row is denoted as N,h, that is, on the hth row, the switch unit 211 comprises the PMOS tube P,h and the NMOS tube N,h. In each row, the gate of the PMOS tube serves as the control end of the corresponding switch unit 211, and is connected to the first control signal CIM CSL, and the source and the drain are connected between the bit lines of the corresponding row, that is, the connection of the switch unit 211 on the bit line is disconnected at one side of the disconnection position, and the other side of the disconnection position is provided with a parameter storage unit, which can also be referred to as a storage row, and a readout branch, the source of the PMOS tube is connected to one end of the disconnection position, and the drain of the PMOS tube is connected to the other end of the disconnection position; the drain of the PMOS tube is also connected to the drain of the NMOS tube; the gate of the NMOS tube is connected to the first control signal CIM CSL, and the source is grounded. When the first control signal CIM CSL connected to the control end of the switch unit 211 is 1, the NMOS tube is turned off, and the PMOS tube is turned on, so that the storage row and the corresponding bit line can be connected, and the readout unit 212 can obtain the electrical signal representing the corresponding weight parameter from the bit line; when the first control signal CIM CSL connected to the control end of the switch unit 211 is 0, the PMOS tube is turned off, and the NMOS tube is turned on, and the drain of the NMOS tube outputs a low-level signal to the logic operation unit 22 through the bit line of the corresponding row. The first control signal CIM CSL is determined according to the weight read enable signal and the input feature data.

[0059] Specifically, the first control signal CIM CSL is the result of the AND operation of the weight read enable signal and the input feature data. Accordingly, as shown in FIG. 7, the storage readout circuit 21 further comprises a first AND gate 213; one input end of the first AND gate 213 is used to connect the input feature data, and the other input end is used to connect the weight read enable signal, and the output end is connected to the control end of each switch unit 211, and is used to output the first control signal CIM CSL; for the weight read enable signal wordline,j and the input feature data fmi,j of the jth column, when the weight read enable signal wordline,j and the input feature data fmi,j are both 1, the first control signal CIM CSL is 1; when at least one of the weight read enable signal wordline,j and the input feature data fmi,j is 0, the first control signal CIM CSL is 0.

[0060] As shown in FIG. 7, the readout unit 212 can comprise read-write MOS tubes and amplifiers and the like components, and the control end such as the gate of the read-write MOS tube can also be connected to a write enable signal WE and other control signals, and each row of bit lines is used to read out the electrical signal representing the weight parameter on the row of bit lines when the readout unit 212 is turned on, so as to read out the corresponding product.

[0061] By comparing the conventional DRAM storage array with the storage readout circuit 21 provided in the present example, it is found that in the conventional DRAM storage array, the readout structure of each row usually includes PMOS tubes in the readout unit and the switch unit. The readout branch provided in the present example only needs to add an NMOS tube in each row, so that when the first control signal CIM_CSL input at the control end of the switch unit 211 is 0, the switch unit 211 can disconnect the storage row and the bit line of the corresponding row, and the NMOS tube in each row of the switch unit can pull down the bit line, so that the storage readout circuit 21 does not perform the readout operation, and directly outputs a low-level signal to the logic operation unit 22.

[0062] Specifically, the storage and calculation integrated module corresponding to the storage readout circuit 21 shown in FIG. 7 can refer to FIG. 8, and a component such as a sense amplifier and / or an equalizer for improving reliability can be arranged on each row of bit lines. Accordingly, the above-mentioned storage and calculation integrated module can further include a sense amplifier and / or an equalizer arranged on each row of bit lines; the input end of the sense amplifier is connected to the output end of the corresponding parameter storage unit through the bit line, and the output end is connected to the readout branch corresponding to the bit line through the equalizer; the sense amplifier is used for amplifying the electrical signal output by the parameter storage unit to the corresponding bit line; and the equalizer is used for equalizing the amplified electrical signal; in this way, the electrical signal output to the corresponding readout branch through the bit line can be more smooth and stable, and the reliability in the above-mentioned electrical signal readout process can be improved.

[0063] In the working process of the storage-computing integrated module: the weight read enable signals corresponding to the 1~N columns can be recorded as wordline,1~N (also referred to as chip select signals CSL), and the input feature data corresponding to the 1~N columns is recorded as fmi,1~N. In the storage-computing integrated module shown in FIG. 8, the first control signal CIM_CSL is obtained through the AND operation of fmi,1~N and wordline,1~N. The first control signal CIM_CSL can control the selection of the bit line through the P,1~P,K (i.e., the PMOS tube in each switching unit). When fmi (i.e., the input feature data) is 0 and wordline (i.e., the weight read enable signal) is 1, the P,1~P,K disconnects the corresponding row of the storage row from the bit line, and the first control signal CIM_CSL simultaneously pulls down the bit line through N,1~N,K. The storage-computing integrated module shown in FIG. 8 avoids the bit line of the traditional DRAM needing to be in the precharge (precharge), aceess (access), sense (sensing), and restore (restore) four operations to complete the read operation. At this time, the bit line needs to be flipped among the 0, VCC / 2, and VCC three states, which is the main source of data read power consumption and system delay. When fmi and wordline are both 1, the P,1~P,K is turned on, the N,1~N,K is disconnected, and the read branch normally works to send the read weight parameters to the logic operation unit 22 to complete the multiply-accumulate operation. It can be seen that the example only adds one transistor in each row of the DRAM storage array without adding control signal lines, so that the storage-computing integration scheme based on the sparsity of the input data can be completed, and the data read power consumption during the convolution operation can be reduced. Especially for the bit-level data sparsity, the power consumption of the convolution operation can be greatly reduced, and the bit width of the output data can be well compressed, greatly reducing the data output amount and the data transmission bandwidth requirement.

[0064] In an example, as shown in FIG. 9, the logic operation unit 22 is also used to access the input feature data and identify whether the bit of the input feature data is 0. When the bit of the input feature data is 0, the accumulation result of the logic operation unit 22 is unchanged. In the embodiment, the logic operation unit 22 directly accesses the input feature data. When the bit of the input feature data is 0, the accumulation result is unchanged, and the accumulation operation does not need to be performed, which can further save power consumption.

[0065] Specifically, one weight read enable signal and one input feature data correspond to one column of parameter storage units. For the weight read enable signal wordline,j and the input feature data fmi,j corresponding to the jth column, when wordline,j = 1 and fmi,j = 1, the logic operation unit 22 performs accumulation or other operations on the signal representing the weight parameter on the bit line, and the convolution operation sum obtained by the logic operation unit 22 is: wherein S represents an accumulated sum, fmi,j represents the input feature data corresponding to the jth column, W(h,j) represents the weight parameter stored in the parameter storage unit of the hth row and the jth column, 1≤h≤K, 1≤j≤N. When fmi,j=0, the product of the corresponding weight parameter and fmi,j is 0 regardless of the value of the weight parameter, the accumulated result of the logic operation unit 22 is unchanged, the weight parameter read operation of the jth column can be shielded by fmi,j=0, the voltage of the bit line does not flip, and the readout power consumption of the storage readout circuit 21 is mainly generated when the bit line flips, so there is no data readout power consumption when fmi,j=0. In addition, the logic operation unit 22 directly outputs the accumulated result, without transmitting the intermediate data in the convolution operation process, and the output data bit width is reduced from the original K-bit data bit width to the accumulated data bit width log2(K). If higher accumulation operations are implemented in the logic operation unit 22, the output data bit width can be further compressed.

[0066] In the above storage-computing integrated module, the storage readout circuit 21 reads the weight parameter to the logic operation unit 22 when the bit of the input feature data is 1, and does not perform the readout operation when the bit of the input feature data is 0, so that the storage readout circuit 21 does not generate power consumption when the bit of the input feature data is 0, thereby reducing the power consumption of the storage readout circuit 21; the output data of the logic operation unit 22 is the accumulated result, without transmitting the intermediate data in the convolution operation process, and the data bit width of the output data is reduced, and if higher accumulation operations are implemented in the logic operation unit 22, the output data bit width can be further compressed. It can be seen that the present application can reduce the power consumption of a large amount of concurrent data reading, and reduce the requirement for data output quantity and data transmission bandwidth.

[0067] In another exemplary embodiment, a chip is also provided, which includes the storage-computing integrated module based on the sparsity of input data in any of the above embodiments, and can include an AI computing chip, a wireless communication chip, a wired communication chip, a digital signal processing chip, a mobile phone main control chip, an AR / VR main control chip, and the like, which needs to perform convolution operation.

[0068] The chip includes the storage-computing integrated module based on the sparsity of input data in any of the above embodiments, and has all the beneficial effects of the storage-computing integrated module based on the sparsity of input data in any of the above embodiments, which will not be repeated here.

[0069] In another exemplary embodiment, an electronic device is also provided, which includes the storage-computing integrated module based on the sparsity of input data in any of the above embodiments or the chip in any of the above embodiments, which has all the beneficial effects of the storage-computing integrated module based on the sparsity of input data in any of the above embodiments, which will not be repeated here.

[0070] While the present application has been illustrated and described in relation to one or more implementations, equivalent variations which do not depart from the scope of the present application are intended to be within the scope of the present application. The present application includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the above described components, the terms (e.g., "means for") used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary implementation of the present application. The above specification, examples and data

[0071] The various embodiments in the specification are described in progressive order with reference to each individual embodiment. However, it is to be understood that the ordering of the embodiments described is not an imposition of a starting point or an ending point to the specification and one of ordinary skill in the art will understand that the various embodiments and layouts described can be combined, interchanged or deleted, unless otherwise specifically noted.

[0072] The principles and implementations of the present application have been described above with the specific examples. The above description of the embodiments is only to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An in-memory computing module based on sparsity of input data, the module comprising: The memory-compute integrated module based on input data sparsity comprises a weight parameter storage array, a bit multiplier, a storage readout circuit and a logic operation unit; The weight parameter storage array is used for storing weight parameters; The bit multiplier is used for receiving a weight read enable signal and input feature data, and when the weight read enable signal is enabled, selecting a weight parameter in the weight parameter storage array according to the weight read enable signal, and multiplying a bit of the input feature data with the selected weight parameter; The storage readout circuit is used for reading out the multiplied product to the logic operation unit when the bit of the input feature data is 1, and not performing a readout operation when the bit of the input feature data is 0; The logic operation unit is used for accumulating the product read out by the storage readout circuit when the bit of the input feature data is 1, so as to realize multiplication and accumulation of the input feature data and the weight parameter.

2. The in-memory computing module based on the sparsity of input data according to claim 1, wherein, The storage readout circuit comprises K rows of bit lines, and a readout branch is arranged on each row of bit lines; The readout branch is used for reading out the multiplied product to the logic operation unit when the weight read enable signal is enabled and the bit of the input feature data is 1; each readout branch comprises a switch unit and a readout unit electrically connected to the switch unit; The switch unit is used for outputting a high-level signal to the bit line where the switch unit is located when the weight read enable signal and the bit of the input feature data are both 1; The readout unit is used for reading an electrical signal representing a corresponding product through the bit line where the readout unit is located when the bit line provides a high-level signal; the switch unit outputs a low-level signal to the bit line where the switch unit is located when at least one of the weight read enable signal and the bit of the input feature data is 0, so that the storage readout circuit does not perform a readout operation.

3. The in-memory computing module based on the sparsity of input data according to claim 2, wherein, Each switch unit comprises a PMOS tube and an NMOS tube; The gate of the PMOS tube is connected to a first control signal, the source and the drain of the PMOS tube are connected between the bit lines of a corresponding row, and the drain of the PMOS tube is also connected to the drain of the NMOS tube; the gate of the NMOS tube is connected to the first control signal, and the source of the NMOS tube is grounded; the first control signal is determined according to the weight read enable signal and the input feature data.

4. The in-memory computing module based on the sparsity of input data according to claim 2, wherein, The storage readout circuit further comprises a first AND gate; One input end of the first AND gate is used for inputting the input feature data, the other input end is used for inputting the weight read enable signal, and the output end is used for outputting the first control signal.

5. The in-memory computing module based on the sparsity of input data according to claim 1, wherein, The weight parameter storage array comprises K rows of N-column parameter storage units; K rows of parameter storage units are arranged on K rows of bit lines; the weight read enable signal is used for enabling one column of parameter storage units in the N-column parameter storage units, so that the parameter storage units output an electrical signal representing a weight parameter to a corresponding bit line.

6. The in-memory computing module based on the sparsity of input data according to claim 5, wherein, The parameter storage unit comprises a parameter storage area and a switch tube; The parameter storage area is used for storing an electrical signal representing the weight parameter; The switch tube is used for being turned on when a second control signal is 1, so as to output the electrical signal representing the weight parameter to the corresponding bit line, and being turned off when the second control signal is 0. The second control signal is determined according to the weight read enable signal and the input feature data.

7. The in-memory computing module based on input data sparsity according to claim 1, wherein, The bit multiplier comprises N second AND gates connected with the weight parameter storage array respectively; One input end of the second AND gate is used for accessing the input feature data, the other input end is used for accessing the weight read enable signal, and the output end is used for outputting the second control signal.

8. The in-memory computing module based on the sparsity of input data according to claim 1, wherein, The logic operation unit is also used for accessing the input feature data and identifying whether the bit of the input feature data is 0, and when the bit of the input feature data is 0, the accumulated result is kept unchanged.

9. A chip, characterized by The memory-compute integrated module based on input data sparsity according to any one of claims 1 to 8.

10. An electronic device, comprising: The memory-compute integrated module based on input data sparsity according to any one of claims 1 to 8 or the chip according to any one of claim 9.

Images