Memory device and operating method thereof

Abstract

The present invention provides a memory device and an operating method thereof. The memory device includes: a plurality of page buffers for temporarily storing input data; a plurality of memory planes coupled to the page buffers, wherein a plurality of weights are stored in the memory planes. Based on a plurality of addresses received from the memory planes, the memory planes perform bitwise multiplications on the weights and the input data of the page buffers in parallel to obtain a plurality of bitwise multiplication results, which are stored back in the page buffers; and at least one accumulation circuit coupled to the page buffers for performing a bitwise accumulation operation on the bitwise multiplication results of the memory planes in parallel or sequentially to obtain a multiplication-accumulation result.

Description

Technical Field

[0001] The present invention relates to a memory device with in-memory computing (IMC) and an operating method thereof. Background Art

[0002] Artificial intelligence (AI) has become a highly effective solution in many fields. The key operation of AI is to perform multiply-accumulation (MAC) operations on large amounts of input data (such as input feature maps) and weight values.

[0003] However, current AI architectures are prone to encountering IO bottlenecks and inefficient MAC operation flows.

[0004] To achieve high accuracy, MAC operations with multi-bit inputs and multi-bit weights can be performed. However, the I / O bottleneck becomes more severe and efficiency is lower.

[0005] In-Memory-Computing (IMC) can be used to accelerate MAC operations because IMC can reduce the complex arithmetic logic unit (ALU) required in the central processing unit architecture and provide high parallelism of MAC operations in memory.

[0006] When performing IMC, if the "operation speed" can be increased, it will be beneficial to IMC performance.

[0007] Public content

[0008] According to an example of the present invention, a memory device is proposed, including: multiple page buffers for temporarily storing input data; multiple storage planes coupled to these page buffers, multiple weights stored in these storage planes according to multiple addresses received from these storage planes, these storage planes performing bit multiplications on these weights and the input data of these page buffers in parallel to obtain multiple bit multiplication results in parallel, and these bit multiplication results are stored back in these page buffers; and at least one accumulation circuit coupled to these page buffers, performing bit accumulation operations on these bit multiplication results of these storage planes in parallel or sequentially to obtain a multiplication-addition operation result.

[0009] According to another example of the present invention, a method for operating a memory device is proposed, including: temporarily storing input data in multiple page buffers; performing bit multiplications in parallel on the stored multiple weights and the input data based on the multiple addresses received from these storage planes to obtain multiple bit multiplication results in parallel, and these bit multiplication results are stored back in these page buffers; and performing bit accumulation operations on these bit multiplication results of these storage planes in parallel or sequentially to obtain a multiplication-addition operation result.

[0010] In order to better understand the above and other aspects of the present invention, the following embodiments are specifically described in detail with reference to the accompanying drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 FIG. 4 is a functional block diagram of a memory device with an in-memory computing (IMC) function according to a first embodiment of the present invention.

[0012] Figure 2 A functional block diagram of a storage plane and an accumulation circuit according to a first embodiment of the present invention is shown.

[0013] Figure 3 A schematic diagram showing data mapping according to a first embodiment of the present invention is shown.

[0014] Figures 4A to 4C Several examples of one-dimensional data mapping according to the first embodiment of the present invention are shown.

[0015] Figure 5 A schematic diagram illustrating an exemplary multiplication operation according to the first embodiment of the present invention.

[0016] Figure 6A and Figure 6B A schematic diagram showing a grouping operation (majority operation) and bitwise counting according to a first embodiment of the present invention is shown.

[0017] Figure 7A and Figure 7B The MAC operation flow of the first embodiment of the present invention is shown.

[0018] Figure 8 FIG. 4 is a functional block diagram of a memory device with in-memory operation function according to a second embodiment of the present invention.

[0019] Figure 9A and Figure 9B The MAC operation flow of the second embodiment of the present invention is shown.

[0020] Figure 10A flow chart showing an operating method of a memory device according to a third embodiment of the present invention is shown.

[0021] Description of Reference Numerals

[0022] 100: Memory device

[0023] MP0~MP3: storage plane

[0024] PB0~PB3: Page buffer

[0025] ADC0-ADC3: conversion unit 120: accumulation detection circuit

[0026] 130: Output latch 140: Accumulation circuit

[0027] MP: memory plane PB: page buffer

[0028] 210: Memory block 220: Multiplication circuit

[0029] 240: Grouping circuit 250: Counting unit

[0030] 211: Memory unit 221: Multiplication unit

[0031] 221A: Input latch

[0032] 221B: Sense Amplifier

[0033] 221C: Output latch

[0034] 221D: Common data latch

[0035] 241: Clustering unit

[0036] 401A, 403A, 401B, 403B, 411A, 413A, 411B, 413B: bits

[0037] 402, 412, 414: weight values ​​505: latch

[0038] 510: Bit line switch

[0039] T11~T46: Timing

[0040] 800: Memory device

[0041] 820-0~820-3: Accumulation detection circuit

[0042] 830: Output latch

[0043] 840-0~840-3: Accumulation circuit

[0044] 1010~1030: Steps DETAILED DESCRIPTION

[0045] The technical terms used in this specification refer to the customary terms in the technical field. If some terms are explained or defined in this specification, the interpretation of these terms shall be based on the explanations or definitions in this specification. Each embodiment of the present disclosure has one or more technical features. Under the premise of possible implementation, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

[0046] First embodiment

[0047] Figure 1 A functional block diagram of a memory device 100 with in-memory computing (IMC) functionality according to a first embodiment of the present invention is shown. Memory device 100 includes multiple memory planes, multiple page buffers, multiple conversion units, an accumulation detection circuit 120, an output latch 130, and an accumulation circuit 140. The conversion unit may be, for example, but not limited to, an analog-to-digital converter (ADC).

[0048] For the convenience of explanation, Figure 1 In the embodiment, the memory device 100 includes four memory planes MP0-MP3, four page buffers PB0-PB3, four conversion units ADC0-ADC3, an accumulation detection circuit 120, an output latch 130, and an accumulation circuit 140. However, it should be understood that the present invention is not limited thereto.

[0049] Page buffers PB0-PB3 are used to temporarily store input data IN and transmit it to memory planes MP0-MP3. The multiple bit multiplication results BM0-BM3 generated by memory planes MP0-MP3 are stored back in page buffers PB0-PB3. Furthermore, controlled by a page buffer select signal PB_SEL, output latch 130 selects the bit multiplication results BM0-BM3 stored in the corresponding page buffers PB0-PB3 for transmission to accumulation circuit 140.

[0050] The memory planes MP0 to MP3 are coupled to the page buffers PB0 to PB3. The memory planes MP0 to MP3 can perform bitwise multiplication (e.g., bitwise AND operation) on the stored weights and the input data IN of the page buffers PB0 to PB3 in parallel to obtain a plurality of bitwise multiplication results BM0 to BM3 in parallel. These bitwise multiplication results BM0 to BM3 are stored back in the page buffers PB0 to PB3. In addition, it is also possible to determine which memory cell strings of the memory planes MP0 to MP3 are to be selected to enable the sensing operation. In addition, when the stored weights are subjected to bitwise multiplication (e.g., bitwise AND operation) with the input data IN of the page buffers PB0 to PB3, the plurality of memory cells of the memory planes MP0 to MP3 further generate memory cell currents IMC0 to IMC3, which are commonly input to the corresponding conversion units ADC0 to ADC3.

[0051] The conversion units ADC0-ADC3 are coupled to the memory planes MP0-MP3. The memory cell currents IMC0-IMC3 of the memory planes MP0-MP3 are input to the conversion units ADC0-ADC3 respectively. The conversion units ADC0-ADC3 convert the memory cell currents IMC0-IMC3 of the memory planes MP0-MP3 to obtain a plurality of conversion results AMACO0-AMACO3.

[0052] Accumulation detection circuit 120 is coupled to conversion units ADC0-ADC3. Accumulation detection circuit 120 compares the conversion results AMACO0-AMACO3 of conversion units ADC0-ADC3 with a threshold value to generate a page buffer select signal PBSEL for output latch 130 and an accumulation enable signal ACC EN for accumulation circuit 140. When the conversion results AMACO0-AMACO3 are greater than the threshold value, output latch 130 selects the bit multiplication results BM0-BM3 stored in the corresponding page buffers PB0-PB3 in response to page buffer select signal PB_SEL and transmits them to accumulation circuit 140.

[0053] When at least one of the conversion results AMACO0 - AMACO3 is higher than the threshold, the accumulation enable signal ACC_EN is in an enabled state; otherwise, the accumulation enable signal ACC_EN is in a disabled state.

[0054] Output latch 130 is coupled to accumulation detection circuit 120 and page buffers PB0-PB3. In response to page buffer select signal PB_SEL, output latch 130 selects the bit-wise multiplication results BM0-BM3 stored in the corresponding page buffers PB0-PB3 for transmission to accumulation circuit 140. For example, when conversion results AMACO0 and AMACO1 from conversion units ADC0 and ADC1 are above the threshold, output latch 130 selects the bit-wise multiplication results BM0-BM1 stored in the corresponding page buffers PB0 and PB1 for transmission to accumulation circuit 140 in response to page buffer select signal PB_SEL.

[0055] The accumulation circuit 140 is coupled to the output latch 130 and the accumulation detection circuit 120. When enabled by the accumulation enable signal ACC_EN, the accumulation circuit 140 performs a bit accumulation operation on the bit multiplication results BM0-BM3 transmitted from the output latch 130 to obtain a product-accumulate operation result OUT, which will be described in detail below.

[0056] Figure 2 A functional block diagram of the memory plane MP and the accumulation circuit 140 according to the first embodiment of the present invention is shown. Figure 2 The memory plane MP can be used to implement Figure 1 The storage planes MP0 to MP3. Figure 2 As shown, the memory plane MP includes a memory block 210 and a multiplication circuit 220. The accumulation circuit 140 includes a grouping circuit 240 and a counting unit 250. The multiplication circuit 220 is analog, while the accumulation circuit 140, the grouping circuit 240 and the counting unit 250 are digital.

[0057] The memory block 210 includes a plurality of memory cells 211. In one embodiment of the present invention, the memory cells 211 are, for example but not limited to, non-volatile memory cells. When performing a MAC operation, the memory cells 211 can be used to store weight values.

[0058] Multiplication circuit 220 is coupled to memory block 210. Multiplication circuit 220 includes a plurality of single-bit multiplication units 221. Each single-bit multiplication unit 221 includes an input latch 221A, a sense amplifier (SA) 221B, an output latch 221C, and a common data latch (CDL) 221D. Input latch 221A is coupled to memory block 210. Sense amplifier 221B is coupled to input latch 221A. Output latch 221C is coupled to sense amplifier 221B. Common data latch 221D is coupled to output latch 221C. The bit multiplication result BM obtained by multiplication circuit 220 during the multiplication operation can be stored back in page buffer PB.

[0059] When the weight value stored in the memory cell 211 is logic 1 and the corresponding input data is also logic 1, the memory cell 211 may generate a cell current. These cell currents of the memory cells 211 may be accumulated to obtain a memory cell current IMC.

[0060] In the first embodiment of the present invention, "digital accumulation" means enabling the accumulator circuit 140 but not enabling the conversion units ADC0-ADC3. "Hybrid accumulation" means enabling both the accumulator circuit 140 and the conversion units ADC0-ADC3. That is, in the first embodiment of the present invention, the conversion units ADC0-ADC3 can be selectively triggered.

[0061] In one embodiment of the present invention, triggering conversion units ADC0-ADC3 quickly filters out useless data, thereby increasing MAC operation speed. Accumulation circuit 140 accumulates unfiltered data to enhance MAC accuracy. Hybrid accumulation, by using low-resolution quantization, reduces the influence of variation, avoids accumulation of useless data, and maintains resolution.

[0062] The grouping circuit 240 is coupled to the output latch 130. The grouping circuit 240 includes a plurality of grouping units 241. These grouping units 241 perform grouping operations on these bit multiplication results BM0 to BM3 to obtain a plurality of grouping results. In one possible embodiment of the present invention, the grouping operation can be implemented, for example, by a majority technique, such as a majority function technique. The grouping circuit 240 is implemented by a majority grouping circuit based on the majority function technique, and the grouping unit 241 is implemented by a distributed majority grouping unit, but the present invention is not limited thereto. The grouping technology can be implemented by other similar technologies.

[0063] Counting unit 250 is coupled to grouping circuit 240. Counting unit 250 is configured to count or accumulate bits of the grouping result (e.g., majority result) from grouping circuit 240 to generate a multiplication-accumulation operation result OUT. In one embodiment of the present invention, counting unit 250 can be implemented using a conventional counting circuit, such as, but not limited to, a ripple counter. In this description, counting and accumulating are essentially synonymous, and counter and accumulator are essentially synonymous.

[0064] In the first embodiment of the present invention, by Figure 1 and Figure 2 The architecture can independently quantize the current of each memory plane. Moreover, the accumulation circuit 140 can be shared by these memory planes, thereby reducing the circuit area.

[0065] Please refer to Figure 3 , which shows a data mapping diagram according to an embodiment of the present invention. Figure 3 As shown, each input data (or each weight value) is taken as an example of 8 bits with N dimensions (N is a positive integer) (but it should be understood that the present invention is not limited thereto).

[0066] The following description is made by taking the data mapping of input data as an example, but it should be understood that the present invention is not limited thereto. The following description is also applicable to the data mapping of weight values.

[0067] When the input data is represented as 8 bits in binary, the input data (or weight value) is divided into a most significant bit (MSB) vector and a least significant bit (LSB) vector. The MSB vector of the 8-bit input data (or weight value) includes 4 bits B7-B4, while the LSB vector includes 4 bits B3-B0.

[0068] Each bit of the MSB vector and LSB vector of the input data is represented using unary coding (i.e., numeric format). For example, bit B7 of the MSB vector of the input data can be represented as B70-B77, bit B6 of the MSB vector of the input data can be represented as B60-B63, bit B5 of the MSB vector of the input data can be represented as B50-B51, and bit B4 of the MSB vector of the input data can also be represented as B4.

[0069] The MSB vector of the input data, expressed in unary code (numeric form), and the LSB vector of the input data are repeated multiple times to form an unfolding dot product (unFDP) format. For example, the MSB vector of the input data is repeated (24-1) times, and similarly, the LSB vector of the input data is repeated (24-1) times. This allows the input data to be represented in unfolded dot product form.

[0070] Perform multiplication operation on the input data (in expanded product form) and the weight value to obtain the multiplication result.

[0071] For ease of understanding, an example is used below for illustration, but it should be understood that it is not intended to limit the present invention.

[0072] Please refer to Figure 4A , which shows an example of one-dimensional data mapping according to the first embodiment of the present invention. Figure 4A As shown, input data = (IN l , IN2) = (2, 1), and weight values ​​= (We1, We2) = (1, 2). Representing the MSB and LSB of the input data in binary form, IN1 = 10 and IN2 = 01. Similarly, representing the MSB and LSB of the weight values ​​in binary form, We1 = 01 and We2 = 10.

[0073] The MSB and LSB of the input data, as well as the MSB and LSB of the weight value, are encoded as unary codes (numeric form). That is, the MSB of the input data is encoded as 110, and the LSB of the input data is encoded as 001. Similarly, the MSB of the weight value is encoded as 001, and the LSB of the weight value is encoded as 110.

[0074] Then, the bits of the MSB (110) of the input data encoded as a unary code and the bits of the LSB (001) of the input data encoded as a unary code are repeated multiple times to form an unfolding dot product (unFDP). For example, the bits of the MSB (110) of the input data are repeated three times, so the unfolding dot product of the MSB of the input data is 111111000. The bits of the LSB (001) of the input data are repeated three times, so the unfolding dot product of the LSB of the input data is 000000111.

[0075] Perform a MAC operation on the input data (in expanded product form) and the weight value to obtain the MAC operation result. The MAC operation result is: 1*0=0, 1*0=0, 1*1=1, 1*0=0, 1*0=0, 1*1=1, 0*0=0, 0*0=0, 0*1=0, 0*1=0, 0*1=0, 0*1=0, 0*0=0, 0*1=0, 0*1=0, 0*0=0, 1*1=1, 1*1=1, 1*0=0. Adding these values ​​yields: 0+0+1+0+0+1+0+0+0+0+0+0+0+0+0+1+1+0=4.

[0076] From the above, it can be seen that if the input data is i bits and the weight value is j bits (i and j are both positive integers), the number of memory cells used is: (2i-1)*(2j-1).

[0077] Please refer to Figure 4B , which shows another possible example of data mapping according to the first embodiment of the present invention. Figure 4BIn the example, the input data is (IN1) = (2), and the weight value is (We1) = (1). The input data and weight value are 4 bits.

[0078] When the input data is expressed in binary format, IN1 = 0010. Similarly, when the weight value is expressed in binary format, We1 = 0001.

[0079] The input data and weight values ​​are encoded into unary codes (numeric form). For example, the highest bit "0" of the input data is encoded as "00000000", and the lowest bit "0" of the input data is encoded as "0", and so on. Similarly, the highest bit "0" of the weight value is encoded as "00000000", and the lowest bit "1" of the weight value is encoded as "1".

[0080] Each bit of the input data encoded into the unary code is replicated multiple times to form a spread product. For example, the most significant bit 401A of the input data encoded into the unary code is replicated 15 times to form bit 403A; and the least significant bit 401B of the input data encoded into the unary code is replicated 15 times to form bit 403B.

[0081] The weight value 402 encoded into the unary code is also replicated 15 times to be expressed in the spread product form.

[0082] The MAC operation result is generated by multiplying the input data represented in the form of a spread product with the weight value represented in the form of a spread product. Specifically, bit 403A of the input data is multiplied by the weight value 402; bit 403B of the input data is multiplied by the weight value 402, and so on. The sum of the multiplication values ​​produces the MAC operation result ("2").

[0083] Please refer to Figure 4C , which shows another possible example of data mapping according to the first embodiment of the present invention. Figure 4C In the example, the input data is (IN1) = (1), and the weight value is (We1) = (5). The input data and weight value are 4 bits.

[0084] When the input data is expressed in binary format, IN1 = 0001. Similarly, when the weight value is expressed in binary format, We1 = 0101.

[0085] Encode the input data and weight values ​​into unary code (numeric form).

[0086] Each bit of the input data encoded into a unary code is replicated multiple times to form a spread product. Figure 4CIn the example, when each bit of the input data and each bit of the weight value are copied, a "0" bit is added. For example, the most significant bit 411A of the input data encoded in the unary code is copied 15 times and a "0" bit is added to become bit 413A. Also, the least significant bit 411B of the input data encoded in the unary code is copied 15 times and a "0" bit is added to become bit 413B. This represents the input data in a spread product form.

[0087] Similarly, the weight value 412 encoded in the unary code is also replicated 15 times, and an additional bit "0" is added to each weight value 414. This allows the weight value to be expressed in a spread product form.

[0088] The MAC operation result is generated by multiplying the input data represented in the form of a spread product by the weight value represented in the form of a spread product. Specifically, bit 413A of the input data is multiplied by weight value 414; bit 413B of the input data is multiplied by weight value 414, and so on. The sum of the multiplication values ​​produces the MAC operation result ("5").

[0089] In conventional technology, a MAC operation is performed on 8-bit input data and an 8-bit weight value. If a direct MAC operation algorithm is used, the number of memory cells used is 255*255*512=33,292,822.

[0090] In contrast, as described above, in the embodiment of the present invention, when performing a MAC operation on 8-bit input data and an 8-bit weight value, the number of memory cells used is 15*15*512*2=115,200*2=230,400. Therefore, the number of memory cells used in the MAC operation in the embodiment of the present invention is approximately 0.7% of that in conventional technology.

[0091] In the embodiment of the present invention, the use of unFDP-style data mapping can reduce the number of memory cells used in operations, thereby reducing computational costs and error correction code (ECC) costs. In addition, it can also tolerate fail-bit effects.

[0092] Please refer to Figure 1 and Figure 2 In the embodiment of the present invention, when performing a multiplication operation, the weight values ​​(transduced values) are stored in the memory cells 211 of the memory block 210, while the input data (voltage) is read from the page buffer and transmitted to the common data latch 221D. The common data latch 221D transmits the input data to the input latch 221A.

[0093] To better understand the multiplication operation of the first embodiment of the present invention, please refer to Figure 5 , which shows an exemplary schematic diagram of the multiplication operation of the first embodiment of the present invention. Figure 5 Applied to memory devices to support the selected bit-line read function. Figure 5 , the input latch 221A includes a latch 505 and a bit line switch 510 .

[0094] like Figure 5 As shown, the weight value is expressed in unary code (numerical form) (such as Figure 3 ). Therefore, the highest bit of the weight value is stored in 8 memory cells 211, the second highest bit of the weight value is stored in 4 memory cells 211, the third highest bit of the weight value is stored in 2 memory cells 211, and the lowest bit of the weight value is stored in 1 memory cell 211.

[0095] Similarly, the input data is represented as a unary code (numeric form) (such as Figure 3 ), therefore, the most significant bit of the input data is stored in eight common data latches 221D, the second most significant bit of the input data is stored in four common data latches 221D, the third most significant bit of the input data is stored in two common data latches 221D, and the least significant bit of the input data is stored in one common data latch 221D. The input data is sent from the common data latches 221D to latch 505.

[0096] At Figure 5 In the embodiment, the plurality of bit line switches 510 are coupled between the memory cell 211 and the sense amplifier 221B. The bit line switches 510 are controlled by the latch 505. For example, when the latch 505 outputs a bit 1, the bit line switches 510 are turned on, and when the latch 505 outputs a bit 0, the bit line switches 510 are turned off.

[0097] Furthermore, when the weight value in the memory cell 211 is bit 1 and the bit line switch 510 is on (the input data is bit 1), the sense amplifier 221B senses the memory cell current to produce a multiplication result of "1." When the weight value in the memory cell 211 is bit 0 and the bit line switch 510 is on (the input data is bit 1), the sense amplifier 221B does not sense the memory cell current. When the weight value in the memory cell 211 is bit 1 and the bit line switch 510 is off (the input data is bit 0), the sense amplifier 221B does not sense the memory cell current to produce a multiplication result of "0." When the weight value in the memory cell 211 is bit 0 and the bit line switch 510 is off (the input data is bit 0), the sense amplifier 221B does not sense the memory cell current.

[0098] That is, through Figure 5In this layout, when the input data is bit 1 and the weight value is bit 1, sense amplifier 221B senses the memory cell current, generating a multiplication result of "1." In other cases, sense amplifier 221B does not sense the memory cell current, generating a multiplication result of "0." The multiplication result from sense amplifier 221B is input to output latch 221C. The output results of output latch 221C (used to form bit multiplication result BM) are then stored back into page buffer PB via common data latch 221D.

[0099] The memory cell currents IMC generated by the memory cells 211 are commonly input to one of the corresponding conversion units ADC0 - ADC3 .

[0100] The relationship between input data, weight value, multiplication result and simulated memory cell current IMC is shown in the following table:

[0101] Input Data Weight value Multiplication result of numbers IMC 0 0(HVT) 0 0 0 +1(LVT) 0 0 1 0(HVT) 0 IHVT 1 +1(LVT) 1 ILVT

[0102] In the table above, HVT and LVT represent high-threshold and low-threshold memory cells, respectively. IHVT and ILVT represent the simulated memory cell currents (IMC) generated by the high-threshold and low-threshold memory cells (weights 0 (HTV) and +1 (LTV), respectively) when the input data is logic 1.

[0103] In an embodiment of the present invention, a selected bit line read (SBL-read) instruction can be repeatedly used when performing a multiplication operation, thereby reducing the influence of variation caused by single-bit representation.

[0104] Please refer to Figure 6A , which shows a schematic diagram of grouping operation (majority operation) and bitwise counting according to the first embodiment of the present invention. Figure 6AAs shown, reference symbol GM1 represents the first multiplication result obtained by bitwise multiplication of the first MSB vector of the input data and the weight value; reference symbol GM2 represents the second multiplication result obtained by bitwise multiplication of the second MSB vector of the input data and the weight value; reference symbol GM3 represents the third multiplication result obtained by bitwise multiplication of the third MSB vector of the input data and the weight value; reference symbol GL represents the fourth multiplication result obtained by bitwise multiplication of the LSB of the input data and the weight value. After the grouping operation (majority operation), the grouping result of the first multiplication result GM1 is the first grouping result CB1 (whose cumulative weight is 2 2 ); The result of grouping the second multiplication result GM2 is the second grouping result CB2 (whose cumulative weight is 2 2 ); The result of grouping the third multiplication result GM3 is the third grouping result CB3 (whose cumulative weight is 2 2 ); and the result of grouping the fourth multiplication result GL is the fourth grouping result CB4 (whose cumulative weight is 2 0 ).

[0105] Figure 6B show Figure 4C Please refer to the cumulative example of Figure 4C and Figure 6B .like Figure 6B As shown, input data ( Figure 4C ) is multiplied by the weight value 414. Figure 4C ) is multiplied by the weight value 414, the first four bits ("0000") of the multiplication result generated are grouped as the first multiplication result "GM1". Similarly, from the input data ( Figure 4C The fifth to eighth bits ("0000") of the multiplication result generated by multiplying the bit 413B of the input data ( Figure 4C The ninth to twelfth bits ("1111") of the multiplication result generated by multiplying the bit 413B of the input data ( Figure 4C The thirteenth to sixteenth bits ("0010") of the multiplication result generated by multiplying the bit 413B of ) by the weight value 414 are directly counted.

[0106] After the grouping operation (majority decision operation), the first grouping result CB1 is "0" (its cumulative weight is 2 2 ); The second clustering result CB2 is "0" (its cumulative weight is 2 2 ); The third clustering result CB3 is "1" (its cumulative weight is 22 ). When counting, these grouping results CB1 to CB4 are multiplied by individual cumulative weights and accumulated to generate product-accumulation operation results OUT. For example, Figure 6B As shown, the product accumulation result OUT is CB1*2 2 +CB2*2 2 +CB3*2 2 +CB4*2 0 =0*2 2 +0*2 2 +1*2 2 +1*2 0 =0000 0000 0000 0000 0000 0000 0000 0101=5.

[0107] In one embodiment of the present invention, the grouping principle (majority rule) may be as follows:

[0108] Group bits Grouping results (majority decision) 1111 (Situation A) 1 1110 (Situation B) 1 1100 (Situation C) 1 or 0 1000 (Situation D) 0 0000 (Situation E) 0

[0109] In the table above, for case A, since the groups are all correct ("1111" has no error bits), the majority result is 1. For case E, since the groups are all correct ("0000" has no error bits), the majority result is 0.

[0110] In case B, since one bit in the group is incorrect (the "0" in "1110" is incorrect), "1110" can be determined to be "1" by majority vote. In case D, since one bit in the group is incorrect (the "1" in "0001" is incorrect), "0001" can be determined to be "0" by majority vote.

[0111] In case C, there are 2 bits in the group that are wrong (“00” in “1100” is wrong, or “11” in “1100” is wrong). By majority vote, “1100” can be determined as “1” or “0”.

[0112] Therefore, in the embodiment of the present invention, the error bits can be reduced through the grouping (majority decision) function.

[0113] The grouping result of the grouping circuit 240 is input to the counting unit 250 for bit counting.

[0114] When counting, the count result of the multiplication result of the MSB vector and the count result of the multiplication result of the LSB vector are accumulated. Figure 6A In the case of , two accumulators are used. The first accumulator is assigned a higher accumulation weight value (e.g. 2 2The first type of accumulator accumulates: (1) "the result of grouping (majority vote) of the multiplication result GM1: 1 bit" plus "the result of grouping (majority vote) of the multiplication result GM2: 1 bit" plus "the result of grouping (majority vote) of the multiplication result GM3: 1 bit". The counting result obtained by the first type of accumulator is then multiplied by a higher accumulation weight value (for example, 2 2 The second type of accumulator is assigned a lower cumulative weight value (e.g. 2 0 ). The second type of accumulator directly counts the multiplication result GL (multi-bit). The multiplication accumulation result OUT is obtained by adding the two accumulated results weighted by the accumulation weight. For example, the grouping result obtained by grouping the multiplication result GM1 is 1 (1 bit), the grouping result obtained by grouping the multiplication result GM2 is 0 (1 bit), and the grouping result obtained by grouping the multiplication result GM3 is 1 (1 bit). The counting result (1+0+1) obtained by the first accumulator is multiplied by 2 2 , which is equal to 2*2 2 = 8. The multiplication result GL is 4 (3 bits), which can be counted directly. The two cumulative results weighted by the cumulative weight are added together to obtain the multiplication and accumulation result OUT, which is 8 + 4 = 12.

[0115] As can be seen from the above, in the embodiments of the present invention, when performing counting or accumulation, since the input data has been expanded into the unFDP format, the data stored in the common data latch can be grouped (i.e., divided into an MSB vector and an LSB vector). This grouping mechanism (majority voting mechanism) can reduce the number of error bits in the MSB vector / LSB vector.

[0116] Furthermore, even with a conventional accumulator (counter), the present invention can still reduce counting / accumulation time. This is because the present invention utilizes digital counting instructions (error bit counting) and assigns different accumulation weights to the accumulation results of different vectors (MSB vector and LSB vector). For example, the accumulation operation time can be reduced by approximately 40%.

[0117] Figure 7A and Figure 7B The MAC operation flow of the first embodiment of the present invention is shown. Figure 7A The MAC operation process in which the display conversion units ADC0 to ADC3 are triggered can also be called a hybrid MAC operation. Figure 7BThe MAC operation process in which the conversion units ADC0-ADC3 are not triggered can also be referred to as a digital MAC operation. When the conversion units ADC0-ADC3 are not triggered, the output latch 130 transmits the bit multiplication results BM0-BM3 to the accumulation circuit 140 for bit accumulation (i.e., the output latch 130 is no longer controlled by the page buffer select signal PB_SEL, and the accumulation circuit 140 is no longer controlled by the accumulation enable signal ACC_EN).

[0118] by Figure 7A As can be seen, in timing T11, input broadcasting (receiving input data) is performed. In timing T12, the individual addresses of these storage planes are received. These storage planes can perform operations for different addresses. However, in case multiple storage planes use the same address, an embodiment of the present invention needs to avoid this situation and allow these storage planes to use different addresses. In timing T13, bit multiplication is performed based on these addresses of the received storage planes. In timing T14, the bit multiplication result is output (for example, from the storage plane to the accumulation circuit). In timing T15, bit accumulation is performed. When performing bit accumulation, the bit accumulation of different storage planes is performed sequentially. For example, Figure 7A As can be seen, since the conversion results AMACO0 and AMACO2 are higher than the threshold, the bit accumulation of the memory planes MP0 and MP2 is performed in sequence, while the bit accumulation of the memory planes MP1 and MP3 is not performed. Figure 7A ,by Figure 7A In the case of

[0065] , bit accumulation can be performed on memory plane MP2 first, followed by bit accumulation on memory plane MP0, which is also within the scope of the present invention. At time sequence T16, the product-add operation result is output, along with the next address used by these memory planes.

[0119] like Figure 7B As shown, the timings T21 to T24 and T26 are similar or identical to the timings T11 to T14 and T16. In the timing T25, since the conversion units ADC0 to ADC3 are not triggered, the bit accumulation of all the storage planes is performed in sequence when performing bit accumulation. Figure 7B From the above, the bit accumulation of the memory planes MP0, MP1, MP2 and MP3 is performed in sequence. Figure 7B , there may be other changes, which are also within the scope of the spirit of the present invention.

[0120] Depend on Figure 7A and Figure 7BIt can be seen that in the first embodiment of the present invention, bit multiplication operations are performed in parallel, and bit accumulation operations are performed sequentially. That is, in the first embodiment of the present invention, the operations are performed in a centralized manner, which can reduce circuit area and power consumption.

[0121] Depend on Figure 7A As shown in Figure 7B, the MAC operation of the present embodiment can be divided into two sub-operation types. The first sub-operation type is a multiplication operation, which multiplies the input data by a weight value and is performed based on the selected bit line read instruction. The second sub-operation type is accumulation (data counting), specifically, fail bit counting. In other possible embodiments of the present invention, more counting units can be added to accelerate the counting / accumulation operations.

[0122] Second embodiment

[0123] Figure 8 A functional block diagram of a memory device 800 with in-memory arithmetic capabilities according to a second embodiment of the present invention is shown. Memory device 800 includes multiple memory planes, multiple page buffers, multiple conversion units, multiple accumulation detection circuits, an output latch 830, and multiple accumulation circuits. The conversion unit may be, for example, but not limited to, an analog-to-digital converter (ADC).

[0124] For the convenience of explanation, Figure 8 In the embodiment, the memory device 800 includes four memory planes MP0-MP3, four page buffers PB0-PB3, four conversion units ADC0-ADC3, four accumulation detection circuits 820-0-820-3, an output latch 830, and four accumulation circuits 840-0-840-3. However, the present invention is not limited thereto.

[0125] Page buffers PB0-PB3 can be used to temporarily store input data ON and transmit the input data IN to memory planes MP0-MP3. In addition, the bit multiplication results BM0-BM3 obtained by memory planes MP0-MP3 are stored back in the page buffers PB0-PB3 for transmission to the accumulation circuits 840-0-840-3.

[0126] The memory planes MP0 to MP3 are coupled to the page buffers PB0 to PB3. The memory planes MP0 to MP3 can perform bitwise multiplication on the stored weights and the input data IN of the page buffers PB0 to PB3 in parallel to obtain a plurality of bitwise multiplication results BM0 to BM3 in parallel. These bitwise multiplication results BM0 to BM3 are stored back in the page buffers PB0 to PB3. In addition, it can also be determined which memory cell series of the memory planes MP0 to MP3 are to be selected to enable the sensing operation. In addition, when the stored weights are bitwise multiplied (for example, a bitwise AND operation) with the input data IN of the page buffers PB0 to PB3, the plurality of memory cells of the memory planes MP0 to MP3 further generate memory cell currents IMC0 to IMC3, which are input together to the corresponding conversion units ADC0 to ADC3.

[0127] The conversion units ADC0-ADC3 are coupled to the memory planes MP0-MP3. The memory cell currents IMC0-IMC3 of the memory planes MP0-MP3 are input to the conversion units ADC0-ADC3 respectively. The conversion units ADC0-ADC3 convert the memory cell currents IMC0-IMC3 of the memory planes MP0-MP3 to obtain a plurality of conversion results AMACO0-AMACO3.

[0128] The accumulation detection circuits 820-0 through 820-3 are coupled to the conversion units ADC0 through ADC3. The accumulation detection circuits 820-0 through 820-3 compare the conversion results AMACO0 through AMACO3 of the conversion units ADC0 through ADC3 with a threshold value to generate accumulation enable signals ACC_EN0 through ACC_EN3 for the accumulation circuits 840-0 through 840-3. When the conversion results AMACO0 through AMACO3 exceed the threshold value, the accumulation enable signals ACC_EN0 through ACC_EN3 are enabled; otherwise, the accumulation enable signals ACC_EN0 through ACC_EN3 are disabled.

[0129] The accumulation circuits 840-0 to 840-3 are coupled to the accumulation detection circuits 820-0 to 820-3. When enabled by the accumulation enable signals ACC_EN0 to ACC_EN3, the accumulation circuits 840-0 to 840-3 perform bit accumulation operations on the bit multiplication results BM0 to BM3 transmitted from the memory planes MP0 to MP3 to obtain a plurality of digital accumulation results DMACO0 to DMACO3.

[0130] The output latch 830 is coupled to the accumulation circuits 840-0 to 840-3 and outputs the digital accumulation results DMACO0 to DMACO3 of the accumulation circuits 840-0 to 840-3 as a multiplication-accumulation operation result OUT.

[0131] As for the circuit structure and operation of these accumulation circuits 840-0 to 840-3, please refer to Figure 2 The accumulation circuit 140 is not repeated here. Figure 2 The memory plane MP can be used to implement Figure 8 Storage planes MP0 to MP3.

[0132] In the second embodiment of the present invention, the current of each memory plane can be quantified independently.

[0133] The data mapping of the second embodiment may refer to the data mapping of the first embodiment and will not be repeated here.

[0134] Figure 9A and Figure 9B The MAC operation flow of the second embodiment of the present invention is shown. Figure 9A The MAC operation process in which the display conversion units ADC0 to ADC3 are triggered can also be called a hybrid MAC operation. Figure 9B The MAC operation process in which the conversion units ADC0-ADC3 are not triggered can also be referred to as a digital MAC operation. When the conversion units ADC0-ADC3 are not triggered, the accumulation circuits 840-0-840-3 perform bit accumulation on the bit multiplication results BM0-BM3 (i.e., the accumulation circuits 840-0-840-3 are no longer controlled by the accumulation enable signals ACC_EN0-ACC_EN3).

[0135] by Figure 9A From the above, the timings T31 to T34 and T36 are similar or the same as the timings T11 to T14 and T16. In the timing T35, bit accumulation is performed. When performing bit accumulation, the bit accumulation of different storage planes is performed simultaneously. For example, Figure 9A From this perspective, since the conversion results AMACO0 and AMACO2 are higher than the threshold, the bit accumulation of the memory planes MP0 and MP2 is performed simultaneously, while the bit accumulation of the memory planes MP1 and MP3 is not performed.

[0136] like Figure 9B As shown, timings T41-T44 and T46 are similar or identical to timings T11-T14 and T16. In timing T45, since the conversion units ADC0-ADC3 are not triggered, the bit accumulation of all memory planes is performed simultaneously.

[0137] Depend on Figure 9A and Figure 9B It can be seen that in the second embodiment of the present invention, the bit multiplication operation is performed in parallel, and the bit accumulation operation is performed in parallel. In other words, in the second embodiment of the present invention, the control operation is performed in a distributed manner, which can further speed up the MAC operation speed.

[0138] Figure 3 、 Figures 4A to 4C 、 Figure 5 、 Figures 6A to 6B It can also be applied to the second embodiment.

[0139] Third embodiment

[0140] Figure 10 A flow chart showing an operating method of a memory device according to a third embodiment of the present invention is provided. The operating method of the memory device according to the third embodiment of the present invention includes: temporarily storing input data in a plurality of page buffers (1010); performing bitwise multiplications on the stored weights and the input data in parallel based on the obtained addresses of the plurality of storage planes to obtain a plurality of bitwise multiplication results in parallel, and storing the bitwise multiplication results back in the page buffers (1020); and performing bitwise accumulation operations on the bitwise multiplication results of the storage planes in parallel or sequentially to obtain a multiplication-accumulation result, outputting the obtained multiplication-accumulation result and the next address used by the storage planes (1030). The details of these steps 1010-1030 are as described above and will not be repeated here.

[0141] As is known, the read voltage will affect the ADC output value and the reading of bit 1. Therefore, in the first to third embodiments of the present invention, the read voltage can be periodically calibrated according to operating conditions (such as, but not limited to, programming cycle, temperature, or read disturb) to maintain high accuracy and reliability.

[0142] The first to third embodiments of the present invention can be applied to NAND flash memory, or memory devices that are sensitive to retention and thermal changes, such as but not limited to, NOR flash memory, phase change (PCM) flash memory, magnetic random access memory (Magnetic RAM), or resistive RAM.

[0143] The first to third embodiments of the present invention can be applied to 3D memory and 2D memory, such as but not limited to, 2D / 3D NAND flash memory, 2D / 3D NOR flash memory, 2D / 3D phase change (PCM) flash memory, 2D / 3D magnetic random access memory (magnetic RAM), or 2D / 3D resistive RAM.

[0144] Although the input data and / or weight values ​​are divided into an MSB vector and an LSB vector (two vectors) in the first to third embodiments described above, the present invention is not limited thereto. In other possible embodiments of the present invention, the input data and / or weight values ​​may be divided into more vectors, which is also within the scope of the present invention.

[0145] The first to third embodiments of the present invention may not only apply the majority decision grouping technique, but may also apply other grouping techniques to accelerate accumulation.

[0146] The first to third embodiments of the present invention can be applied to, for example but not limited to, AI technologies such as face recognition.

[0147] In the first to third embodiments of the present invention, the conversion unit may be a current-mode analog-to-digital conversion unit, a voltage-mode analog-to-digital conversion unit, or a mixed-mode analog-to-digital conversion unit.

[0148] The first to third embodiments of the present invention are applicable not only to serial MAC operations but also to parallel MAC operations.

[0149] The first to third embodiments of the present invention are applicable not only to non-volatile memories but also to volatile memories.

[0150] In summary, although the present invention has been disclosed above with reference to the embodiments, these are not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the scope defined in the appended claims.

Claims

1. A memory device, characterized in that: include: A plurality of page buffers temporarily storing input data; A plurality of memory planes are coupled to the page buffers, a plurality of weights are stored in the memory planes, and based on a plurality of addresses received from the memory planes, the memory planes perform bitwise multiplications in parallel on the weights and the input data of the page buffers to obtain a plurality of bitwise multiplication results in parallel, and the bitwise multiplication results are stored back in the page buffers; as well as At least one accumulation circuit is coupled to the page buffers and performs a bit accumulation operation on the bit multiplication results of the memory planes in parallel or in sequence to obtain a multiplication-accumulation operation result.

2. The memory device according to claim 1, wherein Also includes: A plurality of conversion units are coupled to the memory planes, and the conversion units convert the currents of the plurality of memory cells of the memory planes to obtain a plurality of conversion results; an accumulation detection circuit coupled to the conversion units, the accumulation detection circuit respectively comparing the conversion results of the conversion units with a threshold value to generate a page buffer selection signal and an accumulation enable signal; as well as An output latch is coupled to the accumulation detection circuit. In response to the page buffer selection signal, the output latch selects the bit multiplication results stored in the corresponding page buffers and transmits them to the at least one accumulation circuit. In response to the accumulation enable signal, the at least one accumulation circuit sequentially performs bit accumulation operations on the bit multiplication results transmitted from the output latch to obtain the product-accumulate operation result.

3. The memory device according to claim 2, wherein: When the conversion units are triggered, the memory device performs a mixed multiply-accumulate operation, and the at least one accumulation circuit sequentially performs bit accumulation on at least one target conversion result of the conversion results, wherein the at least one target conversion result is higher than the threshold; and When the conversion units are not triggered, the memory device performs a digital multiplication-accumulation operation, and the at least one accumulation circuit sequentially performs bit accumulation on the conversion results, regardless of whether the conversion results are higher than the threshold. The at least one accumulation circuit includes: a grouping circuit that performs a grouping operation on the bit multiplication results of the storage planes to obtain a plurality of grouped results; and a counting unit coupled to the grouping circuit that performs bit counting on the grouped results to obtain the product-accumulation operation result; wherein the plurality of bits of each of the input data or each of the weight values ​​are divided into a plurality of bit vectors; Converting each bit of the bit vectors from a binary representation to a unary representation; Repeating each bit of the bit vectors represented by the unary code multiple times to form an expanded product form; and The storage planes perform multiplication operations on the input data in the form of the spread product and the weight values ​​in the form of the spread product to obtain the bit multiplication operation results.

4. The memory device according to claim 1, wherein: Also includes: A plurality of conversion units are coupled to the memory planes, and the conversion units convert the currents of the plurality of memory cells of the memory planes to obtain a plurality of conversion results; A plurality of accumulation detection circuits are coupled to the conversion units, and the accumulation detection circuits respectively compare the conversion results with a threshold value to generate a plurality of accumulation enable signals; as well as An output latch, The at least one accumulation circuit includes a plurality of accumulation circuits coupled to the accumulation detection circuits. When enabled by the accumulation enable signals, the accumulation circuits perform bit accumulation operations on the bit multiplication results transmitted from the storage planes in parallel to obtain a plurality of digital accumulation results. as well as The output latch is coupled to the accumulation circuits and outputs the digital accumulation results of the accumulation circuits as the multiplication and accumulation operation results.

5. The memory device according to claim 4, wherein: When the conversion units are triggered, the memory device performs a mixed multiply-accumulate operation, and the accumulation circuits perform bit accumulation in parallel on at least one target conversion result of the conversion results, wherein the at least one target conversion result is higher than the threshold; and When the conversion units are not triggered, the memory device performs digital multiplication and accumulation operations, and the accumulation circuits perform bit accumulation on the conversion results in parallel, regardless of whether the conversion results are higher than the threshold. Each of the accumulation circuits includes: a grouping circuit that performs a grouping operation on the bit multiplication results of the storage planes to obtain a plurality of grouped results; and a counting unit coupled to the grouping circuit that performs bit counting on the grouped results to obtain the product-accumulation operation result; wherein the plurality of bits of each of the input data or each of the weight values ​​are divided into a plurality of bit vectors; Converting each bit of the bit vectors from a binary representation to a unary representation; Repeating each bit of the bit vectors represented by the unary code multiple times to form an expanded product form; and The storage planes perform multiplication operations on the input data in the form of the spread product and the weight values ​​in the form of the spread product to obtain the bit multiplication operation results.

6. A method for operating a memory device, characterized in that: include: temporarily storing input data in a plurality of page buffers; According to the received addresses of the memory planes, the memory planes perform bitwise multiplications on the stored weights and the input data in parallel to obtain a plurality of bitwise multiplication results in parallel, and store the bitwise multiplication results back in the page buffers; as well as A bit accumulation operation is performed in parallel or sequentially on the bit multiplication results of the memory planes to obtain a multiplication-accumulation operation result.

7. The method for operating a memory device according to claim 6, wherein: Also includes: Converting the currents of the multiple memory cells of the storage planes to obtain multiple conversion results; Comparing the conversion results with a threshold value respectively to generate a page buffer selection signal and an accumulation enable signal; as well as In response to the page buffer selection signal and the accumulation enable signal, the bit multiplication results stored in the corresponding page buffers are selected to perform bit accumulation operations in sequence to obtain the product-accumulate operation result.

8. The method for operating a memory device according to claim 7, wherein: The memory device performs a mixed multiply-accumulate operation to sequentially perform bit accumulation on at least one target conversion result of the conversion results, wherein the at least one target conversion result is higher than the threshold; or The memory device performs digital multiplication and accumulation operations, and sequentially performs bit accumulation on the conversion results, regardless of whether the conversion results are higher than the threshold. The bit multiplication results of the storage planes are grouped to obtain a plurality of grouped results; and the bit counts of the grouped results are performed to obtain the product-accumulation operation result. wherein the plurality of bits of each of the input data or each of the weight values ​​are divided into a plurality of bit vectors; Converting each bit of the bit vectors from a binary representation to a unary representation; Repeating each bit of the bit vectors represented by the unary code multiple times to form an expanded product form; and The input data in the form of the spread product are multiplied by the weight values ​​in the form of the spread product to obtain the bit multiplication operation results.

9. The method for operating a memory device according to claim 6, wherein: Also includes: Converting the currents of the multiple memory cells of the storage planes to obtain multiple conversion results; Comparing the conversion results with a threshold value respectively to generate a plurality of accumulation enable signals; In response to the accumulation enable signals, performing bit accumulation operations in parallel on the bit multiplication results transmitted from the storage planes to obtain a plurality of digital accumulation results; as well as These digital accumulation results are output as the product-addition operation results.

10. The method for operating a memory device according to claim 9, wherein: The memory device performs a mixed multiply-accumulate operation to perform bit accumulation in parallel on at least one target conversion result of the conversion results, wherein the at least one target conversion result is higher than the threshold; or The memory device performs a digital multiply-accumulate operation to perform bit accumulation on the conversion results in parallel, regardless of whether the conversion results are above the threshold. The bit multiplication results of the storage planes are grouped to obtain a plurality of grouped results; and the bit counts of the grouped results are performed to obtain the product-accumulation operation result. wherein the plurality of bits of each of the input data or each of the weight values ​​are divided into a plurality of bit vectors; Converting each bit of the bit vectors from a binary representation to a unary representation; Repeating each bit of the bit vectors represented by the unary code multiple times to form an expanded product form; and The input data in the form of the spread product are multiplied by the weight values ​​in the form of the spread product to obtain the bit multiplication operation results.

Images