A charge domain 3D NAND storage in-memory computation method

Abstract

The application provides a charge domain 3D NAND storage in-memory computing method, and belongs to the field of large language model hardware acceleration and storage-computing integration. The method realizes weight storage and charge domain multiplication by using the storage characteristics and switching characteristics of a part of NAND transistors in a string without changing the 3D NAND array structure and manufacturing process, realizes charge domain accumulation based on the charge sharing principle by using the gate capacitance of another part of NAND transistors in the string, realizes independent calculation of the two steps of charge domain multiplication and accumulation by using the switching characteristics of string selection tubes and ground selection tubes, thereby completing the general matrix vector multiplication operation in the storage in the charge domain, eliminating the direct current in the traditional current domain calculation, and relieving the heat aggregation effect and IR drop problem; meanwhile, the multiplication result carrier is replaced from the transistor current to the transistor gate capacitance charge, and the fluctuation problem of the transistor current is overcome. The application can improve the parallelism and accuracy of the storage in-memory computing technology, and provides a highly potential solution for the end-side deployment of the large language model.

Description

Technical Field

[0001] This invention relates to the field of hardware acceleration and in-memory computing for large language models, specifically to a 3DNAND in-memory computing method for charge domains. Background Technology

[0002] The deployment of Large Language Models (LLMs) on edge devices is becoming a mainstream trend due to their faster response times, better privacy, and lower operating costs. When edge device memory cannot accommodate the massive parameters of LLMs (hundreds of gigabytes), offloading the model parameters to 3D NAND-based solid-state drives (SSDs) offers a practical and low-cost solution. On the other hand, the autoregressive decoding stage of LLM inference is memory-bound with high read intensity and low arithmetic intensity, requiring high memory access bandwidth (greater than several hundred GB / s). However, currently, the interface bandwidth of SSDs is generally less than 10 GB / s, which is insufficient to meet this requirement.

[0003] To alleviate this bandwidth bottleneck, in-memory compute (ISC) has emerged as a highly promising solution. ISC can select a single layer within a 3D NAND array and perform generalized matrix-vector multiplication (GEMV) operations in situ with high parallelism at the page level. This fully utilizes the internal bandwidth of the memory, allowing the limited interface bandwidth to be used to transmit GEMV results rather than the original weights, which is beneficial for meeting the high memory access bandwidth requirements of LLM inference.

[0004] Existing ISC technology maps weights to NAND string conductance or transistor thresholds, performing calculations in the current domain based on Ohm's law and Kirchhoff's laws, and representing the GEMV result as accumulated current. This approach introduces the following drawbacks: First, write fluctuations in weight conductance or thresholds limit the accuracy of GEMV calculations based on current; second, high-parallel computation within the array generates massive DC currents, leading to severe heat accumulation and IR drop issues caused by interconnect resistance and parasitic effects, further limiting computational parallelism. Therefore, the performance of existing ISC technology is severely inadequate, hindering its use for accelerating edge-side LLM inference. Summary of the Invention

[0005] To address the problems in the prior art described above, the purpose of this invention is to propose a charge domain-based 3D NAND in-memory computing method that can eliminate DC computing current to alleviate heat accumulation effects and improve computing parallelism, while achieving higher computing accuracy, thus providing a high-performance in-memory computing solution for edge-side LLM inference.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for in-cell computation of charge domain in 3D NAND storage, wherein in a 3D NAND array with a total of L word lines (WL), the a words closest to the string select line (SSL) (denoted as CT layers, i.e., CT1 ~ CT) a ) and the a layers (denoted as CB1 ~ CB) closest to the ground selection line (GSL) a ) is used as a calculation capacitor unit, where L and a are positive integers greater than 1 and L > 2a; the unit threshold is pre-written so that its gate capacitance is in a stable inversion state when the gate voltage is 0, that is, when the channel potential is in [-V m V m When the voltage is within the inversion region, and the gate capacitance C0 is independent of the voltage, the calculated capacitance (V) is calculated using this gate capacitance as the charge domain. m (The maximum value of the input voltage amplitude); the remaining L-2a WL layers in the middle part (denoted as WL1~WL) L-2a As a weight storage unit, each unit stores a binary weight w = 0 or 1, where w = 0 encodes a high threshold state (V). th,H ), w = 1 is encoded as a low threshold state (V th,L In the WL, CT, and CB layers mentioned above, the subscript numbers increase from the bit line (BL) towards the source line (SL).

[0008] Furthermore, in L-2a WL layers (WL1 ~ WL) L-2a The invention selects a layer of weights to participate in a single GEMV operation; it can realize the operation of BL input and SL output, and a complete operation process is divided into three steps: reset, multiplication, and accumulation.

[0009] A1) Reset: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Layer applied V pass Gate voltage (V) pass > V th,H Make the channel at that point conductive, and at the same time ground both BL and SL to reset the channel potential of all strings to 0;

[0010] A2) Multiplication: Apply a multi-valued input voltage V proportional to the input x to BL. x (-V) m < V x < V m Apply V at the SSL layer; CC A gate voltage turns on the series selector, and applying zero gate voltage to the GSL layer turns off the ground selector; in CT1 ~ CTa Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Apply V to the unselected WL layer pass Gate voltage makes the channel conductive at that location; apply V to the selected WL layer (weight w). r Gate voltage (V) th,L < V r < V th,H When w = 1, select WL layer channel conduction, CB1 ~ CB a The channel at the layer is charged to V x The upper plates (conductive channels) of each of the a CB capacitors are charged with C0V. x When w = 0, select WL layer channel shutdown, CB1 ~ CB a The channel at the layer is maintained at 0 potential, and the upper plates of the a CB computational capacitors are all charged with 0; thus, the multiplication of the multi-valued input x and the binary weight w is completed.

[0011] A3) Accumulation: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying a zero gate voltage to the layer enables channel conduction at that location; if the selected WL layer in step A2) belongs to WL1 ~ WL L-2a-1 If one layer is applied, then V is applied to that layer. r Gate voltage, in WL1 ~ WL L-2a-1 Apply V to the remaining layers in the layer pass Gate voltage, in WL L-2a Layer applied V off Gate voltage (V) off < V th,L ) shut off the channel at that location; if the selected WL layer in step A2) is WL L-2a If a layer is applied, then V is applied to that layer. off The gate voltage shuts off the channel at that location, between WL1 and WL. L-2a-1 Layer applied V pass Gate voltage; SL floating, the upper plates of the CB layer of multiple strings mounted on the same SL share charge on the SL, and the accumulation operation is completed. The GEMV result is reflected as the final SL potential.

[0012] Furthermore, in addition to operations with BL input and SL output, this invention can also perform operations with SL input and BL output. A complete operation still consists of three steps: reset, multiplication, and accumulation.

[0013] B1) Reset: The reset process is completely consistent with that of BL input and SL output operations;

[0014] B2) Multiplication: Apply a multi-valued input voltage V proportional to the input x to SL. x (-V) m < V x < V m Apply V to the GSL layer; CC A gate voltage turns on the ground selector, and applying a zero gate voltage to the SSL layer turns off the series selector; in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Apply V to the unselected WL layer pass Gate voltage makes the channel conductive at that location; apply V to the selected WL layer (weight w). r Gate voltage (V) th,L < V r < V th,H When w = 1, select WL layer channel conduction, CT1 ~ CT a The channel at the layer is charged to V x The upper plates (conductive channels) of each of the a CT calculation capacitors are all charged with C0V. x When w = 0, select WL layer channel shutdown, CT1 ~ CT a The channel at the layer is maintained at 0 potential, and the upper plates of the a CT calculation capacitors are all charged with 0; thus, the multiplication of the multi-valued input x and the binary weight w is completed.

[0015] B3) Accumulation: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying a zero gate voltage to the layer enables channel conduction at that location; if the selected WL layer in step B2) belongs to WL2 ~ WL L-2a If one layer is applied, then V is applied to that layer. r Gate voltage, in WL2 ~ WL L-2a Apply V to the remaining layers in the layer pass Gate voltage, applying V to the WL1 layer off Gate voltage (V) off < V th,L ) Shut down the channel at that location; if the selected WL layer in step B2) is WL1 layer, then apply V to that layer. off The gate voltage shuts off the channel at that location, between WL2 and WL. L-2a Layer applied V passGate voltage; BL floating, the upper plates of the CT layer calculation capacitors of multiple strings mounted on the same BL share charge on the BL, and the accumulation calculation is completed. The GEMV result is reflected as the final BL potential.

[0016] Furthermore, this invention implements multi-value extension of weights in the blocks of a 3D NAND array: s strings sharing the same BL within the same block (with the same SL) are grouped together (s is a positive integer). Each WL layer's s units within the group are encoded in a number-by-number manner to jointly represent an s+1 value weight W. Each group can represent a total of L-2a s+1 value weights. W = b corresponds to b units being written as w = 1, and the remaining sb units being written as w = 0 (0 ≤ b ≤ s, b is a natural number).

[0017] Furthermore, since both BL and SL can be used as inputs and outputs, this invention can implement transposeable GEMV operations, making it highly suitable for accelerating feedforward networks (FFNs) in LLMs that include upsampling and downsampling layers: the upsampling matrix in the FFN (dimension d) model ×d FFN ) and the downsampling matrix (dimension d) FFN ×d model The dimensions of the two are transposes of each other, they are stored in two WL layers of the 3D NAND and their storage directions are transposes of each other, and they occupy the same BL (e.g., both are BL1 ~ BL). dFFN ) and SL (e.g., both are SL1 ~SL) dmodel When performing FFN acceleration, the upsampling matrix is ​​first input from SL and the result of the upsampling layer GEMV, represented as the BL potential, is activated by the peripheral circuit at BL and then input from BL in place to perform the downsampling matrix. The final result of FFN is represented as the SL potential.

[0018] The beneficial technical effects of this invention are as follows:

[0019] This invention, without altering the mature commercial 3D NAND array structure and manufacturing process, utilizes the storage and switching characteristics of a portion of the NAND transistors in the string to achieve weight storage and charge domain multiplication. It leverages the gate capacitance of another portion of the NAND transistors in the string based on the charge-sharing principle to achieve charge domain accumulation. Furthermore, it utilizes the switching characteristics of the string selector and ground selector to achieve independent calculations for the charge domain multiplication and accumulation steps. This allows for GEMV computation to be completed in the charge domain, eliminating the DC current required in traditional current domain calculations and mitigating heat accumulation and IR drop issues. Simultaneously, this invention replaces the transistor current as the carrier of the multiplication result with the transistor gate capacitance charge, overcoming transistor current fluctuation problems. This invention improves the parallelism and accuracy of ISC technology, providing a highly promising solution for edge-side LLM. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the in-storage computation of the charge domain in 3D NAND storage according to the present invention.

[0021] Figure 2 This is a schematic diagram illustrating the multiplication principle of the present invention for performing calculations within 3D NAND storage of the charge domain.

[0022] Figure 3 This is a schematic diagram illustrating the accumulation principle of 3D NAND storage computation in the charge domain according to the present invention. Detailed Implementation

[0023] The present invention will be further clearly and completely described below with reference to the accompanying drawings and specific embodiments.

[0024] The charge domain 3D NAND storage computation implemented in this invention is as follows: Figure 1 As shown, among the L WL layers of the 3D NAND array: the a WL layers closest to the SSL (denoted as CT1 ~ CT) a The layer) and the a WL layers (denoted as CB1 ~ CB) close to the GSL. a (Layer) is used as the computational capacitance cell; the cell threshold is pre-written so that its gate capacitance is in a stable inversion state when the gate voltage is 0, that is, when the channel potential is in [-V m V m When the voltage is within the inversion region, and the gate capacitance C0 is independent of the voltage, the calculated capacitance (V) is calculated using this gate capacitance as the charge domain. m (The maximum value of the input voltage amplitude); the remaining L-2a WL layers in the middle part (denoted as WL1 ~ WL) L-2a As a weight storage unit, each unit stores a binary weight w = 0 or 1, where w = 0 encodes a high threshold state (V). th,H ), w = 1 is encoded as a low threshold state (V th,LThe value of the number of computing capacitor units 2a satisfies a trade-off relationship and can be configured according to actual needs. When a is large, the number of weight storage units L-2a is small, resulting in a decrease in storage density. However, a larger computing capacitor value can alleviate the charge injection effect caused by transistor switching during operation. When a is small, a smaller computing capacitor value will cause a charge injection effect during operation due to transistor switching. However, it has a higher weight storage density. Further, multi-value extension of weights is implemented in the block of the 3D NAND array. s strings sharing the same BL in the same block are grouped together (s is a positive integer). The s units of each WL layer in the group are encoded by number to jointly represent an s+1 value weight W. Each group can represent a total of L-2a s+1 value weights. W = b corresponds to b units in the s units being written as w = 1, and the remaining sb units being written as w = 0 (0 ≤ b ≤ s, b is a natural number).

[0025] This embodiment uses BL input and SL output as an example to implement the multiplication principle of charge domain 3D NAND storage calculation as follows: Figure 2 As shown: proportional to the input x i Multi-value input voltage V xi From BL i Apply (-V) m < V xi < V m (1 ≤ i ≤ M, where the number of BLs M is an integer greater than 1); apply V to the SSL layer. CC A gate voltage turns on the series selector; applying zero gate voltage to the GSL layer turns off the ground selector; applying zero gate voltage to the two CT layers and the two CB layers turns on the channel at that location; applying V gate voltage to the unselected WL2 layer... pass The gate voltage makes the channel conductive at that location; a V voltage is applied to the selected WL1 layer. r Gate voltage (V) th,L < V r < V th,H ); BL i The corresponding weight w i When V = 1, the WL1 layer channel is activated, and the channel at the CB layer is charged to V. xi BL i The corresponding weight w i When the value is 0, the channel in layer WL1 is turned off, and the channel at layer CB maintains a potential of 0; thus, the multiplication of the multivalued input x and the binary weight w is completed.

[0026] This embodiment uses BL input and SL output as an example to implement the accumulation principle of charge domain 3D NAND storage calculation as follows: Figure 3 As shown: Applying V to the SSL layer and GSL layer CCA gate voltage connects the series pass and the ground pass. A zero gate voltage is applied to the two CT layers and the two CB layers to connect the channel at that location. A V voltage is applied to the WL1 layer. r Gate voltage, applying V to the WL2 layer off The gate voltage turns off the channel at that point; SL1 is floating, and the upper plates (conducting channels) of the CB layers of multiple strings attached to SL1 share charge on SL1, completing the accumulation calculation. The GEMV result is reflected in the final SL1 potential V. SL .

[0027] Finally, it should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A method for in-storage computation in 3D NAND flash memory with charge domain, characterized in that, In a 3D NAND array with a total of L word line layers, the a word line layers closest to the serial select line SSL are denoted as CT layers, i.e., CT1 ~ CT2. a The a-thousand-word line layers closest to the ground selection line GSL are denoted as CB layers, i.e., CB1 ~ CB2. a These are used as capacitance calculation units, where L and a are positive integers greater than 1 and L > 2a; for the CT and CB layers, the unit thresholds are pre-written so that their gate capacitance is in a stable inversion state when the gate voltage is 0, i.e., when the channel potential is in [-V m V m When the voltage is within the inversion region, and the gate capacitance C0 is independent of the voltage, this gate capacitance is used as the calculated capacitance for the charge domain, where V m The maximum value of the input voltage amplitude; the remaining L-2a word line layers in the middle section are denoted as WL1~WL L-2a As a weight storage unit, each unit stores a binary weight w = 0 or 1, where w = 0 encodes a high threshold state V. th,H w = 1 is encoded as the low-threshold state V th,L ; in WL1 ~ WL L-2a The selected layer of weights participates in a single GEMV operation.

2. The 3D NAND memory in-memory computing method as described in claim 1, characterized in that, For CT1 ~ CT a , CB1 ~CB a WL1~WL L-2a The subscript numbers of the layers increase from the bit line BL to the source line SL. A complete operation of BL input and SL output is achieved through the following three steps: reset, multiplication, and accumulation. A1) Reset: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Layer applied V pass The gate voltage makes the channel open at that point, V pass > V th,H At the same time, both BL and SL are grounded to reset the channel potential of all strings to 0; A2) Multiplication: Apply a multi-valued input voltage V proportional to the input x to BL. x -V m < V x < V m Apply V to the SSL layer CC A gate voltage turns on the series selector, and applying zero gate voltage to the GSL layer turns off the ground selector; in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Apply V to the unselected WL layer pass Gate voltage enables channel conduction at that location; V is applied to the selected WL layer with weight w. r Gate voltage, V th,L < V r < V th,H When w = 1, the WL layer channel is selected for conduction, CB1 ~ CB a The channel at the layer is charged to V x The upper plates of each of the a-CB calculated capacitors are charged with C0V. x When w = 0, select WL layer channel shutdown, CB1 ~ CB a The channel at the layer is maintained at 0 potential, and the upper plates of the a CB computational capacitors are all charged with 0; thus, the multiplication of the multi-valued input x and the binary weight w is completed. A3) Accumulation: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying a zero gate voltage to the layer enables channel conduction at that location; if the selected WL layer in step A2) belongs to WL1 ~ WL L-2a-1 If one layer is applied, then V is applied to that layer. r Gate voltage, in WL1 ~ WL L-2a-1 Apply V to the remaining layers in the layer pass Gate voltage, in WL L-2a Layer applied V off The gate voltage shuts off the channel at that location, V off < V th,L If the selected WL layer in step A2) is WL L-2a If a layer is applied, then V is applied to that layer. off The gate voltage shuts off the channel at that location, between WL1 and WL. L-2a-1 Layer applied V pass Gate voltage; SL floating, the upper plates of the CB layer of multiple strings mounted on the same SL share charge on the SL, and the accumulation operation is completed. The GEMV result is reflected as the final SL potential.

3. The 3D NAND memory in-memory computing method as described in claim 1, characterized in that, For CT1 ~ CT a , CB1 ~CB a WL1~WL L-2a The subscript numbers of the layers increase from the bit line BL towards the source line SL. A complete operation of SL input and BL output is achieved through the following three steps: reset, multiplication, and accumulation. B1) Reset: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Layer applied V pass The gate voltage makes the channel open at that point, V pass > V th,H At the same time, both BL and SL are grounded to reset the channel potential of all strings to 0; B2) Multiplication: Apply a multi-valued input voltage V proportional to the input x to SL. x -V m < V x < V m Apply V to the GSL layer CC A gate voltage turns on the ground selector, and applying a zero gate voltage to the SSL layer turns off the series selector; in CT1 ~ CT a Layers and CB1 ~ CB a Applying zero gate voltage to the layer enables channel conduction at that location, in WL1 ~ WL L-2a Apply V to the unselected WL layer pass Gate voltage enables channel conduction at that location; V is applied to the selected WL layer with weight w. r Gate voltage, V th,L < V r < V th,H When w = 1, select WL layer channel conduction, CT1 ~ CT a The channel at the layer is charged to V x The upper plates of the a CT calculation capacitors are all charged with C0V. x When w = 0, select WL layer channel shutdown, CT1 ~ CT a The channel at the layer is maintained at 0 potential, and the upper plates of the a CT calculation capacitors are all charged with 0; thus, the multiplication of the multi-valued input x and the binary weight w is completed. B3) Accumulation: Apply V to the SSL layer and GSL layer CC The gate voltage connects the series selector to the ground selector, in CT1 ~ CT a Layers and CB1 ~ CB a Applying a zero gate voltage to the layer enables channel conduction at that location; if the selected WL layer in step B2) belongs to WL2 ~ WL L-2a If one layer is applied, then V is applied to that layer. r Gate voltage, in WL2 ~ WL L-2a Apply V to the remaining layers in the layer pass Gate voltage, applying V to the WL1 layer off The gate voltage shuts off the channel at that location, V off < V th,L If the selected WL layer in step B2) is WL1, then apply V to that layer. off The gate voltage shuts off the channel at that location, between WL2 and WL. L-2a Layer applied V pass Gate voltage; BL floating, the upper plates of the CT layer calculation capacitors of multiple strings mounted on the same BL share charge on the BL, and the accumulation calculation is completed. The GEMV result is reflected as the final BL potential.

4. The 3D NAND memory in-memory computing method as described in claim 1, characterized in that, Implement multi-value extension of weights in a 3D NAND array block: Define a block as having the same SL, and group s strings that share the same BL within the same block. Each WL layer s units in the group represent an s+1 value weight W in a number-based encoding manner. Each group can represent a total of L-2a s+1 value weights. W = b corresponds to b units in the s units being written as w = 1, and the remaining sb units being written as w = 0. Here, s is a positive integer, b is a natural number, and 0 ≤ b ≤ s.

5. The application of the 3D NAND storage in-process computing method according to claim 1 in large language models, characterized in that, The 3D NAND in-memory computation method is applied to a feedforward network containing upsampling and downsampling layers. In the feedforward network, the dimensions of the upsampling matrix and the downsampling matrix are transposes of each other. They are stored in two WL layers of the 3D NAND with transposes of each other and occupy the same BL and SL. When the feedforward network is accelerated, it is first input from SL, and the upsampling matrix is ​​calculated according to the method of claim 3. The upsampling layer GEMV result, represented as the BL potential, is activated by the peripheral circuit at BL and then input from BL in place. The downsampling matrix is ​​calculated according to the method of claim 2. The final result of the feedforward network is represented as the SL potential.

Images