Memory, in-memory computing methods and electronic devices

By introducing masked storage strings into the storage array, the problem of crosstalk in the calculation results of storage cell strings is solved, parallel computing within memory is realized, and computing efficiency is improved.

CN120932695BActive Publication Date: 2026-11-17PEKING UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511039626.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-25
Publication Date
2026-11-17
Estimated Expiration
2045-07-25

AI Technical Summary

Technical Problem

In existing in-memory computing technologies, the calculation results of memory cell strings are prone to crosstalk, resulting in low computational efficiency and the inability to perform parallel computing, thus limiting the efficiency and flexibility of in-memory computing.

Method used

In the storage array, a mask storage string is set in each storage cell string. The mask vector of the mask storage string is used to define whether the storage cell string outputs the calculation result of the weight vector. Parallel computing is realized by determining whether the calculation result is output through the mask storage string.

Benefits of technology

By setting a masked storage string, crosstalk between storage cell strings is avoided, improving the computational efficiency of in-memory computation and enabling parallel computation of storage cell strings in the memory, thus enhancing computational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120932695B_ABST
    Figure CN120932695B_ABST
Patent Text Reader

Abstract

This application relates to a memory, an in-memory computing method, and an electronic device. The memory includes a memory array; the memory array includes multiple sets of memory cell strings spaced apart along a first direction; each set of memory cell strings includes at least two memory cell strings spaced apart along a second direction, the second direction being parallel to and intersecting the first direction on a substrate; wherein, the memory cell string includes: a weight storage string and a mask storage string sequentially connected in a direction perpendicular to the substrate; the weight storage string is used to store weight vectors; the mask storage string is used to store mask vectors, the mask vectors being pre-programmed logical decision vectors used to define whether a memory cell string outputs a weight vector. The memory using this application can improve the computational efficiency of in-memory computing technology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of new energy technology, and in particular to a memory, an in-memory computing method, and an electronic device. Background Technology

[0002] Modern computers typically employ the von Neumann architecture, which separates computation from storage, requiring data to be transferred between the processor and memory. However, with the rapid development of artificial intelligence, the scale of neural network models continues to expand, and the computational and data transfer requirements during inference have increased dramatically. The von Neumann architecture, which requires frequent data transfers, is inefficient, while in-memory computing technology promises to solve this problem by embedding computation within memory.

[0003] The core idea of ​​in-memory computing technology is to embed computing functions inside the memory, so that data storage and processing are completed inside the memory. Compared with traditional architecture, in-memory computing reduces the frequent movement of data between the processor and memory, which greatly reduces the latency and energy consumption caused by data transmission.

[0004] However, in-memory computing technology suffers from low computational efficiency. Summary of the Invention

[0005] Based on this, this application provides a memory, an in-memory computing method, and an electronic device, which can improve the computing efficiency of in-memory computing technology.

[0006] In a first aspect, this application provides a memory, including: a memory array;

[0007] The memory array includes multiple sets of memory cell strings spaced apart along a first direction; each set of memory cell strings includes at least two memory cell strings spaced apart along a second direction, the second direction being parallel to and intersecting the substrate with the first direction;

[0008] The storage cell string includes a weight storage string and a mask storage string sequentially connected in series along the direction perpendicular to the substrate; the weight storage string is used to store the weight vector; the mask storage string is used to store the mask vector, which is a pre-programmed logical decision vector used to define whether the storage cell string outputs the weight vector calculation result.

[0009] In some embodiments, the storage array further includes:

[0010] Multiple bit lines are spaced apart along the first direction;

[0011] Multiple first-selection circuits, arrayed together;

[0012] In this group of storage cell strings, each mask storage string is connected to a bit line through a corresponding first gating circuit;

[0013] The storage cell string is used to: respond to the mask vector and weight vector when the first gating circuit is turned on, output the calculation result to the bit line or reset the calculation result to zero.

[0014] In some embodiments, the storage array further includes:

[0015] The source line and the array of multiple second gating circuits are all located on the side of the weight storage string that is away from the mask storage string;

[0016] Each weighted storage string is connected to the source line through a corresponding second gating circuit;

[0017] The storage cell string is used to: respond to the mask vector and weight vector when both the first and second gating circuits are turned on, output the calculation result to the bit line or reset the calculation result to zero.

[0018] In some embodiments, the weight storage string includes a plurality of weight storage units sequentially connected in series along the direction perpendicular to the substrate; each weight storage unit is used to store weight data of the corresponding bit in the weight vector;

[0019] The mask storage string includes at least one mask storage unit; each mask storage unit is used to store the mask data of the corresponding bit in the mask vector; when the mask storage string includes multiple mask storage units, the mask storage units are connected in series in sequence along the direction perpendicular to the substrate.

[0020] The storage array also includes:

[0021] Multiple weight word lines; weight storage cells with the same sequence number along the vertical substrate direction in each weight storage string are set on the same layer and connected to the same weight word line;

[0022] At least one mask word line; mask storage cells with the same sequence number along the vertical substrate direction in each mask storage string are arranged on the same layer and connected to the same mask word line.

[0023] In some embodiments, the mask data in multiple mask storage units in the same layer constitute a mask pattern; the mask pattern includes a pass area and a shielding area complementary to the pass area, the pass area and shielding area in different mask patterns are distributed in different positions, any two mask patterns are orthogonal, and the mask data in at least one mask storage unit of each mask storage string includes at least one piece of data for indicating the calculation result of the output weight vector.

[0024] In some embodiments, the memory further includes:

[0025] The weight driving and decoding circuit is connected to the weight word line and is used to output a read voltage to the target weight word line so that the weight storage unit connected to the target weight word line is in the selected state. The weight data stored in the selected weight storage unit is used to determine the calculation result.

[0026] The mask driving and decoding circuit is connected to the mask word line and is used to output a read voltage to the target mask word line so that the mask storage cell connected to the target mask word line is in the selected state. The selected mask storage cell is used to output the calculation result to the bit line or to reset the calculation result to zero.

[0027] In some embodiments, the memory further includes:

[0028] The mask selection circuit connects the mask driver and decoder circuits. It is used to respond to mask calculation requirements, select the target mask pattern from the pre-programmed mask pattern group, and output the read voltage to the target mask word line based on the target mask pattern.

[0029] In some embodiments, the mask storage unit is composed of redundant storage units in the storage array.

[0030] Secondly, this application provides an in-memory computing method applied to a memory as described in any of the first aspects, the method comprising:

[0031] Program the weight vector into the weight storage string in memory;

[0032] Program the mask vector into the mask storage string in memory;

[0033] The response mask vector and weight vector control whether the memory cell string outputs the weight vector calculation result.

[0034] Thirdly, this application provides an electronic device that includes a memory as described in any of the first aspects.

[0035] In the technical solution provided by this application embodiment, since a mask storage string is additionally set in each storage cell string in the storage array, and the mask vector in the mask storage string is a pre-programmed logical determination vector used to define whether the storage cell string outputs the calculation result of the weight vector, it is possible to determine whether the calculation result of the weight vector of the storage cell string is output through the mask storage string in each storage cell string. That is, the calculation result of the weight vector in any required storage cell string in the memory can be calculated in parallel, avoiding the problem in related technologies that require serial calculation of the calculation result of the weight vector in the required storage cell in the memory in order to avoid crosstalk between the calculation results of the weight vector in the same group of storage cell strings, resulting in low computational efficiency of in-memory computing technology. Therefore, this application embodiment can improve the computational efficiency of in-memory computing technology. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A schematic diagram of the memory structure provided in the first embodiment;

[0038] Figure 2 A schematic diagram of the structure of a storage cell string provided in the first embodiment;

[0039] Figure 3 A schematic diagram of the memory structure provided in the second embodiment;

[0040] Figure 4 A schematic diagram of the structure of a storage cell string provided in the second embodiment;

[0041] Figure 5 A schematic diagram illustrating the relationship between the mask storage unit of the mask layer and the weight storage unit of the weight layer in some embodiments;

[0042] Figure 6 A schematic diagram of the memory structure provided in the third embodiment;

[0043] Figure 7 A schematic diagram of the memory structure provided in the fourth embodiment;

[0044] Figure 8 A schematic diagram of the memory structure provided in the fifth embodiment;

[0045] Figure 9 A flowchart illustrating an in-memory computation method provided for some embodiments;

[0046] Figure 10 This is a schematic diagram of the structure of an electronic device provided for some embodiments. Detailed Implementation

[0047] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0049] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, and "multiple groups" means two or more, unless otherwise explicitly defined. In the description of the embodiments of this application, "each" means each of the multiple options, unless otherwise explicitly defined.

[0050] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0051] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0052] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0053] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0054] Modern computers typically employ the von Neumann architecture, which separates computation from storage, requiring data to be transferred between the processor and memory. In recent years, with the rapid development of artificial intelligence, the scale of neural network models has continuously expanded. Large Language Models (LLMs) based on Transformers often have billions to trillions of parameters, leading to a dramatic increase in computational demands and data transfer requirements during inference. Simultaneously, the demand for deploying large models on edge devices such as the Artificial Intelligence of Things (AIoT), smartphones, and autonomous driving systems continues to grow. Given the limited storage and computing resources on edge devices, significant power constraints, and stringent latency requirements, the von Neumann architecture, which requires frequent data transfers, is inefficient. In-memory computing technology promises to solve this problem by embedding computation within memory.

[0055] Existing in-memory computing solutions mostly employ resistive random access memory (RRAM) and static random access memory (SRAM) as storage media. However, these media have low storage density, limited array size, and high manufacturing costs, making them unsuitable for the potential computational demands of large-scale model inference on the edge. 3D-NAND Flash memory, on the other hand, possesses significant potential for storing model parameters, providing inexpensive computing power, and saving chip area due to its advantages such as non-volatility, large storage capacity, high storage density, and low cost. In research on in-memory computing using 3D-NAND Flash, a current-domain operation strategy can be employed. This strategy's computational method is similar to that in the 3D-NAND Flash memory scenario, where weights are stored in the array, selected memory cells are determined by applying voltage, and the multiplication and accumulation results are output as the sum of currents.

[0056] However, in-memory computing strategies still face some challenges. For example, multiple weight matrices stored separately in multi-level storage units often cannot be computed in parallel due to crosstalk from other units sharing input or output ports. Therefore, these problems severely restrict the efficiency and flexibility of in-memory computing, resulting in low utilization efficiency of equivalent bandwidth and limiting actual computing power; that is, related technologies suffer from low computational efficiency in in-memory computing.

[0057] Figure 1 A schematic diagram of the memory structure provided in the first embodiment is shown below. Figure 1 As shown, the memory includes a storage array 10, and the storage array 10 includes multiple sets of storage cell strings 11 spaced apart along a first direction. Figure 1 The example illustrates three groups; in other embodiments, the number of groups of storage cell strings 11 can be other than the number described in this application. Each group of storage cell strings 11 includes at least two storage cell strings 111 arranged at intervals along a second direction, the second direction being parallel to and intersecting the substrate with the first direction. The storage cell string 111 includes a weight storage string and a mask storage string sequentially connected in a direction perpendicular to the substrate. The weight storage string stores a weight vector; the mask storage string stores a mask vector, which is a pre-programmed logical decision vector defining whether the storage cell string 111 outputs a weight vector.

[0058] For example, the memory in the embodiments of this application may include 3D-NAND memory, and the storage array 10 may include a 3D-NAND storage array.

[0059] The first direction can be the X direction, and the second direction can be the Y direction. Figure 1 In the illustrated embodiment, the X direction is perpendicular to the Y direction; however, in other embodiments, the angle between the X and Y directions is acute or obtuse. For example, the angle between the X and Y directions is in the range of 30 degrees to 150 degrees.

[0060] For example, the storage cell string 111 extends along the Z direction. In some embodiments, the Z direction may be perpendicular to the X direction and the Z direction may be perpendicular to the Y direction.

[0061] In some embodiments, the interval between any two adjacent storage cell strings 111 in each of the multiple sets of storage cell strings 11 is a first interval, meaning that the interval between any two adjacent storage cell strings 111 in different sets is the same. In other embodiments, the interval between at least some of the adjacent storage cell strings 111 is different.

[0062] A storage unit is a basic hardware structure used to store binary data (0s and 1s), and is the smallest independent unit of operation in a memory (such as RAM, hard disks, registers, etc.). It enables the writing, reading, and saving of data and is a fundamental component for computer information processing. Each storage unit can store a fixed number of bits of binary data; exemplarily, the fixed number of bits can range from 1 to 4 bits. For example, the fixed number of bits may include 1, 2, 3, or 4 bits, depending on the hardware design, so that the data stored in each storage unit is binary or multi-valued. Exemplarily, a storage unit may include a transistor.

[0063] In some embodiments, the interval between any two adjacent storage unit strings 111 in at least two storage unit strings 111 of a set of storage unit strings 11 is a second interval, that is, the interval between different adjacent storage unit strings 111 in a set of storage unit strings 11 is the same. In other embodiments, the interval between at least some adjacent storage unit strings 111 in a set of storage unit strings 11 is different.

[0064] In some embodiments, the same set of memory cell strings 11 shares a bit line. In some embodiments, all memory cell strings 11 in the memory array share a source line.

[0065] In some embodiments, a memory cell string 111 includes a plurality of memory cells sequentially connected in series along a direction perpendicular to the substrate. The interval between any two adjacent memory cells in a memory cell string 111 is a third interval, meaning that the interval between different adjacent memory cells in a memory cell string 111 is the same. In other embodiments, the interval between at least some adjacent memory cells in a memory cell string 111 is different.

[0066] For example, the memory array 10 can be stacked on a substrate, that is, the substrate can serve as the basic support structure of the memory array 10, i.e., the carrier.

[0067] The weight storage string may include at least two weight storage units concatenated in sequence, each weight storage unit being used to store weight data. For example, the weight data may include binary data (0 or 1) or multi-valued data.

[0068] The mask storage string may include a single mask storage unit or at least two mask storage units sequentially connected in series, each mask storage unit storing mask data. For example, the mask data may include binary data (0 or 1) or multi-valued data.

[0069] In the memory, the number of weight storage units in the weight storage strings of different storage unit strings 111 is the same, and the number of mask storage units in the mask storage strings of different storage unit strings 111 is the same. In the memory, storage units with the same sequence in different storage unit strings 111 are either all weight storage units or all mask storage units. Storage units with the same sequence are storage units at the same layer; that is, a layer of storage units is either all weight storage units (i.e., one weight layer) or all mask storage units (i.e., one mask layer). In this embodiment, the order of the storage unit sequence can be along the Z-direction or along a direction opposite to the Z-direction.

[0070] The dimension of the weight vector (representing the number of elements in the vector) is the same as the number of weight storage units in the weight storage string. The dimension of the mask vector is the same as the number of mask storage units in the mask storage string. Each element in the weight vector is weight data, and each element in the mask vector is mask data.

[0071] For example, taking the mask data as binary data: for a storage cell string 111, if the mask storage string includes a single mask storage cell, and the single mask storage cell is selected, if the value of the mask data in the single mask storage cell is 1, then the storage cell string 111 can output the calculation result of the weight vector (i.e., the current that can characterize the calculation result of the weight vector). If the value of the mask data in the single mask storage cell is 0, then the storage cell string 111 cannot output the calculation result of the weight vector.

[0072] In this embodiment of the application, the storage unit string 111 cannot output the calculation result of the weight vector, which can be understood as: the storage unit string 111 masks the calculation result of the weight vector, or in other words, the calculation result of the weight vector is masked.

[0073] For example, taking binary mask data as an example: For a storage unit string 111, if the mask storage string includes multiple mask storage units, and the value of the mask data in the selected mask storage unit is 1, then the storage unit string 111 can output the calculated result of the weight vector. If the value of the mask data in the selected mask storage unit is 0, then the storage unit string 111 cannot output the calculated result of the weight vector. The sequence of the selected mask storage units is the same across different storage units.

[0074] For example, the mask data includes 0 and non-zero values. For example, when the mask data is binary data, the non-zero value is 1. When the mask data is multi-valued data, the non-zero value includes 1 and other integers greater than 1, and this application embodiment does not limit this.

[0075] In some embodiments, the result output by storage unit string 111 is the product between the mask data in the mask storage unit of the selected state and the calculation result of the weight vector.

[0076] In this embodiment of the application, for a string of storage cells, the calculation result of the weight vector may include the product of the weight data in the weight storage cell of the selected state in the string of storage cells and the input data (exemplarily, the input data may be the voltage applied to the first gating circuit).

[0077] Figure 2 A schematic diagram of the structure of a storage cell string provided in the first embodiment is shown below. Figure 2 As shown, the storage cell string includes a mask storage cell and multiple weight storage cells (N weight storage cells, where N is an integer greater than or equal to 2). The mask storage cell and multiple weight storage cells are connected in series. The weight data stored in the multiple weight storage cells constitute a weight vector. When a read voltage is applied to the weight storage cells... In this case, it indicates that the weight storage cell is selected, and an on-state voltage is applied to the weight storage cell. In this case, it indicates that the weight storage unit is in the conducting state. The weight storage unit in the conducting state is responsible for conducting, and the weight data stored in it does not participate in the calculation. The weight data stored in the selected weight storage unit is multiplied with the input data to obtain the calculated weight vector W. Due to the application of a read voltage to the mask storage unit... If M is selected, the mask storage unit is in the selected state. The mask data in the selected mask storage unit is M. When M is 1, the output result Iread of this storage unit string is non-zero, meaning the weight vector calculation result can be output. When M is 0, the output result Iread of this storage unit string is 0, meaning the weight vector calculation result cannot be output. For example, M is binary, with 0 corresponding to a high threshold voltage state and 1 corresponding to a low threshold voltage state; W can be binary or multi-valued.

[0078] Figure 2 Taking a storage cell string comprising a mask storage cell string as an example, in other embodiments, where the storage cell string comprises multiple mask storage cells, only one mask storage cell will be applied. Voltage, thus, with only one mask storage unit selected, can the calculated result of the weight vector be output? Figure 2Similar to the embodiments, the embodiments in this application will not be described in detail.

[0079] In the technical solution provided by this application embodiment, since a mask storage string is additionally set in each storage cell string in the storage array, and the mask vector in the mask storage string is a pre-programmed logical determination vector used to define whether the storage cell string outputs the calculation result of the weight vector, it is possible to determine whether the calculation result of the weight vector of the storage cell string is output through the mask storage string in each storage cell string. That is, the calculation result of the weight vector in any required storage cell string in the memory can be calculated in parallel, avoiding the problem in related technologies that require serial calculation of the calculation result of the weight vector in the required storage cell in the memory in order to avoid crosstalk between the calculation results of the weight vector in the same group of storage cell strings, resulting in low computational efficiency of in-memory computing technology. Therefore, this application embodiment can improve the computational efficiency of in-memory computing technology.

[0080] Figure 3 A schematic diagram of the memory structure provided in the second embodiment is shown below. Figure 3 As shown, Figure 3 Compared to the example Figure 1 The difference in this embodiment is that the storage array further includes multiple bit lines (BLs) and multiple first gating circuits (included in the first gating layer). The multiple bit lines are spaced apart along a first direction; the multiple first gating circuits are arrayed; each mask storage string in a set of storage cell strings is connected to a bit line through a corresponding first gating circuit; the storage cell string is used to: respond to the mask vector and weight vector when the first gating circuit is turned on, output the calculation result to the bit line or reset the calculation result to zero.

[0081] Multiple first gating circuits are connected one-to-one with multiple memory cell strings in the memory. For example, if the memory includes A groups of memory cell strings, and each group of memory cell strings includes B memory cell strings, then the memory includes A×B memory cell strings, and the multiple first gating circuits include A×B first gating circuits. The A×B first gating circuits are connected one-to-one with the A×B memory cell strings, and each B memory cell string is connected to a bit line.

[0082] The first gating circuit in this embodiment may include a top select transistor. A top select transistor is a switching device in a memory array used to control the connection state between memory cell strings and bit lines. A top select transistor typically includes a metal-oxide-semiconductor field-effect transistor (MOSFET), which can be controlled to be turned on or off by the voltage signal of its corresponding top select transistor control gate (TSG).

[0083] like Figure 3 As shown, the multiple up-selector tubes include multiple sets of up-selector tubes spaced apart along a first direction. Each set of up-selector tubes includes at least two up-selector tubes. Up-selector tubes of the same sequence in the multiple sets of up-selector tubes are connected by an up-selector tube control gate line. That is, the up-selector tube control gate line extends along the first direction and is correspondingly connected to a row of up-selector tubes arranged in a row along the first direction. In this embodiment, the sequence of up-selector tubes can be ordered along the Y direction or along a direction opposite to the Y direction.

[0084] Bit lines are critical signal transmission paths in a memory array, used to transfer data between memory cells and peripheral circuits. The calculation results output by a string of memory cells can be analog domain calculation results. Each mask memory string in a set of memory cells is connected to a bit line through a corresponding first gating circuit, which allows a single bit line to accumulate and output multiple calculation results when multiple calculation results are acquired simultaneously.

[0085] In the technical solution provided in this application embodiment, each mask storage string in a set of storage cell strings is connected to a bit line through a corresponding first gating circuit. When the first gating circuit is turned on, if each mask storage string outputs a calculation result, it can independently output the calculation result to its dedicated bit line. If it does not output a calculation result, it independently performs the operation of zeroing the calculation result, which effectively avoids the risk of signal crosstalk between different storage cell strings in a set of storage cell strings due to sharing a bit line.

[0086] Continue reading Figure 3 The storage array also includes a source line (SL) and multiple array-distributed second gating circuits (included in the second gating layer). The source line and the array-distributed second gating circuits are all located on the side of the weight storage string opposite to the mask storage string; each weight storage string is connected to the source line through its corresponding second gating circuit. The storage cell string is used to: respond to the mask vector and weight vector, output the calculation result to the bit line or reset the calculation result to zero when both the first and second gating circuits are turned on.

[0087] Multiple second gating circuits distributed in the array are connected one-to-one with multiple strings of memory cells in the memory. For example, the multiple second gating circuits include A×B second gating circuits. Each of the A×B second gating circuits is connected to one of the A×B strings of memory cells, and all A×B second gating circuits are connected to the source line.

[0088] For example, the second gating circuit in this embodiment may include a bottom select transistor. A bottom select transistor is a switching device in a memory array used to control the connection state between the memory cell string and the source line. A bottom select transistor typically includes a MOSFET, which can be controlled to be turned on or off by the voltage signal of its corresponding bottom select transistor control gate (BSG).

[0089] For example, the plurality of down-select transistors in the storage array include a plurality of groups of down-select transistors spaced apart along a first direction, each group of down-select transistors including at least two down-select transistors, wherein the plurality of down-select transistors in the storage array are all connected by down-select transistor control gate lines.

[0090] In some embodiments, the source line may be grounded.

[0091] Figure 4 A schematic diagram of the structure of a storage cell string provided in the second embodiment is shown below. Figure 4 As shown, Figure 4 Compared to the example Figure 2 The difference in this embodiment is that the storage cell string further includes a first gating circuit and a second gating circuit. The weight storage string and the mask storage string are positioned between the first gating circuit and the second gating circuit. Only when both the first gating circuit and the second gating circuit are active can the storage cell string support outputting the calculation result, for example, outputting the calculation result to the connected bit line. Figure 4 In the middle, V SG This is the voltage used to drive the first and second gating circuits to conduct.

[0092] In the technical solution provided in this application embodiment, the weight storage string is connected to the source line through a dedicated second gating circuit, which can ensure that when each weight storage string participates in the calculation, the conduction state between it and the source line is controlled by the corresponding second gating circuit alone, thus avoiding signal interference or potential crosstalk between different weight storage strings due to sharing the source line path.

[0093] In some embodiments, a weight storage string includes a plurality of weight storage cells sequentially connected in series along a direction perpendicular to the substrate; each weight storage cell is used to store weight data of a corresponding bit in a weight vector; a mask storage string includes at least one mask storage cell; each mask storage cell is used to store mask data of a corresponding bit in a mask vector; when the mask storage string includes a plurality of mask storage cells, the mask storage cells are sequentially connected in series along a direction perpendicular to the substrate; wherein, the storage array further includes: a plurality of weight word lines and at least one mask word line. In each weight storage string, weight storage cells with the same sequence number along the direction perpendicular to the substrate are arranged on the same layer and connected to the same weight word line; in each mask storage string, mask storage cells with the same sequence number along the direction perpendicular to the substrate are arranged on the same layer and connected to the same mask word line.

[0094] exist Figure 3 In the illustrated embodiment, the mask storage string is disposed between the weight storage string and the bit line. In other embodiments, the weight storage string is disposed between the mask storage string and the bit line.

[0095] Please see Figure 3 In mask layer WL1, all mask storage cells are connected to the same mask word line. For example, the control terminals (e.g., gates) of all mask storage cells in mask layer WL1 are connected to the same mask word line, so that all mask storage cells in mask layer WL1 can be controlled to be in the selected or on state through a single mask word line. For weight layers WL2 to WL... n In any weight layer (where n is an integer greater than or equal to 3), all weight memory cells in that weight layer are connected to the same weight word line. For example, the control terminals (e.g., gates) of all weight memory cells in any weight layer are connected to the same weight word line, so that all weight memory cells in that weight layer can be controlled to be in a selected or on state through a single weight word line.

[0096] In the technical solution provided in this application embodiment, by applying a corresponding voltage signal to a mask word line, all mask storage cells in the corresponding mask layer can be synchronously controlled to enter the selected state or the on state, without the need to control each mask storage cell in the mask layer individually, which greatly reduces the number of control signals and the wiring complexity; by applying a corresponding voltage signal to a weight word line, all weight storage cells in the corresponding weight layer can be synchronously controlled to enter the selected state or the on state, without the need to control each weight storage cell in the weight layer individually, which greatly reduces the number of control signals and the wiring complexity.

[0097] In some embodiments, mask data in multiple mask storage cells of the same layer constitute a mask pattern. The mask pattern includes passable areas and complementary shielding areas; the passable and shielding areas are distributed in different positions in different mask patterns. Any two mask patterns are orthogonal. At least one mask storage cell in each mask storage string includes at least one data point indicating the calculation result of the output weight vector.

[0098] The mask pattern in this embodiment can be a mask matrix (size A×B), where the element values ​​can be binary data (0 or 1) or multi-valued values. In other embodiments, the mask pattern may include A×B positions, some of which contain a mask, and the positions containing the mask correspond to positions in the mask matrix where the element value is 1 or 0. In still other embodiments, the mask pattern may include A×B positions, and the mask pattern contains different types of masks, with different types of masks corresponding to multiple values ​​in the matrix.

[0099] For example, each of the multiple mask layers (i.e., multiple mask storage units) stores one mask pattern. That is, there is a one-to-one correspondence between the multiple mask layers and the multiple mask patterns. For example, any two mask patterns are orthogonal, and the multiple mask patterns are structurally complementary. For example, the multiple mask patterns are mask patterns with a cyclic matrix structure.

[0100] Figure 5 A schematic diagram illustrating the relationship between the mask storage unit of the mask layer and the weight storage unit of the weight layer provided in some embodiments, such as... Figure 5 As shown, as the position of the mask data with a value of 1 (i.e., mask 1) in the mask matrix corresponding to the mask storage unit of the mask layer changes, the position of the corresponding active weight storage unit also changes accordingly. Conversely, as the position of the mask data with a value of 0 (i.e., mask 0) in the mask matrix changes, the position of the corresponding masked weight storage unit also changes accordingly. Each mask storage unit in the mask layer corresponds one-to-one with a weight storage unit in the weight layer, and these weight storage units are labeled U1 to U16. The active weight storage units illustrated in U1 to U16 can be the weight storage units corresponding to the mask storage units with mask data of 1 in the selected mask layer, and the masked weight storage units illustrated in U1 to U16 can be the weight storage units corresponding to the mask storage units with mask data of 0 in the selected mask layer. The active weight storage unit is the weight storage unit whose weight vector calculation result can be output from the weight storage unit in the storage unit string. The masked weight storage unit is the weight storage unit whose weight vector calculation result cannot be output from the weight storage unit in the storage unit string.

[0101] In the technical solution provided in this application embodiment, the mask data in multiple mask storage units in the same layer constitute a mask pattern. In this way, multiple mask storage units correspond one-to-one with multiple mask patterns, and each layer of mask storage unit stores a mask pattern. Therefore, when a read voltage is applied to a certain layer of mask storage unit, the calculation result of whether each storage unit string outputs the weight vector can be controlled according to the mask pattern stored in that layer of mask storage unit, thereby improving the effectiveness of control.

[0102] Figure 6 A schematic diagram of the memory structure provided in the third embodiment is shown below. Figure 6 As shown, the memory includes not only the aforementioned storage array, but also peripheral circuitry (also known as control circuitry). The peripheral circuitry includes, for example, weight driving and decoding circuitry, and mask driving and decoding circuitry.

[0103] The weight driving and decoding circuit, connected to the weight word line, outputs a read voltage to the target weight word line to select the weight storage cell connected to the target weight word line. The weight data stored in the selected weight storage cell is used to determine the calculation result. The target weight word line may include one or more weight word lines.

[0104] The mask driving and decoding circuit, connected to the mask word line, outputs a read voltage to the target mask word line to select the mask storage cell connected to the target mask word line. The selected mask storage cell is used to output the calculation result to the bit line or to reset the calculation result to zero. The target mask word line may include one or more mask word lines.

[0105] In some embodiments, the weight driving and decoding circuit is further configured to output a turn-on voltage to other weight word lines besides the target weight word line, so as to turn on the weight storage cell connected to other weight word lines.

[0106] In some embodiments, the weight driving and decoding circuit is further configured to output a voltage to the first gating circuit for driving the first gating circuit to conduct, and also to output a voltage to the second gating circuit for driving the second gating circuit to conduct.

[0107] In some embodiments, the mask driving and decoding circuit is further configured to output a conduction voltage to other mask word lines besides the target mask word line, so as to put the mask storage cell connected to other mask word lines into a conducting state.

[0108] In some embodiments, the weight driving and decoding circuit is further used to obtain the weight layer address, determine the corresponding weight word line number based on the weight layer address, and determine the target weight word line based on the weight word line number.

[0109] In some embodiments, the mask driving and decoding circuit is further used to obtain the mask layer address, determine the corresponding mask word line number based on the mask layer address, and determine the target mask word line based on the mask word line number.

[0110] For example, the weight layer address and / or mask layer address can be determined based on the current computation task.

[0111] In the technical solution provided in this application embodiment, a mask driving and decoding circuit is additionally set on the basis of the weight driving and decoding circuit, so as to realize the state control of the mask storage unit in the mask layer, without the need to improve the existing weight driving and decoding circuit. This not only improves the effectiveness of the control of the mask storage unit, but also reduces the design complexity of the peripheral circuit of the memory.

[0112] exist Figure 6In the illustrated embodiment, the peripheral circuitry further includes a mask selection circuit. The mask selection circuit is connected to the mask driving and decoding circuits, and is used to select a target mask pattern from a pre-programmed mask pattern set in response to mask calculation requirements, and output a read voltage to the target mask word line based on the target mask pattern.

[0113] In the technical solution provided by the embodiments of this application, by adding a mask selection circuit, a target pattern can be selected from the mask pattern group according to the calculation requirements, and a read voltage is output to the target mask word line based on the target mask pattern. Therefore, the target mask pattern can be quickly switched according to the calculation task without reprogramming the mask data in the mask storage unit, thus avoiding the problem of calculation delay caused by reprogramming the mask data and improving the calculation efficiency of memory calculation.

[0114] Figure 7 A schematic diagram of the memory structure provided in the fourth embodiment is shown below. Figure 7 As shown, the memory includes a storage array and peripheral circuitry. The peripheral circuitry includes input / output control circuitry, drive and decoding circuitry, sensitive amplifier, current detection and quantization circuitry, page buffer, and mask selection circuitry.

[0115] Figure 8 A schematic diagram of the memory structure provided in the fifth embodiment is shown below. Figure 8 As shown, the memory includes a storage array 10 and peripheral circuitry 20. Peripheral circuitry 20 includes a weight driving and decoding circuit (included in the driving and decoding circuitry), a mask driving and decoding circuit (included in the driving and decoding circuitry), and other circuits. These other circuits include the aforementioned input / output control circuitry, sensitive amplifier, current detection and quantization circuitry, page buffer, and mask selection circuitry, etc.

[0116] For example, such as Figure 8 As shown, the storage array 10 includes a bit line layer 12, a first gating layer 13, a mask layer 14, a weight layer 15, a second gating layer 16, and a source line 17 stacked in sequence. Figure 8 The number of mask layers 14 and weight layers 15 shown is only for example. The actual number of mask layers 14 and weight layers 15 can be flexibly set according to the actual situation.

[0117] For example, the input / output control circuit acts as a communication bridge between the storage array and external devices, primarily responsible for the bidirectional transmission of data and instructions. When an external device needs to send computational tasks, input data, or control commands to the array, this circuit converts this information into electrical signals that the array can recognize. Simultaneously, it also converts the computational results output by the array or the stored data read into a format that the external device can understand and transmits it out, ensuring smooth and orderly interaction between the internal and external systems.

[0118] For example, the electrical signals generated by the memory array are often very weak and easily distorted by noise. The sensitive amplifier can capture these weak signals and amplify them to make them clear and stable, ensuring that subsequent circuits can accurately identify the information represented by the signal, such as distinguishing between different levels representing "0" and "1".

[0119] In in-memory computation, the result of multiplication and accumulation in the analog domain is usually represented by the magnitude of the current on the bit line. The current detection and quantization circuit first accurately detects the value of this current and then converts it into a digital signal. This process is like converting a continuously changing current into a discrete value, which facilitates subsequent digital domain operations and processing, such as performing combined calculations like shifting and summing.

[0120] Please see Figure 3 For any string of memory cells in the memory array, if both the first and second gating circuits are turned on, the voltage applied across the string of memory cells is 1 (e.g., the voltage applied to the first gating circuit). An exemplary method for determining the multiplication-accumulation result is illustrated below:

[0121] If the mask data stored in a mask storage cell in the selected state is 1 (represented by the conductance value of the mask storage cell being 1), and the weight data stored in a weight storage cell in the selected state is 1 (represented by the conductance value of the weight storage cell being 1), then the current output by the string of storage cells is 1 / 2 (i.e., the product of the voltage applied to the first gating circuit and the total conductance of the string of storage cells), and the output current is the current representing the calculation result of the weight vector. For example, the current detection and quantization circuit can determine the calculation result of the weight vector based on the magnitude of the current, the mask data in the mask storage cell, and the weight data stored in the weight storage cell.

[0122] If the mask data stored in a mask storage cell in the selected state is 0, then regardless of whether the weight data stored in a weight storage cell in the selected state is 0 or 1, the string of storage cells will have no output current, that is, the calculation result of the weight data will be masked.

[0123] For each bit line, since the multiple memory cells it connects to are connected in parallel, if multiple currents are obtained, the sum of the multiple currents (analog domain) is output. The sum of the multiple currents in the analog domain is the result of multiplication and accumulation in the analog domain.

[0124] During storage operations, the page buffer temporarily stores a page of data, such as weight data to be written to the array or information just read from the array, to prevent frequent data exchange with external systems that could reduce efficiency. During computation, it can also cache intermediate calculation results, accumulating them until a certain amount is reached before proceeding to the next step, thus acting as a buffer and temporary storage mechanism to improve overall operational efficiency.

[0125] In some embodiments, the mask storage unit is composed of redundant storage units in the storage array.

[0126] In some embodiments, to avoid the influence of the gate on the information of the storage cell, the storage cells adjacent to the gate are usually set as redundant storage cells to achieve reliable storage.

[0127] Therefore, in related technologies, storage arrays not only include multiple layers of weighted storage units, but also at least one layer of redundant (dummy) storage units. The embodiments of this application utilize redundant storage units, obtaining at least one layer of masked storage units by programming mask data into some or all of the redundant storage units.

[0128] In the technical solution provided in this application embodiment, the mask storage unit is composed of redundant storage units in the storage array, thereby avoiding the impact on the weight storage unit. This ensures that the weight storage unit can focus on storing and participating in the weight data required for calculation, ensuring that the integrity and stability of the weight data are not affected. At the same time, there is no need to open up new storage space to specifically place mask information. Thus, without increasing hardware costs or sacrificing the original storage density of the storage array, precise control of the calculation path is achieved, effectively improving the resource utilization efficiency and overall reliability of the storage array in the in-memory calculation process.

[0129] In some embodiments of this application, a 3D-NAND in-memory computation method and its hardware architecture based on a preprogrammed mask mechanism are disclosed. This method achieves configurable matrix multiplication computation without changing the existing structure of the 3D-NAND array itself, and significantly improves the performance of in-memory computation in edge neural network acceleration tasks such as large model inference.

[0130] The basic principle of this application's embodiments is based on the serial characteristics of 3D-NAND memory structures. In 3D-NAND, multiple cells on the same memory string are connected in series, and current is only generated on the string when all selected memory cells are in the ON state. Therefore, when each cell stores single-bit weight information representing "ON" or "OFF", the process is essentially equivalent to implementing an AND operation in the memory array, i.e., calculating the Hadamard product in the Boolean logic sense.

[0131] Based on this principle, this application proposes introducing a type of pre-programmed masked storage cell in the storage array to physically gate the computation path of the weighted storage cell. Since the final calculation result of the analog domain multiplication and accumulation operation is the sum of the storage string current on the bit lines, by using the pre-coding unit to control the on / off state of the storage string current, this method can flexibly control the activation and masking of the summation path without introducing crosstalk.

[0132] The storage cells are functionally divided into two categories: one for storing weight information in the neural network, and the other for gating. The former is called a weight transistor (included in the aforementioned weight storage cell), and the latter is called a mask transistor (included in the aforementioned mask storage cell). By simultaneously applying a read voltage to the gates of the target weight transistor (included in the target weight storage cell, which is the weight storage cell connected to the target weight word line) and the target mask transistor (included in the target mask storage cell, which is the mask storage cell connected to the target mask word line), the mask transistor can determine whether the weight storage cell it controls participates in the calculation of the current round. If the value stored in the mask transistor is 1, the corresponding weight storage cell can participate in the calculation normally; if the value stored in the mask transistor is 0, even if the corresponding weight storage cell is selected, the path will not contribute to the in-memory multiplication and accumulation operation. This mechanism can not only effectively avoid crosstalk problems when multiple selected cells share input and output paths, but also achieve precise path-level control while maintaining high parallel activation.

[0133] It should be noted that the information stored by mask transistors is not limited to binary, but rather has the capability to support multiple values. However, even if the mask storage cell itself stores multiple values ​​of information, its gating behavior can exhibit both on and off states. The gating behavior can be dynamically switched by adjusting the gate read voltage, thereby giving the same set of mask transistors multiple gating effects. Similarly, weight transistors can also support the storage of multiple weight values.

[0134] To further improve resource utilization efficiency, this application proposes a spatial reuse mechanism that allows mask layers to be reused across multiple computation cycles. This means that the same set of mask storage units can participate in inference calculations for multiple layers of different weight matrices. Furthermore, this application proposes a programmable storage mechanism for mask information (i.e., the aforementioned mask data) in a dummy word line layer within the array, avoiding the occupation of additional effective weight storage resources and thus significantly improving the overall effective storage density of the array. This application is highly compatible with traditional 3D-NAND read mechanisms at the operational level, requiring no significant modifications to the existing control logic framework and demonstrating good engineering feasibility.

[0135] This application also proposes a complementary mask group mechanism for efficient parallel traversal and computation of multiple weight matrices within a single array, and to mask unwanted activations on non-target paths. This mechanism predefines and stores a set of orthogonal, structurally complementary mask patterns and maps them to multiple redundant word line layers (i.e., multiple mask layers). This allows the control logic to dynamically select different mask modes as needed, precisely activating the summation path required for the current task. The mask patterns can be designed using a cyclic matrix structure, ensuring that each mask group, while activating a specific weight subset, covers all weight matrices within the computation cycle, achieving both fast and complete scanning. Compared to the traditional serial processing method of weight matrix activation, this mechanism can, in optimal conditions, reduce the overall computation time to the order of its square root, significantly improving computational throughput.

[0136] To support the effective operation of the aforementioned computation mechanism, this application also proposes a complete hardware implementation scheme. In addition to conventional 3D-NAND Flash peripheral circuits such as sensitive amplifiers, input / output control, and driver circuits, it also includes mask driving and decoding circuits, current detection and quantization circuits, and mask selection circuits. The conventional 3D-NAND Flash peripheral circuits include input / output control circuits, sensitive amplifiers, page buffers, and weight driving and decoding circuits. The mask driving and decoding circuits support independent gating of mask layer word lines, allowing mask storage units and weight storage units to be selected in parallel. The current detection and quantization circuit is connected to the array bit line output and is used to detect and quantize the analog current results. In some embodiments, it can also perform merging calculations such as shifting and summing. The mask selection circuit is responsible for dynamically selecting the appropriate mask layer according to the current task and controlling its participation in the current round of computation, ensuring the controllability of the computation path and the flexibility of task scheduling.

[0137] To address the need for simultaneous selection of two cell layers in a 3D-NAND array (i.e., the mask storage cell of the mask layer and the weight storage cell of the weight layer), this application proposes a novel 3D-NAND hardware driver circuit design that supports independent control of the mask layer and weight layer in the 3D-NAND. This scheme equips the array with two independent driving and decoding circuits (weight driving and decoding circuit, and mask driving and decoding circuit), which can independently convert the corresponding mask layer address and weight layer address into the corresponding word line number. This allows for the application of the appropriate word line selection voltage during read or calculation operations, while ensuring that other word line gates are subjected to on-state voltage.

[0138] In summary, the embodiments of this application provide a programmable masking mechanism and its corresponding hardware system architecture for large-scale neural network inference tasks on the edge, which has the following significant advantages: First, it supports the parallel deployment and computation of multiple weight matrices, improving array storage and computation efficiency; second, it is fully compatible with current mainstream 3D-NAND manufacturing processes and has good feasibility; finally, by providing computation path control at the level of each storage cell string, it provides a flexible and efficient computational foundation for new neural networks such as sparse networks represented by hybrid expert models, expanding the application boundaries of in-memory computing in edge computing scenarios.

[0139] The memory array used in this application employs a typical 3D-NAND architecture, consisting of numerous vertically aligned strings of memory cells. The core of each string is a series of memory cells (e.g., FLASH cells), and data writing and erasing are achieved by adjusting the threshold voltage of these cells. All memory cells on the same layer of the array share the same word line, which allows for unified control of the gates of all devices on that layer. Each string of memory cells also has an upper select transistor and a lower select transistor at its top and bottom ends, respectively controlled by the upper and lower select transistor control gate lines, with the TSG extending along the X-direction. The memory cell string extends upwards via bit lines and downwards via source lines.

[0140] In traditional current-domain in-memory computing architectures, computation operations are typically based on selecting a word line and applying a read voltage. The multiply-accumulate calculation is achieved by applying an on / off voltage as the input vector to the control gate (TSG) of the upper selector transistor and detecting the summation current at the bit line. In this strategy, the most basic memory unit is a storage unit that stores weights, and the most basic computation path is a string of storage units. The key to this embodiment is the introduction of a set of controllable pre-programmed mask storage units onto the storage unit string as additional logical decision conditions, thereby enhancing the control over whether each computation path participates in the summation calculation. Figure 3 As shown. The specific design of the mask layer is to select several layers of storage cells on the upper part of the storage array as mask storage cells ( Figure 3 In WL1, all other layers are still used to store weights ( Figure 3 WL2~WL n During computation, preset read voltages are applied simultaneously to the mask layer and the weight layer, forming a computation path on the memory string dominated by both memory cells. The read voltage... It should be between the two threshold voltage states to be distinguished, ensuring that the memory cell with the high threshold voltage is turned off and the memory cell with the low threshold voltage is turned on.

[0141] The above operations are implemented based on the in-memory AND logic proposed in the embodiments of this application. Please refer to [link / reference]. Figure 4 It is possible to apply read voltage to the gates of both the mask memory cell and the weight memory cell simultaneously. To ensure that the mask storage unit and the weight storage unit are selected, the storage unit string can only output current if and only if both storage units are selected and the data stored in both storage units is not zero, that is, the mask data of the selected mask storage unit is not zero (e.g., it is 1) and the weight data of the selected weight storage unit is not zero (e.g., it is 1). As the most basic calculation path, if the mask storage unit on the storage unit string is in a low threshold voltage state, the calculation path is turned on, and the read current depends entirely on the information stored in the weight storage unit and the corresponding TSG input voltage. At this time, the corresponding weight layer information is active. If the mask layer is in a high threshold voltage state, the calculation path is turned off without current. Regardless of what information the weight layer stores or what voltage the TSG input is, there is no current output on the storage string. At this time, the corresponding weight layer information is masked. In this embodiment of the application, for the sake of convenience of description, the storage unit can be set to work in Single-Level Cell (SLC) mode, that is, there are only two possible states: programming state and erase state. However, this simplification does not mean that this application is limited to this.

[0142] To meet the requirement of efficient calculation of stored weight information, embodiments of this application further introduce a complementary mask group design. A schematic diagram of this design is shown below. Figure 5 As shown, the upper layer is the mask layer, and the lower layer is the weight layer. Each sub-block / element (e.g.) , ... Each weight layer represents a computational unit capable of independently performing multiplication and addition operations; that is, one weight layer corresponds to one weight matrix. Correspondingly, the mask layer consists of several binary storage units, arranged strictly aligned with the weight matrix below. When a mask storage unit is set to 1, it indicates that the corresponding weight storage unit below is active in the current computation cycle, and its corresponding multiplication result will participate in the addition operation on the bit lines; conversely, a value of 0 indicates that the corresponding weight storage unit below is masked. For the 16-element weight matrix shown in the figure, in the most extreme case of related technologies (where all weight matrices have different inputs), completing all calculations requires 16 computation cycles. In this embodiment, to shorten the total computation time, these matrices are divided into groups of four along the diagonal, and all calculations are completed in four cycles. To match this design goal, a corresponding mask pattern needs to be designed for each group of matrices, and all these patterns satisfy the following mathematical relationship:

[0143] ;

[0144] Where U represents the union, Representing the A mask pattern (i.e., a mask matrix). The value range is from 1 to n. This represents a square matrix where all elements are 1. Different mask patterns do not overlap, meaning they are spatially orthogonal, and can collectively cover all weight matrices.

[0145] Figure 5 This demonstrates a set of four mask patterns that meet the above requirements. Each mask pattern activates a spatial subset of the weight matrix and does not overlap with other patterns. All mask patterns jointly cover the entire weight layer, ensuring that each sub-block belongs to exactly one set of activation paths. Any one of these mask patterns can be loaded as needed to precisely activate the weight matrix required for the current computation task. Unselected weight matrices are not included in the calculation, thus preventing crosstalk to the results. Figure 5 As shown, after introducing the pre-programmed mask structure, the worst-case computation delay was successfully reduced to 4 cycles. The aforementioned mask information can be written through a standard memory cell programming process to realize the pre-programmed mask structure. This process can be performed simultaneously with or sequentially with weight programming, implemented through layer-by-layer programming. The bias voltage and circuitry used in the writing process are similar to those of conventional programming methods, and will not be described again in the embodiments of this application.

[0146] like Figure 7 As shown in the embodiments of this application, current detection and quantization circuits, mask selection circuits, etc., are added to the peripheral circuits of 3D-NAND. The current detection and quantization circuit includes a digital-to-analog converter, a shift-and-sum circuit, etc., used to detect the analog summation current on the bit lines and convert it into digital values, and then perform shift-and-sum normalization on the digital values ​​on different bit lines. The mask selection circuit is mainly responsible for dynamically selecting the appropriate mask layer according to the current task and controlling its participation in the current round of computation, ensuring the controllability of the computation path and the flexibility of task scheduling.

[0147] like Figure 8 As shown, modifications were made to the array ladder structure and the driving and decoding models in the peripheral circuitry. Specifically, the word line leads in the array are divided into two parts according to the mask layer and the weight layer, and are respectively connected to independent weight driving and decoding circuits and mask driving and decoding circuits in the peripheral circuitry via hybrid bonding technology. The mask driving and decoding circuits are used to add the ability to select and access the word lines of the mask layer, allowing programming and reading operations to also affect the mask layer word line units. The mask driving and decoding circuits are independent of the weight layer word line driving circuits; they can operate independently or together, enabling simultaneous reading of the mask layer and weight layer.

[0148] Figure 9 A flowchart illustrating an in-memory computation method provided in some embodiments, such as Figure 9 As shown, the method is applied to the memory shown in any of the above embodiments, and the method includes:

[0149] S901. Program the weight vector into the weight storage string in memory.

[0150] S902. Program the mask vector into the mask storage string in memory.

[0151] In some embodiments, S901 and S902 may be performed simultaneously. In other embodiments, S901 may be executed before or after S902.

[0152] S903, response mask vector and weight vector, controls whether the storage cell string in the memory outputs the calculation result of the weight vector.

[0153] In some embodiments, both the weight vector and the mask vector can be generated according to computational requirements.

[0154] In some embodiments, the in-memory calculation method further includes: responding to a mask pattern, controlling at least one memory cell string to output a calculation result, and each bit line performing an addition calculation based on the calculation result output by the at least one memory cell string. In some embodiments, each bit line can send the result of the addition calculation to a current detection and quantization circuit in an external circuit.

[0155] In one exemplary embodiment, Figure 10This is a schematic diagram of the structure of an electronic device provided for some embodiments. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the electronic device is used for exchanging information between the processor and external devices. The communication interface of the electronic device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wireless Fidelity (WIFI), mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an in-memory computing method. The display unit of the electronic device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the electronic device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the electronic device, or external keyboards, touchpads, or mice, etc.

[0156] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0157] For example, an electronic device includes the memory found in any of the above embodiments.

[0158] For example, electronic devices may include one or more of the following: USB flash drives, portable hard drives, servers, smart IoT devices, mobile phones, devices in autonomous driving systems, tablets, computers with transceiver capabilities, handheld computers, desktop computers, personal digital assistants, portable media players, smart speakers, navigation devices, smartwatches, smart glasses, smart necklaces and other wearable devices, pedometers, digital TVs, virtual reality (VR) devices, augmented reality (AR) devices, devices in industrial control, devices in self-driving, drones, devices in remote medical surgery, devices in smart grids, devices in transportation safety, devices in smart cities, devices in smart homes, vehicles, in-vehicle devices, in-vehicle modules, etc.

[0159] The functional circuits in any embodiment of this application may include integrations of one or more of the following: general-purpose processors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), central processing units (CPUs), graphics processing units (GPUs), embedded neural network processing units (NPUs), controllers, microcontrollers, microprocessors, programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A memory, characterized in that, include: Storage array; The storage array includes multiple sets of storage cell strings spaced apart along a first direction; each set of storage cell strings includes at least two storage cell strings spaced apart along a second direction, the second direction being parallel to and intersecting the substrate with the first direction. The storage cell string includes a weighted storage string and a mask storage string that are sequentially connected in series along the direction perpendicular to the substrate. The weight storage string includes multiple weight storage units sequentially connected in series along the direction perpendicular to the substrate; each weight storage unit is used to store the weight data of the corresponding bit in the weight vector; The mask storage string includes at least one mask storage unit; each mask storage unit is used to store mask data of corresponding bits in the mask vector; when the mask storage string includes multiple mask storage units, each mask storage unit is sequentially connected in series along the direction perpendicular to the substrate; the mask vector is a pre-programmed logical determination vector used to define whether the storage unit string outputs the calculation result of the weight vector; The storage array further includes: Multiple weight word lines; weight storage cells with the same sequence number along the vertical substrate direction in each weight storage string are arranged on the same layer and connected to the same weight word line; At least one mask word line; the mask storage cells with the same sequence number along the vertical substrate direction in each of the mask storage strings are arranged on the same layer and connected to the same mask word line; The memory also includes: A weight driving and decoding circuit is connected to the weight word line and is used to output a read voltage to the target weight word line so that the weight storage unit connected to the target weight word line is in a selected state. The weight data stored in the selected weight storage unit is used to determine the calculation result. The mask driving and decoding circuit is connected to the mask word line and is used to output a read voltage to the target mask word line so that the mask storage unit connected to the target mask word line is in a selected state. The selected mask storage unit is used to output the calculation result to the bit line or to reset the calculation result to zero.

2. The memory according to claim 1, characterized in that, The storage array also includes: Multiple bit lines are spaced apart along the first direction; Multiple first-selection circuits, arrayed together; In this group of storage cell strings, each of the mask storage strings is connected to a bit line through the corresponding first gating circuit; The storage cell string is used to: when the first gating circuit is turned on, respond to the mask vector and the weight vector, output the calculation result to the bit line or reset the calculation result to zero.

3. The memory according to claim 2, characterized in that, The storage array also includes: The source line and the array of multiple second gating circuits are all located on the side of the weight storage string that is away from the mask storage string; Each of the weight storage strings is connected to the source line through a corresponding second gating circuit; The storage unit string is used to: when both the first gating circuit and the second gating circuit are turned on, respond to the mask vector and the weight vector, output the calculation result to the bit line or reset the calculation result to zero.

4. The memory according to any one of claims 1-3, characterized in that, The mask data in multiple mask storage units in the same layer constitute a mask pattern; the mask pattern includes a pass area and a shielding area complementary to the pass area. The distribution positions of the pass area and the shielding area in different mask patterns are different. Any two mask patterns are orthogonal. The mask data in at least one mask storage unit of each mask storage string includes at least one piece of data for indicating the calculation result of the output weight vector.

5. The memory according to any one of claims 1-3, characterized in that, Also includes: A mask selection circuit, connected to the mask driving and decoding circuit, is used to respond to mask calculation requirements, select a target mask pattern from a pre-programmed mask pattern group, and output the read voltage to the target mask word line based on the target mask pattern.

6. The memory according to any one of claims 1-3, characterized in that, The mask storage unit is constructed by reusing redundant storage units in the storage array.

7. An in-memory computation method, characterized in that, Applied to the memory as described in any one of claims 1 to 6, the method comprises: Program a weight vector into the weight storage string in the memory; Program a mask vector into the mask storage string in the memory; In response to the mask vector and the weight vector, the system controls whether the storage cell string in the memory outputs the calculation result of the weight vector.

8. An electronic device, characterized in that, The electronic device includes a memory as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • In-memory computing architecture and operating policy thereof

    CN119847983A

  • In-memory computing memory device and in-memory computing method

    CN119964621A