Memory device and operating method thereof

Abstract

A method of operating a memory device is provided, the method comprising: generating a weight feature to be stored in a second memory different from the first memory based on at least one weight stored in a first memory, wherein the weight feature is associated with the number of repeating bits in the at least one weight that are adjacent to and the same as the most significant bit; and accessing the first memory and the second memory according to the weight feature and the address of the at least one weight to transfer the at least one weight to multiplication and accumulation circuitry for operation of a first neural network layer.

Description

Prior Art

[0001] Near-memory compute macros, designed to improve energy efficiency by performing computations close to memory storage, face significant challenges in the context of convolutional neural networks (CNNs). The problem lies in the significant energy consumption associated with memory readouts, with nearly half of the total energy dedicated to data retrieval. In CNNs, all inputs within a layer correspond to the same set of weights, resulting in repeated memory reads of the same data. This redundancy in data retrieval significantly increases energy consumption. Simple diagram description

[0002] The various aspects of the present disclosure will be best understood when the following detailed description is read in conjunction with the accompanying drawings. It should be noted that, in accordance with standard practice in the industry, various features are not drawn to scale. In fact, the dimensions of various features may be increased or decreased for clarity of discussion. FIG. 1 is a schematic diagram of a memory device according to some embodiments of the present disclosure. FIG. 2 is a schematic diagram of a portion of the memory device 10 corresponding to FIG. 1 according to some embodiments of the present disclosure. FIG. 3 is a flowchart of a method for operating the memory device shown in FIG. 1 and FIG. 2 according to some embodiments of the present disclosure. FIG4A is a timing diagram of neural network layer operations corresponding to short channels according to some embodiments of the present disclosure. FIG. 4B illustrates operations and signal waveforms of the memory device 10 corresponding to FIG. 4A according to some embodiments of the present disclosure. FIG5A is a timing diagram illustrating the operation of a neural network layer corresponding to a medium channel according to some embodiments of the present disclosure. FIG. 5B illustrates operations and signal waveforms of the memory device 10 corresponding to FIG. 5A according to some embodiments of the present disclosure. FIG6A is a timing diagram of neural network layer operations corresponding to long channels according to some embodiments of the present disclosure. FIG. 6B illustrates operations and signal waveforms of the memory device 10 corresponding to FIG. 6A according to some embodiments of the present disclosure. FIG. 7 illustrates operations and signal waveforms of the memory device 10 during a neural network layer operation corresponding to a short channel according to some embodiments of the present disclosure. FIG. 8 illustrates operations and signal waveforms of the memory device 10 during a neural network layer operation corresponding to a medium channel according to some embodiments of the present disclosure. FIG. 9 illustrates operations and signal waveforms of the memory device 10 during a neural network layer operation corresponding to a long channel according to some embodiments of the present disclosure. FIG. 10 illustrates the operation of the memory device 10 in a neural network layer operation according to some embodiments of the present disclosure. FIG. 11 illustrates the operation of the memory device 10 in a neural network layer operation according to some embodiments of the present disclosure. Implementation Method

[0003] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components, materials, values, steps, arrangements, or the like are described below to simplify the disclosure. Of course, these are merely examples and are not intended to be limiting. Other components, materials, values, steps, arrangements, or the like are contemplated. For example, the following description of a first feature being formed over or on a second feature may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features may be formed between the first and second features, thereby preventing the first and second features from being in direct contact. Furthermore, the disclosure may reuse reference numbers and / or letters in various examples. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed.

[0004] Furthermore, for ease of description, spatially relative terms such as "below," "beneath," "lower," "above," "upper," and similar terms may be used herein to describe one element or feature in relation to another element or feature illustrated in the figures. These spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein should be interpreted accordingly. The terms mask, photolithography mask, reticle, and mask plate are used to refer to the same item.

[0005] Terms used in the following description and throughout the claims generally have their ordinary meanings as determined in the art or in the specific context in which each term is used. Those skilled in the art will appreciate that a component or process may be referred to by different names. The many different embodiments described in detail in this specification are merely illustrative and in no way limit the scope or spirit of the present disclosure or any illustrative terms.

[0006] It is important to note that terms such as "first" and "second" used herein to describe various elements or processes are intended to distinguish one element or process from another. However, the elements, processes, and their order should not be limited by these terms. For example, a first element could be referred to as a second element, and similarly, a second element could be referred to as a first element without departing from the scope of this disclosure.

[0007] In the following discussion and claims, the terms "comprising," "including," "containing," "having," "involving," and similar terms should be understood as open-ended, i.e., to include, but not limited to. The term "and / or" as used herein includes any one of the associated listed items and all combinations of one or more of the associated listed items, and is not mutually exclusive.

[0008] According to some embodiments, the present application relates to a method for optimizing weight reading in neural network systems by exploiting the inherent properties of a zero-centered Gaussian weight distribution. This distribution tends to show a higher frequency in sequences of numbers characterized by repeated 0s or 1s. Specifically, relatively small weight values ​​are common, resulting in many leading 1s or 0s in two's complement representation.

[0009] The method was introduced to take advantage of a specific characteristic of the weighted data, namely the recurring most significant bit (MSB). This approach aims to selectively skip reading certain bits during the search process, thereby reducing the energy consumption associated with reading each bit.

[0010] The method includes extracting a weight signature during an initial readout, characterized by a run-length of the MSB exceeding a predetermined threshold. This information is then encoded and stored in a memory address, forming a record of the weight signature. In certain embodiments, the weight signature varies depending on the number of weights accessed across different neural network layers. During subsequent readouts, if the read address matches the declared weight signature, bits associated with the predetermined run-length threshold are systematically omitted. This strategic omission effectively reduces the overall read energy per bit, helping to improve the efficiency of neural network operations.

[0011] Reference is now made to FIG. FIG. 1 is a schematic diagram of a memory device 10 according to some embodiments of the present disclosure. In some embodiments, the memory device 10 is configured as a compute-in-memory (CIM) system for neural network operations. For purposes of illustration, the memory device 10 includes a memory 101, a word line driver 120, a control circuit 130, a bit line multiplexer 140, an input / output circuit 150, a weight feature read circuit 155, and a multiply and accumulate (MAC) circuit 160.

[0012] In some embodiments, memory 101 includes a memory array 110 comprised of a plurality of bit cells, referred to as memory cells. Memory cells are located at the intersections of columns and rows in 110. In some embodiments, memory array 110 may be a non-volatile memory array and include static random access memory (SRAM) cells. In various embodiments, memory array 110 includes resistance-based random access memory (RAM) cells. Resistance-based RAM may include resistive RAM (ReRAM), magnetoresistive RAM (MRAM), ferroelectric RAM (FeRAM), dielectric RAM, or any suitable array or combination of any suitable memory devices. In some embodiments, memory array 110 is configured to store a plurality of weights for access by a neural network.

[0013] The word line driver (WLDR) 120 is configured to generate word line signals in response to a control signal associated with an address, thereby driving a word line for accessing the memory array 110 to read bits from / write bits to the memory array 110, wherein the address indicates specific memory cells in the memory array 110 where bits are stored. Specifically, in some embodiments, the word line driver 120 selects and enables specific memory cells in the memory array 110 based on the address.

[0014] The control circuit 130 is configured to control the word line driver 120, the bit line multiplexer 140, the input / output circuit 150, the weight feature read circuit 155, and the multiplication and accumulation circuit 160 to perform conventional memory access (e.g., reading and writing specific addresses) and CIM operations. In some embodiments, the control circuit 130 includes an x-decoder for the word lines and a y-decoder for the bit lines and / or sense lines. The control circuit 130 also includes timing control for read and write operations. In some embodiments, the control circuit 130 is configured to generate control signals to the word line driver 120, the bit line multiplexer 140, the input / output circuit 150, and the multiplication and accumulation circuit 160 in response to the address for performing access operations (e.g., reading and writing operations to the memory array 110).

[0015] Bit line multiplexer (MUX) 140 is coupled to memory 101 and configured to enable rows of memory array 110 by selecting bit lines (BL) and / or sense lines based on control signals from control circuit 130. In some embodiments, bit line multiplexer 140 includes precharge circuitry. For example, during memory access, the precharge circuitry precharges bit lines for read operations.

[0016] Input / output (IO) circuitry 150 is configured to transmit data to be written to memory array 110 and / or read data stored in memory array 110. For example, IO circuitry 150 transmits weights stored in memory array 110 to weight signature readout circuitry 155 for further use. In some embodiments, IO circuitry 150 includes sense amplifier circuitry for input / output operations from memory array 110.

[0017] The weight characteristic read circuit 155 is coupled to the input / output circuit 150 and is configured to extract the weight characteristics of the weights in the memory 101. In some embodiments, the weight characteristic read circuit 155 is further configured to control the precharge circuitry in the bit line multiplexer 140 and the sense amplifier circuit in the input / output circuit 150 based on the extracted weight characteristics, so as to access the weights in the memory 101 and / or output the weights stored in the weight characteristic read circuit 155 during a weight read operation.

[0018] In some embodiments, the multiplication and accumulation circuit 160 provides functional units (eg, adders, multipliers, registers, etc.) for performing MAC operations based on the weights transmitted from the weight feature read circuit 155 .

[0019] Reference is now made to FIG. FIG. 2 is a schematic diagram of a portion of memory device 10 corresponding to FIG. 1 , according to some embodiments of the present disclosure. For ease of understanding, like reference numerals are used to represent like elements in FIG. 2 , as with the embodiment of FIG. For the sake of brevity, the detailed operations of like elements discussed in detail in previous paragraphs are omitted herein.

[0020] For purposes of illustration, the memory array 110 is coupled to local multiplexers and precharge circuits 141[0] and 141[1]. Each of the local multiplexers and precharge circuits 141[0] and 141[1] is coupled to one of the sense amplifier circuits 151[0] and 151[1]. In some embodiments, the local multiplexers and precharge circuits 141[0] and 141[1] are configured for, for example, the bit line multiplexer 140 of FIG. 1 . The local multiplexer and precharge circuit 141[0] includes an 8-to-1 multiplexer and precharge circuit coupled to bit lines BL0 to BL7 for 8-bit data BL[7:0]. The configuration of the local multiplexer and precharge circuit 141[1] is similar to that of the local multiplexer and precharge circuit 141[0]. Therefore, repeated descriptions are omitted here.

[0021] The sense amplifier circuits 151[0] and 151[1] are configured for example for the input / output circuit 150 of FIG. 1 and are configured to access the memory array 110. For example, the sense amplifier circuit 151[0] includes a sense amplifier controller SAC[0] and a sense amplifier VSA[0]. In some embodiments, the sense amplifier controller SAC[0] is configured to control the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] by an enable signal EN[0]. For example, the sense amplifier controller SAC[0] temporarily disables the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] from accessing the memory array 110 during a read operation in response to the disable signal Dis[0], and further resumes the read operation by enabling the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0]. In some embodiments, sense amplifier VSA[0] is configured to access memory array 110 to obtain weights from memory array 110 to generate readout signal SAOUT[0]. Sense amplifier circuit 151[1] is configured similarly to sense amplifier circuit 151[0]. Therefore, repeated descriptions are omitted here.

[0022] Sense amplifier circuits 151[0] and 151[1] are further coupled to weight read circuits WFAR[0] and WFAR[1], respectively. For purposes of illustration, weight read circuit WFAR[0] includes a weight read controller FARC, a weight extraction circuit BTF, a memory circuit 211, and an address decoder WAP. The configuration of weight read circuit WFAR[1] is similar to that of weight read circuit WFAR[0]. Therefore, repeated descriptions are omitted here.

[0023] The weight read controller FARC is coupled to the sense amplifier controller SAC[0] and is configured to generate a disable signal Dis[0] for the sense amplifier controller SAC[0] in response to the data f_c0[1:0] and the data Conf[7:0]. In some embodiments, the control circuit 130 of FIG1 is configured to generate the data Conf[7:0] according to the layer implementing the neural network operation. The detailed configuration is discussed with reference to FIG3 to FIG10.

[0024] The weight extraction circuit BTF of the weight read circuit WFAR[0] is coupled to the sense amplifier VSA[0] and further coupled to the memory circuit 211 via the switch S1. In some embodiments, the weight extraction circuit BTF of the weight read circuit WFAR[0] generates a signal FE_flag0 based on the read signal SAOUT[0] received from the sense amplifier VSA[0] and transmits the signal FE_flag0 to the memory circuit 211. In some embodiments, the weight extraction circuit BTF extracts a weight characteristic of the weight of the read signal SAOUT[0] to generate the signal FE_flag0.

[0025] Memory circuit 211 includes a control circuit FF-RWC and a flip-flop circuit WFB-DFF. The control circuit FF-RWC is coupled to the address decoder WAP of the weight read circuit WFAR[0] and multiplexers 221 and 222. The control circuit FF-RWC is configured to control read and write operations on the flip-flop circuit WFB-DFF based on the mapped address WA[3:0] and the data f_d0[1:0], and further control a reset operation of the flip-flop circuit WFB-DFF based on the signal rst. In some embodiments, the memory circuit 211 generates data f_c0[1:0] to the weight read controller FARC and generates data f_d0[1:0] to the multiplexer 221.

[0026] The address decoder WAP of the weight read circuit WFAR[0] is configured to generate a mapping address WA[3:0] based on the y-address YA[2:0] and the x-address XA[7:0] of the weight stored in the memory array 110. In some embodiments, the address decoder WAP indicates the location of the weight feature corresponding to the weight stored in the flip-flop circuit WFB-DFF. The detailed configuration is discussed in the following paragraphs.

[0027] As shown in the embodiment of FIG2 , the memory circuit 211 is coupled to the weight read controller FARC of the weight read circuit WFAR[0] via switch S2. The weight read controller FARC is further coupled to the multiplexer 221 via switch S3. The weight read controller FARC in the weight read circuit WFAR[1] is coupled to the multiplexer 221 via switch S5 and is coupled to the memory circuit 212 in the weight read circuit WFAR[1] via switch S7. The weight extraction circuit BTF of the weight read circuit WFAR[1] is coupled to the memory circuit 212 via switch S6.

[0028] Specifically, the weight read circuit WFAR[1] further includes a logic gate circuit 231. In some embodiments, the logic gate circuit 231 includes an AND gate 232 having a first input coupled to the weight extraction circuit BTF of the weight read circuit WFAR[0] and a second input coupled to the weight extraction circuit BTF of the weight read circuit WFAR[1]. The output of the AND gate 232 is coupled to the memory circuits 211 and 212 via switches S4 and S8, respectively. In some embodiments, the AND gate 232 is configured to generate the signal FE_Lflag based on the signal FE_flag0 and the signal FE_flag1.

[0029] Reference is now made to FIG. FIG. 3 is a flowchart of method 30 for operating the memory device shown in FIG. 1 and FIG. 2 , according to some embodiments of the present disclosure. It should be understood that, for alternative embodiments of method 30, additional operations / stages may be provided before, during, and after the process shown in FIG. 3 , and some of the operations / stages described below may be replaced or eliminated. Method 30 includes operations S310, S320, S330, and S340, and will be discussed in the following paragraphs with reference to FIG. 1-2 and FIG. 4A-10 .

[0030] In operation S310, as shown in Figures 3, 4A, 5A, and 6A, a read operation is performed on the memory array 110 to obtain the weight stored in the memory array 110 during the multiplication and accumulation operation period MAC OG1 of the neural network layer operation LN. For example, the weight W0 has 8 bits, which are represented as "00001101" and are stored in the memory cells of the memory array 110 coupled to the bit lines BL0 to BL7. The control circuit 130 controls the sense amplifier circuit 151[0] to access the memory array 110 to generate a readout signal SAOUT[0] indicating the weight.

[0031] In operation S320, a weight feature is generated based on the weight. In some embodiments, the weight read circuit (e.g., WFAR[0]) is configured to operate in response to the data Conf[7:0] to extract and store weight features of each layer in the neural network, wherein each layer has a different number of weights. For example, as shown in Table 1 below, a layer including a relatively small number of weights (e.g., 144 weights) is referred to as a short channel, and the weight read circuit WFAR[0] generates an 8-bit weight feature based on the weight. In another embodiment, a layer including a medium number of weights (e.g., 288 weights) is referred to as a medium channel, and the weight read circuit WFAR[0] generates a 2-bit weight feature. In yet another embodiment, a layer including a relatively large number of weights (e.g., 576 weights) is referred to as a long channel, and the weight read circuit WFAR[0] generates a 1-bit weight feature. aisle (layer) Short Channel Medium channel Long corridor Bits of weight features# 8-bit 2-bit 1 bit operate Storing weights in WFB-DFF WFB value Nc Reducing Access WFB value Nc Reducing Access 0 0 0 0 0 0 1 1 1 2 2 2 1 ≥2 2 3 ≥3 3 Table I The WFB value corresponds to the weight characteristic of the memory circuit to be stored in the weight read circuits WFAR[0] and / or WFAR[1]. The number Nc corresponds to a predetermined value associated with the number of repeated bits in the weight that are identical to the MSB and are located in adjacent positions. The number of reduced accesses corresponds to the number of bits in the weight that will not be read from the memory array 110 and will be discussed later with reference to Figures 7 to 9.

[0032] 4A-4B and Table I. For an embodiment in which weight signatures are extracted from the weights in a short-channel extraction phase (FE) after reading the weights from the memory array 110, adjacent weight read circuits (e.g., WFAR[0] and WFAR[1]) operate independently to generate weight signatures based on the weights in the data BL[7:0] (e.g., "00001101") and the weights in the data BL[15:8] (e.g., "11100001").

[0033] Taking the embodiment of the weight read circuit WFAR[0] as an example, the weight extraction circuit BTF receives the readout signal SAOUT[0] and transmits the readout signal SAOUT[0] as the flag signal FE_flag0 to the memory circuit 211 by turning on the switch S1. The control circuit FF-RWC further controls the flip-flop circuit WFB-DFF to store the entire 8-bit weight as the weight feature. Similarly, the weight extraction circuit BTF of the weight read circuit WFAR[1] receives the readout signal SAOUT[1] and transmits the readout signal SAOUT[1] as the flag signal FE_flag1 to the memory circuit 212 by turning on the switch S6. The control circuit FF-RWC in the memory circuit 212 further controls the flip-flop circuit WFB-DFF to store the entire 8-bit weight as a weight feature in a position of the flip-flop circuit WFB-DFF, where the position is pointed to by the corresponding mapping address WA[3:0] generated by the x-address XA[7:0] and y-address YA[2:0] of the weight.

[0034] Referring now to FIG. 5A-5B and Table 1, weight characteristics for extracting weights in the extraction phase (FE) of a mid-channel after reading the weights from the memory array 110 are shown. In some embodiments, the weight extraction circuit BTF is configured to generate a flag signal that changes from a first voltage level to a second voltage level in response to a bit change in the weight of the readout signal. The control circuit in the memory circuit is further configured to store the weight characteristics based on the flag signal.

[0035] For example, as shown in FIG5B , the sense amplifier VSA[0] transmits a readout signal SAOUT[0] characterized by a weight “00001101”. Specifically, the weight “00001101” has a most significant bit (MSB) “0” and three repeated bits “000” of the MSB at adjacent positions. The weight extraction circuit BTF adjusts the voltage level of the signal FE_flag0 from a low level to a high level in response to the change of the readout signal SAOUT[0], while the eighth bit [7] to the fifth bit [4] of the weight are pre-read as “0” and the fourth bit [3] is “1”. In some embodiments, according to Table I, when the number (3) of repeated bits identical to the MSB at adjacent positions in the weight is equal to a predetermined value Nc “3”, the control circuit FF-RWC further controls the flip-flop circuit WFB-DFF in response to the signal FE_flag0 to store a 2-bit weight characteristic having “11”.

[0036] In various embodiments, as shown in FIG5B , the sense amplifier VSA[1] transmits a readout signal SAOUT[1] characterized by a weight “11100001”. Specifically, the weight “11100001” has a most significant bit (MSB) “1” and two repeated bits “1” of the MSB at adjacent positions. The weight extraction circuit BTF adjusts the voltage level of the signal FE_flag1 from a low level to a high level in response to the change of the readout signal SAOUT[1], while the eighth bit [7] to the sixth bit [5] of the weight are pre-read as “1” and the fifth bit [4] is “0”. In some embodiments, according to Table I, when the number (2) of repeated bits identical to the MSB at adjacent positions in the weight is equal to another predetermined value Nc “2”, the control circuit FF-RWC further controls the flip-flop circuit WFB-DFF in response to the signal FE_flag1 to store a 2-bit weight characteristic having “10”.

[0037] In some embodiments, according to Table I, when the number of repeated bits (eg, 4) exceeds a threshold value (eg, 3), the flip-flop circuit WFB-DFF stores a 2-bit weight feature "11".

[0038] 6A-6B and Table 1, weight signatures for extracting weights in the extraction phase (FE) of a long channel. In some embodiments, weight read circuits WFAR[0] and WFAR[1] are configured to cooperate to generate a weight signature based on a first weight from sense amplifier VSA[0] and a second weight from sense amplifier VSA[1].

[0039] In some embodiments, the configuration for generating signals FE_flag0 and FE_flag1 is similar to the configuration shown in FIG5A and FIG5B . For example, weight read circuit WFAR[0] determines whether a weight (e.g., "00001101") includes repeated bits based on readout signal SAOUT[0] to generate signal FE_flag0, and weight read circuit WFAR[1] determines whether a weight (e.g., "11100001") includes repeated bits based on readout signal SAOUT[1] to generate signal FE_flag1. The determined configuration is similar to the configuration shown in FIG5A and FIG5B . Therefore, repeated descriptions are omitted here.

[0040] As shown in FIG6B , AND gate 232 generates a 1-bit shared weight signature by performing an AND operation on signals FE_flag0 and FE_flag1 from weight extraction circuits BTF in weight read circuits WFAR[0] and WFAR[1]. Because weights "00001101" and "11100001" both include at least two repeated bits that are identical in their MSBs and located adjacent to each other, AND gate 232 transmits the shared weight signature "1" to one of the memory circuits in weight read circuits WFAR[0] and WFAR[1] (e.g., memory circuit 211).

[0041] In some embodiments, the WFB value is not equal to the predetermined value Nc. As shown in Table II below, in the example of a medium channel, when the WFB value is equal to 1 and corresponds to a weight characteristic of "10," the predetermined value Nc is equal to N1, where N1 is not equal to 1 (e.g., 2). Therefore, in some embodiments, according to Table II, when the number of repeated bits in the weight that are identical to the MSB in adjacent positions (e.g., 2) is equal to the predetermined value N1 (e.g., 2), the control circuit FF-RWC controls the flip-flop circuit WFB-DFF to store a 2-bit weight characteristic of "10." In some embodiments of a medium channel, N1 is the smallest and N3 is the largest among N1 to N3. For example, N1 is 2, N2 is 4, and N3 is 5. In some embodiments of a long channel, N4 can be any suitable positive integer (e.g., 3). For example, when the WFB value is equal to 1 and corresponds to a weight characteristic of "1," and the predetermined value Nc is equal to N4, it indicates the number of repeated bits in the weight that are identical to the MSB in adjacent positions (e.g., 3). aisle (layer) Short Channel Medium channel Long corridor Bits of weight features# 8-bit 2-bit 1 bit operate Storing weights in WFB-DFF WFB value Nc Reducing Access WFB value Nc Reducing Access 0 0 0 0 0 0 1 N1 N1 2 N2 N2 1 ≥N4 N4 3 ≥N3 N3 Table II

[0042] 3 , in operation S330, based on the weight characteristics and the weight addresses, the weight read circuits WFAR[0], WFAR[1], the local multiplexer and precharge circuits 141[0], 141[1], and the sense amplifier circuits 151[0], 151[1] access the memory array 110 and the memory circuit (e.g., 211) to transfer the weights to the multiplication and accumulation circuit 160 for neural network operation.

[0043] For example, in an embodiment of the feature and weight read phase (FD) in a short channel, as shown in FIG4A and FIG7 , the weight read controller FARC in the weight read circuit WFAR[0] generates a disable signal Dis[0] having a high level to the sense amplifier circuit 151[0] in response to the data Conf[7:0] instructing the weight read circuit WFAR[0] to operate on the weight of the short channel. The sense amplifier controller SAC[0] in the sense amplifier circuit 151[0] further generates an enable signal EN[0] in response to the disable signal Dis[0] to disable the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0]. Alternatively, as shown in FIG4A , in the multiplication and accumulation operation period MAC OG2, the sense amplifier VSA[0] does not access the memory array 110 to obtain the weight to generate the readout signal SAOUT[0]. In addition, in response to the x-address XA[7:0] and y-address YA[2:0] of the weight, a read operation is performed on the memory circuit 211 to output the weight feature "0001101" stored in the memory circuit 211 as the weight to the multiplication and accumulation circuit 160.

[0044] Specifically, the control circuit FF-RWC controls the flip-flop circuit WFB-DFF in response to the mapping address WA[3:0] to output the data f_d0[1:0] including the stored weight characteristics as weights to the multiplexer 222. The multiplexer 222 further transmits the data f_d0[0] as the signal To_MAC[0] to the multiplication and accumulation circuit 160 in response to the signal Ch[1:0]. In some embodiments, the signal Ch[1:0] is generated by the control circuit 130 based on the selected channel (e.g., one of the short channel, the medium channel, and the long channel).

[0045] The configuration of the weight read circuit WFAR[1] is similar to that of the weight read circuit WFAR[0]. Therefore, repeated descriptions are omitted here.

[0046] In an embodiment of the feature and weight read phase (FD) in the medium channel, as shown in FIG5A and FIG8, during the multiplication and accumulation operation period MAC OG2 for obtaining the weight "00001101", a read operation is performed on the memory circuit 211 to obtain the weight feature "11", wherein the weight feature "11" indicates three repeated bits identical to the MSB in the weight to be read and is transmitted via the data f_c0[1:0]. The weight read controller FARC in the weight read circuit WFAR[0] controls the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] in response to the data Conf[7:0] and the data f_c0[1:0] to perform a read operation on the memory array 110 to read the MSB of the weight to be read, as shown in FIG8.

[0047] Furthermore, as shown in FIG8 , the weight read controller FARC disables the read operation on the memory array 110 for a number of cycles (e.g., 3 cycles) using the disable signal Dis[0] based on the weight characteristic "11," and controls the sense amplifier VSA[0] to output the MSB "0" as part of the weight bit, instead of reading the remaining weight bits from the memory array 110. Therefore, referring to Table I, a reduction of "3" accesses to the memory array 110 is achieved, saving energy corresponding to the read operation on the memory array 110.

[0048] After disabling the local multiplexer and precharge circuit 141[0] and the sense amplifier circuit 151[0] for three cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "1101" in the weight "00001101").

[0049] Similarly, in order to read the weight "11100001" by the weight read circuit WFAR[1], the weight read controller FARC disables the read operation on the memory array 110 for a number of cycles (e.g., 2 cycles) by the disable signal Dis[1] according to the weight characteristic "10", and controls the sense amplifier VSA[1] to output the MSB "1" as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, the number of accesses to the memory array 110 is reduced to "2", saving the read energy corresponding to the memory array 110.

[0050] After disabling the local multiplexer and precharge circuit 141[1] and the sense amplifier circuit 151[1] for two cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "00001" in the weight "11100001").

[0051] In an embodiment of the feature and weight read stage (FD) in a long channel, as shown in Figures 6A and 9, during the multiplication and accumulation operation period MAC OG2, the weight read controller FARC in the adjacent weight read circuit WFAR[0] and the weight read circuit WFAR[1] is configured to control the corresponding read operation in response to the same weight feature "1" transmitted by the multiplexer 221 through the data f_d0[1:0].

[0052] Specifically, to obtain the weight "00001101" via the weight read circuit WFAR[0], a read operation is performed on the memory circuit 211 to obtain the weight characteristic "1" based on the data f_cL[1:0] associated with the data f_d0[1:0] and the data f_c0[1:0]. The weight read controller FARC in the weight read circuit WFAR[0] controls the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] in response to the data Conf[7:0] and the data f_c0[1:0] to perform a read operation on the memory array 110 to read the MSB of the weight to be read. In addition, as shown in FIG9 , the weight read controller FARC disables the read operation on the memory array 110 for a number of cycles (e.g., 2 cycles) based on the weight feature “1” by means of the disable signal Dis[0], and controls the sense amplifier VSA[0] to output the MSB “0” as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, a reduction in the number of accesses to the memory array 110 of “2” is achieved, saving the read energy corresponding to the memory array 110. After disabling the local multiplexer and precharge circuit 141[0] and the sense amplifier circuit 151[0] for two cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (e.g., “01101” in the weight “00001101”).

[0053] Similarly, in order to obtain the weight "11100001" by the weight read circuit WFAR[1], a read operation is performed on the memory circuit 212 to obtain the weight characteristic "1" based on the data f_cL[1:0] associated with the data f_d0[1:0] and the data f_c1[1:0]. The weight read controller FARC in the weight read circuit WFAR[1] controls the local multiplexer and precharge circuit 141[1] and the sense amplifier VSA[1] in response to the data Conf[7:0] and the data f_c1[1:0] to perform a read operation on the memory array 110 to read the MSB of the weight to be read. In addition, as shown in FIG9 , the weight read controller FARC prohibits the reading operation of the memory array 110 for a number of cycles (e.g., 2 cycles) by disabling the signal Dis[1] according to the weight feature “1”, and controls the sense amplifier VSA[1] to output the MSB “1” as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, a reduction in the number of accesses to the memory array 110 of “2” is achieved, saving the reading energy corresponding to the memory array 110. After disabling the local multiplexer and precharge circuit 141[1] and the sense amplifier circuit 151[1] for two cycles, the weight read controller FARC controls the resumption of the reading operation of the memory array 110 to obtain the remaining bit data in the weight (e.g., “00001” in the weight “11100001”).

[0054] In operation S340, the weight read circuit (eg, WFAR[0]) transmits the weight to the multiplication and accumulation circuit 160 for neural network layer operation.

[0055] In some embodiments, referring to Figures 4A, 5A, 6A, and 10, method 30 further includes the following operation: resetting a memory circuit (e.g., memory circuit 211) to erase weight features for use in a second neural network layer operation. For example, after the last multiplication and accumulation operation period MAC OGFinal among the multiple multiplication and accumulation operation periods of neural network layer operation LN, control circuit 130 of Figure 1 generates a signal rst to memory circuit 211 to erase the data stored in memory circuit 211. As a result, memory circuit 211 is ready to store weight features for use in the next neural network layer operation LN+1. In some embodiments, the configuration of neural network layer operation LN+1 is similar to that of neural network layer operation LN. Therefore, repeated description is omitted here.

[0056] The configurations of Figures 2 through 10 are provided for illustrative purposes. Various implementations are contemplated within the scope of the present disclosure. For example, in the embodiment of Figure 11 , four adjacent weight read circuits WFAR[0] through WFAR[3] are configured to collaborate to generate a weight signature based on a first weight from sense amplifier VSA[0], a second weight from sense amplifier VSA[1], a third weight from sense amplifier VSA[2], and a fourth weight from sense amplifier VSA[3].

[0057] Specifically, each of the weight read circuits WFAR[0] to WFAR[3] determines whether the corresponding received weight includes a repeated bit based on the received readout signal to generate a corresponding one of the signals FE_flag0 to FE_flag3. As shown in FIG11, the AND gate 232 of the weight read circuit WFAR[3] generates the signal FE_Lflag1 by performing an AND operation on the signals FE_flag2 and FE_flag3 from the weight extraction circuits BTF in the weight read circuits WFAR[2] and WFAR[3], and further transmits the signal FE_Lflag1 to the AND gate 233 in the weight read circuit WFAR[1]. The AND gate 232 generates a 1-bit common weight feature to be stored in the memory circuit 211 by performing an AND operation on the signals FE_flag0, FE_flag1, and FE_Lflag1.

[0058] The present application provides a memory device with weight feature extraction for near-memory computations. Furthermore, the present invention discloses a method for operating a memory device, comprising minimizing the frequency of non-volatile memory reads. This is achieved by skipping read operations that correspond to repeated bits corresponding to the most significant bit (MSB) and outputting the MSB based on extracted weight features. In contrast to certain approaches, using the configuration of the present application, certain neural network models (e.g., the Residual Neural Network (ResNet) 20 model and the ResNet 32 ​​model using the Canadian Institute for Advanced Research (CIFAR)-100 dataset) save approximately 30% of operating energy per unit area.

[0059] A method for operating a memory device is disclosed, the method comprising: generating a weight feature to be stored in a second memory different from the first memory based on at least one weight stored in the first memory, wherein the weight feature is associated with the number of repeated bits that are adjacent to and identical to the most significant bit of the at least one weight; and accessing the first memory and the second memory according to the weight feature and the address of the at least one weight to transmit the at least one weight to a multiplication and accumulation circuit for a first neural network layer operation.

[0060] In some embodiments, generating the weight feature includes: transmitting the at least one weight as the weight feature to a second memory.

[0061] In some embodiments, accessing the first memory and the second memory includes: performing a read operation on the second memory in response to the address of the at least one weight to output the stored weight feature to the multiplication and accumulation circuit.

[0062] In some embodiments, generating the weight feature includes generating a weight feature equal to the number of repeated bits in the at least one weight.

[0063] In some embodiments, generating a weight feature includes: generating a weight feature having a first value when the number of repeated bits in the at least one weight is equal to a first predetermined value; and generating a weight feature having a second value different from the first value when the number of repeated bits in the at least one weight is greater than a second predetermined value, wherein the second predetermined value is greater than the first predetermined value.

[0064] In some embodiments, accessing the first memory and the second memory includes: performing a first read operation on the second memory to obtain a weight characteristic from the second memory; performing a second read operation on the first memory to read the most significant bit of the at least one weight; prohibiting the second read operation on the first memory within a number of cycles based on the weight characteristic, and outputting the most significant bit as the bit data in the at least one weight to the multiplication and accumulation circuit; and resuming the second read operation on the first memory to obtain the remaining bit data in the at least one weight.

[0065] In some embodiments, the at least one weight includes a first weight and a second weight, and generating a weight feature includes: determining whether the first weight includes repeated bits, and determining whether the second weight includes repeated bits; and when both the first weight and the second weight include repeated bits, generating a weight feature with a non-zero value.

[0066] In some embodiments, accessing the first memory and the second memory includes: performing a first read operation on the second memory to obtain a weight characteristic from the second memory; performing a second read operation on the first memory to read the most significant bit of the first weight; prohibiting the second read operation on the first memory within a number of cycles based on the weight characteristic, and outputting the most significant bit as the bit data in the first weight to the multiplication and accumulation circuit; and resuming the second read operation on the first memory to obtain the remaining bit data in the first weight.

[0067] In some embodiments, accessing the first memory and the second memory further includes: performing a third read operation on the second memory to obtain a weight feature from the second memory; performing a fourth read operation on the first memory to read the most significant bit of the second weight; prohibiting the fourth read operation on the first memory within the number of cycles, and outputting the most significant bit of the second weight as the bit data in the second weight to the multiplication and accumulation circuit; and resuming the fourth read operation on the first memory to obtain the remaining bit data in the second weight.

[0068] In some embodiments, the method further includes: resetting the second memory to erase weight features for the second neural network layer operation.

[0069] A memory device is also disclosed. The memory device includes: a non-volatile memory array configured to store a plurality of weights; a plurality of sense amplifier circuits configured to access the non-volatile memory array; and a plurality of weight read circuits coupled to the plurality of sense amplifier circuits, each of the plurality of weight read circuits including: a weight extraction circuit configured to generate a flag signal in response to a readout signal, the readout signal being associated with a corresponding one of the plurality of weights and generated by a corresponding one of the plurality of sense amplifier circuits; a memory circuit configured to store a weight characteristic based on the flag signal; and a weight read controller configured to control the corresponding sense amplifier circuit to output a first bit of the corresponding one of the weights as part of a bit in the corresponding one of the weights in response to control data and characteristic data associated with the weight characteristic.

[0070] In some embodiments, the memory circuit includes a flip-flop circuit.

[0071] In some embodiments, the memory device further includes: an address decoder configured to generate a mapping address based on the address of the corresponding weight to be read, the mapping address indicating the location of the weight characteristic stored in the flip-flop circuit. The memory circuit further includes: a control circuit configured to control the flip-flop circuit to output characteristic data to a weight read controller in response to the mapping address. The weight read controller is further configured to prohibit the corresponding sense amplifier circuit from accessing the non-volatile memory array when the corresponding sense amplifier circuit outputs the first bit as the portion of the bit in the corresponding weight.

[0072] In some embodiments, the first bit is the most significant bit in the corresponding weight.

[0073] In some embodiments, the memory device further includes: a logic gate circuit configured to generate a common weight feature based on a flag signal from a weight extraction circuit in at least two of the multiple weight reading circuits.

[0074] In some embodiments, the logic gate circuit is further configured to transmit the common weight characteristic to a memory circuit in one of the at least two of the plurality of weight reading circuits.

[0075] In some embodiments, the memory device further includes: a first AND gate configured to generate an output signal based on flag signals from a first weight read circuit and a weight extraction circuit in a second weight read circuit among the plurality of weight read circuits; and a second AND gate configured to generate a common weight signature based on the output signal from the first AND gate and flag signals from a weight extraction circuit in a third weight read circuit and a fourth weight read circuit among the plurality of weight read circuits. The second AND gate is further configured to transmit the common weight signature to a memory circuit in one of the first to fourth weight read circuits.

[0076] A method for operating a memory device is also disclosed. The method includes: extracting a weight feature of at least one weight and storing the weight feature in a first memory during a first multiplication and accumulation operation period of a neural network layer operation; based on the weight feature, prohibiting a sense amplifier from accessing a second memory for at least one cycle during a second multiplication and accumulation operation period of the neural network layer operation, and accessing the first memory, the second memory, or a combination of the first and second memories to output the at least one weight; and erasing the first memory during a final multiplication and accumulation operation period of the neural network layer operation.

[0077] In some embodiments, when the neural network layer operation corresponds to a short channel, accessing the first memory, the second memory, or a combination of the first memory and the second memory to output the at least one weight includes: performing a read operation on the first memory in response to the address of the at least one weight to output a weight feature as the at least one weight.

[0078] In some embodiments, when the neural network layer operation corresponds to a medium channel or a long channel, disabling the sense amplifier from accessing the second memory based on the weight characteristics and accessing the first memory, the second memory, or a combination of the first memory and the second memory to output the at least one weight includes: performing a read operation on the second memory in a first cycle to obtain the most significant bit of the at least one weight; disabling the sense amplifier in a number of cycles and outputting the most significant bit in the number of cycles; and resuming the read operation on the second memory in the remaining cycles to obtain the remaining bits in the at least one weight.

[0079] The above summarizes the features of several embodiments so that those skilled in the art can better understand the various aspects of the present disclosure. Those skilled in the art will appreciate that they can readily use this disclosure as a basis for designing or modifying other processes and structures to perform the same purposes and / or achieve the same advantages as the embodiments described herein. Those skilled in the art will also recognize that these equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they can make various changes, substitutions, and modifications herein without departing from the spirit and scope of the present disclosure.

[0080] 10: Memory device 30: Method 101: Memory 110:Memory Array 120: word line driver (WLDR) 130. FF-RWC: Control Circuit 140: Bit line multiplexer (MUX) 141[0], 141[1]: Local multiplexer and precharge circuit 150: Input / Output (IO) Circuit 151[0], 151[1]: Sense amplifier circuit 155: Weight feature reading circuit 160: Multiplication and Accumulation (MAC) Circuit 211, 212: Memory circuit 221, 222: Multiplexer 231:Logic Gate Circuit 232, 233: and gate BL[7:0], BL[15:8], Conf[7:0], f_c0[1:0], f_c1[1:0], f_cL[1:0], f_d0[0], f_d0[1:0]: Data BTF: Weight Extraction Circuit Ch[1:0], FE_flag0, FE_flag1, FE_flag2, FE_flag3, FE_Lflag, FE_Lflag1, rst, To_MAC[0]: signal Dis[0], Dis[1]: Disable signal EN[0]: enabling signal FARC: Weight Read Controller FD: Feature and weight reading stage FE: Extraction stage FF-RWC: Control Circuit LN, LN+1: Neural network layer operations MAC OG1: Multiplication and accumulation operation period MAC OG2: Multiplication and accumulation operation period MAC OGFinal: The last multiplication and accumulation operation period S1, S2, S3, S4, S5, S6, S7, S8: switches S310, S320, S330, S340: Operation SAC[0]: Sense Amplifier Controller SAOUT[0], SAOUT[1]: readout signal VSA[0], VSA[1]: sense amplifier W0: weight WA[3:0]: mapping address WAP: Address Decoder WFAR[0], WFAR[1], WFAR[2], WFAR[3]: weight reading circuit WFB-DFF: Flip-flop circuit XA[7:0]: x address YA[2:0]:y address

Claims

1. A method for operating a memory device, comprising: Based on at least one weight stored in a first memory, a weight feature is generated to be stored in a second memory different from the first memory, wherein the weight feature is associated with the number of repeating bits that are adjacent to the most significant bit in the at least one weight and are the same as the most significant bit; and the first memory and the second memory are accessed according to the weight feature and the address of the at least one weight to transfer the at least one weight to multiplication and accumulation circuitry for operation of the first neural network layer.

2. The method as described in claim 1, wherein generating the weight features includes: The at least one weight is transmitted to the second memory as the weight feature.

3. The method as described in claim 2, wherein accessing the first memory and the second memory includes: A read operation is performed on the second memory in response to the address of the at least one weight, so as to output the stored weight features to the multiplication and accumulation circuit.

4. The method as described in request item 1, wherein generating the weight feature comprises: The weight feature is generated to be equal to the number of repeating bits in the at least one weight.

5. The method as described in claim 1, wherein generating the weighted features comprises: When the number of repeated bits in the at least one weight is equal to a first predetermined value, the weight feature having the first value is generated; And when the number of repeating bits in the at least one weight is greater than a second predetermined value, a weight feature having a second value different from the first value is generated, the second predetermined value being greater than the first predetermined value.

6. The method as described in claim 5, wherein accessing the first memory and the second memory comprises: Perform a first read operation on the second memory to obtain the weighted features from the second memory; A second read operation is performed on the first memory to read the most significant bit of the at least one weight; the second read operation on the first memory is prohibited within a number of loops according to the weight characteristics, and the most significant bit is output as bit data in the at least one weight to the multiplication and accumulation circuit; And resume the second read operation on the first memory to obtain the remaining bit data in the at least one weight.

7. A memory device, comprising: A non-volatile memory array is configured to store multiple weights; Multiple sense amplifier circuits are configured to access the non-volatile memory array; The system also includes multiple weight readout circuits coupled to the multiple sense amplifier circuits, each of which comprises: a weight extraction circuit configured to generate a flag signal in response to a readout signal, the readout signal being associated with a corresponding weight among the multiple weights and generated by the corresponding sense amplifier circuit among the multiple sense amplifier circuits; a memory circuit configured to store weight features based on the flag signal; and a weight readout controller configured to control the corresponding sense amplifier circuit in response to control data and feature data associated with the weight features to output the first bit of the corresponding weight as a subset of bits of the corresponding weight.

8. The memory device as claimed in claim 7, wherein the memory circuitry includes a flip-flop circuitry.

9. The memory device as claimed in claim 8, further comprising: An address decoder is configured to generate a mapped address based on the address of the corresponding weight to be read, the mapped address indicating the location of the weight feature stored in the flip-flop circuit, wherein the memory circuit further includes: a control circuit configured to control the flip-flop circuit to output the feature data to the weight read controller in response to the mapped address, wherein the weight read controller is further configured to disable the corresponding sense amplifier circuit from accessing the non-volatile memory array when the corresponding sense amplifier circuit outputs the first bit as a portion of the corresponding weight.

10. A method of operating a memory device, comprising: During the first multiplication and accumulation operation period of the neural network layer operation, the weight features of at least one weight are extracted and the weight features are stored in the first memory. Based on the weight characteristics, during the second multiplication and accumulation operation period of the neural network layer operation, the sensor amplifier is disabled from accessing the second memory and accessing the first memory, the second memory, or a combination of the first memory and the second memory within at least one loop to output the at least one weight; and the first memory is erased during the last multiplication and accumulation operation period of the neural network layer operation.

Images