Memory device and operating method thereof

Abstract

A method of operating a storage device includes the operations of generating a weight feature to be stored in a second memory different from a first memory based on at least one weight stored in the first memory, wherein the weight feature is associated with the number of duplicate bits which are located at the adjacent position of the most significant bit and are the same as the most significant bit in the at least one weight; and accessing the first memory and the second memory according to the weight feature and the address of the at least one weight to transmit the at least one weight to a multiplication and accumulation circuit for operation of a first neural network layer.

Description

Technical Field

[0001] The present invention relates to a storage device and an operating method thereof. Background Art

[0002] Near-memory compute macros, designed to improve energy efficiency by performing computations close to memory storage, face significant challenges that are particularly prominent in the context of convolutional neural networks (CNNs). The problem is that a large amount of energy is consumed due to memory readouts, with nearly half of the total energy dedicated to fetching data. In CNNs, all inputs within a layer correspond to the same group of weights, resulting in repeated memory reads of the same data. This redundancy in data retrieval significantly increases energy consumption. Summary of the invention

[0003] According to an embodiment of the present invention, a method for operating a storage device is provided, the method comprising: generating a weight feature to be stored in a second memory different from the first memory based on at least one weight stored in the first memory, wherein the weight feature is associated with the number of repeated bits that are adjacent to the most significant bit in the at least one weight and are the same as the most significant bit; and accessing the first memory and the second memory according to the weight feature and the address of the at least one weight to transmit the at least one weight to a multiplication and accumulation circuit for a first neural network layer operation.

[0004] According to an embodiment of the present invention, a storage device is provided, comprising: a non-volatile memory array configured to store multiple weights; multiple sense amplifier circuits configured to access the non-volatile memory array; and multiple weight read circuits coupled to the multiple sense amplifier circuits, and each of the multiple weight read circuits comprises: a weight extraction circuit configured to generate a flag signal in response to a read signal, the read signal being associated with a corresponding one of the multiple weights and generated by a corresponding sense amplifier circuit in the multiple sense amplifier circuits; a memory circuit configured to store weight characteristics according to the flag signal; and a weight read controller configured to control the corresponding sense amplifier circuit to output the first bit of the corresponding one weight as part of the bit in the corresponding one weight in response to control data and characteristic data associated with the weight characteristics.

[0005] According to an embodiment of the present invention, a method for operating a storage device is provided, the method comprising: in a first multiplication and accumulation operation period of a neural network layer operation, extracting a weight feature of at least one weight and storing the weight feature in a first memory; based on the weight feature, in a second multiplication and accumulation operation period of the neural network layer operation, prohibiting a sense amplifier from accessing a second memory and accessing the first memory, the second memory, or a combination of the first memory and the second memory within at least one cycle to output the at least one weight; and erasing the first memory in a last multiplication and accumulation operation period of the neural network layer operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a schematic diagram of a storage device according to some embodiments of the present disclosure.

[0007] Figure 2 According to some embodiments of the present disclosure Figure 1 A schematic diagram of a portion of the corresponding storage device 10 .

[0008] Figure 3 According to some embodiments of the present disclosure, Figure 1 and Figure 2 A flowchart of a method for storing a device is shown.

[0009] Figure 4A is a timing diagram of the operation of a neural network layer corresponding to a short channel according to some embodiments of the present disclosure.

[0010] Figure 4B The present disclosure shows some embodiments of the present invention. Figure 4A The corresponding operation and signal waveforms of the storage device 10.

[0011] Figure 5A is a timing diagram of the operation of a neural network layer corresponding to a medium channel according to some embodiments of the present disclosure.

[0012] Figure 5B The present disclosure shows some embodiments of the present invention. Figure 5A The corresponding operation and signal waveforms of the storage device 10.

[0013] Fig. 6A is a timing diagram of neural network layer operations corresponding to long channels according to some embodiments of the present disclosure.

[0014] Figure 6B The present disclosure shows some embodiments of the present invention. Fig. 6A The corresponding operation and signal waveforms of the storage device 10.

[0015] Figure 7The operation and signal waveform of the storage device 10 in the neural network layer operation corresponding to the short channel according to some embodiments of the present disclosure are shown.

[0016] Figure 8 The operation and signal waveforms of the storage device 10 in a neural network layer operation corresponding to a medium channel according to some embodiments of the present disclosure are shown.

[0017] Fig. 9 The operation and signal waveforms of the storage device 10 in a neural network layer operation corresponding to a long channel according to some embodiments of the present disclosure are shown.

[0018] Fig.10 The operation of the storage device 10 in a neural network layer operation according to some embodiments of the present disclosure is shown.

[0019] Fig.11 The operation of the storage device 10 in a neural network layer operation according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0021] The following disclosure provides many different embodiments or examples for implementing the different features of the provided target. The following describes specific examples of components, materials, values, steps, arrangements or similar elements to simplify the disclosure. Of course, these are only examples and are not intended to be limiting. It is expected that there are other components, materials, values, steps, arrangements or similar elements. For example, the following description may include an embodiment in which the first feature and the second feature are formed to be in direct contact, and may also include an embodiment in which additional features may be formed between the first feature and the second feature so that the first feature and the second feature may not be in direct contact. In addition, the disclosure may reuse reference numbers and / or letters in various examples. This repetition is for the purpose of simplicity and clarity, rather than the relationship between the various embodiments and / or configurations discussed by itself.

[0022] In addition, for ease of description, spatially relative terms such as "under", "beneath", "lower", "over", "upper" and the like may be used herein to describe the relationship of one component or feature to another (other) component or feature as shown in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein should be similarly interpreted accordingly. The terms mask, photolithography mask, reticle, and mask plate are used to refer to the same item.

[0023] The terms used in the following description and throughout the claims generally have the ordinary meanings in the art or as clearly determined in the specific context in which each term is used. One of ordinary skill in the art will understand that a component or process may be referred to by different names. The many different embodiments described in detail in this specification are merely illustrative and in no way limit the scope and spirit of the present disclosure or any illustrative term.

[0024] It is worth noting that the terms such as "first" and "second" used herein to describe various components or processes are intended to distinguish one component or process from another component or process. However, the components, processes and their order should not be limited by these terms. For example, a first component may be referred to as a second component, and similarly a second component may be referred to as a first component without departing from the scope of the present disclosure.

[0025] In the following discussion and claims, the terms "comprising", "including", "containing", "having", "involving" and similar terms should be understood as open-ended, i.e., interpreted as including but not limited to. The term "and / or" used herein includes any one of the associated listed items and all combinations of one or more of the associated listed items, rather than being mutually exclusive.

[0026] According to some embodiments, the present application is directed to a method for optimizing weight reading processes in a neural network system by exploiting the inherent properties of a zero-centered Gaussian weight distribution. Such a distribution tends to show higher frequencies in sequences of numbers characterized by repeated 0s or 1s. Specifically, relatively small weight values ​​are common, resulting in many leading "1s" or "0s" in 2's complement representation.

[0027] The method is introduced to take advantage of a specific characteristic of the weight data, namely the recurring most significant bit (MSB). This approach aims to selectively skip the reading of certain bits during the retrieval process, thereby reducing the energy consumption associated with reading each bit.

[0028] The method includes extracting a weight signature during an initial readout, the weight signature being characterized by a run-length of the MSB exceeding a predetermined threshold. This information is then encoded and stored in a memory address to form a record of the weight signature. In certain embodiments, the weight signature varies depending on the number of weights accessed across different neural network layers. During subsequent readouts, if the read address is consistent with the declared weight signature, the bits associated with the predetermined run-length threshold are systematically omitted. This strategic omission effectively reduces the overall read energy per bit, helping to improve the efficiency of neural network operations.

[0029] Now refer to Figure 1 . Figure 1 1 is a schematic diagram of a memory device 10 according to some embodiments of the present disclosure. In some embodiments, the memory device 10 is configured as a compute-in-memory (CIM) system for neural network operations. For illustration, the memory device 10 includes a memory 101, a word line driver 120, a control circuit 130, a bit line multiplexer 140, an input / output circuit 150, a weight feature read circuit 155, and a multiply and accumulate (MAC) circuit 160.

[0030] In some embodiments, memory 101 includes a memory array 110 composed of a plurality of bit cells, referred to as memory cells. Memory cells are located at the intersections of columns and rows in 110. In some embodiments, memory array 110 may be a non-volatile memory array and include static random access memory (SRAM) cells. In various embodiments, memory array 110 includes resistance-based random access memory (RAM) cells. Resistance-based RAM may include resistive-RAM (ReRAM), magnetoresistive RAM (MRAM), ferroelectric RAM (FeRAM), dielectric RAM, any suitable array of any suitable storage device, or any combination thereof. In some embodiments, memory array 110 is configured to store a plurality of weights for neural network access.

[0031] The word line driver (WLDR) 120 is configured to generate a word line signal in response to a control signal associated with an address to drive a word line for accessing the memory array 110 to read bits from / write bits to the memory array 110, wherein the address indicates some specific memory cells storing bits in the memory array 110. Specifically, in some embodiments, the word line driver 120 selects and enables specific memory cells in the memory array 110 according to the address.

[0032] The control circuit 130 is configured to control the word line driver 120, the bit line multiplexer 140, the input / output circuit 150, the weight feature read circuit 155, and the multiplication and accumulation circuit 160 to perform conventional memory access (e.g., reading and writing of a specific address) and CIM operations. In some embodiments, the control circuit 130 includes an x-decoder for the word line and a y-decoder for the bit line and / or the sense line. The control circuit 130 also includes timing control for read operations and write operations. In some embodiments, the control circuit 130 is configured to generate control signals for the word line driver 120, the bit line multiplexer 140, the input / output circuit 150, and the multiplication and accumulation circuit 160 in response to the address for performing access operations (e.g., reading operations and writing operations to the memory array 110).

[0033] The bit line multiplexer (MUX) 140 is coupled to the memory 101 and is configured to enable the rows of the memory array 110 by selecting the bit line (BL) and / or the sense line based on the control signal from the control circuit 130. In some embodiments, the bit line multiplexer 140 includes a precharge circuit system. For example, in memory access, the precharge circuit system precharges the bit line for a read operation.

[0034] The input / output (IO) circuit 150 is configured to transmit data to be written into the memory array 110 and / or read out data stored in the memory array 110. For example, the input / output circuit 150 transmits the weight stored in the memory array 110 to the weight feature reading circuit 155 for further application. In some embodiments, the input / output circuit 150 includes a sense amplifier circuit for input / output operations from the memory array 110.

[0035] The weight characteristic reading circuit 155 is coupled to the input / output circuit 150 and is configured to extract the weight characteristic of the weight in the memory 101. In some embodiments, the weight characteristic reading circuit 155 is further configured to control the precharge circuit system in the bit line multiplexer 140 and the sense amplifier circuit in the input / output circuit 150 according to the extracted weight characteristic, so as to access the weight in the memory 101 and / or output the weight stored in the weight characteristic reading circuit 155 in the weight reading operation.

[0036] In some embodiments, the multiplication and accumulation circuit 160 provides functional units (eg, adders, multipliers, buffers, etc.) for performing MAC operations based on the weights transmitted from the weight feature read circuit 155 .

[0037] Now refer to Figure 2 . Figure 2 According to some embodiments of the present disclosure Figure 1 A schematic diagram of a portion of the corresponding storage device 10. Figure 1 For ease of understanding, the same reference numerals are used to represent Figure 2 For the sake of brevity, the specific operations of similar components that have been discussed in detail in the above paragraphs are omitted in this article.

[0038] For purposes of illustration, memory array 110 is coupled to local multiplexer and precharge circuits 141[0] and 141[1]. Each of local multiplexer and precharge circuits 141[0] and 141[1] is coupled to one of sense amplifier circuits 151[0] and 151[1]. In some embodiments, local multiplexer and precharge circuits 141[0] and 141[1] are coupled to, for example, sense amplifier circuits 151[0] and 151[1]. Figure 1 The local multiplexer and precharge circuit 141[0] includes an 8-to-1 multiplexer and precharge circuit coupled to the bit lines BL0 to BL7 for the 8-bit data BL[7:0]. The configuration of the local multiplexer and precharge circuit 141[1] is similar to that of the local multiplexer and precharge circuit 141[0]. Therefore, repeated description is omitted here.

[0039] Sense amplifier circuits 151[0] and 151[1] are used for example Figure 1The sense amplifier circuit 151[0] is configured by the input / output circuit 150 of the memory array 110 and is configured to access the memory array 110. For example, the sense amplifier circuit 151[0] includes a sense amplifier controller SAC[0] and a sense amplifier VSA[0]. In some embodiments, the sense amplifier controller SAC[0] is configured to control the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] through the enable signal EN[0]. For example, the sense amplifier controller SAC[0] temporarily disables the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] from accessing the memory array 110 in response to the disable signal Dis[0] during the read operation, and further restores the read operation by enabling the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0]. In some embodiments, the sense amplifier VSA[0] is configured to access the memory array 110 to obtain a weight from the memory array 110 to generate a readout signal SAOUT[0]. The configuration of the sense amplifier circuit 151 [ 1 ] is similar to that of the sense amplifier circuit 151 [ 0 ]. Therefore, repeated descriptions are omitted here.

[0040] The sense amplifier circuits 151[0] and 151[1] are further coupled to weight read circuits WFAR[0] and WFAR[1], respectively. For illustration, the weight read circuit WFAR[0] includes a weight read controller FARC, a weight extraction circuit BTF, a memory circuit 211, and an address decoder WAP. The configuration of the weight read circuit WFAR[1] is similar to that of the weight read circuit WFAR[0]. Therefore, repeated descriptions are omitted here.

[0041] The weight read controller FARC is coupled to the sense amplifier controller SAC[0] and is configured to generate a disable signal Dis[0] for the sense amplifier controller SAC[0] in response to the data f_c0[1:0] and the data Conf[7:0]. In some embodiments, Figure 1 The control circuit 130 is configured to generate data Conf[7:0] according to the layer implementing the neural network operation. Figures 3 to 10 The detailed configuration is discussed.

[0042] The weight extraction circuit BTF of the weight reading circuit WFAR[0] is coupled to the sense amplifier VSA[0], and is further coupled to the memory circuit 211 through the switch S1. In some embodiments, the weight extraction circuit BTF of the weight reading circuit WFAR[0] generates a signal FE_flag0 according to the read signal SAOUT[0] received from the sense amplifier VSA[0], and transmits the signal FE_flag0 to the memory circuit 211. In some embodiments, the weight extraction circuit BTF extracts the weight feature of the weight of the read signal SAOUT[0] to generate the signal FE_flag0.

[0043] The memory circuit 211 includes a control circuit FF-RWC and a flip-flop circuit WFB-DFF. The control circuit FF-RWC is coupled to the address decoder WAP of the weight read circuit WFAR[0] and multiplexers 221 and 222. The control circuit FF-RWC is configured to control the read operation and the write operation of the flip-flop circuit WFB-DFF according to the image address WA[3:0] and the data f_d0[1:0], and further control the reset operation of the flip-flop circuit WFB-DFF according to the signal rst. In some embodiments, the memory circuit 211 generates data f_c0[1:0] to the weight read controller FARC and generates data f_d0[1:0] to the multiplexer 221.

[0044] The address decoder WAP of the weight read circuit WFAR[0] is configured to generate the image address WA[3:0] based on the y address YA[2:0] and the x address XA[7:0] of the weight stored in the memory array 110. In some embodiments, the address decoder WAP indicates the location of the weight feature corresponding to the weight stored in the flip-flop circuit WFB-DFF. The detailed configuration will be discussed in the following paragraphs.

[0045] like Figure 2 As shown in the embodiment of the present invention, the memory circuit 211 is coupled to the weight read controller FARC of the weight read circuit WFAR[0] via switch S2. The weight read controller FARC is further coupled to the multiplexer 221 via switch S3. The weight read controller FARC in the weight read circuit WFAR[1] is coupled to the multiplexer 221 via switch S5, and is coupled to the memory circuit 212 in the weight read circuit WFAR[1] via switch S7. The weight extraction circuit BTF of the weight read circuit WFAR[1] is coupled to the memory circuit 212 via switch S6.

[0046] Specifically, the weight reading circuit WFAR[1] further includes a logic gate circuit 231. In some embodiments, the logic gate circuit 231 includes an AND gate 232, wherein the AND gate 232 has a first input coupled to the weight extraction circuit BTF of the weight reading circuit WFAR[0] and a second input coupled to the weight extraction circuit BTF of the weight reading circuit WFAR[1]. The output of the AND gate 232 is coupled to the memory circuits 211 and 212 through switches S4 and S8, respectively. In some embodiments, the AND gate 232 is configured to generate the signal FE_Lflag based on the signal FE_flag0 and the signal FE_flag1.

[0047] Now refer to Figure 3 . Figure 3 According to some embodiments of the present disclosure, Figure 1 and Figure 2 It should be understood that for other embodiments of the method 30, the method 30 may be Figure 3 Additional operations / stages are provided before, during, and after the process shown, and some of the operations / stages described below may be replaced or eliminated. Method 30 includes operations S310, S320, S330, and S340, and will be referred to in the following paragraphs. Figure 1 to Figure 2 as well as FIG. 4A to FIG. 10 Discuss.

[0048] In operation S310, Figure 3 and Figure 4A , Figure 5A and Fig. 6A As shown, a read operation is performed on the memory array 110 to obtain the weight stored in the memory array 110 in the multiplication and accumulation operation period MACOG1 of the neural network layer operation LN. For example, the weight W0 has 8 bits, and the 8 bits are represented as "00001101" to be stored in the memory cells of the memory array 110 coupled to the bit lines BL0 to BL7. The control circuit 130 controls the sense amplifier circuit 151[0] to access the memory array 110 to generate a read signal SAOUT[0] indicating the weight.

[0049] In operation S320, a weight feature is generated based on the weight. In some embodiments, a weight read circuit (e.g., WFAR[0]) is configured to operate in response to data Conf[7:0] to extract and store weight features of each layer in a neural network, in which each layer has a different number of weights. For example, as shown in Table I below, a layer including a relatively small amount of weights (e.g., 144 weights) is referred to as a short channel, and the weight read circuit WFAR[0] generates an 8-bit weight feature based on the weight. In another embodiment, a layer including a medium amount of weights (e.g., 288 weights) is referred to as a medium channel, and the weight read circuit WFAR[0] generates a 2-bit weight feature. In yet another embodiment, a layer including a relatively large amount of weights (e.g., 576 weights) is referred to as a long channel, and the weight read circuit WFAR[0] generates a 1-bit weight feature.

[0050]

[0051] Table I

[0052] The WFB value corresponds to a characteristic of a weight in the memory circuit to be stored in the weight read circuit WFAR[0] and / or WFAR[1]. The number Nc corresponds to a predetermined value associated with the number of repeated bits in the weight that are identical to the MSB in adjacent positions. The number of reduced accesses corresponds to the number of bits in the weight that will not be read out of the memory array 110 and will be referred to later. Figures 7 to 9 Discuss.

[0053] Now refer to FIG. 4A to FIG. 4B and Table I. For an embodiment in which weight features of weights are extracted in an extract phase (FE) of a short channel after reading the weights from the memory array 110, adjacent weight read circuits (e.g., WFAR[0] and WFAR[1]) are operated separately to generate weight features based on the weights in the data BL[7:0] (e.g., "00001101") and the weights in the data BL[15:8] (e.g., "11100001").

[0054] Taking the embodiment of the weight reading circuit WFAR[0] as an example, the weight extraction circuit BTF receives the read signal SAOUT[0] and transmits the read signal SAOUT[0] as the flag signal FE_flag0 to the memory circuit 211 through the turned-on switch S1. The control circuit FF-RWC further controls the flip-flop circuit WFB-DFF to store the entire 8-bit weight as the weight feature. Similarly, the weight extraction circuit BTF of the weight reading circuit WFAR[1] receives the read signal SAOUT[1] and transmits the read signal SAOUT[1] as the flag signal FE_flag1 to the memory circuit 212 through the turned-on switch S6. The control circuit FF-RWC in the memory circuit 212 further controls the flip-flop circuit WFB-DFF to store the entire 8-bit weight as a weight feature in a position of the flip-flop circuit WFB-DFF, where the position is pointed to by the corresponding image address WA[3:0] generated by the x address XA[7:0] and y address YA[2:0] of the weight.

[0055] Now refer to FIG. 5A to FIG. 5B and Table I for extracting the weight characteristics of the weights in the extraction stage (FE) of the mid-channel after reading the weights from the memory array 110. In some embodiments, the weight extraction circuit BTF is configured to generate a flag signal that changes from a first voltage level to a second voltage level in response to a bit change of the weight of the read signal. The control circuit in the memory circuit is further configured to store the weight characteristics according to the flag signal.

[0056] For example, Figure 5B As shown, the sense amplifier VSA[0] transmits a read signal SAOUT[0] characterized by a weight "00001101". Specifically, the weight "00001101" has a most significant bit (MSB) "0" and three repeated bits "000" of the MSB at adjacent positions. The weight extraction circuit BTF adjusts the voltage level of the signal FE_flag0 from a low level to a high level in response to the change of the read signal SAOUT[0], and the eighth bit [7] to the fifth bit [4] of the weight are pre-read as "0" and the fourth bit [3] is "1". In some embodiments, according to Table I, when the number of repeated bits (3) in the weight that are identical to the MSB at adjacent positions is equal to the predetermined value Nc "3", the control circuit FF-RWC further controls the flip-flop circuit WFB-DFF to store a 2-bit weight characteristic of "11" in response to the signal FE_flag0.

[0057] In various embodiments, such as Figure 5BAs shown, the sense amplifier VSA[1] transmits a read signal SAOUT[1] characterized by a weight "11100001". Specifically, the weight "11100001" has a most significant bit (MSB) "1" and two repeated bits "1" of the MSB at adjacent positions. The weight extraction circuit BTF adjusts the voltage level of the signal FE_flag1 from a low level to a high level in response to the change of the read signal SAOUT[1]. At the same time, the eighth bit [7] to the sixth bit [5] of the weight are pre-read as "1" and the fifth bit [4] is "0". In some embodiments, according to Table I, when the number of repeated bits (2) in the weight that are identical to the MSB at adjacent positions is equal to another predetermined value Nc "2", the control circuit FF-RWC further controls the flip-flop circuit WFB-DFF to store a 2-bit weight characteristic of "10" in response to the signal FE_flag1.

[0058] In some embodiments, according to Table I, when the number of repeated bits (eg, 4) exceeds a threshold value (eg, 3), the flip-flop circuit WFB-DFF stores a 2-bit weight feature "11".

[0059] Now refer to FIG. 6A to FIG. 6B and Table I, weight signatures for extracting weights in an extraction phase (FE) of a long channel. In some embodiments, weight read circuits WFAR[0] and WFAR[1] are configured to collaborate to generate a weight signature based on a first weight from sense amplifier VSA[0] and a second weight from sense amplifier VSA[1].

[0060] In some embodiments, the configuration of generating signals FE_flag0 and FE_flag1 is similar to FIG. 5A to FIG. 5B For example, the weight read circuit WFAR[0] determines whether the weight (e.g., "00001101") includes a repeated bit based on the read signal SAOUT[0] to generate the signal FE_flag0, and the weight read circuit WFAR[1] determines whether the weight (e.g., "11100001") includes a repeated bit based on the read signal SAOUT[1] to generate the signal FE_flag1. The configuration determined is similar to FIG. 5A to FIG. 5B Therefore, repeated descriptions are omitted here.

[0061] like Figure 6BAs shown, AND gate 232 generates a 1-bit shared weight feature by performing an AND operation on signals FE_flag0 and FE_flag1 from weight extraction circuits BTF in weight reading circuits WFAR[0] and WFAR[1]. Since weights "00001101" and weights "11100001" both include at least two repeated bits that are the same as their MSBs and are located at adjacent positions of the MSBs, AND gate 232 transmits the shared weight feature "1" to one of the memory circuits in weight reading circuits WFAR[0] and WFAR[1] (e.g., memory circuit 211).

[0062] In some embodiments, the WFB value is not equal to the predetermined value Nc. As shown in Table II below, in the example of a medium channel, when the WFB value is equal to 1 and corresponds to the weight feature "10", the predetermined value Nc is equal to N1, where N1 is not equal to 1 (for example, 2). Therefore, in some embodiments, according to Table II, when the number of repeated bits that are the same as the MSB in adjacent positions in the weight (for example, 2) is equal to the predetermined value N1 (for example, 2), the control circuit FF-RWC controls the flip-flop circuit WFB-DFF to store a 2-bit weight feature of "10". In some embodiments of the medium channel, N1 is the smallest and N3 is the largest among N1 to N3, for example, N1 is 2, N2 is 4, and N3 is 5. In some embodiments of the long channel, N4 can be any suitable positive integer (for example, 3). For example, when the WFB value is equal to 1 and corresponds to the weight feature "1" and the predetermined value Nc is equal to N4, it indicates the number of repeated bits that are the same as the MSB in adjacent positions in the weight (for example, 3).

[0063]

[0064] Table II

[0065] Continue to refer to Figure 3 In operation S330, according to the weight characteristics and the weight address, the weight read circuits WFAR[0], WFAR[1], the local multiplexer and pre-charge circuits 141[0], 141[1], and the sense amplifier circuits 151[0], 151[1] access the memory array 110 and the memory circuit (e.g., 211) to transfer the weights to the multiplication and accumulation circuit 160 for neural network operation.

[0066] For example, for an embodiment of the feature and weight reading stage (FD) in a short channel, Figure 4A and Figure 7As shown, the weight read controller FARC in the weight read circuit WFAR[0] generates a disable signal Dis[0] with a high level to the sense amplifier circuit 151[0] in response to the data Conf[7:0] indicating that the weight read circuit WFAR[0] operates on the weight of the short channel. The sense amplifier controller SAC[0] in the sense amplifier circuit 151[0] further generates an enable signal EN[0] in response to the disable signal Dis[0] to disable the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0]. As another option, as Figure 4A As shown, in the multiplication and accumulation operation period MACOG2, the sense amplifier VSA[0] does not access the memory array 110 to obtain the weight to generate the read signal SAOUT[0]. In addition, in response to the x address XA[7:0] and y address YA[2:0] of the weight, the memory circuit 211 is read to output the weight feature "0001101" stored in the memory circuit 211 as the weight to the multiplication and accumulation circuit 160.

[0067] Specifically, the control circuit FF-RWC controls the flip-flop circuit WFB-DFF in response to the mapping address WA[3:0] to output the data f_d0[1:0] including the stored weight feature as the weight to the multiplexer 222. The multiplexer 222 further transmits the data f_d0[0] as the signal To_MAC[0] to the multiplication and accumulation circuit 160 in response to the signal Ch[1:0]. In some embodiments, the signal Ch[1:0] is generated by the control circuit 130 based on the selected channel (e.g., one of the short channel, the medium channel, and the long channel).

[0068] The configuration of the weight read circuit WFAR[1] is similar to that of the weight read circuit WFAR[0]. Therefore, repeated description is omitted here.

[0069] In an embodiment of the feature and weight reading stage (FD) in the medium channel, as Figure 5A and Figure 8 As shown, during the multiplication and accumulation operation period MACOG2 for obtaining the weight "00001101", a read operation is performed on the memory circuit 211 to obtain the weight feature "11", wherein the weight feature "11" indicates three repeated bits that are the same as the MSB in the weight to be read and are transmitted through the data f_c0[1:0]. The weight read controller FARC in the weight read circuit WFAR[0] controls the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] to perform a read operation on the memory array 110 in response to the data Conf[7:0] and the data f_c0[1:0] to read the MSB of the weight to be read, as shown in FIG. Figure 8shown.

[0070] In addition, if Figure 8 As shown, the weight read controller FARC prohibits the read operation of the memory array 110 for a number of cycles (e.g., 3 cycles) through the disable signal Dis[0] according to the weight feature "11", and controls the sense amplifier VSA[0] to output the MSB "0" as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, the number of accesses to the memory array 110 is reduced to "3", saving the read energy corresponding to the memory array 110.

[0071] After disabling the local multiplexer and precharge circuit 141[0] and the sense amplifier circuit 151[0] for three cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "1101" in the weight "00001101").

[0072] Similarly, in order to read the weight "11100001" through the weight read circuit WFAR[1], the weight read controller FARC prohibits the read operation of the memory array 110 for a number of cycles (e.g., 2 cycles) through the disable signal Dis[1] according to the weight feature "10", and controls the sense amplifier VSA[1] to output the MSB "1" as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, a reduction of the number of accesses to the memory array 110 to "2" is achieved, saving the read energy corresponding to the memory array 110.

[0073] After disabling the local multiplexer and precharge circuit 141[1] and the sense amplifier circuit 151[1] for two cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "00001" in the weight "11100001").

[0074] In an embodiment of the feature and weight reading stage (FD) in the long channel, as Fig. 6A and Fig. 9 As shown, during the multiplication and accumulation operation period MACOG2, the weight read controller FARC in the adjacent weight read circuit WFAR[0] and the weight read circuit WFAR[1] is configured to control the corresponding read operation in response to the same weight feature "1" transmitted by the multiplexer 221 through the data f_d0[1:0].

[0075] Specifically, in order to obtain the weight "00001101" through the weight read circuit WFAR[0], a read operation is performed on the memory circuit 211 to obtain the weight feature "1" based on the data f_cL[1:0] associated with the data f_d0[1:0] and through the data f_c0[1:0]. The weight read controller FARC in the weight read circuit WFAR[0] controls the local multiplexer and precharge circuit 141[0] and the sense amplifier VSA[0] to perform a read operation on the memory array 110 in response to the data Conf[7:0] and the data f_c0[1:0] to read the MSB of the weight to be read. In addition, as Fig. 9 As shown, the weight read controller FARC prohibits the read operation on the memory array 110 within a number of cycles (for example, 2 cycles) according to the weight feature "1" through the disable signal Dis[0], and controls the sense amplifier VSA[0] to output the MSB "0" as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, a reduction in the number of accesses to the memory array 110 of "2" is achieved, saving the read energy corresponding to the memory array 110. After disabling the local multiplexer and precharge circuit 141[0] and the sense amplifier circuit 151[0] for two cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "01101" in the weight "00001101").

[0076] Similarly, in order to obtain the weight "11100001" through the weight read circuit WFAR[1], a read operation is performed on the memory circuit 212 to obtain the weight characteristic "1" based on the data f_cL[1:0] associated with the data f_d0[1:0] and through the data f_c1[1:0]. The weight read controller FARC in the weight read circuit WFAR[1] controls the local multiplexer and precharge circuit 141[1] and the sense amplifier VSA[1] in response to the data Conf[7:0] and the data f_c1[1:0] to perform a read operation on the memory array 110 to read the MSB of the weight to be read. In addition, as Fig. 9As shown, the weight read controller FARC prohibits the read operation on the memory array 110 within a number of cycles (for example, 2 cycles) through the disable signal Dis[1] according to the weight feature "1", and controls the sense amplifier VSA[1] to output the MSB "1" as part of the bit in the weight, instead of reading the remaining bits of the weight from the memory array 110. Therefore, referring to Table I, a reduction in the number of accesses to the memory array 110 of "2" is achieved, saving the read energy corresponding to the memory array 110. After disabling the local multiplexer and precharge circuit 141[1] and the sense amplifier circuit 151[1] for two cycles, the weight read controller FARC controls the resumption of the read operation on the memory array 110 to obtain the remaining bit data in the weight (for example, "00001" in the weight "11100001").

[0077] In operation S340 , the weight read circuit (eg, WFAR[0]) transmits the weight to the multiplication and accumulation circuit 160 for neural network layer operation.

[0078] In some embodiments, reference Figure 4A , Figure 5A , Fig. 6A and Fig.10 , the method 30 further includes the following operation: resetting the memory circuit (e.g., the memory circuit 211) to erase the weight feature for the second neural network layer operation. For example, after the last multiplication and accumulation operation period MACOGFinal in the plurality of multiplication and accumulation operation periods of the neural network layer operation LN, Figure 1 The control circuit 130 generates a signal rst to the memory circuit 211 to erase the data stored in the memory circuit 211. Therefore, the memory circuit 211 is ready to store the weight features used in the next neural network layer operation LN+1. In some embodiments, the configuration of the neural network layer operation LN+1 is similar to the neural network layer operation LN. Therefore, repeated descriptions are omitted here.

[0079] Figures 2 to 10 The configuration is provided for illustrative purposes. Various implementations are within the intended scope of the present disclosure. For example, in Fig.11 In an embodiment, four adjacent weight read circuits WFAR[0] to WFAR[3] are configured to collaborate to generate a weight signature based on a first weight from sense amplifier VSA[0], a second weight from sense amplifier VSA[1], a third weight from sense amplifier VSA[2], and a fourth weight from sense amplifier VSA[3].

[0080] Specifically, each of the weight reading circuits WFAR[0] to WFAR[3] determines whether the corresponding received weight includes a repeated bit based on the received read signal to generate a corresponding one of the signals FE_flag0 to FE_flag3. Fig.11 As shown, the AND gate 232 of the weight reading circuit WFAR[3] generates the signal FE_Lflag1 by performing an AND operation on the signals FE_flag2 and FE_flag3 from the weight extraction circuits BTF in the weight reading circuits WFAR[2] and WFAR[3], and further transmits the signal FE_Lflag1 to the AND gate 233 in the weight reading circuit WFAR[1]. The AND gate 232 generates a 1-bit shared weight feature to be stored in the memory circuit 211 by performing an AND operation on the signals FE_flag0, FE_flag1, and FE_Lflag1.

[0081] The present application provides a storage device with weight feature extraction for near memory computing. In addition, the present invention shows a method of operating a storage device, the method comprising minimizing the read frequency of a non-volatile memory. This is achieved by skipping the read operation of repeated bits corresponding to the most significant bit (MSB) and outputting the MSB based on the extracted weight feature. In contrast to some methods, using the configuration of the present application, some neural network models (e.g., Residual Neural Network (Residual Neural Network, ResNet) 20 model and ResNet32 model with Canadian Institute for Advanced Research (CIFAR)-100 data set) save approximately 30% of operating energy per unit area.

[0082] A method for operating a storage device is disclosed, the method comprising the following operations: generating a weight feature to be stored in a second memory different from the first memory based on at least one weight stored in a first memory, wherein the weight feature is associated with the number of repeated bits that are adjacent to the most significant bit and are the same as the most significant bit in the at least one weight; and accessing the first memory and the second memory according to the weight feature and the address of the at least one weight to transmit the at least one weight to a multiplication and accumulation circuit for a first neural network layer operation.

[0083] In some embodiments, generating the weight feature includes: transmitting the at least one weight as the weight feature to a second memory.

[0084] In some embodiments, accessing the first memory and the second memory includes: performing a read operation on the second memory in response to an address of the at least one weight to output the stored weight feature to the multiplication and accumulation circuit.

[0085] In some embodiments, generating the weight characteristic includes generating a weight characteristic equal to the number of repeated bits in the at least one weight.

[0086] In some embodiments, generating a weight feature includes: generating a weight feature having a first value when the number of repeated bits in the at least one weight is equal to a first predetermined value; and generating a weight feature having a second value different from the first value when the number of repeated bits in the at least one weight is greater than a second predetermined value, the second predetermined value being greater than the first predetermined value.

[0087] In some embodiments, accessing the first memory and the second memory includes: performing a first read operation on the second memory to obtain a weight characteristic from the second memory; performing a second read operation on the first memory to read the most significant bit of the at least one weight; prohibiting the second read operation on the first memory within a number of cycles based on the weight characteristic, and outputting the most significant bit as the bit data in the at least one weight to the multiplication and accumulation circuit; and resuming the second read operation on the first memory to obtain the remaining bit data in the at least one weight.

[0088] In some embodiments, the at least one weight includes a first weight and a second weight, and generating a weight feature includes: determining whether the first weight includes a repeated bit, and determining whether the second weight includes a repeated bit; and when both the first weight and the second weight include repeated bits, generating a weight feature with a non-zero value.

[0089] In some embodiments, accessing the first memory and the second memory includes: performing a first read operation on the second memory to obtain a weight characteristic from the second memory; performing a second read operation on the first memory to read the most significant bit of the first weight; prohibiting the second read operation on the first memory within a number of cycles based on the weight characteristic, and outputting the most significant bit as the bit data in the first weight to the multiplication and accumulation circuit; and resuming the second read operation on the first memory to obtain the remaining bit data in the first weight.

[0090] In some embodiments, accessing the first memory and the second memory further includes: performing a third read operation on the second memory to obtain a weight feature from the second memory; performing a fourth read operation on the first memory to read the most significant bit of the second weight; prohibiting the fourth read operation on the first memory within the number of cycles, and outputting the most significant bit of the second weight as the bit data in the second weight to the multiplication and accumulation circuit; and resuming the fourth read operation on the first memory to obtain the remaining bit data in the second weight.

[0091] In some embodiments, the method further includes: resetting the second memory to erase weight features for a second neural network layer operation.

[0092] A storage device is also disclosed. The storage device includes: a non-volatile memory array configured to store a plurality of weights; a plurality of sense amplifier circuits configured to access the non-volatile memory array; and a plurality of weight reading circuits coupled to the plurality of sense amplifier circuits, and each of the plurality of weight reading circuits includes: a weight extraction circuit configured to generate a flag signal in response to a readout signal, the readout signal being associated with a corresponding one of the plurality of weights and generated by a corresponding one of the plurality of sense amplifier circuits; a memory circuit configured to store a weight characteristic according to the flag signal; and a weight reading controller configured to control the corresponding sense amplifier circuit to output the first bit of the corresponding one of the weights as part of the bit in the corresponding one of the weights in response to control data and characteristic data associated with the weight characteristic.

[0093] In some embodiments, the memory circuit includes a flip-flop circuit.

[0094] In some embodiments, the storage device further includes: an address decoder configured to generate an image address based on the address of the corresponding one of the weights to be read, the image address indicating the location of the weight feature stored in the flip-flop circuit. The memory circuit further includes: a control circuit configured to control the flip-flop circuit to output feature data to a weight read controller in response to the image address. The weight read controller is further configured to prohibit the corresponding sense amplifier circuit from accessing the non-volatile memory array when the corresponding sense amplifier circuit outputs the first bit as the portion of the bit in the corresponding one of the weights.

[0095] In some embodiments, the first bit is the most significant bit in the corresponding one of the weights.

[0096] In some embodiments, the storage device further comprises: a logic gate circuit configured to generate a shared weight feature according to a flag signal from a weight extraction circuit in at least two of the plurality of weight reading circuits.

[0097] In some embodiments, the logic gate circuit is further configured to transmit the shared weight characteristic to a memory circuit in one of the at least two of the plurality of weight reading circuits.

[0098] In some embodiments, the storage device further includes: a first AND gate configured to generate an output signal according to a flag signal from a first weight reading circuit and a weight extraction circuit in a second weight reading circuit among the plurality of weight reading circuits; and a second AND gate configured to generate a shared weight feature according to the output signal from the first AND gate and a flag signal from a weight extraction circuit in a third weight reading circuit and a fourth weight reading circuit among the plurality of weight reading circuits. The second AND gate is further configured to transmit the shared weight feature to a memory circuit in one of the first weight reading circuit to the fourth weight reading circuit.

[0099] A method for operating a storage device is also disclosed. The method includes: extracting a weight feature of at least one weight and storing the weight feature in a first memory in a first multiplication and accumulation operation period of a neural network layer operation; based on the weight feature, prohibiting a sense amplifier from accessing a second memory in at least one cycle in a second multiplication and accumulation operation period of the neural network layer operation, and accessing the first memory, the second memory, or a combination of the first memory and the second memory to output the at least one weight; and erasing the first memory in a last multiplication and accumulation operation period of the neural network layer operation.

[0100] In some embodiments, when the neural network layer operation corresponds to a short channel, accessing the first memory, the second memory, or a combination of the first memory and the second memory to output the at least one weight includes: performing a read operation on the first memory in response to the address of the at least one weight to output a weight feature as the at least one weight.

[0101] In some embodiments, when the neural network layer operation corresponds to a medium channel or a long channel, disabling the sense amplifier from accessing the second memory based on the weight characteristics and accessing the first memory, the second memory, or a combination of the first memory and the second memory to output the at least one weight includes: performing a read operation on the second memory within a first cycle to obtain the most significant bit of the at least one weight; disabling the sense amplifier within a number of cycles and outputting the most significant bit in the number of cycles; and resuming the read operation on the second memory within the remaining cycles to obtain the remaining bits in the at least one weight.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for operating a storage device, characterized in that: include: generating a weight characteristic to be stored in a second memory different from the first memory based on at least one weight stored in the first memory, wherein the weight characteristic is associated with the number of repeated bits that are adjacent to a most significant bit in the at least one weight and are identical to the most significant bit; as well as The first memory and the second memory are accessed according to the weight characteristic and the address of the at least one weight to transmit the at least one weight to the multiplication and accumulation circuit for the first neural network layer operation.

2. The method according to claim 1, characterized in that: The weight features include: The at least one weight is transmitted to the second memory as the weight feature.

3. The method according to claim 2, characterized in that Accessing the first memory and the second memory includes: A read operation is performed on the second memory in response to the address of the at least one weight to output the stored weight feature to the multiplication and accumulation circuit.

4. The method according to claim 1, characterized in that Generating the weight feature includes: The weight characteristic is generated to be equal to the number of the repeated bits in the at least one weight.

5. The method according to claim 1, characterized in that Generating the weight feature includes: When the number of the repeated bits in the at least one weight is equal to a first predetermined value, generating the weight feature having a first value; and The weight feature having a second value different from the first value is generated when the number of the repeated bits in the at least one weight is greater than a second predetermined value, the second predetermined value being greater than the first predetermined value.

6. The method according to claim 5, characterized in that Accessing the first memory and the second memory includes: Performing a first read operation on the second memory to obtain the weight feature from the second memory; performing a second read operation on the first memory to read the most significant bit of the at least one weight; prohibiting the second read operation on the first memory within a number of cycles according to the weight feature, and outputting the most significant bit as the bit data in the at least one weight to the multiplication and accumulation circuit; and Resume performing the second read operation on the first memory to obtain remaining bit data in the at least one weight.

7. A storage device, characterized in that: include: a non-volatile memory array configured to store a plurality of weights; a plurality of sense amplifier circuits configured to access the non-volatile memory array; as well as A plurality of weight reading circuits are coupled to the plurality of sense amplifier circuits, and each of the plurality of weight reading circuits comprises: a weight extraction circuit configured to generate a flag signal in response to a readout signal associated with a corresponding one of the plurality of weights and generated by a corresponding sense amplifier circuit of the plurality of sense amplifier circuits; a memory circuit configured to store a weight feature according to the flag signal; as well as The weight read controller is configured to control the corresponding sense amplifier circuit to output the first bit of the corresponding one of the weights as part of the bits in the corresponding one of the weights in response to control data and characteristic data associated with the weight characteristics.

8. The storage device according to claim 7, characterized in that: The memory circuit includes a flip-flop circuit.

9. The storage device according to claim 7, characterized in that: Also includes: an address decoder configured to generate an image address based on the address corresponding to the one weight to be read, the image address indicating the location of the weight feature stored in the flip-flop circuit, The memory circuit further comprises: a control circuit configured to control the flip-flop circuit to output the feature data to the weight reading controller in response to the image address, The weight read controller is further configured to prohibit the corresponding sense amplifier circuit from accessing the non-volatile memory array when the corresponding sense amplifier circuit outputs the first bit as the portion of the bits in the corresponding one weight.

10. A method for operating a storage device, characterized in that: include: In a first multiplication and accumulation operation period of the neural network layer operation, extracting a weight feature of at least one weight and storing the weight feature in a first memory; Based on the weight feature, in a second multiplication and accumulation operation period of the neural network layer operation, prohibiting the sense amplifier from accessing the second memory and accessing the first memory, the second memory, or a combination of the first memory and the second memory for at least one cycle to output the at least one weight; as well as The first memory is erased during a last multiplication and accumulation operation period of the neural network layer operation.

Images