Method of in-memory computing and device thereof

Abstract

Embodiments of the invention provide a method of in-memory computation that includes monitoring partial sums of multiply-accumulate computations for certain conditions. When certain conditions are met, memory contents are read using reduced read energy instead of using regular read energy. The reduced read energy can be provided by lowering a pre-charge voltage, suppressing a pre-charge voltage, or providing a ground signal, and / or by reducing a voltage hold time (i.e., reducing the time during which a pre-charge voltage is provided and / or discharged). Embodiments of the invention also provide a device for in-memory computation.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of electronic circuits, and more specifically, to methods and devices for performing in-memory computation. Background Technology

[0002] A multiply-accumulate register can be used to multiply input data with corresponding weighted data in a word-by-word, bit-by-bit manner. Input data is read from memory, multiplied by the weights, and the result is stored in the multiply-accumulate register. This result can be used in various applications, such as for artificial intelligence computations. Summary of the Invention

[0003] One aspect of the present invention provides a method for performing in-memory computation, comprising: determining whether a partial sum of an in-memory computation (CIM) operation is positive to obtain a first result; determining a selected position of the partial sum to be converted from 0 to 1 to obtain a second result; and adjusting the read configuration of a read operation of a storage unit of the CIM in response to both the first result and the second result being true.

[0004] Another aspect of the present invention provides a method for performing in-memory computation, comprising: reading a first set of bits from a set of weighted vectors from memory using a first read energy; multiplying a set of inputs by the first set of bits to obtain a first product; adding the first product to an accumulated sum of products; enabling a reduced read energy signal when the accumulated sum of products is positive and a bit condition of the accumulated sum of products changes from 0 to 1; and reading a second set of bits from the weighted vector set from memory using a second read energy less than the first read energy.

[0005] Another aspect of the present invention provides an apparatus for performing in-memory computation, comprising: a computer-readable memory storing an input set and a corresponding weighted vector set; a multiply-accumulate device including an adder, a multiplier, and a partial sum (PS) register configured to store an accumulated result of an iterative product sum operation from the input set and the corresponding weighted vector set; a multiplexer configured to provide a bias voltage to a sense amplifier to read the weighted vectors; and dynamic readout logic configured to evaluate the partial sum, determine whether a reduced readout energy (RRE) signal should be enabled, and enable the reduced readout energy signal, providing the reduced readout energy signal to the multiplexer. Attached Figure Description

[0006] The various aspects of the invention can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that, in accordance with standard industrial practice, the various components are not drawn to scale. In fact, for clarity of discussion, the dimensions of the various components may be arbitrarily increased or decreased.

[0007] Figure 1 and Figure 2 The diagram illustrates the input nodes, weighted vectors, and summations that can be used according to some embodiments.

[0008] Figures 3-6 The various stages of multiply-accumulate computation (MAC) according to some embodiments are illustrated.

[0009] Figure 7 A diagram of an in-memory computing (CIM) system for providing MAC operations is shown according to some embodiments.

[0010] Figure 8 A high-level block diagram for dynamic read operations is shown according to some embodiments.

[0011] Figure 9 An example implementation of MAC block 160 is shown.

[0012] Figure 10 A flowchart of a process flow 200 for performing MAC operations is shown according to some embodiments.

[0013] Figure 11 and Figure 12 A flowchart of a process flow 240 for evaluating whether a portion and a PS meet dynamic read conditions is shown according to some embodiments.

[0014] Figure 13 An example implementation of a DYNR block for evaluating and determining whether the RRE signal is enabled, according to some embodiments, is shown.

[0015] Figure 14 This illustrates a set of instance logic conditions that can be enabled according to some embodiments, rather than a one-to-one input of the selection bits of the PS.

[0016] Figures 15 to 22 Examples of calculations and demonstrations of the operation of the DYNR block according to some embodiments are shown.

[0017] Figure 23 A graph illustrating the reduced read power that can be achieved when reduced read power is enabled is provided according to some embodiments.

[0018] Figure 24 The relationship between read voltage and sensing yield is shown according to some embodiments.

[0019] Figure 25 A simplified schematic diagram illustrating a read path of an I / O associated with an array, according to some embodiments, is shown.

[0020] Figure 26 Illustrations are shown according to some embodiments Figure 25Enlarged image.

[0021] Figure 27 A timing diagram and a view of a sense amplifier are shown according to some embodiments.

[0022] Figure 28 This shows a view of the logic circuitry that does not provide pre-charge if reduced read power is enabled. Detailed Implementation

[0023] This invention provides numerous different embodiments or instances for implementing various features of this disclosure. Specific examples of components and arrangements are described below to simplify the invention. These are, of course, merely examples and are not intended to limit the invention. For example, in the following description, forming a first component on or over a second component may include embodiments where the first and second components are formed in direct contact, and may also include embodiments where an additional component may be formed between the first and second components, such that the first and second components are not in direct contact. Furthermore, reference numerals and / or characters may be repeated in various instances of the invention. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed. It should be understood that signals may be enabled as high 1 or low 0, and unless the context or convention otherwise indicates, “1” as used herein is understood to mean “affirmative,” and “0” as used herein is understood to mean “undetermined” unless the context or convention otherwise indicates. Depending on the device and design, those skilled in the art can readily reverse these signals as needed.

[0024] In the field of artificial neural networks, machine learning acquires input data, performs some calculations on the input data, and then applies activation functions to process the data. The output of the activation function is essentially a simplified representation of the input data. The input data can be data nodes in a node layer. Figure 1 An example of a 3×3 convolution is shown, which can be used to process image data in machine learning. Image 10 consists of individual pixels 11. The image can be represented in a color space, such as RGB (Red-Green-Blue) or HSL (Hue-Saturation-Emissivity), with each pixel assigned a value for each color space variable. Node 12 of the image is a 3x3 pixel block, and each pixel 11 in node 12 has an input value I for each color space variable of pixel 11 in node 12. 1-9 One possible computation method for a 3x3 convolution is to use a product-sum, where each input value I... 1-9 Multiply by the weighting value W of weighting matrix 14 respectively 1-9With each multiplication, the sum of each product can be maintained. This product summation calculation can be called Multiplicative Accumulation Calculation / Operation (MAC)16. During the calculation, the intermediate values ​​can be called Accumulated Product Sum (APS). At the end of the calculation, the APS is given as the output of MAC 16. This output can then be provided to the activation function for evaluation.

[0025] Figure 2 In a more general way, that is, for any input node of length N, the following is explained: Figure 1 The concepts shown in the diagram. Input I0–I N-1 Each is multiplied by the weighted vector W0–W N-1 These values ​​are then summed using a product summation (MAC) calculation. The MAC can then be output as O and optionally provided to the activation function or used in other ways.

[0026] It is possible to write computer programs that will execute on a general-purpose processor, including, for example, a for-loop that performs MAC operations on the INPUT and WEIGHT arrays, as shown in the following pseudocode:

[0027] Initialize a counter integer to 0.

[0028] Initialize a storing variable (eg, APS) to 0.

[0029] Provide an INPUT array having the length n with input values.

[0030] Provide a WEIGHT array having the length n with signed weightvalues.

[0031] For counter = 0, counter <n,counter++{

[0032] APS = APS + (INPUT * WEIGHT).

[0033] }

[0034] MAC = APS.

[0035] Provide the MAC address as output.

[0036] To improve efficiency, this algorithm can be implemented in dedicated hardware, such as in an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). However, implementing this logic in dedicated hardware (e.g., an ASIC) involves using binary mathematics within a digital logic block. Such a hardware implementation can be referred to as an in-memory computation (CIM) implementation. A CIM implementation involves reading data from memory, including input data and weight data, and performing simple operations on them, including MAC operations. As described in this paper, a hardware-based CIM implementation uses binary mathematics to compute MACs.

[0037] Figure 4 The binary representation of MAC input data, weighted vectors, and MAC is shown for implementation in hardware via an algorithm. The hardware implementation is discussed in more detail below in conjunction with the dynamic read module. For data points in a node, the input data is represented as unsigned node values, such as magnitude. The length of the input data is N bits. N can be, for example, 4 bits, 8 bits, 16 bits, etc. For example, if N is 8, each input value is between 0 and 255. The weighted vector is a signed weighted value in two's complement format. Therefore, negative numbers will start with 1 in the most significant bit (MSB). The length of each weighted vector is K bits. N can be equal to K or can be different values. For example, if K is 8 bits, each weight value may be between -128 and 127. In the representation, for input values, the i-th input corresponds to the input index of the input data point in the node. Each weight will have a corresponding i-th weight index of the weighted vector. In other words, there is a one-to-one correlation between the i-th input and the i-th weighted vector.

[0038] The length of each i-th input can differ from that of each i-th weighted vector. The inputs are sorted from least significant bit (LSB) to most significant bit (MSB). For example, the r-th value of the i-th input is equal to I. i，r ×2 r The weighted vectors are ordered in reverse order of the input, i.e., from MSB to LSB. For example, the j-th value of the i-th weighted vector is equal to W. i，j ×2 K-j-1 In the input, bit k = 0 is the least significant bit (LSB) and has a value I for the i-th input. i，0 ×2 0 .

[0039] like Figure 3As shown, the total number of bits produced by MAC is equal to the logarithm of N plus K plus M (base 2), rounded up to the nearest integer. For example, if the input number in the node is 9 (e.g., corresponding to a point convolution of 9) and N and K are both 8, then the number of bits in the MAC output is 8 + 8 + Roundup(log29) = 20. This value can also be expressed as Roundup(N + K + log2M).

[0040] Given these relationships, Figure 4 The mathematical formula for processing input values ​​and weighted vectors bit by bit is shown. Each input value is multiplied by each bit of the weighted vector, and the sum is taken after each iteration. The left side of the equation is a general formula for the sum of the products of the i inputs and the corresponding i weighted vectors. This sum can be decomposed into the right side of the equation, which includes a first term for processing the sign bit of the weighted vectors and a second term for processing the remaining bits.

[0041] The first term represents the sum of the products of the N-bit unsigned input and the sign bits of each of the K-bit weighted signed vectors. For example... Figure 3 As shown, the MSB of the weighted vector stores the sign bit and is represented as the 0th bit of the weighted vector, where bit j = 0. The first term multiplies the input by the 0th bit of the weighted vector (representing the sign bit), and multiplies this result by the 0th bit, which equals 2. K-1 The place value is then recorded as a negative value. Essentially, the product of the input and the sign bit determines the maximum negativeness of the weighted vector. For example, if the weighted vector is 8 bits and negative, W... i，0 =1, then the sign bit represents 2. 7 The position value is "1". In binary mathematics, this is equivalent to taking the 2's complement of the input and shifting it left by 7 times. This is for each input I i The iteration is complete; the first term represents the sum of all these products. This occurs when the corresponding weighted vector is not negative, i.e., W. i，0 If the value is 0, a zero will be added.

[0042] The second item includes two implementation options. In the first option, the second item includes two nested summation operations. The inner summation represents the weighted vector W. i Each of the remaining j bits is multiplied by the input I. i Then multiply by the weighted vector W i The sum of the results for the corresponding j-th position value. In other words, for a specific input I... i The entire input I i Multiply by each j-th bit of the weighted vector and the corresponding j-th position value of the j-th bit (2)K-j-1 And add the products together. The outer summation is repeated for each input I. i and weighted vector W i The internal summation is performed, and all these summations are added together.

[0043] In the second option, the second term includes two nested summation operations; however, they are in the reverse order of those used in the first option. The inner summation represents each input I... i Multiply by the sum of the specific weighted vector bit values ​​of each of the K-bit weighted vectors. Add these values ​​together. Then, for each input I... i Multiply by the next weighted vector bit of each bit in the K-bit weighted vector. In this way, all weighted bits of each position value have been processed before moving to the next position value.

[0044] Figure 5 It shows Figure 4 An example implementation of the summation formula is shown. It uses a single input I and a single weighted vector W, where M = 1, N = 8, and K = 8. I0 = 77 (01001101) and W0 = 116 (01110100). In the summation, the first term can be rearranged as –(77·0·2) 7 ) = 0000 0000. The second term can be rearranged as 77·(1·2) 6 )+77·(1·2 5 )+77·(1·2 4 )+77·(0·2 3 )+77·(1·2 2 )+77·(0·2 1 )+7·(0·2 0 ) = 77·2 6 +77·2 5 +77·2 4 +77·2 2 =4928(1 00110100 0000)+2464(1001 0000)+1232(1001 10100000)+308(1 0011 0100)=8932(00100010 0110 0100). The sum of the first term (0) and the second term is 8932(0010 0010 1110 0100).

[0045] Conversely, if the weighted vector is negative, i.e., -116(1000 1100), then the result is as follows: –(77·1·2 7 )=–(0100 1101)·2 7 =1011 0011·2 7=101 1001 1000 0000. The second term can be rearranged as 77·(0·2) 6 )+77·(0·2 5 )+77·(0·2 4 )+77·(1·2 3 )+77·(1·2 2 )+77·(0·2 1 )+77·(0·2 0 ) = 77·2 3 +77·2 2 =616(0010 0110 1000)+308(00010011 0100)=924(0011 1001 1100). Adding the first and second terms together gives a total of -8932(1101 1101 0001 1100).

[0046] As seen in this example, when the weighted vector is negative, bitwise math sets the weighted vector to -128 times the input, and then adds the positive parts back to the negative number (making the negative number smaller) until the final result is reached. When the weighted vector is positive, the first term will result in "0", and the second term will be the bitwise sum of the remaining bits of the weighted vector.

[0047] Figure 6 Will Figure 4 The right-hand term is decomposed into two parts to represent the computational state at a given point, for example, after processing the n bits of the weighted vector W. The first part ( The partial sum of the MAC operation is provided by the nth bit of the weighted vector W. (Part Two) This represents the remaining unknown partial sum from position n+1 to position K-1 of the weighted vector W. At any given position n, the known partial sums are collected as cumulative partial sums, while the unknown remaining sums have not yet been calculated.

[0048] The embodiments evaluate the known partial sum to determine whether the remaining computation can be performed by reading the weighted bits used in subsequent computations from memory using reduced read energy. Using reduced read energy increases the likelihood of incorrect memory reads, or, as noted below regarding some embodiments, forces the remaining unread bits to "0". This allowed error effectively leads to an ordering estimate of the unknown remaining sum. This error may be allowed for several reasons. First, because the weighting vector is processed from MSB to LSB, the unknown remaining sum is typically much smaller than the known partial sum and contributes much less to the final MAC value than the earlier evaluated bits represented by the known partial sum. For example, in the following regarding... Figures 15-22In the example calculation, if fully computed, the MAC output will be 38865. In this value, the last bit of the weighted vector contributes only 253, the last two bits only 1317, the last three bits only 2641, the last four bits 6017, and the last five bits 15601. These represent 0.7%, 3.4%, 6.8%, 15.5%, and 40.1% of the MAC output value 38865, respectively. While these percentages and values ​​are specific to the inputs and weighted vectors shown below, they indicate (as one would expect) that the contribution of the lower significant bits of the weighted vector has a relatively small impact on the final MAC value. Secondly, the MAC output is interpreted as some representation of the input data (rather than the actual data itself), so some errors may be tolerable because the final representation is itself a derived representation of the input data. Therefore, the embodiment provides the ability to test the accumulated product summation to determine whether the bits used to compute the unknown residual sum can be read with reduced read energy.

[0049] Using a reduced read energy (RRE) signal, the embodiments provide a way to reduce the computational energy of a multiply-accumulate function by monitoring partial sums and accumulations, and reducing memory read energy used to read input values ​​from memory for the remaining computations if certain conditions are met. Reducing memory read energy increases the risk of reading erroneous values ​​but lowers energy costs. As described above, this effectively results in an estimated or approximate final accumulated value. Because these conditions are monitored, precise values ​​are not required, and the estimates are considered sufficient for input processing purposes. When the conditions for reducing read energy are met, the embodiments can implement dynamic read operations to reduce read energy consumption by reducing read voltage, shortening read latency, or skipping read operations. These embodiments will be described in detail below.

[0050] For example, suppose a nominal voltage of 0.2V is the read voltage (or bias voltage) used to read memory locations. If the read voltage can be reduced to 0.1V, the total energy required to perform a multiply-accumulate operation can be significantly reduced, provided that the following conditions are met. For example, the average read energy can be characterized by the following equation:

[0051] RE AVG =P1×E1+P2×E2

[0052] Where P1 is the probability of the read voltage being the nominal read voltage Vi (e.g., 0.2V), E1 is the energy consumption when the read voltage is the nominal read voltage V1, P2 is the probability of the read voltage being a reduced read voltage V2 (e.g., 0.1V), and E2 is the energy consumption when the read voltage is a reduced read voltage V2. As an example of energy consumption, for MRAM devices, E1 can be approximately 256 fj / bit, and E2 can be approximately 144 fj / bit. If P1 = P2 = 50%, then the average read energy is 0.5 × 256 + 0.5 × 144 = 200 fj / bit. In this case, the energy saving will be (256 - 200) / 256 = 22%. Of course, one will understand that these values ​​are merely examples, and other values ​​can be used, depending on the memory type, read voltage, and energy consumption at that read voltage.

[0053] Figure 7 A CIM system diagram for providing MAC operation according to some embodiments is shown. This system may be referred to as MAC system 100. MAC system 100 includes several blocks. Memory array 110 (or memory 110 or storage device 110) stores input values ​​and weighting vectors. Memory array 110 can be any suitable array of any suitable storage device. For example, memory array 110 may include resistive RAM (RRAM), magnetic RAM (MRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), phase-change RAM (PCRAM), etc., or combinations thereof. Word line driver (WLDR) 120 can be used to drive word lines to access bits from memory array 110. Control block 130 includes x-decoders for word lines and y-decoders for bit lines and sense lines. It also includes timing control for read and write operations. Multiplexer (MUX) 140 selects bit lines and sense lines based on decoded signals from the control. Input / output (IO) block provides sense amplifiers for input / output operations from memory array 110. The Multiply-Accumulate Unit (MAC) block 160 provides functional units for performing MAC operations, such as adders, multipliers, registers, etc. The Dynamic Read (DYNR) block 170 calculates whether the reduced read energy condition is met and enables the RRE signal based on whether the reduced read energy condition is met.

[0054] Figure 8A high-level block diagram for a dynamic read operation is shown according to some embodiments. In a dynamic read operation, several system blocks work together to determine whether data provided to MAC block 160 is read using reduced read energy or nominal read energy. Dynamic Read (DYNR) block 170 provides a reduced read energy (RRE) signal to multiplexer (MUX) block 140. The initial conditions of the input may depend on whether the read configuration desires greater energy efficiency or reliability. According to some embodiments, depending on the input, multiplexer block 140 provides a dynamic read bias voltage V1 or V2 for pre-charging the bit-line sense amplifier input of input / output (IO) block 150. IO block 150 is used to read bits of a weighted vector W from a memory device, and these bits are provided to multiply-accumulate compute (MAC) block 160. Input I is also provided to MAC block 160. Input vector I and weighted vector W have a one-to-one correspondence such that the number M of input vectors is equal to the number M of weighted vectors. A partial sum PS (either a portion of the entire partial sum (i.e., selected bits) or the entire partial sum) is provided to DYNR block 170, which can use this partial sum PS to test the partial sum against a set of conditions that determine whether the RRE signal is enabled from DYNR block 170 and returned to MUX 140 for further processing. In some embodiments, weighted vectors are processed separately, one complete weighted vector at a time, and the sum is accumulated as a partial sum PS. In such an embodiment, the output of the MAC is another partial sum accumulated in another MAC register. In other embodiments, such as those discussed in detail below, each weighted vector is partially processed such that all j bits of each weighted vector are processed for each input, then j+1 bits of each weighted vector are processed, and so on.

[0055] Figure 9 An example implementation of MAC block 160 is shown. W0 to W M-1 W in each j Bits are provided to weight register 161. Inputs I0 to I M-1 A set of input registers 162 are provided. Each of these inputs is multiplied at multiplication block 163 by W of each weighted vector. j Bitwise multiplication. The result is provided to adder block 164, which, after shifting the previously stored partial sum, adds the multiplication result to the previously stored partial sum. The result is then stored back into the partial sum register 165. The partial sum PS can be provided to DYNR block 170.

[0056] It should be understood that the sub-blocks of MAC block 160 can be configured in various ways. In some embodiments, input register 162 holds one input vector at a time, while in other embodiments, input register 162 may hold all input vectors of the data node. In some embodiments, weight register 161 holds a signed weighted vector or corresponding bits from each weighted vector, while in other embodiments, weight register 161 holds one bit from the weighted vector at a time. Multiplication block 163 may utilize a shift register to multiply the input vector by the weighted vector bitwise, from the most significant bit to the least significant bit of the weighted vector. Then, after the input vector is multiplied by the weighted vector, the result can be provided to adder block 164, and then to partial sum block 165.

[0057] Figure 10 A flowchart of a process flow 200 for performing MAC operations according to some embodiments is shown. At block 210, if the Reduce Read Energy (RRE) signal is valid, the next weighted bit is read using the energy reduction process; if the RRE signal is invalid, the next weighted bit is read using the nominal process. As described above, the energy reduction process may include using a reduced bias voltage, shortened timing, and / or skipped reads (e.g., by reducing the bias voltage to 0, causing the remaining bits to be read as '0'). At block 220, as part of the MAC multiplication-accumulation summation process, the partial sum-accumulation process is performed in a word-input and bit-weighted manner. At block 230, the validity of the RRE is evaluated. If it is invalid, the partial sum (PS) is evaluated at block 240 against the dynamic read conditions. If the RRE is valid, then in some embodiments, the RRE signal remains valid and will not return to an invalid state unless it is reset. Therefore, if the RRE is valid, the process can jump to block 270 to determine whether all weighted bits have been processed. Again at block 250, if PS meets the conditions for enabling dynamic read operations, then at block 260, RRE will be set to valid; otherwise, the process can proceed to 270 and evaluate whether all weighted bits have been processed. If all weighted bits have been processed, then PS is output as MAC at block 280. If all weighted bits have not been processed, then at block 290 the system advances to the next weighted bit of the weighted vector.

[0058] Figure 11 A process flow 240 is shown for evaluating whether the PS meets the dynamic readout conditions (see [link]). Figure 10 The flowchart is shown. In block 241, data is received from the PS. The received data can be the entire APS or the selection bits from the PS. In block 242, bit 19 of the PS is checked (PS...). 19The sign bit (or the sign bit) determines whether the value of PS is positive or negative. If PS is negative, the process can jump to block 247, thus determining that PS does not meet the dynamic read condition. If PS is positive, further evaluation can proceed. If PS is not 20 bits long, the chosen bit can be any sign bit of PS. For example, if PS is 24 bits long, the sign bit would be the sign bit of PS. 23 Procedure blocks 243, 244, 245, and 246 each test a specific bit of PS to determine if it has moved from 0 to 1. Specifically, block 243 tests PS... 11 Block 244 test PS 12 Block 245 test PS 13 And block 246 test PS 14 These bit values ​​are merely examples. More or fewer than four PS bits can be used for testing. Furthermore, the bit indices for testing may differ from bits 11, 12, 13, and 14. After exploring an example of this process, we will discuss the selection of test bits in further detail below.

[0059] In some embodiments, such as in Figure 11 As shown, one or more of the shown bits 11, 12, 13, and / or 14 can be tested. In some embodiments, the test element can be enabled or disabled as needed for each bit. Testing an earlier bit will cause the PS to satisfy the dynamic read condition at block 248 at an earlier stage of the process. Once an earlier bit has been tested, for example, bit 11 has been tested and the condition has been met, there is no need to test subsequent bits, and therefore the process can immediately move to process block 248, i.e., the PS satisfies the dynamic read condition.

[0060] exist Figure 12 In other embodiments, logical combinations of bits can be used. The logical combination shown is just an example, and any logical combination can be used as needed. Similar elements are labeled with similar reference numerals. However, in block 244, bit PS 11 and position PS 12 Both are checked to determine if they have moved from 0 to 1. In block 245, bit PS 11 PS 12 and position PS 13 They are all checked to determine if they have all moved from 0 to 1. In block 246, bit PS 11 PS 12 PS 13 and position PS 14 All are checked to determine if they have all moved from 0 to 1. When one of these conditions is met, the flow moves to block 248 to determine if PS meets the dynamic read condition.

[0061] Figure 13An exemplary implementation of DYNR block 170 is shown for evaluating and determining whether the RRE signal is enabled. DYNR block 170 accepts an input including a reset input RST, which, when enabled, indicates that the MAC process has been reset. The RST signal can be enabled, for example, by control block 130 after the MAC process has completed. When the RST signal is 1, the MAC process should be reset. When the RST signal is zero, the MAC process can continue. DYNR block 170 also accepts an input NZ, which indicates that the input is not zero. If NZ is 0, no calculation should be performed because the input multiplied by the weighted vector will always result in a zero output. If NZ is 1, the input is not zero, and the MAC process may continue. 19 Assuming a 20-bit portion and 165 (see...) Figure 9 If the part and 165 have another bit length b, then the sign bit will be PS. b-1 This will be the bit being checked, not the bit PS. 19 Check bit PS 19 This determines whether the part sum and 165 are negative, i.e., "1". If the part sum and 165 are negative, the RRE signal is not enabled. If the part sum and 165 are positive, the RRE signal may be enabled, depending on the values ​​of the other bits in the part sum and 165.

[0062] Figure 13 Bit PS according to some embodiments is also shown. 11 PS 12 PS 13 and PS 14 These bits can be received by DYNR block 170. Each of these bits can also have a corresponding enable bit signal from control block 130, which enables the transmission gate for the corresponding bit signal. For example, the transmission gate TPS 11 It can have an enable input, which allows the transmission gate to switch from input PS. 11 Transmit to output PS X TPS 11 The enable input can also originate from an input, but is not shown for simplicity. This enable input can come from control block 130 or can be generated internally. The enable input allows for use with PS 11 PS 12 PS 13 and PS 14 The signal is selectively transmitted to the output signal PS. X For example, DYNR block 170 can test the least significant bit (PS) when j=0. 11 When j=1, test the next bit (PS) 12 When j=2, test the next bit (PS) 13 When j≥3, test the next bit (PS) 14)。Or in another example, the DYNR block 170 may test the least significant bit PS when j = ≤ 1 11 , the next bit (PS when j = 2 12 ), the next bit (PS when j = 3 13 ), and the next bit (PS when j ≥ 4 14 ). Other configurations are possible. For example, in some embodiments, the selected bits may be based on the sum value of the inputs. The maximum sum is (N 8 - 1) × M, where N is the bit length of the input and M is the number of inputs. For N = 8 and M = 9, the maximum input sum IS is 2295. In one embodiment, for example, if the input sum IS is in the bottom quartile (1 < IS < 573), the least significant bit PS 11 may be enabled to select to the output signal PS X . If the input sum IS is in the second quartile (574 < IS < 1147), the next bit PS 12 may be enabled. If the total input sum IS is in the third quartile (1148 < IS < 1721), the next bit PS 13 may be enabled. If the total input sum IS is in the fourth quartile (1722 < IS < 2295), the next bit PS 14 may be enabled.

[0063] It should be understood that the bits (PS 11 , PS 12 , PS 13 and PS 14 ) used for testing are based on the assumed partial sum 165 of 20 bits. If the number of inputs M is larger or smaller or the bit length N of the input is larger or smaller, then other bits of the test partial sum 165 are suitable. For example, the index of the least significant bit tested may be equal to the number of bits N + Roundup(log2M) - 1. Then the next three bits may be indexed starting from that bit. In the described example, this would result in 8 + 4 - 1 = 11, and the next three indices 12, 13, and 14. Since the partial sum PS 165 is iteratively constructed, the PS stores values that are iteratively shifted left as each weight bit of the weighted vector is processed. This means that the bits to be tested should be based on the bit length of the input, the bit length of the weighted vector, and the number of inputs in the input node. In cases where the size of the partial sum is also determined based on these factors, the test bits may be approximated based on the length of the partial sum. In some embodiments, the test bits may be in the upper half of the partial sum, but other bits may also be used.

[0064] Still referring to Figure 13 , the output PS X and PS 19The inverted signals are provided together to the NAND gate. If both are 1, the output of the NAND gate will be 0; otherwise, it will be 1. This output is fed to the S side of the SR latch, and the R side of the SR latch receives the inverted signal of the RST signal. The outputs Q and Q' of the SR latch, along with the RST and NZ signals, are provided to the corresponding NOR gates. The outputs of the NOR gates provide RRE and RZ respectively. <1> or RRE <0> Signal. That is, the inverted output of the NOR gate represents RRE. <1> and RRE <0> The value of . When the RST signal is 0 and the NZ signal is 1, only one of these outputs can be "1" at a time, because they are based on the opposite signals Q and Q' from the SR latch. The RRE is described below. <0> When = 0, the canonical conditions of the Vread bias are used. When RRE <1> When = 0, a risky read using the Vread bias is employed. If RRE <0> =0 and RRE <1> =0, which is considered a high-priority read and will use a higher Vread. Unless otherwise stated, for RRE <1> The reference indicates RRE <1> =0 and RRE <0> =1, thus achieving a reduced bias voltage, i.e., a risky read. Similarly, for RRE <0> The reference indicates RRE <0> =0 and RRE <1> =1, thus enabling the standard bias voltage, i.e., safe reading. This is understandable. Figure 13 The logic provided is just one instance; other implementations are also possible.

[0065] The truth table below shows the signals RST, NZ, and PS. 19 PS X 、S、R、Q、Q'、RRE <1> and RRE <0> The relationship between them is as follows: The letter X indicates that the output is independent of the signal, and the letter NC indicates that there is no change.

[0066] RST NZ <![CDATA[PS 19 ]]> <![CDATA[PS X ]]> s R Q Q' RRE<1> RRE<0> 1 1 0 0 0 1 0 1 0 0 0 2 X 0 X X X X X X 0 0 3 0 1 1 X 1 1 NC NC 1 0 4 0 1 0 0 1 1 NC NC 1 0 5 0 1 0 1 0 1 0 1 0 1

[0067] Table 1

[0068] In the first row of Table 1, the RST signal is activated, simultaneously resetting the SR latch; RRE <0> and RRE <1> Both are equal to 0, therefore the higher voltage will be used for Vread bias. In row 2 of Table 1, the input is 0, causing NZ to equal 0; RRE <0> and RRE <1> Both are equal to 0, therefore the higher voltage will be used for Vread bias. In row 3 of Table 1, the partial sum and PS are negative; RRE is used. <0> Therefore, secure reads will be used for Vread bias. In row 4 of Table 1, the part and PS are positive, but the selected part and bit PS... X =0; RRE was used <0> Therefore, secure reads will be used for Vread bias. In row 5 of Table 1, the portion and PS are positive, and the selected portion and bit PS... XThe value is 1; RRE was used. <1> Therefore, risky reads will be used for Vread bias.

[0069] Figure 14 This shows a set of instance logic conditions that can be enabled, instead of a one-to-one input of the partial and 165 select bits. The logic implementation comes from... Figure 12 The flow of blocks 243, 244, 245, and 246. Other logical conditions can be used, and the logical conditions shown are only for the purpose of using logical combinations to determine PS. X Examples of signals.

[0070] Figures 15 to 22 Example computations and demonstrations of the operation of DYNR block 170 are shown. At the top of these graphs is a set of M=9 inputs I of length N=8 and a set of M weighted vectors W of length K=8. The first column at the bottom of each of these graphs contains the input values ​​listed again, multiplied by the W being processed in the second column. i，j The corresponding weight bits of the weighted vector. The immediate sum is provided in the third column of values. The fourth column of values ​​represents the positional value multiplier, or in other words, the 2^j bits used for the weighted vector W being processed. K-1-j The fifth column is the product of the i-th input multiplied by the j-th weight bit of the i-th weighted vector multiplied by the bit value multiplier. The bottom of the third and fifth columns displays the sum of intermediate sums and the sum of value sums, respectively. Intermediate sums and partial sums are accumulated. Partial sum register 165 is shown to display the current partial sum PS value. The previous partial sum PSp, carried over from the previous value, is also provided, showing the partial sum PS before the shift. PS is retrieved and provided separately from the partial sum PS. 19 PS 14 PS 13 PS 12 PS 11 . Figures 16 to 22 The calculations for the current intermediate sum and the previous intermediate sum (shifted), as well as the calculations for the previous sum and the current sum, are also provided at the bottom of each graph. These aspects will be explained in more detail below.

[0071] exist Figure 15The first term, 32, is provided for calculating 30. This term calculates the sign bit of the product of the input I and the weighted vector W. If any weighted vector is negative, the result will be negative; otherwise, the result will be zero. Since the weighted vector W uses signed two's complement format, the MSB of the negative weighted vector will be "1", and the MSB of the positive weighted vector will be "0". Therefore, multiplying the input I by the negative weighted vector W will result in a potentially most negative final value. The sum of the values ​​after calculating the sign bit will be equivalent to a weighted vector value of -128 (10000000). Any other bit in the weighted vector being "1" instead of "0" will ultimately result in the final product sum being less negative. Figure 15 As shown, input I0 and bit W 0，0 Multiplication, input I1 and bit W 1，0 Multiplication, input I2 and bit W 2，0 Multiply, and so on, until the input I8 is multiplied by the weight W. 8，0 The only weighted vector bit that is "1" corresponds to W. 5，0 W 7，0 and W 8，0 The products of the corresponding inputs and these weights are -21, -98, and -108, respectively. Adding these together provides a partial sum of -227, which is stored as a partial sum (1111 1111 1111 0001 1101) in the partial sum PS register 165. The value for the sum, -29056, is also provided. PS 19 PS 14 PS 13 PS 12 PS 11 Each is equal to 1. Because PS 19 The bit represents a negative number, so RRE <0> The signal remains at 0, indicating that reduced read energy should not be used.

[0072] exist Figures 16 to 22 In the calculation of 30, the second term 34 has already begun, for example, for the values ​​of j≥1 in the weighted vector. Figure 16 In this context, the corresponding bit of the weighted vector W at j=1 is multiplied by the corresponding input. For example... Figure 16 As shown, input I0 and bit W 0，1 Multiplication, input I1 and bit W 1，1 Multiplication, input I2 and bit W 2，1 Multiply, and so on, until the input I8 is multiplied by the weight W. 8，1 The only weighted vector bit that is "1" corresponds to W. 0，1 W 1，1 W 2，1 W 5，1 W 6，1 and W 8，1The corresponding inputs and the products of these weights are 164, 137, 43, 21, 110, and 108, respectively. These are added together to provide an intermediate sum of 583. The previous partial sum PSP, i.e., -227, is left-shifted to -454 and added to the intermediate sum 583 to provide a new partial sum PS, i.e., 129, which is stored as a partial sum (0000 000000001000 0001) in the partial sum PS register 165. The bit value of this sum, i.e., 8256, is also provided (e.g., if the bit values ​​are also multiplied). PS 19 The bit is currently equal to 0, indicating that PS is positive. However, PS 14 PS 13 PS 12 and PS 11 The bit is now also equal to 0. Although the bit PS 19 Indicates a positive number, but RRE <0> The signal is still 0 because bit PS 14 PS 13 PS 12 and PS 11 None of them used PS X The trigger value is 1. Therefore, the next read should not use the reduced read energy.

[0073] exist Figure 17 In the weighted vector W, the corresponding bit j=2 is multiplied by the corresponding input. For example... Figure 17 As shown, input I0 and bit W 0，2 Multiplication, input I1 and bit W 1，2 Multiplication, input I2 and bit W 2，2 Multiply, and so on, until input I8 is multiplied by weight W. 8，2 The only weighted vector bit that is "1" corresponds to W. 0，2 W 2，2 W 3，2 W 5，2 W 7，2 and W 8，2 The products of the corresponding inputs and the products of these weights are 164, 43, 35, 21, 98, and 108, respectively. These are added together to provide an intermediate sum of 469. The previous partial sum PSP, 129, is left-shifted to 258 and added to the intermediate sum 469 to provide a new partial sum PS, 727, which is stored as a partial sum (0000 0000 0010 1101 0111) in PS register 165. The bit value of this sum is also provided, i.e., 8256 + 15008 = 23264 (e.g., if the bit-place value is also multiplied and added to the previous partial sum). 19 A value of 0 indicates that PS is positive. However, PS 14 PS 13PS 12 and PS 11 The bit is still equal to 0. Although the bit PS 19 Indicates a positive number, but RRE <0> The signal is still 0 because PS 14 PS 13 PS 12 and PS 11 The PS will not be triggered. X The value is 1. Therefore, the next reading should not use the reduced reading energy.

[0074] exist Figure 18 In this case, the corresponding bit of the weighted vector W at j=3 is multiplied by the corresponding input. For example... Figure 18 As shown, input I0 and bit W 0，3 Multiplication, input I1 and bit W 1，3 Multiplication, input I2 and bit W 2，3 Multiply, and so on, until input I8 is multiplied by weight W. 8，3 The only weighted vector bit that is "1" corresponds to W. 1，3 W 3，3 W 4，3 W 6，3 W 7，3 and W 8，3 The corresponding inputs and the products of these weights are 137, 35, 111, 110, 98, and 108, respectively. These additions provide an intermediate sum of 599. The previous partial sum PSP, 727, is shifted left to 1454 and added to the intermediate sum 599 to provide a new partial sum PS, 2053, which is stored as a partial sum (00000000 1000 000 0101) in the partial sum PS register 165. The bit value of this sum, 23264 + 9584 = 32848, is also provided (e.g., if the bit value is also multiplied and added to the previous partial sum). 19 A value of 0 indicates that PS is positive. (PS is a bitwise variable.) 14 PS 13 and PS 12 It still equals 0, but the bit PS 11 Triggered to 1. If enable bit PS 11 The transmission gate, then bit PS 11 Transmit to PS X And RRE <1> The signal will be provided (RRE) <1> =0), thus reducing the read energy for the next read. For illustrative purposes, we can assume a transmission gate TPS (transmission gate percentage). 11 Not enabled, therefore PS X It remains at 0. Therefore, the reduced read energy will not be used for the next read.

[0075] exist Figure 19In this case, the corresponding bit of the weighted vector W at j=4 is multiplied by the corresponding input. For example... Figure 19 As shown, input I0 and bit W 0，4 Multiplication, input I1 and bit W 1，4 Multiplication, input I2 and bit W 2，4 Multiply, and so on, until the input I is reached. 8与 Weight W 8，4 Multiply. The weighted vector bit that is uniquely "1" corresponds to W. 1，4 I2, W 4，4 W 5，4 and W 6，4 The corresponding inputs and the products of these weights are 137, 43, 111, 21, and 110, respectively. These are added together to provide an intermediate sum of 422. The previous partial sum PSP, 2053, is left-shifted to 4106 and added to the intermediate sum 422 to provide a new partial sum PS, 4528, which is stored as a partial sum (0000 0001 00011011 0000) in the partial sum PS register 165. The bit value of this sum, 32848 + 3376 = 36224, is also provided (e.g., if the bit value is also multiplied and added to the previous partial sum). 19 A value of 0 indicates that PS is positive. 14 PS 13 And (now) PS 11 Bit is equal to 0, but bit PS 12 Triggered to 1. If bit PS is enabled. 12 The transmission gate, then bit PS 12 Transmit to PS X And RRE <1> This will provide a signal, thereby reducing the read energy for the next read. For illustrative purposes, it can be assumed that bit PS... 12 The transmission gate is not enabled, therefore PS X It remains at 0. Therefore, the reduced read energy will not be used for the next read.

[0076] exist Figure 20 In this case, the corresponding bit of the weighted vector W at j=5 is multiplied by the corresponding input. For example... Figure 20 As shown, input I0 and bit W 0，5 Multiplication, input I1 and bit W 1，5 Multiplication, input I2 and bit W 2，5 Multiply, and so on, until the input I8 is multiplied by the weight W. 8，5 The only weighted vector bit that is "1" corresponds to W. 0，5 W 3，5 W 4，5 and W 6，5The corresponding inputs and the products of these weights are 164, 35, 111, and 21, respectively. These additions provide an intermediate sum, 331. The previous partial sum PSp, 4528, is left-shifted to 9056 and added to the intermediate sum 331 to provide a new partial sum PS, 9387, which is stored as a partial sum (00000010010010101011) in the partial sum PS register 165. The bit value of this sum is also provided, 36224 + 1324 = 37548 (e.g., if the bit value is also multiplied and added to the previous partial sum). 19 A bit value of 0 indicates that PS is positive. 19 And (now) PS 12 and PS 11 Bit equals 0, but PS 13 The bit has been triggered to 1. If PS is enabled. 13 A bit transmission gate, then PS 13 The bits will be transmitted to PS X Bit and RRE <1> This will provide a signal, thereby reducing the read energy for the next read. For illustrative purposes, it can be assumed that bit PS... 13 The transmission gate is not enabled, therefore PS X It remains at 0. Therefore, the reduced read energy will not be used for the next read.

[0077] exist Figure 21 In this case, the corresponding bit of the weighted vector W at j=6 is multiplied by the corresponding input. For example... Figure 21 As shown, input I0 and bit W 0，6 Multiplication, input I1 and bit W 1，6 Multiplication, input I2 and bit W 2，6 Multiply, and so on, until the input I8 and the weight W are multiplied. 8，6 Multiply. The weighted vector bit that is uniquely "1" corresponds to W. 1，6 W 2，6 W 3，6 W 4，6 W 7，6 and W 8，6 The corresponding inputs and the products of these weights are 137, 43, 35, 111, 98, and 108, respectively. Adding these together provides the intermediate sum 532. The previous partial sum PSP, 9387, is shifted left to become 18774 and added to the intermediate sum 532 to provide the new partial sum PS. 19 That is, 306, which is stored as a partial sum (00000100 100 1011 1010) in partial sum register 165. The bit value of this sum is also provided, i.e., 37548 + 532 = 38612 (e.g., if the bit value is also multiplied and added to the previous partial sum). Bit PS 19 A value of 0 indicates that PS is positive.14 It is now triggered at 1. If the PS bit is enabled. 14 The transmission gate, then bit PS 14 Will be transmitted to PS X Bit and provide RRE <1> This signal reduces the read energy required for the next read. For illustrative purposes, we can assume that bit PS... 14 The transmission gate is enabled, therefore PS X It now becomes 1. Therefore, the next read will use the reduced read energy RRE. <1> .

[0078] exist Figure 22 In this process, the corresponding bit of the weighted vector W at j=7 is multiplied by the corresponding input. However, due to the enabling of RRE... <1> The signal, therefore, uses reduced read energy to read W. i，7 The weighted vector W is used to reduce the total power consumption. Figure 22 This shows that the value is W i，7 The case where all weighted vectors are read as equal to 0. In some embodiments, this may occur intentionally to enable skipped read conditions. In such embodiments, the memory location is not actually read and is assumed to be 0. Figure 22 If the MAC process has already completed, the difference between the calculated PS and the actual MAC value is 253, resulting in an error of 0.65%. Figure 22 It also provides an option to determine if the maximum value (all W) is observed. i，7 The value when = 1) results in a median of 827 and a difference from the actual MAC value of 574, leading to an error of 1.48%. For this particular set of calculations, this might be considered the worst-case scenario because it provides the maximum possible deviation from the actual MAC value.

[0079] As the preceding calculations show, later calculations contribute significantly less to the PS than earlier calculations. Since the earlier calculations are left-shifted, they have greater significance in each iteration. Therefore, it can be seen that although reducing read energy carries a higher risk of read errors, the trade-off in reducing energy savings may be worthwhile. In fact, the introduced read risk is far lower than the risk associated with... Figure 22 The worst-case scenario will be discussed in more detail below.

[0080] In the above example, by observing bit PS 14 RRE was triggered <1> Signal. At this point, the calculated portion and PS contribute 99.35% of the total MAC value. If the PS bit 13 RRE was triggered <1> The signal, then the partial sum calculated at that point will represent 96.61% of the total MAC value. If the PS bit 12RRE was triggered <1> The signal, then the partial sum calculated at that point will represent 93.2% of the total MAC value. If the PS bit 11 RRE was triggered <1> The signal, then the partial sum calculated at that point, will represent 84.52% of the total MAC value.

[0081] Figure 23 The provided chart illustrates the reduced read energy that may be obtained when RRE1 = 0. In some embodiments, Vread = 0.2V can be considered as the nominal read voltage, i.e., at RRE1 = 0.2V. <0> =0 is used. Energy savings can be achieved by reducing Vread to 0.15V, 0.1V, or lower. This reduces the energy used for the pre-charge, development, and recovery processes of reading memory signals. For example, reducing the pre-charge voltage from 0.2V to 0.15V reduces energy consumption from approximately 15262fJ to approximately 6783fJ. In another example, reducing the pre-charge voltage from 0.2V to 0.1V reduces energy consumption from approximately 15262fJ to approximately 4016fJ. Energy savings have also been observed during development and growth. After summing the total energy usage, the total energy consumption of 255.5fJ per unit can be reduced to 174.1fJ at 0.15V and 144.2fJ at 0.1V. This represents energy savings of 31.9% and 43.6%, respectively. It should be understood that these values ​​are merely examples, and energy consumption can vary based on memory type and process conditions (e.g., operating temperature). In some embodiments, changing the precharge, development, and recovery voltage by 25% can result in energy savings of about 25% to about 35%, and changing the precharge, development, and recovery voltage by 50% can result in energy savings of about 38% to 48%. Figure 23 The charts also show that some energy consumption does not change based on the Vread voltage value; therefore, baseline energy consumption will occur regardless of the Vread value.

[0082] Figure 24 The relationship between read voltage and sensing yield according to some embodiments is shown. When Vread is 0.2V, the sensing yield is essentially error-free. When Vread is 0.15V, the sensing yield drops to 99.6% ± 0.3%, and when Vread is 0.1V, the sensing yield drops to approximately 98.3% ± 0.4%. Essentially, for example, this means that when Vread is 99.6%, there are approximately 4 incorrect reads per 1000 bits, and when Vread is 0.1V, there are approximately 17 incorrect reads per 1000 bits. Furthermore, as... Figure 24As shown, as Vread decreases, read energy also decreases; however, the energy decrease is not proportional to the Vread decrease. Similarly, as Vread increases, sensing yield also increases; however, sensing yield is not proportional to Vread. Therefore, Vread can be selected to balance energy saving and sensing yield (reliability), depending on the designer's fault tolerance and energy saving goals.

[0083] Figure 25 A simplified schematic diagram illustrating the read path of an I / O associated with an array dimension of 1 word line WL, 32 bit lines BL, and 8 common source lines is shown. This schematic should be understood as an example only, and other implementations may be used. The source line MUX 140 includes a global source line pull-down GSL_PD transistor attached to the global source line GSL. The global source line GSL enters a set of source line transmission gates controlled by a set of first source line select SLSEL1 lines. The output of MUX 140 is used to control the common source line CSL of memory 110. In this example, memory 110 is shown as a 1T1MTJ MRAM device with 1 transistor and 1 magnetic tunnel junction; however, other memory devices may be used as discussed above. The word line WL signal is the input from the word line driver WLDR 120 to memory 110. Bit line MUX 140 provides a set of transmit gate inputs from the first bit line select (BLSEL1) signal and the second bit line select (BLSEL2) signal, causing the BL of memory 110 to first flow to the local bit line LBL using the BLSEL1 signal and then to the global bit line GBL using the BLSEL2 signal, to select which bit lines BL are output to IO 150. DYNR block 170 provides an RRE<0:1> signal output to connect to the selected Vread bias voltage (see [link to DYNR block]). Figure 26 The READ gate control signal enables the global bit line GBL to flow to the sense amplifier of bit line SA_BL. A voltage-type sense amplifier VSA is shown, which uses a reference voltage to compare the BL value with the global bit line GBL and amplifies the global bit line GBL to provide the output. The PRECHARGE gate signal enables the Vread bias voltage VBL_RD to precharge the voltage sense amplifier at IO 150. Figure 26 The diagram of the boxed area F26 is provided.

[0084] Figure 26 It shows Figure 25 The expanded view of the dashed box F26. Figure 26In some embodiments, the output of DYNR block 170 is connected to MUX 140 to provide bias for bit line BL. The PRECHARGE signal is a gated signal used to enable the Vread bias voltage. However, DYNR block 170 provides RRE... <1> and RRE <0> Signal according to RRE <1> The signal is enabled (i.e., equal to 1) or disabled (i.e., equal to 0) to provide different Vread bias voltages. Therefore, Figure 26 The logic provides a way to combine the precharge signal with the RRE <1> and RRE <0> The signal connections control the method of using which Vread bias voltage to apply. It is worth noting that alternative embodiments can be used. For example, alternative logic can be used. In some embodiments, the RRE signal is a single line with a value of 1 or 0 depending on whether reduced read energy should be used. Figure 26 In the middle, when the precharge signal is 0, neither gate will conduct. When the precharge signal is 1, if RRE <0> =0, safe read will be used, and the bit line bias BLBias will be biased with the Vread safe bias voltage. If RRE <1> =0, a risky read will be used, and the bit line bias BL Bias will be biased using the Vread risky bias voltage. If, for some reason (e.g., after a MAC reset), RRE <0> and RRE <1> If the value is 0, a higher voltage will be used, namely the Vread safety voltage.

[0085] Figure 27 A timing diagram and a view of a sense amplifier are shown according to some embodiments. In some embodiments, RRE <1> The signal can enable control block 130 to change the timing of the read operation to shorten the time spent performing the read, thereby reducing energy usage. In some embodiments, the length of time for providing the precharge voltage can be reduced, resulting in a reduction in the total power provided during the precharge time. In other embodiments, the length of time for discharging the bit line voltage can be reduced, resulting in a reduction in the total power discharged during the read time. The risk of shortening the read operation latency is that some values ​​may not be read correctly due to the reduced time. Before being sensed by the VSA, the voltages associated with logic "0" and logic "1" of the data (e.g., on bit line BL) are precharged and discharged to be compared with a reference voltage. For example, for MRAM memory device 110, an antiparallel high-resistance state can represent "0", while a parallel low-resistance state can represent logic "1". Similar settings can be made for other memory types. The antiparallel and parallel states are compared with a reference voltage to obtain the stored data in memory device 110. Shortening the read latency can reduce the energy used. Figure 27The timing diagram shown includes three time periods: period 1, P1, for preparing and pre-charging the bit line to Vread; period 2, P2, for releasing the bit line voltage through the storage structure of storage device 110; and period 3, P3, for enabling the sense amplifier and outputting the Q / QB of the sense amplifier. In some embodiments, period P1 can be shortened by shortening the time used to pre-charge the bit line. The risk is that the bit line may not be charged sufficiently to compare the value with the reference voltage to receive a reliable read. In some embodiments, period P2 can be shortened by shortening the time used to discharge the bit line. The risk is that the bit line may not be sufficiently discharged to compare the value with the reference voltage to receive a reliable read.

[0086] Figure 28 This indicates that if RRE <1> If = 0, no view of the pre-charge logic circuit diagram is provided. In some embodiments, when RRE is satisfied... <1> When the precharge is bypassed, the remaining weighted vector W bits can be read as 0. This can be done by forcibly bypassing the precharge. When the precharge is bypassed, all (or most) of the remaining weighted vector bits will be read as 0. Figure 22 An example is provided where, although additional weight bits are available, the remaining bits are treated as 0. It should be noted that in some cases, a 1 can be read even without applying a precharge voltage, although the precharge voltage does not provide energy. When precharge is initiated and RRE... <1> When = 1, precharge will read normally. This can also be achieved by... Figure 26 Setting the Vread risk voltage to ground disables precharge. It should be understood that other logic can be used to bypass precharge. The logic provided here should not be considered as excluding other logic.

[0087] The implementation offers advantages. Dynamic read voltage conditions can be set by monitoring the partial sum in the in-memory calculation MAC operation. When certain conditions of the partial sum are met, memory read energy may be reduced for the remainder of the MAC operation. Energy reduction can be achieved by providing a lower (riskier) precharge bias to the voltage sensing amplifier, shortening the delay time for performing the sensing operation, or by skipping the read of the remaining weighted vector (assuming the remaining weighted vector is 0). Combinations of these operations can also be used. For example, the shortened delay time can be combined with any other strategy. Skipping can also be combined with a lower precharge bias voltage by implementing skipping after monitoring conditions on bits different from those used for risky voltage bias in the partial sum and PS. For example, PS 11 The bit may trigger a risky read condition for Vread. Besides the risky voltage bias, PS 12 The bit may trigger a lower latency. And PS 13 Or PS 14 The bit may trigger the remaining bits to be skipped.

[0088] One embodiment is a method that includes determining whether a partial sum of an in-memory computation (CIM) operation is positive to obtain a first result. The method further includes determining that a selected bit of the partial sum is transferred from 0 to 1 to obtain a second result. The method further includes adjusting a read configuration of a read operation of a memory cell of the CIM in response to both the first and second results being true. In one embodiment, the read configuration is adjusted to reduce timing delays while waiting to read a memory cell. In one embodiment, the read configuration is adjusted to reduce the bias voltage used to read the memory cell. In one embodiment, the read configuration is adjusted to remove the bias voltage used to read the memory cell. In one embodiment, the selected bit is located in the upper half of the partial sum. Another embodiment is a method that includes reading a first set of bits from a set of weighted vectors from memory using a first read energy. The method further includes multiplying a set of inputs by the first set of bits to obtain a first product. The method further includes adding the first product to an accumulated product sum. The method further includes enabling a reduced read energy signal when the accumulated product sum is positive and a bit condition of the accumulated product sum changes from 0 to 1. The method further includes reading a second set of bits from the weighted vector group from memory using a second read energy less than the first read energy. In one embodiment, the method may include shifting the sum of accumulated products before adding the first product to the sum of accumulated products. In one embodiment, reading the second set of bits uses a shorter timing period than the timing period used to read the first set of bits. In one embodiment, reading the second set of bits uses a second precharge voltage for the sense amplifier, which is less than the first precharge voltage used to read the first set of bits. In one embodiment, reading the second set of bits is performed without providing a positive precharge voltage to the sense amplifier. In one embodiment, the bit condition corresponds to a selected position of the sum of accumulated products having a first index, a second index, a third index, or a fourth index, wherein the first index is equal to the bit length of the first input of the input group plus the next integer obtained by carrying up the logarithm of the number of inputs in the input group to base 2, wherein the second index is equal to the first index plus one, wherein the third index is equal to the first index plus two, and wherein the fourth index is equal to the first index plus three. In one embodiment, the bit condition corresponds to a logical combination of two or more selected bits in the sum of the accumulated products. In another embodiment, reading the second set of bits from the weighted vector does not accurately determine the value of one or more of the bits in the second set.

[0089] Another embodiment is a device including a computer-readable memory that stores a set of inputs and a corresponding set of weighted vectors. The device also includes a multiply-accumulate device comprising an adder, a multiplier, and a partial sum (PS) register configured to store the accumulated result of an iterative product sum operation from the set of inputs and the corresponding set of weighted vectors. The device further includes a multiplexer configured to provide a bias voltage to a sense amplifier to read the weighted vectors. The device also includes dynamic readout logic configured to evaluate the partial sum, determine whether a reduced readout energy (RRE) signal should be enabled, and enable the reduced readout energy signal, providing the reduced readout energy signal to the multiplexer. In one embodiment, the device may include a control block, wherein the reduced readout energy signal is also provided to the control block, the control block providing memory access timing, the control block being configured to reduce read latency for reading the memory when the reduced readout energy signal is enabled. In one embodiment, the dynamic readout logic is configured to evaluate the PS by checking the sign bit and the select bit of the PS. In one embodiment, the selected bit corresponds to the bit index of PS. The bit index is incremented by one, two, or three, and the bit index is equal to the bit length of the first input of the input group plus the integer obtained by carrying up the logarithm of the number of inputs in the input group (base 2) minus one. In one embodiment, the multiplexer is configured to select a bias voltage based on the RRE signal, wherein when the RRE signal is enabled, the multiplexer is configured to provide a smaller bias voltage than when the RRE signal is not enabled. In one embodiment, when the RRE signal is enabled, the multiplexer is configured to provide a bias voltage that causes the sense amplifier to output 0. In one embodiment, the dynamic readout logic is configured to evaluate the partial sum by checking the sign bit of the partial sum and a combination of logic at two or more selected bits of the partial sum.

[0090] The foregoing has outlined features of several embodiments to enable those skilled in the art to better understand aspects of this disclosure. Those skilled in the art should understand that they can readily use this disclosure as the basis for designing or modifying other processes and structures to achieve the same purposes and / or advantages as the embodiments described herein. Those skilled in the art should also recognize that such equivalent constructions do not depart from the spirit and scope of this disclosure, and that they can be modified, substituted, and altered in various ways without departing from the spirit and scope of this disclosure.

Claims

1. A method for performing in-memory computation (CIM), comprising: Determine if the partial sum of the in-memory computation operations is positive to obtain the first result; The selected position of the determined part is converted from 0 to 1 to obtain the second result; as well as In response to both the first and second results being true, the read energy required to perform a read operation from the memory cell is reduced. The partial sum is a partial accumulation result generated during the convolution operation, in which each input data in the multiple input data is multiplied and accumulated bit by bit with the corresponding weighted data vector of the multiple weighted data vectors.

2. The method according to claim 1, wherein, Reducing the read energy includes reducing the timing delay while waiting to read the memory cell.

3. The method according to claim 1, wherein, Reducing the read energy includes reducing the bias voltage used to read the memory cell.

4. The method according to claim 1, wherein, Reducing the read energy includes removing the bias voltage used to read the memory cell.

5. The method according to claim 1, wherein, The selected position is located in the upper half of the portion and.

6. A method for performing in-memory computation (CIM), comprising: The first set of bits from a set of weighted vectors is read from the memory using the first read energy; Multiply a set of inputs by the first set of bits to obtain a first product; Add the first product to the sum of the accumulated products; When the sum of the first product and the accumulated products is positive and the bit condition of the sum changes from 0 to 1, the reduced read energy signal is enabled. as well as The second set of bits from the weighted vector group is read from the memory using a second read energy that is less than the first read energy.

7. The method according to claim 6, further comprising: The sum of accumulated products is shifted before adding the first product to the sum of accumulated products.

8. The method according to claim 6, wherein, The second set of bits is read using a shorter timing period than the timing period used to read the first set of bits.

9. The method according to claim 6, wherein, The second set of bits is read using a second precharge voltage for the sensing amplifier, which is less than the first precharge voltage used to read the first set of bits.

10. The method according to claim 6, wherein, The second set of bits was read without providing a positive precharge voltage to the sense amplifier.

11. The method according to claim 6, wherein, The bit condition corresponds to the selected position of the sum of the accumulated products having a first index, a second index, a third index, or a fourth index, wherein the first index is equal to the bit length of the first input of the input group plus the next integer obtained by carrying over the logarithm of the number of inputs in the input group to base 2, wherein the second index is equal to the first index plus one, wherein the third index is equal to the first index plus two, and wherein the fourth index is equal to the first index plus three.

12. The method according to claim 6, wherein, The bit condition corresponds to a logical combination of two or more selected bits in the sum of the accumulated products.

13. The method according to claim 6, wherein, Reading the second set of bits from the weighted vector group does not accurately determine the value of one or more of the bits in the second set of bits.

14. A device for performing in-memory computing (CIM), comprising: A computer-readable storage device that stores an input set and a weighted vector set; A multiply-accumulate device includes an adder, a multiplier, and a partial sum (PS) register configured to store partial accumulation results generated during the bitwise multiplication and accumulation of each input from the input group of the convolution operation with the corresponding weighted vector of the weighted vector group; A multiplexer is configured to provide a bias voltage to a sense amplifier to read the weighted vector; as well as Dynamic read logic is configured to evaluate a portion of the data and determine whether a Reduced Read Energy (RRE) signal should be enabled, and to enable the Reduced Read Energy signal and provide it to the multiplexer.

15. The device according to claim 14, further comprising: A control block, wherein the reduced read energy signal is also provided to the control block, the control block provides memory access timing, and the control block is configured to reduce read latency for reading the memory when the reduced read energy signal is enabled.

16. The device according to claim 14, wherein, The dynamic read logic is configured to evaluate the partial sum by checking the sign bit and the selection bit of the partial sum.

17. The device according to claim 16, wherein, The selected position corresponds to the bit index of the partial sum, the bit index plus one, the bit index plus two, or the bit index plus three. The bit index is equal to the bit length of the first input of the input group plus the integer obtained by carrying over the logarithm of the number of inputs in the input group (base 2) minus one.

18. The device according to claim 14, wherein, The multiplexer is configured to select the bias voltage based on the reduced read energy signal, wherein when the reduced read energy signal is enabled, the multiplexer is configured to provide a smaller bias voltage than when the reduced read energy signal is not enabled.

19. The device according to claim 18, wherein, When the reduced read energy signal is enabled, the multiplexer is configured to provide a bias voltage that causes the sense amplifier to output 0.

20. The device according to claim 14, wherein, The dynamic read logic is configured to evaluate the partial sum by examining the sign bit of the partial sum and a logical combination of two or more selected bits of the partial sum.

Images