Residual number system compute-in-memory (CIM) multiplication

Abstract

A multiplication circuit which multiplies two selected numbers, modulo p in a fast and efficient manner, where p is prime. The multiplication can be implemented by use of a crossbar multiplication circuit. In a folded and split crossbar implementation for modulo p operands in a sign-magnitude format, a crossbar array of size ((p-1) / 2)x(p+1) / 2 can be used to compute the magnitude of the product with logic circuitry used to compute the sign of the product. In additional implementations, a first modulo p operand, such as a weight value of a neural network, is programmed into an array of DRAM memory transistors to which a second modulo p operand is applied as bias levels.

Description

RESIDUAL NUMBER SYSTEM COMPUTE-IN-MEMORY (CIM) MULTIPLICATIONInventor:Michael DibrinoCLAIM OF PRIORITY

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 608,653, entitled “Residual Number System Compute-In-Memory (CIM) Multiplication”, filed December 11 , 2023, which is incorporated by reference herein in its entirety.FIELD

[0002] The following is related generally to the field of electronic circuits and, more specifically, to electronic circuits for performing arithmetic functions such as multiplication.BACKGROUND

[0003] Computer systems frequently include one or more circuits for performing arithmetic functions such as addition, subtraction, multiplication, and division. In general-purpose computer architectures, one or more circuits for performing arithmetic operations such as multiplication may be integrated as execution units within a central processing unit (CPU). For example, computer systems frequently include combinatorial logic circuits (e.g., in a microprocessor) such as a Vector Unit, or VU, an arithmetic logic unit, or ALU, and / or a floating-point unit, or FPU, often referred to as a math coprocessor. One or more such units may include multiplication circuits (multiplication units) to perform multiplication operations. In general-purpose computer architectures, one or more VUs, ALUs and / or FPUs may be integrated as execution units within the central processing unit. One way to improve the efficiency of multiplication operations is through use of a Residual, or Residue, Number System (RNS) multiplication operations in which the operands are represented by a set of remainders for a set of moduli. Although this approach can provide advantages, asusually implemented it can require m x m lookup tables for a modulus value of m, need to use division for performing modular reduction for general values of m, or be restricted to specific values such as 2'-1 or 2'+1 , where i is an integer. The need for division can significantly slow the process, the need for 2-dimensional lookup tables can result in large area and power requirements for larger moduli, and moduli of the form 2'-1 or 2'+1 restrict the possible choices of moduli. Circuits that perform RNS based multiplication operations in an efficient manner may mitigate such delays.SUMMARY

[0004] According to one aspect of the present disclosure, a multiplication circuit comprises: a first array of memory cells connected along rows and columns, the memory cells of the first array storing a first number of a first modulo pi , where pi is a prime number and, for non-zero values of the first number, the first number is stored as a programmed memory cell value on a single memory cell of each row of the first array and each corresponding to a different column of the first array; and a decoder circuit. The decoder circuit is configured to: receive a second modulo pi number; convert the second modulo pi number into a first binary valued vector, wherein, for non-zero values, one of the values of the first vector is one and other values of the first vector are zero; and determine a product of the first pi number and the second pi number by applying the second pi number to the rows of the first array.

[0005] Optionally, in the preceding aspect, the memory cells of the first array are DRAM memory cells.

[0006] Optionally, in any of the preceding aspects, the first array is of size (pi -1 ) / 2 by (pi-1 ) / 2.

[0007] Optionally, in the preceding aspect, the memory cells of the first array are connected along bit lines, each of the bit lines of a plurality of the bit lines of the first array including a first portion connected to the memory cells connected to a first output encoder and a second portion connected to the memory cells connected to a second output encoder.

[0008] Optionally, in any of the preceding aspects, the first array is of size (pi -1 ) by (pi-1 ).

[0009] Optionally, in the preceding aspect, each of the columns of the memory cells of the first array are connected along a corresponding bit line.

[0010] Optionally, in the preceding aspect, the bit lines are connected to an output encoder.

[0011] Optionally, in any of the preceding aspects, the multiplication circuit further comprises: one or more second arrays of memory cells connected along rows and columns, the memory cells of each of the second arrays storing a corresponding third number of modulo pi stored as a programmed memory cell value on a single memory cell of each row of the corresponding second array and each corresponding to a different column of the corresponding second array, and wherein the decoder circuit is further configured to determine a product of the corresponding third pi number and the second pi number by applying the second pi number to the rows of the corresponding second array.

[0012] Optionally, in any of the preceding aspects, the multiplication circuit further comprises: one or more second arrays of memory cells each storing an additional corresponding modulo p2 number, where each of moduli pi and p2 are distinct prime numbers and wherein the decoder circuit is further configured to: receive an input number; convert the input number to a plurality of residual number system components, including the second modulo pi number and corresponding additional modulo p2 components; and further determine the product or the input number and the modulo pi number and additional corresponding modulo p2 numbers.

[0013] Optionally, in the preceding aspect, the multiplication circuit is on a die with the first array and one or more second array located one adjacent to another extending in a first direction on the die in order of decreasing size and aligned along a first side extending in the first direction, and the decoder circuit is located along a side of the array of a largest of the first array and one or more second arrays opposite the side of the largest array adjacent to others of the first array and one or more second arraysand is connected to bias rows of the first array and one or more second arrays according to the converted input number.

[0014] Optionally, in the preceding aspect, the additional plurality of arrays comprises a first additional array of memory cells storing number of modulo pi and one or more second additional arrays of memory cells each storing a corresponding modulo p2 number, the additional array is located one adjacent to another extending in the first direction on the die in order of increasing size and aligned along a side opposing the first side extending in the first direction, and the additional arrays are located adjacent to the first array and one or more second arrays on the side of the first array and one or more second arrays opposing the first side extending in the first direction.

[0015] Optionally, in any of the preceding aspects, the first number is a weight value of a layer of a neural network and the second modulo pi number is an input of the layer of the neural network.

[0016] According to another aspect of the present disclosure, a method of performing multiplication of a first and a second number includes: receiving a first number; converting the first number to one or more residual number system components, each residual number component corresponding to one of the one or more distinct prime numbers; converting each residual number system component of the first number into a binary valued vector, wherein, for non-zero values, one of the values of the vector is one and other values of the vector are zero; and applying the binary valued vector of each component of the first number as a set of bias levels to an array of memory cells storing a corresponding component of a second number converted to the one or more residual number system components.

[0017] Optionally, in the preceding aspect, the method further comprises, comprising prior to receiving the first number: receiving the second number; converting the second number to the one or more residual number system components; and programming each residual number system component of the second number into the corresponding one of the arrays of memory cells.

[0018] Optionally, in any of the preceding two aspects, the second number is a weight value of a layer of a neural network and the first number is an input for the layer of the neural network.

[0019] Optionally, in any of the preceding three aspects, applying the binary valued vector of each component of the first number to the array of memory cells storing the corresponding component of the second number includes concurrently applying the binary valued vector of each component of the first number to the array of memory cells storing the corresponding component of the second number.

[0020] Optionally, in any of the preceding four aspects, for each array of memory cells storing a component of the second number, a size of the array is (p-1 ) / 2 by (p- 1 ) / 2, where p is a corresponding modulus of the component.

[0021] Optionally, in any of the preceding five aspects, for each array of memory cells storing a component of the second number, a size of the array is (p-1 ) by (p-1 ), where p is a corresponding modulus of the component.

[0022] According to a further aspect of the present disclosure, a memory circuit comprises: a plurality of arrays of memory cells connected along rows and columns, each array configured to store a modulus pi component of a residual number system representation of a first number as, for non-zero components of the first number, as a programmed memory cell value on a single memory cell of each row of the array and each corresponding to a different column of the array, where pi is a different prime number for each of the arrays; and a decoder circuit connected to the plurality of arrays and configured to: receive a second number; convert the second number to components in the residual number system; convert each residual number system component of the second number into a binary valued vector, wherein, for non-zero values, one of the values of the vector is one and other values of the vector are zero; and applying the binary valued vector of each component of the second number as a set of bias levels to the rows of the array of memory cells storing the corresponding component of the second number.

[0023] Optionally, in the preceding aspect, the memory circuit further comprises: a plurality of sense amplifier circuits connected to the columns of the arrays; and anencoder connected to the sense amplifier circuits and configured to output a product of the first number and the second number.

[0024] Optionally, in any of the preceding two aspects, the memory cells are DRAM memory cells.

[0025] Optionally, in any of the preceding three aspects, each array is of size (pi - 1 ) by (pi - 1 ) or, alternatively, of size (pi - 1 ) / 2 by (pi - 1 ) / 2.

[0026] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the Background.BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying Figures for which like references indicate elements.

[0028] Figures 1A and 1 B are respectively block diagrams of a computer system and a microprocessor that can be incorporated into such a computer system.

[0029] Figure 2A illustrates a Residual, or Residue, Number System (RNS) number RNS(8|7|5|3) represented in a binary-coded number format.

[0030] Figure 2B illustrates an embodiment for the structure of an adder, subtractor, or multiplier for numbers in the RNS format using the RNS(8|7|5|3) example.

[0031] Figure 3 illustrates components for implementing modular multiplication for moduli not of the form 2kor 2k-1 .

[0032] Figure 4 illustrates the multiplication group representation using a primitive root for the p=11 example.

[0033] Figure 5 illustrates an example of sign-magnitude encoding for the primitive root of 2 for the modulo 13.

[0034] Figure 6 illustrates an example of a barrel shifter efficiently implemented with a crossbar switch.

[0035] Figure 7 illustrates the signed remainder table of (n-1 ) entries for the n=13 example.

[0036] Figures 8 and 9 schematically illustrate implementation of a signed modulo 13 multiplication circuit as full crossbar circuit and as a folded and split crossbar circuit, respectively.

[0037] Figure 10 illustrates the reduction in remainder size of the signed remainder table to just (n-1 ) / 2 entries for the n=13 example.

[0038] Figures 11 A and11 B present an embodiment for a split and folded crossbar multiplier circuit.

[0039] Figure 12 illustrates an NMOS only crossbar shifter circuit implementation than can be used in some embodiments.

[0040] Figure 13 illustrates an example of a level restorer circuit.

[0041] Figure 14 illustrates an embodiment for a cross-connected level restoration circuit for use with the outputs of a split crossbar multiplier.

[0042] Figures 15 and 16 illustrate the operation of the cross-connected level restoration circuit embodiment of Figure 14 for a “0” input and a weak “1” input, respectively.

[0043] Figure 17 is a flowchart to illustrate the operation of the cross-connected level restoration circuit embodiment of Figure 14.

[0044] Figures 18A and 18B illustrate a portion of a further embodiment for a split and folded crossbar multiplier circuit for the n=13 example.

[0045] Figure 19 is a remainder table for modulo 23 multiplication in wrapmagnitude format.

[0046] Figures 20 and 21 respectively illustrate embodiments for sign bit to wrap bit conversion in an NMOS with level restoration and a CMOS embodiment for the modulo 13 example.

[0047] Figure 22 shows an embodiment of a wrap select circuit for product wrap- to-sign conversion for a modulo 13 example.

[0048] Figures 23A and 23B present an embodiment for a folded and split NMOS crossbar multiplier for inputs in a sign-magnitude format.

[0049] Figures 24A and 24B present an embodiment for a folded NMOS crossbar multiplier for inputs in a sign-magnitude format for a non-primitive root of 2.

[0050] Figure 25 is a flowchart for an embodiment of performing modular multiplication based on the embodiments of 11 A, 11 B, 18A, 18B, 23A, 23B, 24A, and 24B.

[0051] Figure 26 is a block diagram to illustrate the use of power-of-2 ordering for both folded and unfolded crossbar embodiments for RNS multiplication.

[0052] Figure 27 is a simplified version of the RNS 13 crossbar architecture of FIGs. 23A and 23B.

[0053] Figure 28 illustrates the mapping of a crossbar multiplier to an array of memory cells for an RNS 13 component in a DRAM embodiment.

[0054] Figure 29 illustrates an RNS crossbar multiple mapped to DRAM in a magnitude only embodiment for a modulo 13 example.

[0055] Figure 30 illustrates an embodiment of an RNS multiplier for an example allowing a single input to applied in parallel to multiple stored weight values.

[0056] Figures 31 -33 present examples of embodiments for the logical organization of wide data-out DRAM dies.

[0057] Figure 34 presents an embodiment for a layout on a die of the RNS component arrays in an RNS [3|5|7|11113| 19|23|29] example.

[0058] Figure 35 presents an embodiment for a layout on a die of paired RNS component arrays in an RNS [3|5|7|11113| 19|23|29] example.

[0059] Figure 36 is a flowchart of an embodiment for residual number system compute-in-memory multiplication.

[0060] Figure 37 illustrates an example of a computing system configured to implement embodiments of the present technology.DETAILED DESCRIPTION

[0061] One technique to improve the efficiency of a multiplication circuit is through use of the Residual, or Residue, Number System (RNS) in which the multiplication operands are represented as a set of remainder values for a set of moduli. This allows for the multiplication to be broken down into a set of channels, one for each of the moduli, that can perform modular arithmetic within each of the channels. Although this approach has advantages, as traditionally implemented, it has several inefficiencies. One is that it uses two-dimensional lookup tables for modular multiplication, where such tables scale as the square of the size of the modulus. Another inefficiency is that for moduli not of the form 2kor 2k-1 , modular multiplication can be slow as it uses division in the modular reduction of the product. Another inefficiency is that for moduli of the form 2k-1 for which somewhat efficient modular multiplications are possible, only a limited number of ‘small’ moduli exist (e.g., 3, 7, 15, 31 for k<5). A limited number of moduli will limit the range of the number which can be represented by the plurality of moduli channels. Additionally, since the moduli must be pairwise co-prime, only a single moduli of the form 2kcan be used. Larger moduli of the form 2k-1 are not desirable due to the inherent delay of carry propagation.

[0062] In embodiments presented below, the multiplication can be implemented by use of a crossbar multiplication circuit. In a folded and split crossbar embodiment, for modulo p operands in a sign-magnitude format, a crossbar array of size ((p-1 ) / 2) x((p+1 ) / 2) can be used to compute the magnitude of the product with logic circuitry used to compute the sign of the product.

[0063] In other aspects, a cross-connected level restoration circuit is presented. For each row of the folded and split crossbar multiplication circuit, two outputs result, one active and at either a low logic level or a degraded high logic level and the other inactive and floating. The cross-connected level restoration circuit restores the logic levels for the active input and returns a low logic level for the floating input.

[0064] In further aspects, the multiplication unit multiplies two selected RNS numbers in DRAM, one number being applied to the row inputs of a DRAM cell array, and the other number being represented by an array of bits previously set to “1” in the DRAM cell array. The aspect can be implemented using (p-1 )x(p-1 ) cell arrays for prime RNS moduli or a “folded” (p-1 )x(p-1 ) / 4 cell array for prime RNS moduli.

[0065] The present embodiments of the disclosure may be implemented in many different forms and the claim scope should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided to convey the inventive embodiment concepts to those skilled in the art. Indeed, the disclosure is intended to cover alternatives, modifications, and equivalents of these embodiments, which are included within the scope and spirit of the disclosure as defined by the appended claims. Furthermore, in the following detailed description of the present embodiments of the disclosure, numerous specific details are set forth in order to provide a thorough understanding. However, it will be clear to those of ordinary skill in the art that the present embodiments of the disclosure may be practiced without such specific details.

[0066] Figures 1 A and 1 B, respectively, are block diagrams of a computer system and a microprocessor that can be incorporated into such a computer system that implements one or more embodiments of the disclosure. In the simplified representation of Figure 1A, the computer system 100 includes a computer 105, one or more input devices 101 and one or more output devices 103. Common examples of input devices 101 include a keyboard or mouse or even a voice recognition system to support voice commands. Common examples of output devices 103 include monitors, speakers or printers. The computer 105 includes a memory 107 and amicroprocessor 120, where in this simplified representation the memory 107 is represented as a single block. The memory 107 can include ROM memory, RAM memory and / or non-volatile memory and, depending on the embodiment, can include separate memory for data and instructions.

[0067] Figure 1 B illustrates one embodiment for the microprocessor 120 of Figure 1A and includes the memory 107. In the representation of Figure 1 B, the microprocessor 120 includes control logic 125, a processing block 140, an input interface 121 , and an output interface 123. The dashed lines represent control signals exchanged between the control logic 125 and the other elements of the microprocessor 120 and the memory 107. The solid lines represent the flow of data and instructions within the microprocessor 120 and between the microprocessor 120 and the memory 107.

[0068] The processing block 140 includes combinatorial logic 143 that is configured to execute instructions and registers 141 in which the combinatorial logic stores instructions and data while executing these instructions. In the simplified representation of Figure 1 B, certain elements or units, such as a vector unit (VII) 146, an arithmetic logic unit (ALU) 147, and a floating-point unit (FPU) processor 149, are shown, while other elements are not explicitly shown in the combinatorial logic 143. The combinatorial logic 143 is connected to the memory 107 to receive and execute instruction and supply back the results. The combinatorial logic 143 is also connected to the input interface 121 to receive input from input device(s) 101 and / or other sources. The combinatorial logic 143 is also connected to the output interface 123 to provide output to output device(s) 103 and / or other destinations.

[0069] A microprocessor, such as the microprocessor 120, may perform a range of different functions including arithmetic operations. For example, a microprocessor may perform addition, subtraction, multiplication, multiply-accumulate, and / or division. Aspects of the present technology may enable efficient multiplication operations to be performed by a microprocessor. More generally, aspects of the present technology can more generally be applied to embodiments for one or more central processing units (CPUs), graphic processing units (GPUs), artificial intelligence (Al) accelerators, Tensor Processing Units (TPUs), and / or any other digital logic that performs multiplication or multiplication as part of a multiply-accumulate operation.

[0070] Multiplication operations may be performed by one or more multiplication circuits in the microprocessor 120. For example, one or more of the VII 146, the ALU 147 and the FPU 149 may include one or more multiplication circuits. Other elements of combinatorial logic 143 may alternatively or additionally include one or more multiplication circuits. The present technology is not limited to multiplication circuits or multiply-accumulate units in any location (in a microprocessor or elsewhere) or used for any particular purpose and can be applied wherever efficient multiplication or multiply-accumulate operations are desired.

[0071] Several different approaches may be used to perform multiplication. The embodiments presented in the following discussion are based on the use of the Residual, or Residue, Number System (RNS). Before discussing these embodiments, some discussion is given of the Residual Number System.

[0072] The residual number system is based on the use of modular arithmetic and a property known as the Chinese Remainder Theorem. If mi, ..., n < are integers greater than 1 (often called moduli or divisors), then the Chinese remainder theorem asserts that if the ny are pairwise coprime, and if ai , ... , ak are integers such that 0 < az< mi for every / , then there is one and only one integer x, such that 0 < x < N and the remainder of the Euclidean division of x by ny is azfor every / . The consequence is that, if M denotes the product of k pairwise relatively prime moduli,M = nik-i X • • • X mi x mo , then M is the number of different representable values in the RNS and is known as its dynamic range. This allows any number within this dynamic range to be expressed as a vector of its values for each of the moduli.

[0073] To take the example of the moduli 8, 7, 5, and 3, The dynamic range in this example is M = 8 x 7 x 5 x 3 = 840, which is the total number of distinct values that are representable in this example 4-modulus RNS. A number in this format is made up of four components, one for each modulus, and referred to an RNS(8|7|5|3) number. The following are some example numbers in RNS(8|7|5| 3) format: Represents 0 or 840 or ■ ■ ■ Represents 1 or 841 or ■ ■ ■Represents 2 or 842 or ■ ■ ■Represents 8 or 848 or ■ ■ ■ Represents 21 or 861 or ■ ■ ■ Represents 64 or 904 or - - Represents -70 or 770 or - -Represents -1 or 839 or ■ ■ ■Other numbers, modulo 840, can be similarly expressed in the RNS(8|7|5|3) format. When performing arithmetic for integers in the RNS format, the operations can be on a component-by-component basis, using the corresponding modulus. Performing arithmetic for integers in this can have a number of advantages.

[0074] Continuing with the example of the moduli 8, 7, 5, and 3, an integer (within the corresponding dynamic range) in a binary-coded number format for RNS(8|7|5|3) is illustrated in Figure 2A. This includes three 3-bit slots for the mod 8, mod 7, and mod 5 components and a 2-bit slot for the mod 3 component. When performing an arithmetic operation between two numbers in this format, operations can individually be performed pairwise on each component. This allows arithmetic operations for numbers in such an RNS format to be separated out into independent RNS channels, as illustrated in Figure 2B.

[0075] Figure 2B illustrates an embodiment for the structure of an adder, subtractor, or multiplier for numbers in the RNS format using the RNS(8|7|5|3) example that may be implemented by a processor. At top are shown two operands, Operand 1 and Operand 2, in the RNS(8|7|5|3) format that can be stored in corresponding registers 2111 and 2112, where these operands will be converted to the RNS format previously in an operation that typically takes a few steps that can be hidden behind other operations. The modulus 8, 7, 5, and 3 components of each operand go to the corresponding mod-8 unit 201 , mod-7 unit 203, mod-5 unit 205, and mod-3 unit 207 to undergo pair-wise operations in each RNS channel using the corresponding modular arithmetic. The components of the results can then be stored in the result register 221 , which can be used in subsequent RNS computations or converted out of the RNS format when needed.

[0076] The RNS approach illustrated with respect to Figure 2B has a number of advantages. It solves the carry-propagation problem, as carry-propagation is limited to a few bits. It is faster, as for example, 4x4 multipliers are considerably more than 4 times simpler than a 16x16 multiplier. It is also smaller as a 4x4 multiplier is ~1 / 16 the size of a 16x16 multiplier (power and area scaled quadratically). Additionally, small residue arithmetic operations can be implemented with direct table lookup. This approach does require conversion between the RNS format and binary, and some arithmetic operations (e.g., division, sign detection, overflow detection) are more difficult, but the following is concerned with multiplication, where it can have a number of advantages.

[0077] Considering the details of implementing efficient RNS multiplication, one detail is the selection of the RNS moduli, where two of the considerations are channel width and the number of channels. If the objective is to reduce the addition / multiplication execution time, then a large number of small moduli is desirable, as the execution time is determined by the largest modulus and Power / Area of multipliers proportional to n2.

[0078] One approach to selecting moduli is to use moduli of the form 2kor 2k-1 for ease of implementation. In this approach the total number of bits can be minimized by using an efficient binary representation — i.e., maximize the dynamic range by selecting a modulus, mi, that equals 2kor is very close to it (2k-1 ), for example. Only a single mi of the form 2kbecause the moduli must be relatively prime. Moduli mi of the form (2k-1 ) work well, since end-around-carry adders may be used for this form; however, not all terms of the form (2k-1 ) may be selected, since for example 2k-1 = (2^ - 1 )(2k / 2+1 ) for even values of k.

[0079] Considering the case of representing a 32-bit integer in the RNS system, this has a range = 232= 4,294,967,296 and using moduli of the form 2kor 2k-1 for ease of implementation. One set of possible moduli are then rrij = 31 , 63, 127, 255, 256. The dynamic range M is adequate as M / 232= 3.7799 and this set of moduli are relatively prime. These five moduli are of 5, 6, 7, 8, and 8 bits respectively for a total of 34 bits. For moduli of the form 2k, an ordinary binary adder can be used, with theresidue read directly from the LSBs. For moduli of the form 2k-1 , an ‘end-around carry’ adder can be used.

[0080] Another approach is to use the smallest prime moduli for which a symmetric complete residue system exists. One example for a 32 bit integer are the nine moduli mj = 3, 5, 7, 11 , 13, 19, 23, 29, 37. The dynamic range M is adequate as M / 232=I .6393 and this set of 9 moduli are of 2, 3, 3, 4, 4, 5, 5, 5, and 6 bits respectively, for a total of 37 bits. Although this has more moduli (and a higher total number of bits), its largest moduli are significantly smaller. This has the advantages of smaller moduli2 begin faster and that area / power are reduced as this is proportional to log2(mi) . However, implementing multiplication for moduli not of the form 2kor 2k-1 has some complications, as can illustrated with respect to Figure 3.

[0081] Figure 3 illustrates components for implementing modular multiplication for moduli not of the form 2kor 2k-1 . Multiplier block 301 receives two n-bit operands A and B and multiplies these together to generate the 2n-bit result of AB. To turn this into the n-bit mod-m product of these numbers, |AB|m, the product AB needs to undergo modular reduction in block 303. However, implementing modular reduction requires division, which is a time-consuming multistep operation that slows down the process. To avoid this problem, the embodiments below present a simple and efficient method to multiply moduli not of the form 2kor 2k-1 .

[0082] The embodiments presented below involve the use of the primitive roots of a number in modular arithmetic: Given gx(mod p) and where p is a prime number, the question is whether there a value of g that makes ensures a unique output from 1 to p-1 , for every value of x from 0 to p-1. If so, this generator (g) is known as a primitive root under modulo p: Y = gx(mod p). The table of Figures 4 illustrates an example.

[0083] Figure 4 illustrates the multiplication group representation using a primitive root for the p=11 example. In the top row, the index (i.e. , power of the primitive root) runs from 0-9, with the second row showing the root g to the power of the index, modI I , notated as | |u . As shown in Figure 6 in the left most column, the primitive roots of 11 are g = 2, 6, 7, or 8. Considering an example using the root of g=7 circled in the oval and computing the product of |5|n * |3|n = 15 = |4|n , 5 corresponds to |g2|i 1 and3 corresponds to |g4|n . Adding the indices (exponents) to lookup the product from product index, 2+4=6, and looking up |g6|n =4, which is the correct result. This can similarly be performed using any group formed by a primitive root g of p and not limited to a primitive root of 2.

[0084] Figure 5 illustrates an example of sign-magnitude encoding for the primitive root of 2 for the modulo 13. The top row presents the index (i.e., power of primitive root), the second row is g=2 to the power of the index, modulo 13, and the third row the resultant magnitude. The fourth row illustrates a sign-magnitude encoding, where remainders greater than 6 are replaced with negatives, so that -1 is same as 12, -2 is same as 11 , and so on. Consequently, the right side is the modulo added inverse of the left side. Note that the wrap bit is not the same as the sign bit, as can be seen with respect to column with {-5, 5}.

[0085] The following discussion considers embodiments for the circuitry to implement the residual number system multiplication techniques described above. Specific embodiments include a crossbar-based multiplication circuit that is both scalable and efficient. Relative to prior art modular multiplication circuits, such as combinational multipliers or Wallace tree multipliers, the crossbar-based approach presents smaller and more power efficient circuitry.

[0086] Figure 6 illustrates an example of a barrel shifter efficiently implemented with a crossbar switch. A barrel shifter is a digital circuit that can shift a data word by a specified number of bits without the use of any sequential logic and can be used to present a relatively simple implementation of a crossbar circuit as an introduction to residual number system crossbar multiplication units presented further below. The example of Figure 6 is a 4X4 example, with an input (xo, xi , X2, xs) that is a 1 -hot vector (i.e., one entry “1”, the rest “0”), a shift count of 0-3, and an output (yo, yi, y2, ys) corresponding to the input shifted (mod 4) by the shift amount. The shift count is received at a decoder 1201 that converts the shift count to another 1 -hot vector specifying a shift amount of either shift 0, shift 1 , shift 2, or shift 3. The barrel shifter is a square array of switches (4x4 in this example) connected between the input (xo, xi , X2, xs) vector and the output vector (yo, yi , y2, ys) and controlled by the shift value. The shift values run diagonally upward across the array, looping back from the bottom. For example, switch 1203 connects input xo to output ys and is controlled by the shift 3input, that an input of (1 , 0, 0, 0) becomes an output of (0, 0, 0, 1 ) in response to a shift of 3. In the example of switch 1205, this connects the input of X3 to the output of yi and is controlled by shift 2 such that an input of (0, 0, 0, 1 ) becomes an output of (0, 1 , 0, 0) for a shift of 2. Note that in the arrangement of Figure 6, the inputs run from xo to X3 left to right, with the shift and output indices increase from bottom to top.

[0087] The array structure of the crossbar switch can be made quite dense, similar to a DRAM device. In terms of RNS multiplication, for a modulo value of n, rather than an (n-1 )x(n-1 ) array, multiplier optimization based on the unwrapped / wrapped approach can reduce this to an (n-1 ) / 2 x (n-1 ) / 2 array for a 75% area savings. By use of a single gate per switch, the circuit can be fast and, such as by using a single NMOS pulldown device, small. Although the size of the array grows as n2, this can be mitigated by use of a “folded” crossbar arrangement based on wrapping, as described below with respect to Figures 8 and 9, for example. The crossbar switch can also result in degraded logic “1” output levels when only NMOS pass gates are used, but level restoring circuitry as presented below can overcome this problem.

[0088] Figure 7 illustrates both wrapped and unwrapped remainder values (wrapped values are the additive inverses of the unwrapped values) for the example of modulus value of 13 when using negative remainders. Figure 7 repeats the mod 13 table of Figure 5, but with the remainders greater than 6 replaced with negative values: for example, -1 replaces 12, -2 replaces 11 , and so on. In this arrangement, the wrapped values (Wrap = 1 row) are the modulo additive inverses of the unwrapped values (Wrap=0 row). Not that in this arrangement, the wrap bit is not always the same as the sign bit, such as the k=3 column where the Wrap = 0 value is -5 and the Wrap = 1 value is +5. This distinction is discussed in more detail below.

[0089] Figures 8 and 9 schematically illustrate implementation of a modulo 13 multiplication circuit as full crossbar circuit and as a folded and split crossbar circuit, respectively. In the full crossbar implementation of Figure 8, the input vector 0 is the (n-1 ) component 1 -hot input vector source 0, the input vector 1 is the (n-1 ) component 1 -hot input vector source 1 , resulting in a crossbar multiplier of size (n-1 )2. Figure 8 shows this n=13 example of Figure 5. with positive magnitude-only values for k=0-11 .

[0090] To reduce the size of the crossbar array, a folded and split crossbar array can be used, in which only the Wrap=0 values for k=0-5 are used. In this case, the crossbar array is folded, in that the product is between only one of + / - 1 , 2, 3, 4, 5, 6 is used, and split, in that array is split into a wrapped and unwrapped halves with the outputs divided between the wrapped and unwrapped values. This reduces the array size to ((n-1 ) / 2)2, or the size of the full crossbar. In the n=13 case, the values used are then 1 , 2, 4, -5, 3, and 6, but where the sign needs to be adjusted as described in more detail below.

[0091] Considering the full crossbar of Figure 8 and the folded and split crossbar of Figure 9 further, in a full crossbar implementation, each non-zero value for both inputs has a corresponding input line after decoding. In a folded crossbar implementation, one or more of the input lines for each input is shared by a pair of input values, where an additional piece of information (e.g., a sign or wrap value) is used to distinguish between the pair of values sharing an input line. As illustrated in Figure 9, and as used in the following discussion, folded is used to refer to when the inputs have been fully folded, or folded in half, so that for each input line for both inputs is shared by two of the non-zero inputs. (For the treatment of zero inputs see, for example, Figures 23A and 23B.) However, intermediate degrees of folding, where less than all of the input values are paired up are possible as alternate embodiments. The crossbar is split in that part of the array corresponds to wrapped values and part corresponds to unwrapped values.

[0092] The use of a folded and split crossbar implementation reduces the size of the needed crossbar array from (n-1 )2to ((n-1 ) / 2)2, or a quarter in size, requiring less space on the logic die. Although the size of the crossbar array is reduced to a quarter, as described with respect to Figures 23A and 23B, for example, additional circuitry is needed to sort out the sign values of the inputs. Consequently, for smaller moduli, a full crossbar implementation may take up less area than a folded and split crossbar implementation. For example, in a RNS arithmetic unit, the optimal implementation may use a full crossbar for, say, modulo 5 and below, with the folded and split crossbar used for modulo 7 and above.

[0093] Figure 10 illustrates the reduction in remainder size of the remainder table to just (n-1 ) / 2 entries for the n=13 example. The recued number of values are thoseused in the folded and split crossbar embodiment of Figure 9 for the inputs, where the wrap or sign value is accounted for by additional logic circuitry, an example of which is shown in Figure 11 B.

[0094] Figuresl 1 A and11 B present an embodiment for a split and folded crossbar multiplier circuit for the n=13 example, where the two figures shows different parts of the same circuit. The circuitry includes the crossbar barrel shifter that calculates a product magnitude and the circuitry to calculate product wrap value, where the magnitude and wrap value together give the final product value. Approximately half of the crossbar cells (those above lower left-to-upper right diagonal with black fill, such as 1711 and 1717) calculate a wrap condition, which indicate that Operand A has been wrapped off the end of the barrel shifter by rotation by Operand B. The other crossbar cells (those on and below the diagonal with no fill, such as 1713 and 1715) are the products that have not wrapped off the end of the barrel shifter.

[0095] The inputs to the crossbar array of operand A and operand B are received as magnitude values, where if not already in the appropriate 1-hot vector format (such as from a previous operation in this format) these are respectively converted in Decoder A 1703 and Decoder B 1701. These inputs can then be applied to the crossbar array as described above with respect to Figure 9. In the folded and split arrangement though, the outputs from the wrapped switches and the unwrapped switches in each row are kept separate in each row to use in determining the product wrap value. For example, in the top row, the output of wrapped element 1711 goes to the lower output line of the first row while the output of unwrapped element 1713 goes to the upper output line of the row. Similarly, in the third row wrapped switch 1717 goes to the lower line and unwrapped switch 1715 goes to the upper line. (As the bottom row has only unwrapped values, it only has a single output.)

[0096] Following the outputs from the barrel shifter on to Figurel 1 B, the pairs of outputs from each row of the barrel shifter go a corresponding one of the OR gates 1721 , so that if the output on either of the lines is a “1”, the combined result is a “1”. This results in the output in a 1-hot vector format. In this unwrapped / wrapped n=13 embodiment, the different components correspond to the shown values of 1 , 2, 4, 5, 3, 6, where, since all elements of the bottom row are unfolded, an OR gate is not needed for the bottom row. At this point, the product magnitude is in the 1-hot vectorformat and, if the magnitude is not to be kept in this format (such as for a subsequent operation), a corresponding one of the set of AND gates 1725 received the values. The second input of each of the AND gates is the corresponding value in a 4-bit decimal value, i.e. , 4’d1 is 1 in the format (0001 ). Here, a value is used as n=13 and 4 bits are needed to express 13 as a binary number. The resultant outputs of the AND gates 1725 are then the product magnitude.

[0097] The series of AND gates 1723 are part of the product wrap determination circuitry. These each receive the output of the corresponding OR gate 1721 with its 1 -hot vector value and the other input corresponding to the lower line of the row. The outputs of the AND gates 1723 are then inputs to the OR gate 1733, so that if any of the AND gates 1723 output a “1 ”, the OR gate provides a “1 ” for the wrap select values. The operands A and B also have, in addition to the magnitude inputs, a wrap input in the folded embodiment. The operand A wrap bit and operand B wrap bit are inputs to the XOR gate 1731 , whose output then is combined with the wrap select value in XOR gate 1735 to generate the product wrap output value. Alternately, the circuitry of Figure 11 B can be replaced with the circuitry of Figure 23B as presented below.{0098} Figure12 illustrates an NMOS only crossbar shifter circuit implementation that can be used in some embodiments. By use of a single NMOS for each element of the crossbar shifter array, such as NMOS 1801 connected between operand A[0] input and output Out[0] and with its control gate connected to operand B[0], the size the array can use a smaller area than an NMOS / PMOS transmission gate implementation. The crossbar circuit array devices can be implemented in any semiconductor technology, including but not limited to planar transistors, finFET transistors, and ribbonFET, gate-all-around (GAA) or nano-wire or nano-sheet transistors.

[0099] Considering the NMOS implementation of Figure 12 further and looking NMOS1801 , when A[0] is at ground or logic ‘0’ and B[0] is at logic T or VDD, Out[0] will also be at ground, and so definitely at “0”. However, when both of A[0] and B[0] are a “1” value of VDD, Out[0] will not be the full VDD value; rather, if the threshold voltage of the NMOS1801 is Vth, then Out[0] will be degraded to VDD-Vth and a “weak” “1” logic value. To maintain the logic values as they propagate through the arithmeticunit, the “1” logic values need to be restored to a strong “1” value of the high logic level VDD by use level restorer.

[0100] Figure 13 illustrates an example of a level restorer circuit. The circuit acts as an inverter, but with well-defined output logic values of either ground for “0” or VDD for “1”, even for a weak “1” input. When receiving a “0” value or ground, PMOS 1903 will be on and NMOS 1901 off, so that the Out value will be VDD and the full “1” logic level and, since Out is high, PMOS 1905 is off and the input stays at ground. When receiving a weak “1” input that is reduced from VDD, NMOS 1901 may not fully be on and PMOS 1903 may not fully be off, resulting in the Out value being pulled down somewhat, but not fully to a well-defined “0” value of ground. As the Out is also connected to the gate of PMOS 1905, this will turn PMOS 1905 at least partially on, raising In value and taking Out fully to a strong “0” value of ground. Although such a level restorer will work with a standard crossbar shifter, the situation is more complicated for the split crossbar arrangement.

[0101] Figure 14 illustrates an embodiment for a cross-connected level restoration circuit for use with the outputs of a split crossbar multiplier. At left of Figure 14 is a portion 2001 of a split crossbar showing four elements of one row. The right most input / SOo (hereis used to represent NOT, which is shown in the figures as a bar on top of the name) is unwrapped and connected to the output of / xbar and the other three inputs are wrapped and connected to the output of inverse of / (xbar_wrap). If these two outputs were independent, they could respectively use the level restoration circuit of 2020 or 2025 to restore weak “1” values of (VDD - Vth) to the full VDD logic level; however, the two values are related in that the gate selects on one output will be always inactive, causing high impedance output on that crossbar output section. To correct these related output values, the cross-connected level restoration circuit is introduced, with an embodiment shown at 2021 .

[0102] The embodiment of Figure 14 includes cross-connected pull-ups to restore low-z state on both outputs. For the xbar output, at the input of level restoration circuit 2023 the pull-ups of PMOSs 2031 and 2033 respectively have their gates connected to the input and the output of level restoration circuit 2025. For the xbar_wrap output, at the input of level restoration circuit 2025 the pull-ups of PMOSs 2035 and 2037 respectively have their gates connected to the input and the output of level restorationcircuit 2023. As will be discussed with respect to Figures 15 and 16, in the embodiment of Figure 14, PMOS 2033 and PMOS 2037 and the other pull-up PMOSs can be weak devices relative to the transistors forming the inverters of 2023 and 2025 and relative to other elements in the multiplication circuit. The circuit 2021 is needed since the split crossbar circuit is being used to compute two separate results with the same crossbar circuit: the crossbar to calculate a product magnitude; and the same crossbar also generates the wrap indication.

[0103] Considering Figure 14 further, as the second input of the split crossbar is a 1 -hot vector, only one of S1o, S1 i , SI2, and SI3 will be high, with the others all at ground, so that only one of the columns will be ON. Thus, if S1o is high, S11 , SI2, and SI3 are all at “0” and the input / (xbar_wrap) will be left to float, while / xbar will be determined by / SOo. Consequently, if S1o is high, if / SOo=O, then / xbar=“0”; and if / SOo=1 , then / xbar is a weak “1”, where, as discussed above, the full “1” value of VDD will be degraded or weakened to the reduced value of (VDD-Vth), where Vth is the threshold voltage of the crossbar NMOS switch. Similarly, as the cross-connected level restoration circuit 2021 is symmetric between its inputs, if S1o is low, / xbar is floating and / (xbar_wrap) will either be a “0” or a weak “1” based on the value of / SO3, / SO2, or / SOi respectively depending on which of S11, SI2, or SI3 is on. For the floating input, the cross-connected level restoration circuit 2021 will output a “0” for either a ”0” or weak “1” at the non-floating input; and for the floating input, a ”0” or weak “1” will respectively result in a (full VDD value) “1” or a “0”. This operation is illustrated with respect to Figures 15 and 16.

[0104] Figures 15 and 16 illustrate the operation of the cross-connected level restoration circuit embodiment of Figure 14 for a “0” input and a weak “1” input, respectively. In Figures 15 and 16, the / (xbar_wrap) input is floating and the / xbar input is either “0” or a weak “1”, corresponding to the S1o=“1” case. Based on the symmetry of the circuit’s topology between the / xbar and / (xbar_wrap) paths, for the S1o= “0” case the value would be swapped between the upper path and the lower path.

[0105] In Figure 15, the input / xbar = “0” so that, as discussed with respect to Figure 13, the output from level restorer 2023 is xbar= “1”. As xbar= “1”, the pull-up PMOS of level restorer 2023 is OFF. Looking now at the cross-coupled pull-up devices andthe / (xbar_wrap) path, as / xbar = “0” is the gate voltage of PMOS 2035, PMOS 2035 is ON and the floating input of level restorer 2025 is now “1”, so that xbas_wrap= “0” and the pull-up of level restorer 2025 is also on. Since xbar= “1”, the pull-up PMOS 2037 is OFF and does not contribute. PMOS 2031 now has its gate set to the high “1” level, it is OFF and does not contribute to the input to the top path level restorer 2023.

[0106] With respect to PMOS 2033, as xbar_wrap is “0”, it will also be ON. However, PMOS 2033 is formed to be a weak device relative to the other transistors setting the / xbar level so that, even though it is ON, its output to the / xbar input to level restorer 2023 is overridden by the “0” value provided at the input by / SOo. For example, PMOS 2033 can be sized to provide low current when ON, as represented by the broken line arrow. Similarly, in the symmetric embodiment of Figures 14, PMOS 2037 will be a similarly weak device. (Depending on the embodiment, the other pull-up PMOSs can also be relatively weak devices.) Consequently, when / xbar= “0” and / (xbar_wrap) is floating, xbar is a strong “1” and xbar_wrap= “0”.

[0107] Figure 16 considers when / xbar is a weak “1” and / (xbar_wrap) is again floating. As discussed with respect to Figure 13, the output from the level restorer 2023 will be xbar= “0”, with the PMOS back to the input (corresponding to PMOS 1905 of Figure 13) ON, taking the input to a strong “1”. As / xbar is now a strong “1”, 2035 is OFF, but as xbar= “0”, PMOS 2037 is ON. However, as PMOS 2037 is a relatively weak device, this results in a weak “1” at the input to level restorer 2025, but this is turned to a strong “1” by level restorer 2025, so that the output xbar_wrap= “0”. Consequently, PMOS 2031 is OFF and does not contribute, while PMOS 2033 is ON, further strengthening the “1” at / xbar. Consequently, when / xbar is a weak “1” and / (xbar_wrap) is floating, xbar= “0” and xbar_wrap= “0”.

[0108] Because of the symmetry of the cross-connected level restoration circuit 2021 , when / (xbar_wrap)= “0” and / xbar is floating, xbar_wrap is a strong “1” and xbar= “0”; and when / (xbar_wrap) is a weak “1” and / xbar is floating, xbar_wrap= “0” and xbar= “0”. In this way the circuit 2021 will return a “0” for the floating input that is to be ignored, while acting as a level restored for the active input, returning a strong “1” for a “0” input and a “0” for a weak “1” input.

[0109] Figure 23 is a flowchart to illustrate the operation of the cross-connected level restoration circuit embodiment of Figure 14. At 2301 , the cross-connected level restoration circuit 2021 receives first and second inputs (e.g., / xbar and / xbar_wrap), where either one of the inputs can be active and the other one of the inputs is inactive. The inactive input is floating and the active input either at a low logic value (“0”) or a degraded high logic value (weak “1”). At 2303 and 2305 the corresponding outputs for the two inputs are generated. For the active input, at 2303 the corresponding output is generated by the level restore circuit of the active input, either 2023 or 2025, resulting full high logic level (“1”) for a low logic level input or a low logic level (“0”) for the weak “1” of the degraded high logic input. The levels on the active input’s path then bias the cross-connected pull-up transistors (e.g., PMOSs 2031 , 2033, 2035, 2037) to set the output corresponding to the inactive input to a low logic level. Although presented here as being used for the split and folded crossbar circuit for modular multiplication, it can be applied to similar circuits, such as other arithmetical circuits such as a modular adder. More generally, when a row of logic circuitry provides a pair of related outputs for two subsets of switches in which only one is active at a time, the cross-connected level restoration circuit can be used.

[0110] Figures 18A and 18B illustrate a portion of an embodiment for a split and folded crossbar multiplier circuit for the n=13 example. Relative to Figure 11 A, Figure 18A has NMOSs for the switches, adds in the cross-connected level restoration circuits, and adds in 0 detection. More specifically, in the split crossbar array, each of the switches of Figurel 1 A is now implemented as a single NMOS, such as a finFET, ribbonFET, gate-all-around (GAA), or nano-wire or nano-sheet transistors, for example. With 0 detection, there are (p+1 ) / 2 second inputs and the split and folded crossbar multiplier circuit is of size (p-1 ) / 2 x (p+1 ) / 2.The operand B magnitude decoder now includes a 0 decode value 2451 . Each also now has a NMOS switch in column 2453 whose gate is connected to the 0 decode value 2451. The zero decode in crossbar ensures low-impedance on one of the two sets of crossbar outputs. Consequently, when operand B has zero magnitude, each of the unwrapped outputs will be pulled high (e.g., VDD-Vth), which will then be turned into a “0” by the subsequent level restores 2443.

[0111] The cross-connected level restorers 2443 are connected to receive the pair of outputs of each row to provide well-defined logic outputs of either ground or VDD, described above with respect to Figure 17. As the cross-connected level restorers 2443 also invert their input values, each operand A input now receives an inverter 2441 to cancel out the inversion of the level restorers 2443. As discussed above with respect to Figurel 1 A, the bottom row of the shifter array has all unwrapped switches, so that a more conventional level restore circuit 2445, as in Figure 13. Alternately, if a cross-connected level restored is used at 2445, to provide the 1 -bit value of the operand A, a “1” value (1 ’b1 ) can supplied to as the second input, as this value will be inactive when the first input is active. The outputs of the level restore circuits 2443 can then pass on to the rest of the circuit, which can be implemented as in Figure 18B.

[0112] Figure 18B repeats the elements of Figure 11 B, which are similarly numbered (e.g., XOR 1741 is now XOR 2431 ), but adds the three input AND gate 2439. AND block 2439 insures that the output product sign is zero when either of the input sources are zero. Put another way, this is a three input AND gate. One input is the output of XOR gate 2435, which represents the output sign when both sources are non-zero. The other two inputs to the AND gate 2439 have inverted inputs, which mean that both of these inputs (Operand A Decode 0, Operand B Decode 0) must be inactive (logic 0) for the output sign bit from the XOR gate 2435 to pass through to the output. If either of these two inverted inputs are logic T (indicating that one or both of the sources equal to zero), then the product sign will be forced to zero. Alternately, the circuitry of Figure 18B can be replaced with the circuitry of Figure 23B as presented below.

[0113] The embodiments for crossbar multipliers discussed so far have used a wrap-magnitude encoding. The following embodiments instead consider a signmagnitude encoding. The sign-magnitude encoding format can be implemented with less decoding than binary encoding in the folded multiply crossbar, since instead of needing to decode (for example) “1” and “12” separately from binary and then OR-ing them, there is only a single decode (for magnitude). Additionally, it is easier to convert operands in a binary encoding to / from the sign-magnitude RNS encoding than trying to convert binary to / from a ‘wrap’-magnitude encoding, as the “wrap” bit does not have any standard meaning outside RNS folded crossbar multiplication, whereas the signbit does. Figures 10 and 19 provide examples of remainder tables that can be used to illustrate converting input sign bits to wrap bits.

[0114] Figures 10 and 19 are respectively remainder tables for modulo 13 and 23 multiplication in wrap-magnitude format. In Figure 10, the “-5” / “5” values are bolded to indicate that the sign bit does not equal the wrap bit. In Figure 19, the -7 / 7, -5 / 5, - 10 / 10, and -11 / 11 values are bolded to indicate remainders for which the sign bit does not equal the wrap bit. Therefore, when decoding the inputs, if the inputs match an entry for which the sign and wrap bits differ, a “flip” bit can be set to be XOR-ed with the sign bit. As the decodings, including the special case of zero, are 1 -hot vector decodings, the multiplier can be implemented as an NMOS network (with level restoration) to perform this logic OR function or as a full CMOS circuit using with 1-hot inputs. Figures 20 and 21 illustrate embodiments for these two implementations.

[0115] Figures 20 and 21 respectively illustrate embodiments for sign bit to wrap bit conversion in an NMOS with level restoration and a CMOS embodiment for the modulo 13 example, where the MOSFETs can again be implemented in a FinFET embodiment. Referring to Figure 10 again and considering the Wrap=0 line, the only negative value is for 5, while the other values (and the unshown 0 value) are all nonnegative. To perform a sign bit to wrap bit conversion in an NMOS embodiment, the 5 input pulls down the output ( / flip) to ground, while the other input values pull up the output up to VDD - Vth. To restore the full logic values, a level restorer 2601 converts the output / flip to flip, so that in the NMOS (with level restoration) implementation the inverses for the input can be used to undo the inversion. For a CMOS embodiment, as in Figure 21 , the inverse of the 5 value is used to supply the gate voltage of a PMOS pull up, this the other (non-negative) values are all applied to a corresponding gate of an NMOS pull down. In the CMOS case of Figure 21 , a level restorer is not needed to provide the final output flip value as the logic values at the output are well-defined and level restoration is not needed.

[0116] Figure 22 shows an embodiment of a wrap select circuit for product wrap- to-sign conversion for a modulo 13 example. At upper left are shown the crossconnected level restoration circuits for the outputs of the crossbar array, which then serve as inputs to the wrap select circuity to determine the product sign. The wrap select circuit includes a determination circuit 2801 in an NMOS embodiment that thenis followed by a level restoration circuit 2803 like those described above to provide well-defined logical values for the wrap select output. Other embodiments can be CMOS based, although these embodiments would typically require greater area and complexity. The circuit of Figure 22 selects the final wrap bit from the crossbar product outputs that the multiplier circuit uses to create the product sign bit and is scalable with the folded crossbar. When the product row that matches the result, either xbar or xbar wrap will be a logic “1”, while for all other product rows, both of xbar and xbar wrap will be a logic “0”. Xbar 5 is the unwrapped output for output value 5 and is used (rather than xbar wrap 5) because the wrap bit for output 5 is different from the sign bit for output 5 (i.e. , +5 is wrapped, while -5 is unwrapped for modulo 13). The determination circuit 2801 also includes the “0” decode to ensure that all product outputs are covered.

[0117] Figures 23A and 23B present an embodiment for a folded and split NMOS crossbar multiplier for inputs in a sign-magnitude format for the n=13 example, where the two figures shows different parts of the same circuit. Figure 23A repeats the elements of Figure 18A, with a decoder 2901 to provide Operand B in a 1 -hot format if received as a magnitude and with a decoder 2903 to provide Operand A in a 1 -hot format if received as a magnitude. The 1 -hot format version is optimal for chained (i.e., sequentially dependent) multiplications as it eliminates 1-hot encoding / decoding to / from binary and if an operand is received the 1 -hot format, the corresponding decoder (2901 , 2903) is not needed. Relative to Figure 18A, the embodiment of Figures 23A includes elements related to the operand sign inputs. For operand B, the input values in the 1 -hot vector format are inputs into the Sign to Wrap circuit 2913, such as illustrated by the embodiments of Figure 20 or 21 . The result sign flip value for operand B and sign value for operand B are then inputs to the XOR gate 2915. Similarly for operand A, the input values in the 1-hot vector format and the 0 value from operand B are inputs into the Sign to Wrap circuit 2911 , that can also be as illustrated by the embodiments of Figure 20 or 21 . (The XOR gate corresponding to 2915 is in Figure 23B.) The operand A sign flip value, the outputs of the lever restorers, and the XOR 2915 continue on to the circuitry of Figure 23B.

[0118] In Figure 23B, relative to Figure11 B the OR gates 2921 and the AND gates 2923 serve the same respective functions as the OR gates 1721 and the AND gates 1725. If the output is provided in a 1 -hot vector format, such as for subsequentmultiplications, the AND gates 1723 for the encoding to binary format is not needed for the product magnitude. The equivalent of the AND gates 1723 are no longer present and the OR 1733 is replaced by wrap select circuit 2925, where Figure 22 is one embodiment for wrap select circuit 2925. Both of the inputs to the OR gates 2921 are provided to the wrap select circuit 2925. The product sign value is generated by the XOR logic circuit 2927, which receives the wrap select value from the wrap select circuit 2925, and AND logic 2929. Additionally, XOR logic circuit 2927 also receives the output of XOR 2915 for the B operand and output of XOR 2917 for the A operand, where XOR has inputs of the operand A sign flip value from sign to wrap circuit 2911 and the operand A sign value. The output of XOR 2927 is then one of the inputs to AND gate 2929, with additional inputs the inverted values of when Operand A decodes as 0 and when operand B decodes as 0. The resultant output of the circuit of Figures 23A and 23B is then a sign value for the product and the product either in 1-hot vector format (if AND gates 2923 are not included) or a magnitude format (if AND gates 2923 are used).

[0119] Figures 23A and 23B present a folded and split crossbar multiplier, specifically for a modulo 13 example. Referring back Figure 11 A, it can be seen that for modulo 13 all remainders are unique and represented for k=0 to 11. To use the modulo 23 example, as can be seen in Figure 9, all remainders are unique within the first half of the range (k = 0 to 10). All remainders from the first half of the range (k = 0 to 10) repeat exactly in the same order in the second half of the range (k = 11 to 21 ).

[0120] In a folded crossbar, the multiplication circuit only uses the first half of the remainders (i.e., for RNS-13, instead of remainder range of 0-12, the circuit of Figures 23A and 23B uses a remainder range of -6 to +6). In RNS-13, as shown in Figure 11 A a remainder of +8 is equivalent to a remainder of -5, so that for any remainder in which a negative appears in the top row (i.e., 8 or 5), that the sign-to-wrap indicator is set to a ‘T, else it is set to a ‘O’. Consequently, for RNS-13, an input of 5 has a sign-to-wrap indicator of “1”, with all other inputs have sign-to-wrap indicator of “0”. For the nonprimitive root of 2 case, such as for RNS-23 as illustrated in Figure 9, inputs of 7, 5, 10, and 11 have a sign-to-wrap indicator of “1”, while all other inputs have a sign-to- wrap indicator of “0”, so that for the product output, the circuit can recover the product sign using a wrap-to-sign which performs the same function as sign-to-wrap.

[0121] To determine the final product sign for moduli with a primitive root of 2, as for the RNS-13 example of Figure 23B, in AND logic 2929 the product sign is determined as:Product Sign = (Sign A XOR Sign-to-Wrap A) XOR(Sign B XOR Sign-to-Wrap B) XOR (Product Wrap-to-Sign) XOR (Product Wrap). For moduli without a primitive root of 2 and for which all remainders are unique in the first half of the table and repeat exactly in the second half of the table, such as RNS- 23, there is no product wrap:Product Sign = (Sign A XOR Sign-to-Wrap A) XOR(Sign B XOR Sign-to-Wrap B) XOR (Product Wrap-to-Sign).

[0122] Figures 24A and 24B present an embodiment for a folded NMOS crossbar multiplier for inputs in a sign-magnitude format for the n=23 example, where the two figures shows different parts of the same circuit. Figures 24A and 24B are laid out similarly to Figures 23A and 23B, although the equivalents of decoders A 2901 and B 2903 for the inputs and 2923 for the output are not shown. The decoded operand A is supplied to both the crossbar array and to the sign to wrap circuitry 3011 (along with the 0 value for operand B) to provide an operand A sign flip. The decoded operand B is supplied to both the crossbar array and to the sign to wrap circuitry 3013 to provide an operand B sign flip, which is an input to XOR 3015 along with the sign input of operand B. The crossbar array is folded, but not wrapped, so that there is only one output per row and a conventional level restorer, illustrated in Figure 13, can be used.

[0123] Moving to Figure 24B, the operand A sign flip value and operand A sign are both received at XOR 3017, whose output, along with that of XOR 3015, are received at XOR 3027. Relative to Figure 23B, the equivalent of the OR circuits 2921 are not needed as array is not wrapped. The third input to the XOR 3027 is from the wrap to sign circuit 3025, which takes an input directly from each row of the cross bar array. Similarly to Figure 23B, the product sign is provided by AND logic 3029 that has as inputs the output of XOR 3027 and the inverted values of when Operand A decodes as 0 and when operand B decodes as 0.

[0124] Relative to the embodiment of Figure 23A and 23B, the non-primitive root- of-2 embodiment of Figures 24A and 24B has no product wrap, so split crossbar array (as no product wrap) and no cross-connected level restore (because there is no splitcrossbar array), so that a more conventional level restore circuitry can be used. Consequently, in this embodiment the non-primitive root of 2 circuit can be simpler than the primitive root of 2 circuit. The non-primitive root of 2 must meet the following criteria: The first half of the moduli table must have unique values which represent exactly half of the remainders; and the second half of the table must exactly repeat the first half of the table in the same order.

[0125] Figure 25 is a flowchart for an embodiment of performing modular multiplication based on the embodiments of11 A, 11 B, 18A, 18B, 23A, 23B, 24A, and 24B, where Figures 23A and 23B will be used as the exemplary embodiment for presenting the steps of Figure 25. Beginning at step 3101 , a first modulo p input number is received in a sign-magnitude format, where p is a prime number. For example, taking the first modulo p input number as operand A, the sign value is received at XOR gate 2917 and the magnitude is received at the decoder 2903, although, if the magnitude is already in a 1 -hot vector format, the input can go directly to the inverters shown below decoder 2903 and the sign to wrap circuit 2911 . In step 3103, which can be concurrent with step 3101 , a second modulo p input number is received in a sign-magnitude format. For example, taking the second modulo p input number as operand B, the sign value is received at XOR gate 2915 and the magnitude is received at the decoder 2901 , although again if the magnitude is already in a 1 -hot vector format, the input can go directly to the switch array and the sign to wrap circuit 2913.

[0126] In step 3105, a magnitude of a product of the first modulo p input number and the second modulo p input number is determined by providing the magnitude of the first modulo p input number and the second modulo p input number as respective first and second inputs of the folded and split crossbar multiplier circuit of Figure 23A, along with the OR gates 2921 of Figure 23B. The sign value of the product of the first modulo p input number and the second modulo p input number is determined from the sign value of the of the first modulo p input number and the sign value of the of the second modulo p input number using the XOR gates 2915, 2917, 2927 and the wrap select circuit 2925 in step 3107.

[0127] In the embodiments presented above, a dense NMOS folded and split crossbar circuit replaces the conventional modular multiplier. By use of a regularNMOS crossbar structure, far less area is needed that for a conventional modular multiplier for a wide variety of RNS moduli. In particular, such embodiments: require no combinatorial multipliers or adders and no division to obtain remainder values; expand available moduli beyond conventional 2n, 2n-1 , or 2n+1 moduli and beyond extremely small moduli using lookup tables; and require minimal extra logic- only decoders and a small number of basic gates. Use of the folded and split architecture uses 75% less area in the crossbar than a “naive” (n-1 )x(n-1 ) implementation of an (un-folded, un-split) crossbar. Use of 1 -hot encoding in crossbar can reduce switching power over conventional modular multiplier.

[0128] To further improve efficiency and layout, the embodiment of Figures 23A and 23B uses a regular 1 -dimensional array of NMOS devices with 1 -hot inputs for wide logic OR’s or selection of multiple decodes; that is, it can help to insure that any remaining ‘glue’ logic scales along with the main crossbar logic. Additionally, the use of sign-magnitude encoding of RNS numbers improves RNS multiplication (as opposed to prior-art unsigned binary encoding). Use of a “cross-connected levelrestoration circuit” enables correct electrical operation of the split crossbar. These various features can significantly provide better area, power, and performance establishes argument for use of signed magnitude RNS encoding for GPUs, deep learning accelerators, autonomous vehicles, DSPs, vector and tensor processors, and CPUs.

[0129] Figure 26 is a block diagram to illustrate the use of remainder (modulo p) of power-of-g ordering (where g is a primitive root of the prime p) for both folded and unfolded crossbar embodiments for RNS multiplication. The inputs Source 0 and Source 1 are initially received and, if in sequential or other, non-power-of-g, ordering at respective source decoders 3701 and 3803. In addition to the input decoding described above with respect to Figures 23A and 23B or 18A and 18B, the decoders 3801 and 3803 also arrange the inputs in power-of-g ordering, where these can be in folded or unfolded format, which are then applied to the crossbar array 3805, which will correspondingly be a full (p-1 )x(p-1 ) array (for non-zero values), a folded and split ((p-1 ) / 2)x((p-1 ) / 2) array (for non-zero values), depending on the embodiment. For an unfolded crossbar array, the output will be unfolded, while for a folded embodiment the output will include a sign (or wrap) value, as well as a magnitude. If in foldedformat, unfolding circuitry 3807 is included for the final output. Some embodiment can also be “partially” folded, where some, but less than all, of the inputs are folded and shared by two unfolded inputs that are folded to a signal magnitude differentiated by a sign or other distinguishing value.

[0130] Referring back to the embodiment Figures 23A and 23B, this corresponds to a folded and split embodiment and the (non-zero) portions of the crossbar array Figure 23A is of size ((p-1 ) / 2)x((p-1 ) / 2) and inputs are of size >2(p-1 ). (With zero included, the size is ((p-1 ) / 2)x((p+1 ) / 2), with the second input size >2(p+1 ).) The circuitry of Figure 23B then corresponds to the unfolding circuitry 3807. In full (i.e., unfolded) embodiment, relative to Figure 23A the (non-zero) portions of the crossbar array is of size (p-1 )x(p-1 ) and inputs each of size (p-1 ). Additionally, the crossconnected level restorers 2443 can be replaced with a standard level restorer as in Figure 13. With respect to Figure 23B, as the operands and output are no longer folded, these elements are not needed in a full crossbar embodiment. In both the folded and full crossbar arrangement, the connections of the crossbar array represent the topology of the connections of the switches, but the actual layout on a die may be arranged in different ways, with corresponding changes in routing.

[0131] The discussion so far has considered the application of the presented embodiments for folded and unfolded residual number system crossbar multiplication units to the general case of two input values that can be changed for each multiplication. The following discussion now considers application to RNS compute- in-memory (CIM) multiplication, such as can be used for inference operations of a neural network in which a set of weights are determined in a training process and then stay fixed, but where the input vectors change during the inferencing process. More specifically, the next portion presents a multiplication unit that multiplies two selected RNS numbers in DRAM, one number being applied to the row inputs of a DRAM memory cell array, and the other number being represented by an array of bits previously set to T in the DRAM cell array. The technique is applicable to nxn cell arrays for prime RNS moduli, or a ‘folded’ (nxn) / 4 cell array, for prime RNS moduli. The discussion will mainly be presented in the context of a neural network when reference is made to a specific example, but the compute-in-memory techniques can more generally be used of other vector-matrix multiplication operations and in usessuch as digital signal processing (DSP) systems, digital filters, graphics accelerators, general-purpose CPUs, finite impulse response (FIR) filters, convolutional neural networks (CNNs) and other neural networks, multiply-accumulate (MAC) Units, and fault-tolerant systems, for example.

[0132] Conventionally, neural networks are implemented using GPUs; however, such approaches have drawbacks. GPUs used for such applications use extremely large and growing numbers of transistors, such as on the order of hundreds of billions. The size dies that can used are reticle limited, so that such GPUs need to rely on advanced packaging for multi-die solutions. Such systems are also power constrained due to GPUs’ to high bandwidth memory power consumption. The DRAM based compute-in-memory embodiments presented here can help to overcome these limitations. Figures 27 and 28 illustrate the use of an RNS sign-magnitude crossbar multiplier and its mapping to a DRAM array for computer-in-memory operations.

[0133] Figure 27 is a simplified version of the RNS 13 crossbar architecture of FIGs. 23A and 23B. More specifically, Figure 27 shows the crossbar array, the operator A decoder 2903, the operator B decoder 2901 , and the output decoders 3905; where the input decoders are simplified, the output decoder 3905 represents all of the elements to the right of the central array in Figures 23A and 23B, and the 0 input circuitry is not shown to simplify the discussion. Additionally, Figure 27 is rotated counterclockwise by a quarter turn relative to Figure 23A, so that Operand A is input from the left and Operand B is input from below.

[0134] Considering the case where the array of Figure 27 is for a compute-in- memory operation between a value of input vector for a layer and a weight value for the layer of neural network, the operand corresponding to the input will vary while the operand corresponding to the weight value will be fixed at a value determined during the training process. Figure 27 illustrates an example where the input is Operand A and the weight is Operand B and has an RNS 13 component of 3, as illustrated by the circling of 3 from the Operand B decoder 2901 and the heavier line weight. The outputs of the Operand B decoder 2901 are all low, except for 3, which is high. Consequently, all of the transistors whose control gates to the 3 value (extending upward diagonally to the left until the Operand A=4 row, then looping around to the right-most column and continuing upward diagonally to the left) are ON. When a valuefor Operand A is input, the output from the decoders 3905 will be the mod-13 product of the input (at Operand A) and 3, as is described in more detail above.

[0135] Although the arrangement of Figure 27 can implement a compute-in- memory determination of an input and a weight, as the weight value is fixed, the crossbar multiplier implementation will include a lot of redundant elements related to Operand B encoding and output decoding. In this situation, the circuitry and its operation can be simplified, and the operation improved by mapping the RNS crossbar multiplier components to programmable memory cells, such as DRAM based memory cells. This is illustrated for the RNS 13 component in Figure 28.

[0136] Figure 28 illustrates the mapping of a crossbar multiplier to an array of memory cells for an RNS 13 component in a DRAM embodiment. The array 4000 has memory cells each including a transistor with a control gate connected to row input from the row decoder 4001 and connected between a bit line (column line) and a top plate of a capacitor whose bottom plate is connected to ground. The DRAM memory cell is programmed by set a charge level on the capacitor. In the embodiments here, a “1” value is set by storing charge on the memory cell’s capacitor and a “0” set by leaving the upper capacitor plate at ground.

[0137] In the folded embodiment of Figure 28, the array is split diagonally into an unwrapped portion and a wrapped portion. As indicated by the broken line running diagonally from upper left to lower right, the bit lines are (except for the right-most) split. Memory cells above the broken line correspond to unwrapped values that are connected to sense amplifiers 4003a, whose outputs are stored in row buffer 4005a and then encoded in encoder 4007a to provide the output product for the unwrapped outputs. Memory cells below the broken line corresponding to wrapped values that are connected to sense amplifiers 4003b, whose outputs are stored in row buffer 4005b and then encoded in encoder 4007b to provide the output product for the wrapped outputs. Relative to Figure 27, there is not an input for Operand B, which is instead programmed into the memory cells. If Operand B were changed from multiplication to multiplication, this could be drawback as the array would need reprogramming; however, in an application where the weights (i.e., Operand B) is unchanging, the weight value can be programmed in to set its value, with only the input value (i.e., Operand A) changing from multiplication to multiplication. Figure 28illustrates, corresponding to the example of Figure 27, the example of the RNS 13 component of the weight equaling 3. The capacitors programmed are represented in a dashed circle and where, comparing to Figure 27, the programmed DRAMs of Figure 28 are in the same location as the ON transistors of Figure 27. For a give input value (i.e. , Operand A), the resultant output will be the same, with the operation and circuitry simplified, although an initial programming of the memory DRAM memory cells is required to set the weight values.

[0138] Figure 28 illustrates a sign-magnitude embodiment, which reduces the size of the array and peripheral circuitry by roughly a quarter. As with the crossbar embodiments, in addition to folded and split embodiments, full array embodiments can be used. This is illustrated in Figure 29, which again uses a modulo 13 example for comparison with the folded and split embodiment of Figure 28.

[0139] Figure 29 illustrates an RNS crossbar multiple mapped to DRAM in a magnitude only embodiment for a modulo 13 example. In this embodiment, the DRAM array is now of size (p-1 )x(p-1 ) for modulo p, with row decoder 4101 now having outputs 1 to (p-1 ), or 1 -12 in this example. As the array is no longer split and folded, each of the (p-1 ) bit lines span all of the rows and there are only the one set of (p-1 ) set sense amplifiers 4103, one for each bit line and here all placed at the bottom of the DRAM array, although these could be all at top or split between top and bottom depending on the selected layout. The sense amplifiers 4103 are then connected to the row buffer 4105 that in turn is connected to the encoder 4107 to provide the product of input Operand A and the Operand B as programmed into the memory cells.

[0140] Figure 29 illustrates an example where the value of 3 has been programmed into the DRAM array, where the memory cells programmed to “1” have their corresponding capacitors circled by a broken line. For example, in the bottom row (corresponding to 7) the memory cell along the bit line corresponding to an output of 8 has its capacitor circled as |7|i3 * |3| 13 = |8|i 3. Similar, the other memory cells written to “1” slope upward to the left until the left most bit line is reached, after which it loops around to the right most bit line and continues to slope upward to the left until reaching the top row. To increase efficiency and parallelism, an RNS input value can be applied to multiple memory cell arrays storing different RNS weight components concurrently. An example of this is illustrated in Figure 30.

[0141] Figure 30 illustrates an embodiment of an RNS multiplier for an example allowing a single input to applied in parallel to multiple stored weight values. In this example, an input of Operand A is applied concurrently to i weight values Bo-Bi stored in i arrays 4200-0 to 4200-(i-1 ), where the modulo 13 example is again used. Arrays 4200-0, 4200-1 , and 4200-(i-1 ) are explicitly shown and store corresponding respective RNS modulo 13 component values of Bo=3, Bi =5, and B( )=4, where these are examples and any modulo 13 weight value can be stored in any of the arrays as determined through the training process. For each of these weight values, the corresponding memory cells programmed to a “1” in each of the arrays has its capacitor circled, as in Figure 30.

[0142] Row decoder 4201 , sense amplifiers 4203-0, row buffer 4205-0, and encoder 4207-0, and the array 4200-0 can be as described above with respect to the similarly numbered elements of Figure 29. As in Figure 29, the stored weight value is again 3 and the resultant output Co is AxBo = Ax3, mod-13. Unlike Figure 29, the row output lines (or word lines) not only span the initial array 4200-0, but continue on to span arrays 4200-1 to 4200-( i-1 ). If Bj is the weight stored in array 4200-j and Cj is the output product, the application of the input of Operand A will compute-in-memory each of the products Cj=AxBj for j=0 to (i-1 ) concurrently.

[0143] Figures 31 -33 present examples of embodiments for the logical organization of wide data-out DRAM dies. If the DRAM device outputs more than one bit at a time (as in Figure 30), the internal organization is that of multiple arrays, each of which provides one bit toward an aggregate data output.

[0144] Figure 31 illustrates an X2, or two by, die 4300 with two DRAM memory arrays 4301. Although here labelled X2, the die can include additional circuitry and arrays, but the labelling refers to a single row decoder 4313 topology driving two memory arrays 4301 to generate two one bit outputs. The bit lines (column lines) from each array 4301 feed out into a corresponding set of sense amplifiers 4305 that can each have a corresponding data buffer 4303 and provide an output to a column decoder 4311 , which can include the output decoding. Referring back to Figure 30, the embodiment of Figure 31 can correspond to an i=2, two embodiment of Figure 30 or, as discussed below with respect to Figure 34, correspond to two different RNS components.

[0145] Figures 32 and 33 respectively present X4 and X8 DRAM embodiments, which could correspond to i=4 and i=8 in Figure 30 or 4 or 8 different RNS components in Figure 34. In Figure 32, the die includes 4 memory arrays 4401 on the memory die 4400, each of which can be programmed to a weight value. The row decoder 4413 can concurrently bias the arrays according to the input value. The bits lines from the 4 memory arrays 4401 provide the resultant product outputs to the corresponding sense amplifiers 4405, each connected to a corresponding data buffer 4403 and to a column decoder 4411 to provide the resultant product values of the RNS component. Similarly, in the X8 example of Figure 33 the die’s 8 memory arrays 4501 on the memory die 4500 can each be programmed to an RNS component of a weight value and subsequently concurrently biased by the row decoder 4513 according to an input value. The bits lines from the 8 memory arrays 4501 provide the resultant product outputs to the corresponding sense amplifiers 4505, each connected to a corresponding data buffer 4503 and to a column decoder 4511 to provide the resultant product values of the RNS component.

[0146] As discussed above with respect to Figures 2A and 2B, when multiplying two numbers in RNS implementation, each of the operands is broken down into RNS channels. In the case of DRAM embodiments of Figures 28-33, the RNS components of the one operand, such as the weight values of a neural network example, are programmed into the memory cells of an array of the corresponding RNS component. The other operand, such as from an input vector of a layer of the neural network, is broken down into RNS components and applied to that channel’s memory arrays. The number of RNS channels needed will depend on the data type, both its size and category, such as being byte wide or word wide, floating point or integer, and so on. For example, RNS [3|5|7|11113| 19|23|29] (i.e., with the 8 RNS channels of p=3, 5, 7, 11 , 13, 19, 23, 29) can provide a range of 1.42 x 227. Figures 34 and 35 consider embodiments for the efficient layout of these different RNS components on a die using the RNS [3|5|7|11113| 19|23|29] example.

[0147] Figure 34 presents an embodiment for a layout on a die of the RNS component arrays in an RNS [3|5|7|11113| 19|23|29] example. The 8 arrays are laid out side by side ordered in decreasing size from RNS 29 down to RNS 3 (labelled “3”) running from left to right in this example and aligned along one of their edges, here thelower edges. In this embodiment, the arrays are for a sign-magnitude encoding and the sign logic is not shown. Considering the RNS 13 component, the array 4000 and sense amplifiers 4003a and 4003b can be as described above with respect to Figure 28. The other RNS component arrays are similarly arranged with sense amplifiers (or “SA” for the smaller arrays) for the wrapped output across the top of each array and for the unwrapped output across the bottom of each array. The layout of the 8 arrays can correspond to the X8 DRAM embodiment of Figure 33, where the sense amplifiers connected on to the data buffers 4503 and column decoder 4511. The row decoder 4601 can correspond to 4513 of Figure 33.

[0148] The different RNS component arrays are programmed, in a neural array inferencing application, with corresponding weight value components. The programming can be any of the programming methods of a DRAM array, such as on a row by row write, but where in this application only a single, different column is written to a “1” in each array. An input value SrcO is then received by the row decoder 4601 , decoded into the RNS components, and used to bias the rows of the arrays by the row decoder. The decoder can be located along the side or sides of the group of arrays, such 4601 located along the side of the largest of the arrays, here the RNS 29 array. The row lines for each of the component arrays can then extend across the array, with the number row lines for each RNS p array having (p-1 ) / 2 inputs in a folded and split embodiment, as discussed above. As the largest p value here is p=29, the outputs of row decoder 4601 run from 0 to 13, which subsequent array having correspondingly fewer corresponding rows.

[0149] The descending staircase arrangement of the RNS component arrays of Figure 34 lends itself to particularly compact layout when there are pairs of such arrays formed on a die. An example of this is illustrated in Figure 35.

[0150] Figure 35 presents an embodiment for a layout on a die of paired RNS component arrays in an RNS [3|5|7|11113| 19|23|29] example. In the lower portion, Figure 35 repeats the elements of Figure 34, but without explicit inclusion of the row decoder 4601 . Above is another set of RNS [3| 517111113| 19123| 29] , but now arranged in reverse order laterally, now starting with the largest array (here RHS 29) at right and decreasing in size from right to left. Further, on the upper set of RNS component arrays, these are aligned along their upper edges. The result is now that the upperset of array are, shape-wise, rotated 180 degrees relative to the lower set, allowing them to be placed in a nested arrangement within a relatively compact rectangular footprint, where the gaps between the lower and upper sets of arrays can be used for peripheral circuitry (data buffers, column encoders, and so on) for the arrays.

[0151] Figure 36 is a flowchart of an embodiment for residual number system compute-in-memory multiplication. At 4801 , a first number is received, where, in the example embodiment, this could be a weight value for a layer of a neural network that can be determined as part of a training process for the neural network. 4803 converts this number into a residual number system representation, decomposing the numbers into one or more mod-p components of different prime numbers as described above with respect to Figures 2A and 2B, which are then (at 4805) are each programmed into corresponding array of memory cells, such as illustrated in Figures 28 and 29 for a DRAM cell embedment. The programming can be performed in any of the various DRAM implementation where, as seen in Figures 28 and 29, for each array a single memory cell on a different column of each array (for non-zero values) a single memory cell is written to memory cell. The dies for such an implementation can be provided to a user to be programmed accordingly or provided to a customer pre-programmed with values previously determined in a training process.

[0152] Once the weight or other first number values are written into the memory arrays, the multiplication process can begin with receiving of a second number, such as a value of an input vector for the layer of the neural network, at 4807. The second number is then converted into the residual number system components (at 4809) and each of these components can then be converted (at 4811 ) into a 1-hot vector format, where both of these can be implemented, for example, but row decoder 4201 , 4313, 4413, 4513, or 4601 in respective embodiments of Figures 30-34. At 4813 the vectors for each component are applied as bias levels to row inputs of the corresponding RNS component array.

[0153] As described above, embodiments can use DRAM storage cells to encode one operand of an RNS encoded multiplicative operand (call it the “multiplier”) and the row address of the DRAM storage cells can provide the other RNS encoded operand (call it the “multiplicand”). The row address can be implemented as the 1 -hot decode of the RNS encoded multiplicand, where the 1 -hot row address decode of the RNSencoded multiplicand is ordered according to the group ordering generated by a primitive root of a prime, where the prime is a prime RNS base or moduli. The product can then be read out as a 1 -hot decode of the bit-lines and is ordered according the group ordering generated by a primitive root of a prime, where, again, the prime is a prime RNS base or moduli. These embodiments are especially suitable for multiplication of activations and weights in a neural network. Successive bit-lines in the RNS encoded memory arrays can be used to compute the neural network product of single activation multiplied by many different weights in which the weights are stored in the RNS encoded DRAM cells. As illustrated with respect to Figures 35 and 37, a die layout for optimally nesting multipliers for different RNS moduli can be used in these embodiments.

[0154] Figure 37 is a high-level block diagram of a computing system 4901 that can be used to implement various embodiments of the microprocessors described above. In one example, the computing system 4901 is a network system 4901. Specific devices may utilize all of the components shown, or only a subset of the components, and levels of integration may vary from device to device. Furthermore, a device may contain multiple instances of a component, such as multiple processing units, processors, memories, transmitters, receivers, etc.

[0155] The network system may comprise a computing system 4901 equipped with one or more input / output devices, such as network interfaces, storage interfaces, and the like. The computing system 4901 may include a central processing unit (CPU) 4910, a memory 4920, a mass storage device 4930, and an I / O interface 4960 connected to a bus 4970, where the CPU can include a microprocessor such as described above with respect to Figures 1A-B. The computing system 4901 is configured to connect to various input and output devices (keyboards, displays, etc.) through the I / O interface 4960. The bus 4970 may be one or more of any type of several bus architectures including a memory bus or memory controller, a peripheral bus or the like.

[0156] The CPU 4910 may comprise any type of electronic data processor, including the microprocessor 120 of Figure 1 B, which includes divide circuits (e.g., any of the divide circuits described above). The CPU 4910 may be configured to implement any of the schemes described herein with respect to division, using any one orcombination of steps described in the embodiments. The memory 4920 may comprise any type of system memory such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), a combination thereof, or the like. In an embodiment, the memory 4920 may include ROM for use at boot-up, and DRAM for program and data storage for use while executing programs.

[0157] The mass storage device 4930 may comprise any type of storage device configured to store data, programs, and other information and to make the data, programs, and other information accessible via the bus 4970. The mass storage device 4930 may comprise, for example, one or more of a solid-state drive, hard disk drive, a magnetic disk drive, an optical disk drive, or the like.

[0158] The computing system 4901 also includes one or more network interfaces 4950, which may comprise wired links, such as an Ethernet cable or the like, and / or wireless links to access nodes or one or more networks 4980. The network interface 4950 allows the computing system 4901 to communicate with remote units via the network 4980. For example, the network interface 4950 may provide wireless communication via one or more transmitters / transmit antennas and one or more receivers / receive antennas. In an embodiment, the computing system 4901 is coupled to a local-area network or a wide-area network for data processing and communications with remote devices, such as other processing units, the Internet, remote storage facilities, or the like. In one embodiment, the network interface 4950 may be used to receive and / or transmit interest packets and / or data packets in an ICN. Herein, the term “network interface” will be understood to include a port.

[0159] The components depicted in the computing system of Figure 37 are those typically found in computing systems suitable for use with the technology described herein, and are intended to represent a broad category of such computer components that are well known in the art. Many different bus configurations, network platforms, and operating systems can be used.

[0160] The technology described herein can be implemented using hardware, firmware, software, or a combination of these. Depending on the embodiment, these elements of the embodiments described above can include hardware only or acombination of hardware and software (including firmware). For example, logic elements programmed by firmware to perform the functions described herein is one example of elements of the described multiplication circuit. A multiplication circuit can include a processor, FGA, ASIC, integrated circuit or other type of circuit. The software used is stored on one or more of the processor readable storage devices described above to program one or more of the processors to perform the functions described herein. The processor readable storage devices can include computer readable media such as volatile and non-volatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer readable storage media and communication media. Computer readable storage media may be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer readable storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information, and which can be accessed by a computer. A computer readable medium or media does (do) not include propagated, modulated or transitory signals.

[0161] Communication media typically embodies computer readable instructions, data structures, program modules or other data in a propagated, modulated or transitory data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as RF and other wireless media. Combinations of any of the above are also included within the scope of computer readable media.

[0162] In alternative embodiments, some or all of the software can be replaced by dedicated hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), special purpose computers, etc. In one embodiment, software (stored on a storage device) implementing one or more embodiments is used to program one or more processors. The one or more processors can be in communication with one or more computer readable media / storage devices, peripherals and / or communication interfaces.

[0163] It is understood that the present subject matter may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this subject matter will be thorough and complete and will fully convey the disclosure to those skilled in the art. Indeed, the subject matter is intended to cover alternatives, modifications and equivalents of these embodiments, which are included within the scope and spirit of the subject matter as defined by the appended claims. Furthermore, in the following detailed description of the present subject matter, numerous specific details are set forth in order to provide a thorough understanding of the present subject matter. However, it will be clear to those of ordinary skill in the art that the present subject matter may be practiced without such specific details.

[0164] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0165] The description of the present disclosure has been presented for purposes of illustration and description but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of thedisclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.

[0166] For purposes of this document, each process associated with the disclosed technology may be performed continuously and by one or more computing devices. Each step in a process may be performed by the same or different computing devices as those used in other steps, and each step need not necessarily be performed by a single computing device.

[0167] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

CLAIMSWhat is claimed is:

1. A multiplication circuit, comprising: a first array of memory cells connected along rows and columns, the memory cells of the first array storing a first number of a first modulo pi, where pi is a prime number and, for non-zero values of the first number, the first number is stored as a programmed memory cell value on a single memory cell of each row of the first array and each corresponding to a different column of the first array; and a decoder circuit configured to: receive a second modulo pi number; convert the second modulo pi number into a first binary valued vector, wherein, for non-zero values, one of the values of the first vector is one and other values of the first vector are zero; and determine a product of the first pi number and the second pi number by applying the second pi number to the rows of the first array.

2. The multiplication circuit of claim 1 , wherein the memory cells of the first array are DRAM memory cells.

3. The multiplication circuit of any of claims 1-2, wherein the first array is of size (pi-1 ) / 2 by (pi-1 ) / 2.

4. The multiplication circuit of claim 3, wherein the memory cells of the first array are connected along bit lines, each of the bit lines of a plurality of the bit lines of the first array including a first portion connected to the memory cells connected to a first output encoder and a second portion connected to the memory cells connected to a second output encoder.

5. The multiplication circuit of any of claims 1-4, wherein the first array is of size (pi-1 ) by (pi-1 ).

6. The multiplication circuit of claim 5, wherein each of the columns of the memory cells of the first array are connected along a corresponding bit line.

7. The multiplication circuit of claim 6, wherein the bit lines are connected to an output encoder.

8. The multiplication circuit of any of claims 1 -7, further comprising: one or more second arrays of memory cells connected along rows and columns, the memory cells of each of the second arrays storing a corresponding third number of modulo pi stored as a programmed memory cell value on a single memory cell of each row of the corresponding second array and each corresponding to a different column of the corresponding second array, and wherein the decoder circuit is further configured to: determine a product of the corresponding third pi number and the second pi number by applying the second pi number to the rows of the corresponding second array.

9. The multiplication circuit of any of claims 1 -8, further comprising: one or more second arrays of memory cells each storing an additional corresponding modulo p2 number, where each of moduli pi and p2 are distinct prime numbers and wherein the decoder circuit is further configured to: receive an input number; convert the input number to a plurality of residual number system components, including the second modulo pi number and corresponding additional modulo p2 components; and further determine the product or the input number and the modulo pi number and additional corresponding modulo p2 numbers.

10. The multiplication circuit of claim 9, wherein the multiplication circuit is on a die with the first array and one or more second array located one adjacent to another extending in a first direction on the die in order of decreasing size and aligned along a first side extending in the first direction, andwherein the decoder circuit is located along a side of the array of a largest of the first array and one or more second arrays opposite the side of the largest array adjacent to others of the first array and one or more second arrays and is connected to bias rows of the first array and one or more second arrays according to the converted input number.

11. The multiplication circuit of claim 10, the additional plurality of arrays comprising a first additional array of memory cells storing number of modulo pi and one or more second additional arrays of memory cells each storing a corresponding modulo p2 number, the additional array located one adjacent to another extending in the first direction on the die in order of increasing size and aligned along a side opposing the first side extending in the first direction, and the additional arrays located adjacent to the first array and one or more second arrays on the side of the first array and one or more second arrays opposing the first side extending in the first direction.

12. The multiplication circuit of any of claims 1 -11 , wherein the first number is a weight value of a layer of a neural network and the second modulo pi number is an input of the layer of the neural network.

13. A method of performing multiplication of a first and a second number, comprising: receiving a first number; converting the first number to one or more residual number system components, each residual number component corresponding to one of the one or more distinct prime numbers; converting each residual number system component of the first number into a binary valued vector, wherein, for non-zero values, one of the values of the vector is one and other values of the vector are zero; and applying the binary valued vector of each component of the first number as a set of bias levels to an array of memory cells storing a corresponding component of a second number converted to the one or more residual number system components.

14. The method of claim 13, further comprising prior to receiving the first number: receiving the second number; converting the second number to the one or more residual number system components; and programming each residual number system component of the second number into the corresponding one of the arrays of memory cells.

15. The method of any of claims 13-14, wherein the second number is a weight value of a layer of a neural network and the first number is an input for the layer of the neural network.

16. The method of any of claims 13-15, wherein applying the binary valued vector of each component of the first number to the array of memory cells storing the corresponding component of the second number includes concurrently applying the binary valued vector of each component of the first number to the array of memory cells storing the corresponding component of the second number.

17. The method of any of claims 13-16, wherein, for each array of memory cells storing a component of the second number, a size of the array is (p-1 ) / 2 by (p-1 ) / 2, where p is a corresponding modulus of the component.

18. The method of any of claims 13-17, wherein, for each array of memory cells storing a component of the second number, a size of the array is (p-1 ) by (p-1 ), where p is a corresponding modulus of the component.

19. A memory circuit, comprising: a plurality of arrays of memory cells connected along rows and columns, each array configured to store a modulus pi component of a residual number system representation of a first number as, for non-zero components of the first number, as a programmed memory cell value on a single memory cell of each row of the array andeach corresponding to a different column of the array, where pi is a different prime number for each of the arrays; and a decoder circuit connected to the plurality of arrays and configured to: receive a second number; convert the second number to components in the residual number system; convert each residual number system component of the second number into a binary valued vector, wherein, for non-zero values, one of the values of the vector is one and other values of the vector are zero; and applying the binary valued vector of each component of the second number as a set of bias levels to the rows of the array of memory cells storing the corresponding component of the second number.

20. The memory circuit of claim 19, further comprising: a plurality of sense amplifier circuits connected to the columns of the arrays; and an encoder connected to the sense amplifier circuits and configured to output a product of the first number and the second number.

21. The memory circuit of any of claims 19-20, wherein the memory cells are DRAM memory cells.

22. The memory circuit of any of claims 19-21 , wherein each array is of size (pi - 1 ) by (pi - 1 ).

23. The memory circuit of any of claims 19-22, wherein each array is of size (pi - 1 ) / 2 by (pi - 1 ) / 2.

Images