An efficient algorithm for 2's power-residue

Abstract

A method for performing scaling operations in a neural network, the method comprising: selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; selecting a k factor such that 2 k is a smallest positive integer satisfying, wherein is a number of moduli in the moduli set; precomputing and storing parameters in memory; and computing a scaled version of a Residue Number System (RNS) number by removing m least significant bits using the parameters and the k factor without binary reconstruction of the RNS number.

Description

Atty. Docket No.4502-85800 (6000726PCT02) AN EFFICIENT ALGORITHM FOR 2’S POWER-RESIDUE CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 691,046 filed September 5, 2024 by Futurewei Technologies, Inc. and titled “An Efficient Algorithm For 2’s Power-Residue,” which is hereby incorporated by reference. TECHNICAL FIELD

[0002] The present disclosure is generally related to improving processing efficiency, and in particular to various embodiments of systems and methods of an efficient algorithm for 2’s power- residue. BACKGROUND

[0003] The residue number system (RNS) is a non-weighted number system that provides efficient arithmetic operations, especially addition, subtraction, and multiplication. RNS represents numbers as a set of remainders (residues) with respect to a set of pairwise coprime moduli. RNS is widely used in digital signal processing and machine learning applications due to its carry-free, parallel nature. However, significant challenges are encountered for algorithms that involves consecutive multiplications, such as in multi-layer convolutional neural networks, where each layer involves a set of dot-products results in dynamic range that necessitates least significant bit (LSB) removal (scaling) to maintain efficiency. Traditional scaling approaches require sign detection and cascading operations, which are latency and hardware intensive. SUMMARY

[0004] A first aspect relates to an apparatus comprising a first set of modulo multipliers corresponding to a moduli set of a Residue Number System (RNS) number x, wherein the first set of modulo multipliers is configured to multiply a value xjin each RNS channel of the RNS numberAtty. Docket No.4502-85800 (6000726PCT02)x by a precomputed modular inverse ^ ^^^ = ^^ mod ^^ to obtain a first set of first results; asecond set of modulo multipliers corresponding to the moduli set, wherein each modulo multiplier of the second set of modulo multipliers is configured to multiply a first result in the first set of results by a precalculated modular constant corresponding to the first result to obtain a second set of second results; a first adder configured to add the second results in the second set of second results to obtain a sum; a first subtractor configured to calculate a partial residue correction value based on a redundant residue and a lower k bits of the sum ^^, wherein k is selected such that 2kis a smallest positive integer satisfying 2^ ≥ ^ , and wherein ^ is a number of moduli in themoduli set; a first multiplier configured to multiply the partial residue correction value and a multiplicative inverse of a dynamic range with respect to the redundant residue to obtain a third result; a second multiplier configured to multiply the third result and a residue of the dynamic range with respect to 2mto obtain correction value, wherein m is a predetermined power-of-2 scaling exponent; a second subtractor configured to calculate a fourth result by subtracting the correction value from the sum; a second adder configured to add 2^to the fourth result to obtain a fifth result when the fourth result is negative; and a multiplexer configured to select either the fourth result or the fifth result as an output result based on a sign bit of the fourth result, wherein the output result is a scaled value ^|^|^^^ computed fully in modular arithmetic and is used in scaling operations in a neural network.

[0005] Optionally, in a first implementation according to the first aspect, the precalculated modular constant is a lower m bits of a cofactor ^^.

[0006] Optionally, in a second implementation according to the first aspect or any implementation thereof, the apparatus further comprises a residue extractor configured to compute^^ = |^^|^^.in a third implementation according to the first aspect or any implementation thereof, the multiplicative inverse of a dynamic range with respect to theredundant residue is ^ ^^^ = ^ ^^^ ^^.Atty. Docket No.4502-85800 (6000726PCT02)

[0008] Optionally, in a fourth implementation according to the first aspect or any implementation thereof, the multiplexer outputs the fifth result as the output result when the sign bit equals 1, or outputs the fourth result as the output result when the sign bit equals 0.

[0009] Optionally, in a fifth implementation according to the first aspect or any implementation thereof, each modulo multiplier in the first set of modulo multiplier or the second set of modulo multiplier is implemented as a carry-free, parallel hardware unit operating independently for each modulus.

[0010] A second aspect relates to an apparatus comprising a first Residue Number System (RNS) encoder configured to encode binary input values into RNS format using a set of moduli to obtain first RNS encoded values; a multi-channel RNS algorithms component configured to perform RNS-based computations on the first RNS encoded values to obtain first RNS result values; a second RNS encoder configured to encode the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; a single-channel RNS algorithms component configured to perform RNS-based computations on the second RNS encoded values to obtain second RNS result values; a calculation component configured to receive the first RNS result values and the second RNS result values; and obtain a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; a third RNS encoder configured to encode the modulus value into RNS format to obtain third RNS encoded values; and a least significant bit (LSB) removal component configured to perform LSB removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.

[0011] Optionally, in a first implementation according to the second aspect, the redundant modulus is a power of 2 value.

[0012] Optionally, in a second implementation according to the second aspect or any implementation thereof, the LSB removal component is configured to subtract the third RNS encoded values from the first RNS result values in a carry-free manner.Atty. Docket No.4502-85800 (6000726PCT02)

[0013] Optionally, in a third implementation according to the second aspect or any implementation thereof, the apparatus further comprises memory configured to store precomputed constants comprising Cofactor of Modulus ^^, Multiplicative Inverse of ^^with respect to ^^, and . lower m bits of ^^.

[0014] A third aspect relates to a method for performing scaling operations in a neural network, the method comprising selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; selecting a k factor such that 2kis a smallest positive integersatisfying 2^ ≥ ^, wherein ^ is a number of moduli in the moduli set; precomputing and storingparameters in memory; and computing a scaled version of a Residue Number System (RNS) number by removing m least significant bits using the parameters and the k factor without binary reconstruction of the RNS number.

[0015] Optionally, in a first implementation according to the third aspect, selecting the power of 2 scaling factor m is based on a number of LSBs to remove.

[0016] Optionally, in a second implementation according to the third aspect or any implementation thereof, the parameters comprise modular inverses and cofactors.

[0017] Optionally, in a third implementation according to the third aspect or any implementation thereof, computing the scaled version the RNS number is implemented by a pipelined hardware scaling engine.

[0018] Optionally, in a fourth implementation according to the third aspect or any implementation thereof, the set of prime moduli are pairwise relatively prime.

[0019] A fourth aspect relates to an apparatus comprising a memory storing instructions; and one or more processors in communication with the memory, the one or more processors configured to execute the instructions to cause the apparatus to: select a dynamic range; select a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; select a power of 2 scaling factor m; select a k factor such that 2kis a smallestpositive integer satisfying 2^ ≥ ^ , wherein ^ is a number of moduli in the moduli set;Atty. Docket No.4502-85800 (6000726PCT02) precompute and store parameters in memory; and compute a scaled version of a Residue Number System (RNS) number by removing m least significant bits using the parameters and the k factor without binary reconstruction of the RNS number for performing scaling operations in a neural network.

[0020] Optionally, in a first implementation according to the fourth aspect, the power of 2 scaling factor m is based on a number of LSBs to remove.

[0021] Optionally, in a second implementation according to the fourth aspect or any implementation thereof, the parameters comprise modular inverses and cofactors.

[0022] Optionally, in a third implementation according to the fourth aspect or any implementation thereof, the apparatus comprises a pipelined hardware scaling engine.

[0023] Optionally, in a fourth implementation according to the fourth aspect or any implementation thereof, the set of prime moduli are pairwise relatively prime.

[0024] A fifth aspect relates to a computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer- executable instructions when executed by a processor of an apparatus, cause the apparatus to perform a method according to any of the preceding aspects or any implementation thereof.

[0025] For clarity, any one of the foregoing aspects may be combined with any one or more of the other foregoing aspects to create a new embodiment within the scope of the present disclosure.

[0026] These and other features, and the advantages thereof, will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF DRAWINGS

[0027] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.Atty. Docket No.4502-85800 (6000726PCT02)

[0028] FIG. 1 is a schematic diagram illustrating a RNS operations using a set of moduli in accordance with an embodiment of the present disclosure.

[0029] FIG.2 is a schematic diagram illustrating a convolutional neural network in accordance with an embodiment of the present disclosure.

[0030] FIG.3A is a schematic diagram that illustrates an implementation process to generate the 2’s power residue in accordance with a disclosed embodiment.

[0031] FIG.3B illustrates a numerical example of the process of FIG.3A in accordance with a disclosed embodiment.

[0032] FIG. 4 is a schematic diagram illustrating a scaling engine configured to perform an efficient algorithm for 2’s power-residue in accordance with an embodiment of the present disclosure.

[0033] FIG. 5 is a block diagram illustrating LSB removal in multi-layer neural network in accordance with an embodiment of the present disclosure.

[0034] FIG. 6 is a flowchart illustrating a scaling process in accordance with a disclosed embodiment.

[0035] FIG.7A is a flowchart illustrating a process for scaling an RNS number.

[0036] FIG.7B is a flowchart illustrating a process for scaling an RNS number in accordance with the disclosed embodiments.

[0037] FIG.8 is a diagram illustrating an apparatus according to an embodiment of the present disclosure. DESCRIPTION OF EMBODIMENTS

[0038] A problem addressed by the disclosed systems and methods below is the efficiency challenge encountered in RNS operations, particularly for algorithms involving consecutive multiplications, such as those found in multi-layer convolutional neural networks. Generally, computational requirements are resulting in ever increasing needs for computational power and memory to support the computational requirements. RNS may be used to reduce theAtty. Docket No.4502-85800 (6000726PCT02) computational resource requirements. While RNS provides significant advantages for arithmetic operations like addition, subtraction, and multiplication due to its carry-free and parallel nature, these benefits are quickly diminished in multi-layered applications where each layer involves dot- products and results in an exponential increase of the dynamic range. This uncontrolled growth in dynamic range necessitates LSB removal, or scaling, to maintain efficiency.

[0039] Traditional LSB removal approaches are problematic because they often rely on sign detection, cascading operations, or converting the RNS number back to a binary format, all of which are latency and hardware intensive. RNS inherently lacks direct magnitude or sign information, making these operations difficult without converting out of the RNS domain. The disclosed embodiments prevent the exponential increase of required dynamic range that would otherwise quickly erode the RNS benefits in applications like convolutional neural network. In particular, the disclosed embodiments overcome these limitations by enabling efficient, direct LSB removal within RNS, specifically using 2's power residue, without requiring binary reconstruction, sign detection, or even / odd detection, thereby preserving RNS's computational advantages for applications like neural networks and digital signal processing. As described herein, operations involving powers of 2 can be performed efficiently in the disclosed hardware implementations because they align with binary positions. This direct operation within RNS, combined with the bitwise simplicity of power-of-2 moduli and the parallel nature of RNS, leads to substantial improvements in hardware efficiency, including reduced latency, power consumption, and area.

[0040] It should be understood at the outset that although an illustrative implementation of one or more embodiments is provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.Atty. Docket No.4502-85800 (6000726PCT02)

[0041] The present disclosure relates generally to systems and methods configured to implement an efficient algorithm for 2’s power-residue |^|^^for scaling operation in RNS without sign detection, parity evaluation, or RNS to binary conversion. This algorithm is derived from theoretical analysis and guarantees to work over the entire range of the RNS, without massive search for parameters to ensure the fidelity of the algorithm. In an embodiment, a method for performing LSB removal directly in RNS includes computing the 2’s power residue from an RNSrepresentation of x (written as ρ = x mod 2m or ρ = |^|^^). The disclosed embodiments reducelatency and power consumption in rescaling operations, and is particularly suited to neural networks and digital signal processing.

[0042] In an embodiment, the algorithm uses a redundant modulus ^^ ≥ ^ (^ is the numberof the moduli of the RNS base) that is significantly smaller than other procedures and guarantees not to cause latency bottleneck in the RNS system. In some embodiments, for efficientimplementation, ^ is equal to 2 t ^^ o the smallest integer ^ such that ^^ = 2 ≥ ^ is chosen.The disclosed embodiment takes advantage of the internal structure of the algorithm and efficient modulo operations with respect to 2’s power, resulting in an efficient solution to the 2’s power residue|^|^^. For instance, the operations of this algorithm are modulo throughout (mostly with respect to 2’s power). Thus, no big numbers are involved, and the process can be performed in parallel without carry across modulus channels. As will be described, another key efficiency feature of the disclosed embodiments is the sharing of the modulo operations with respect to 2^and 2^ for ^ > ^ , the modulo calculation with respect to 2^ is achieved practically withoutrequiring resources.

[0043] The RNS represents numbers using remainders with respect to a set prime moduli. The RNS provides advantages in performing addition, subtraction, and multiplication primarily due to no carry propagation between RNS channels. As an example, FIG. 1 is a schematic diagram illustrating a RNS operations using a set of moduli. In the depicted example, the set of moduli includes prime numbers [47, 37, 29, 23, 19, 13, 11, 7, 5, 3] and is used to perform a multiplication operation (x ∙ y) in RNS where: x=50, y=40, and (x ∙ y) or xy=2000. Each numberAtty. Docket No.4502-85800 (6000726PCT02) is represented by its residues (i.e., remainder) with respect to the set of moduli. For instance, the residues of x mod 29 is 21 (i.e., 50 divided by 29 has a remainder of 21) and y mod 29 is 11. Then (21 × 11) mod 29 is 28. Each operation is performed independently in a separate channel allowing for parallel and carry-free computation (i.e., without having to carry values between digit positions, as in conventional number systems like decimal or binary).

[0044] FIG. 2 is a schematic diagram illustrating a convolutional neural network 200 in accordance with an embodiment of the present disclosure. The convolutional neural network 200 includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, and an output layer. Each of the layers in the convolutional neural network 200 involves a set of dot- products. Without loss of generality, assume the inputs are normalized with their magnitude less than 1. Each point at a layer’s output can be written as the following dot product: ! ^= ^1 2^^^

[0045] where ^^^can be^of the layer, and ^^can be considered as the neural netof that layer. With RNS implementation, both the inputs and the weights are to be quantized, for example into 16-bit integers (16-bit quantization). Without LSB removal or rescaling, the dynamic range will be 22∙16(2∙16 bit-width) at the end of layer 1, 23∙16(3∙16 bit- width) at the end of layer 2, 24∙16(4∙16 bit-width) at the end of layer 3, and 25∙16(5∙16 bit-width) at the end of layer 4. This exponential increase of dynamic range will quickly erode the RNS benefits. For continued efficient operation, at the end of each layer’s dot product operation, a) the RNS base is extended (add more RNS digits) to expand the dynamic range or b) the LSBs are removed (i.e., rescaling) to maintain the dynamic range. Expanding the dynamic range is suited to limited applications, while the LSB removal to maintain the dynamic range is suitable for most engineering applications.

[0046] Several rounding modes can be applied to the scaling, such as flooring, rounding, or ceiling. In the convolutional neural network 200 shown in FIG. 2, the LSB removal maintains the 16-bit precision at the input of each layer by removing the 16 least-significant bits to provideAtty. Docket No.4502-85800 (6000726PCT02) the output of the successive layer. The present disclosure describes a scaling solution which does not convert the RNS number to binary. Example embodiments described herein rescale (divide) by a factor of 216(16-bit LSB removal utilizing flooring, rounding, or ceiling). However, the disclosed embodiments may be applied to other rescaling factors.

[0047] Some LSB removal techniques use cascading ^ scaling-by-2 blocks to achieve scaling by 2^. In these techniques, each scaling-by-2 block uses sign detection and even / odd detection as well. The challenge of these techniques includes the complexity of the sign detection, even / odd detection, and the latency due to the cascading nature of the operations.

[0048] The present disclosure describes an LSB removal solution in RNS with 2’s power residue|^|^^, which is not cascading in nature. Instead, LSB removal is achieved in one step and without using sign detection or even / order detection. The present disclosure also presents three different embodiments for calculating the 2’s power residue |^|^^. The following notations and definitions in Table 1 are used herein. Defining D fi iti S b l C diti E ti V i bl T eAtty. Docket No.4502-85800 (6000726PCT02) Input Number to be |^|%&Primary Input t

[0049] In the present disclosure, let the RNS used in the dot product operation be defined by^ pairwise relatively co-prime moduli ^^ , . = 1,2, … , ^. This RNS can represent any variable^ ∈ 20, ^4 , where ^ = ^^^^ … , ^$ , called therange of the RNS. The RNSrepresentation of a variable in this RNS is 2^$ , ^$^^, … , ^^] , where ^^, . = 1,2, … , ^, is theAtty. Docket No.4502-85800 (6000726PCT02) residue of ^ with respect to the j-th co-prime modulus ^^, where ^ is the regular binary or decimal value. ^^is defined as follows. |^|%&, 78 ^ ≥ 0

[0050] ^^ = 6^ − ^ Equation (1)^ :ℎ<=>7?<derivation of the 2’s power residue inaccordance with a disclosed embodiment. Assume a RNS with its base defined by the moduli set@ = 2^$,^$^^, … , ^^], where ^^ , . = ^, ^ − 1, … ,1 are pairwise relatively co-prime. The RNSrepresentation of a variable ^ ∈ 20, ^4 can be represented by ^ RNS digits 2^$ , ^$^^, … , ^^]with ^ = |^| , . = ∏$^ %& ^, ^ − 1, … ,1, where ^ = ^"^ ^^ called the range of the RNS and

[0052] Based on Chinese Remainder Theorem (CRT), ^ = ,∑$^"^ ^^*^^^^*%& , , there existsCsome integer 0 ≤ =^ < ^ , where ^ is the number of moduli of the RNS base @ =2^$,^$^^, … , ^^], such that

[0053] ^ = ∑$^"^ ^^*^^^^*%& − =^^ Equation (2)

[0054] where ^^ = C%&, and ^^=*^^^^ *%&called the multiplicative inverse of ^^ with respectto ^ , which is defined^^ is an integer that 1 ≤ ^^ < ^

[0055] Taking modulo with respect^ for ^ ≥ 1 on both side of Equation (2) yields the2’s power residue|^|^^in Equation (3).

[0056] |^|^^ = ,∑$^"^ ^^*^^^^* − =^^, Equation (3)

[0057] Equation (4) and Equation (5).

[0058] |^|^^ = ,∑$^"^ ^^*^^^^* , − |=^|^|^^|^^ Equation (4)

[0059]

[0060] |^|^^ = 2^ + |^|^^ Equation (5)

[0061] where|^|^^can be precalculated, which is simply the lower ^ bits of ^. Note that Equation (3) is true for any positive ^. What follows describes how to obtain =^.Atty. Docket No.4502-85800 (6000726PCT02)

[0062] Let ^^be a redundant modulus that is pairwise relatively co-prime with the RNSmoduli set 2^$ , ^$^^, … , ^^] of the RNS base. As stated above, in some embodiments, forefficient implementation, let ^ ^^ = 2 ≥ ^, where ^ is the number of moduli in the RNS base.From Equation (2), the residue of the ^ with respect to the redundant modulus ^^can be written as

[0063] |^|%E = ,∑$^"^ ^^*^^^^*%& − =^^, Equation (6)%E

[0064] as follows.

[0065] |^|%E = F,∑$^"^ ^^*^^^^*%& , − |=^^|%EFEquation (7) %E

[0066] written

[0067] |=^^|%E = F,∑$^"^ , − ^%EFEquation (8)

[0068] multiplication with the multiplicative inverse^ = |1 | | | ^^ / ^ %Eof ^ with respect to ^^, and considering =^ %E = =^ due to ^^ = 2 ≥ ^ > =^,thus

[0069] =^ = F^^ H,∑$^"^ , − ^%EIF Equation (9)^^

[0070] (4) and Equation (5) give the 2’s power|^|^^. For negative value ^, the redundant residue ^%E = |^|^^ is defined by Equation (10).|

[0071] = ^|^^ , 78 ^ ≥ 0^, ^:ℎ<=>7?< Equation (10)[001 ≤ ^ ≤ ^ , |^|^ can be obtained simply by taking ^ leastsignificant bits of ^%E, there is no need of Equation (10), Equation (4),and Equation (5). For themajority of the applications, ^ > ^. What follows describe the efficient implementation detailsfor |^|^^ with ^ > ^.

[0073] In Equation (10) and Equation (4), let

[0074] ^^ = ^"^ ^^*^^^^* Equation (11)Atty. Docket No.4502-85800 (6000726PCT02)

[0075] ^^ = ,∑$^"^ ^^*^^^^*%& , Equation (12)^^

[0076] Equation (4) become Equation (13) and Equation (14).

[0077] =^ = − Equation (13)

[0078] Equation (14)

[0079] From Equation (11) and Equation (12), ^^can be obtained as follows in Equation (15)because both ^^ and ^^ are 2’s power residue of the same variable and ^ > ^.

[0080] ^^ = |^^|^^ Equation (15)

[0081] Furthermore, the modulo operation with respect to 2^in Equation (15) can be achieved simply by taking the lower ^ bits of ^^, thus practically no resource is needed to obtain ^^from ^^.

[0082] Examining Equation (12) for ^^calculation, Equation (12) can be further simplified as follows.

[0083] ^^ = ,∑$^"^ *^^*^^*^^^^*%& , Equation (16)^^

[0084] smaller than ^^, and*^^*^^is simply the lower m bits of ^^.

[0085] Therefore, the efficient solution to the 2’s power residue|^|^^can be concluded as follows. Equations (13), (15), and (16) give =^, and Equation (14) and Equation (4) give|^|^^.

[0086] FIG. 3A is a schematic diagram that illustrates an implementation process 700 to generate the 2’s power residue in accordance with a disclosed embodiment. In the depicted embodiment, 2^, 2^, and ^^in the multiplication blocks and addition blocks indicate the modulus of the modulo operation, and the minus sign outside the modulo addition block is to indicate a modulo subtraction.

[0087] Beginning at the bottom of FIG.3A, x1, …., xn-1, xn is input into the process 700. x1, …., xn-1, xnrepresent the RNS-encoded form of the original number ∣x∣. Each value is multiplied by a precomputed modular inverse αj modulo ^^by a corresponding modulo multiplier. Each ofAtty. Docket No.4502-85800 (6000726PCT02) these intermediate values is then multiplied by the lower m bits of the cofactor Mn, denoted *^^*^^and the results are summed according to Equation 27 to obtain ^^. From ^^, the process extracts the lower k bits, producing ^^. In an embodiment, ^^is used to correct rounding and ensure the scaled value is exact. Using Equation 24, the process then calculates a partial residuecorrection =^ = *^^K^^ − ^%EL*^^ using ^^ , ^%E, and ^^. ^^ is the multiplicative inverse of Mmodulo ^^, where ^^=2kis the redundant modulus. Partial residue correction =^is then multiplied by a precomputed dynamic range residue|^|^^to obtain a temporary correction term. The temporary correction term is used to compute |^|^^. A check is performed to determine whether|^|^^is positive. If|^|^^is positive then|^|^^remains the same. Otherwise, if|^|^^is negative, then 2^is added to|^|^^per Equation (5) to ensure that|^|^^is in range.

[0088] FIG. 3B illustrates a numerical example of the process 700 of FIG. 3A in accordance with a disclosed embodiment. In the depicted example, assume an RNS with 6 moduli,2^ , ^ , … , ^ ] = [19,13,11,7,5,3]. With this RNS, the smal O$ $^^ ^ lest ^ = 3 since 2 ≥ 6. Thus,^^ = 2^ = 8. In the depicted embodiment, to calculate the 2’s power residue with ^ = 8 for^ = 1002 , the redundant residue ^%E = |^|^R = 2. Equations (11)-(16) and (5) are used tocalculate |^|^S with the ^%E = 2 and the RNS representation of ^ = 2^T, ^U, … , ^^] = [14, 1, 1,1, 2, 0].

[0089] To calculate the parameters, begin by calculating ^ = ∏$^"^ ^^ = 285285. Basedon M, then ^^ = C = [15015, 21945, 25935, 40755, 57057, 95095]. The other calculatedparameters are as^^=*^^^^ [4,1,7, 1, 3,1], ^^ = |1 / ^|%E= 5; *^^*^S is [167, 185,79, 51, 225, 119], and

[0090] Second, calculate *^^^^*%& , . = 6,5, … ,1 = [18, 1, 7, 1, 1, 0] in Equation (16). Thencalculate ^^ with Equation (16), resulting in ^^ = 180, and ^^ = 4. Equation (9) is used tocalculate =^, resulting in =^ = 2. =^ is then used to calculate |=^|^|^^|^^ in Equation (14),resulting in |=^|^|^^|^^ = 202. Equation (14) is used to calculate |^|^^, resulting in |^|^^ =Atty. Docket No.4502-85800 (6000726PCT02) −22. Since it is negative, Equation (5) is used to finalize the|^|^^= 234, which is identical tothe direct calculation of |^|^^ = |1002|^S = 234.

[0091] FIG. 4 is a schematic diagram illustrating a scaling engine 400 configured to perform an efficient algorithm for 2’s power-residue in accordance with an embodiment of the present disclosure. For example, the scaling engine 400 may be used to implement the process 700 in FIG.3A. In the depicted embodiment, the scaling engine 400 is configured to perform an RNS- based scaling operation by 28. In this example, the scaling engine 400 takes as input ∣x∣ represented in RNS format using six pairwise prime moduli ^^∈{3,5,7,11,13,19}, with corresponding residues (x1 = 0, x2 = 2, x3 = 1, x4 = 1, x5 = 1, x6 = 14). For each modulus ^^, thescaling engine 400 uses the cofactor ^^ = ^ / ^^ , where ^ is the full dynamic range, themultiplicative inverse ^ ^1^ = ^^ mod ^^, and the modular-scaled cofactor *^^*^S.

[0092] Each input residue xj is multiplied by αj modulo ^^by a corresponding modulo multiplier. The results from each modulo multiplier is then passed to a modular multiplier (labeled like ∣2×8∣28or similar -- Notation means: 2-bit by 8-bit multiplier, modulo 28). The modular multiplier scales the results by the corresponding modular multiplicative constant *^^*^S. This part constructs the scaled contribution of each residue toward the overall number. The results from the modular multiplier are then summed via a 6:1 modular adder (modulo 28) to produce the intermediate partial 2's power residue sum ^^=180.

[0093] Additionally, the scaling engine 400 computes ^^ = |^^|^^=4, which is the lower kbits of ^^. A subtractor then calculates the partial residue correction value =^for ^^(=^=*^^K^^ − ^%EL*^^ )=2. A 3-bit by 3-bit multiplier, modulo 23 then multiples =^ by ^^ =5inverse of ^ with respect to redundant modulus ^^) to obtain result =2. A 3-bit by 8-bit multiplier, modulo 28then multiples the obtained result by|^|^^(residue of dynamic range ^ with respect to 2m) to obtain correction value=202.

[0094] An 8-bit subtractor then subtracts the correction value from the intermediate partial 2's power residue sum ^^=180 (180-202 = -22) to obtain a subtraction result. Because theAtty. Docket No.4502-85800 (6000726PCT02) subtraction result is negative (-22), the two’s complement of the subtraction result is computed=234 using Equation (5).

[0095] A multiplexer receives as input the subtraction result= -22, the most significant bit (MSB) of the subtraction result =1 from the 8-bit subtractor, and the two’s complement=234. The MSB is a sign bit that indicates the sign of the subtraction result. If the sign bit = 0, the number is non-negative (0 or positive). If the sign bit = 1, the number is negative. Thus, the multiplexerdetermines the value of |^|^^ = |1002|^S = 234 per Equation (5). The final output |^|^^ =234 represents the scaled value^|^|^^^, computed fully in modular arithmetic (i.e., in RNS) effectively removes the 8 least significant bits without binary reconstruction. The disclosed embodiment is highly efficient making it ideal for various applications including, but not limited, digital signal processing (DSP), cryptographic scaling, and neural networks.

[0096] FIG. 5 is a block diagram illustrating LSB removal in multi-layer neural network in accordance with an embodiment of the present disclosure. In particular, FIG. 5 illustrates how LSB removal can be performed entirely within RNS without converting back to binary, which significantly improves performance and hardware efficiency. In the depicted embodiment, in a first branch, binary inputs ^^, ^^,…, ^Yare encoded into RNS format at RNS encoding block 502 using a set of relatively prime moduli ^$, ^$^^,…, ^^(e.g., ^$= 47 (47 is the largest prime modulus channel (modulo 47) used to reach 232dynamic range). Two integers are relatively prime (also called coprime) if their GCD is 1. The two integers do not both need to be prime numbers so long as the two integers do not have any prime factor in common. The RNS encoding allows each ^^^value to be processed in parallel across multiple RNS channels for each (i.e., one) layer of the neural network at RNS algorithms (n-channel) block 504. RNS algorithms (n- channel) block 504 performs RNS-based computations such as dot products or weighted sums in parallel across multiple RNS channels. Therefore, block 504 represents a neural network function such as a MAC operation for a single layer of a neural network. The results ^^of these operations are then passed from block 504 to an online calculation block 512 and to an LSB removal in RNS with⌊^ / 2^⌋block 506.Atty. Docket No.4502-85800 (6000726PCT02)

[0097] In a second branch, the binary inputs ^^, ^^,…, ^Yare encoded at RNS encodingblock 508 using a single modulus pr (e.g., pr = 64 = 2T (6 bits & 2’s power modulo channel)).The encoded values ^^^are passed to a RNS algorithms (1-channel pr) block 510, which is configured to perform similar RNS-based computations (performed in block 504) using pr(i.e., only using one power-of-2 modulus). As described above, this operation can be performed efficiently due to shift-and-mask based modulo arithmetic. The result ^^of block 510 is passed to the online calculation block 512.

[0098] The online calculation block 512 takes as input ^^calculated at block 504 and ^^calculated at block 510, and is configured to extract the LSB part of the result using modulo 2m( |^|^^4 (i.e., keep lowest m bits). In an embodiment, the online calculation block 512 isimplemented using the scaling engine 400 of FIG. 4. The result|^|^^is passed to a RNS encoding block 514. RNS encoding block 514 is configured to encode the result back into multi- channel RNS representation using moduli ^$, ^$^^,…, ^^to produce computed value ?^, which is passed to the LSB removal in RNS with⌊^ / 2^⌋block 506.

[0099] The LSB removal in RNS with⌊^ / 2^⌋block 506 is configured to perform rescaling in accordance with the disclosed embodiments. As described above, instead of converting the values back to binary to drop precision (rescale), block 506 performs flooring in the RNS by usingthe 2’s power residue (^^ ^^^) to rescale directly in RNS. An example of the flooring operation isdescribed in FIG.3. For each channel, block 506 is configured to shift ^^right by m bits (logical division by 2m) directly in RNS domain using the computed value ?^. This operation effectively reduces the bit-width of the values (removing lower bits) while in the RNS format, which maintains efficiency and enables further computation. In some embodiments, the block 506 may also be configured to perform rounding (Equation (12)) or ceiling (Equation (13)) operations after flooring.

[0100] The block 506 outputs ^^after rescaling in RNS representation, which provides enough dynamic range for further RNS operations (e.g., processing in a next layer of the neural network). Block 506 provides the scaled result in binary format, and is converted back to RNS format before entering the next layer of the neural network. The disclosed embodiments areAtty. Docket No.4502-85800 (6000726PCT02) significantly more efficient in terms of power, delay, and area than the process of decoding back to binary, LSB removal in binary, then encoding to RNS.

[0101] FIG.6 is a flowchart illustrating a scaling process 600 in accordance with a disclosed embodiment. The scaling process 600 is configured to convert a number in RNS form into a scaled binary value (scaled by a power of 2). The scaling process 600 begins at step 602 by selecting a desired dynamic range Md. The dynamic range is the total range of integers that can be uniquely represented using a given set of moduli. The dynamic range defines how large the final or intermediate number (after scaling) should be.

[0102] At step 604, the scaling process 600 selects n prime moduli pj to achieve ^ =#$^^ >= Md. As defined in Table 2, each ^^ is an RNS modulus used in the RNS. Thus,^"^this step is selecting a set of moduli where the product of all chosen moduli is greater than the dynamic range to ensure that all numbers within the dynamic range can be represented.

[0103] The scaling process 600 at step 606 selects power of 2 scaling factor m (scaling factor will therefore be 2m). For example, if m=16, the scaling factor is 216. This sets the extended modulus pe=2m. As described above, a scaling factor is used to shift or rescale values during base extension in RNS. By choosing the scaling factor to be a power of 2, the disclosed embodiments can perform scaling directly in RNS. To provide the RNS equivalent operation of right-shifting a value in binary. As described above, to extend the RNS representation of a number from one set of moduli to another, the residue of the number is calculated with respect to a new modulus (e.g., pe) without fully reconstructing the number in binary. To do this efficiently, the redundant modulus pr is used to compute intermediate values (like weighted sums or scaled versions of the number), which then can be used to derive the residues in the extended base.

[0104] The scaling process 600 at step 608 selects k so that 2kis the smallest positive integersatisfying 2^ ≥ ^. ^ is a number of moduli in the RNS moduli.

[0105] At step 610, the scaling process 600 pre-calculates the off-line parameters and store as hardware constants or in registers (e.g., αi, *^^*^^, k, ^^).Atty. Docket No.4502-85800 (6000726PCT02)

[0106] The scaling process 600 at step 612 issues primary inputs (online inputs) |^|%&to a (pipelined or non-pipelined) hardware scaling engine (e.g., FIG. 4). Output of the hardware scaling engine will|^|^^, which is the binary value of x scaled by 2m. In some embodiments, this binary value of|^|^^is then converted to equivalent RNS values.

[0107] FIG.7A is a flowchart illustrating a process 700A for scaling an RNS number. In FIG. 7A, Xj is an RNS number (i.e., a number represented by its residues modulo) to be scaled (i.e., multiplied by another number). As shown in FIG.7A, Xjis first converted from RNS format to binary at block 702. The number in binary format is then truncated / scaled at block 704, and then converted from binary back to RNS at block 706 to produce scaled RNS number Xj / 2m.

[0108] FIG. 7B is a flowchart illustrating a process 700B for scaling an RNS number in accordance with the disclosed embodiments. In contrast to FIG. 7A, to scale RNS number Xj, the disclosed embodiments do not have to first convert RNS number Xjto binary. Instead, using LSB removal in RNS with 2’s power residue as described herein, the RNS number Xj can be scaled in RNS at block 708. Block 708 then output the binary value of x scaled by 2m. This binary value is then converted to RNS at block 710 to produce scaled RNS number Xj / 2m. Thus, the disclosed embodiments eliminate the need to perform RNS to binary conversion before truncation. This vastly improves efficiency when performing scaling such as in neural networks.

[0109] In summary, RNS native operations such as integer multiplication, addition, and subtraction are fast and efficient, but encounter significant challenges for algorithms that involve consecutive multiplications, such as in multi-layer convolutional neural networks. RNS representation of a variable provides no visibility to its magnitude. Because of this fundamental barrier, the LSB removal operations in RNS, such as flooring ⌊∙⌋, round (∙), and ceiling ⌈∙⌉, are tremendously difficult, thus limiting its value for practical use. The present disclosure describes the use of the 2’s power residue|^|^^as a bridge between RNS and binary system. The 2’s power residue |^|^^enables efficient LSB removal in RNS, a desirable RNS operation for RNS practical use, without need of sign detection or even / odd detection.Atty. Docket No.4502-85800 (6000726PCT02)

[0110] Additionally, an efficient algorithm and its efficient implementation solution to calculate the 2’s power residue |^|^^is described, which uses the RNS representation{^$ , ^$^^, … , ^^ } and a 2’s power redundant residue ^%E = |^|^^ . The algorithm is derivedtheoretical to ensure that the algorithm works for any ^ ∈ 20, ^] when the redundant modulus^ ≥ ^. The smallest integer ^^ ^ such that ^^ = 2 ≥ ^ for the redundant modulus is used forefficient implementation to take advantage of the efficient modulo operations with respect to 2’s power and reuse of ^^for ^^. In addition, for practical applications, the redundant modulus^^ is smaller than the largest modulus in the moduli set 2^$ , ^$^^, … , ^^]. Thus, the modulochannel associated with the redundant modulus is guaranteed to not create a latency bottleneck in the RNS system.

[0111] FIG. 8 is a diagram illustrating an apparatus 800 according to an embodiment of the present disclosure. The apparatus 800 can be used to implement embodiments of the present disclosure. The apparatus 800 includes receiver units (RX) 820 for receiving data via ingress ports 810. The apparatus 800 also includes transmitter units (TX) 840 for transmitting via data egress ports 850.

[0112] The apparatus 800 includes a memory 860 for storing the instructions and various data. The memory 860 can be any type of, or combination of, memory components capable of storing data and / or instructions. For example, the memory 860 can include volatile and / or non-volatile memory such as read-only memory (ROM), random access memory (RAM), ternary content- addressable memory (TCAM), and / or static random-access memory (SRAM). The memory 860 can also include one or more disks, tape drives, and solid-state drives. In some embodiments, the memory 860 can be used as an over-flow data storage device to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. In some embodiments, the memory 860 can be memory that is integrated with the processor 830.

[0113] The apparatus 800 has one or more processors 830 (e.g., central processing unit (CPU)) to process instructions. The one or more processors 830 may be implemented as one or moreAtty. Docket No.4502-85800 (6000726PCT02) CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The one or more processors 830 are communicatively coupled via a system bus with the ingress ports 810, RX 820, TX 840, egress ports 850, Input / Output (I / O) 880, and memory 860. I / O 880 provide the communication interfaces for enabling the apparatus 800 to receive input (e.g., from a keyboard, mouse, or touchscreen) and output information (e.g., to a display or printer).

[0114] The one or more processors 830 can be configured to execute instructions stored in the memory 860. For instance, the one or more processors 830 can be configured to perform any computational, comparison, determination, initiation, configuration, or any other action corresponding to the claims when the appropriate instruction is executed by the processor 830. As an example, in one embodiment, the memory 860 stores an efficient algorithm for 2’s power- residue module 870. The efficient algorithm for 2’s power-residue module 870 includes data, executable instructions, and / or one more sub-modules for implementing the disclosed embodiments. Alternatively, in some embodiments, the efficient algorithm for 2’s power-residue module 870 may be implemented in specialized hardware (e.g., a hardware scaling engine comprising adders and multipliers as described in FIG.3B and FIG.4) as opposed to be stored the memory 860. Thus, the inclusion of the efficient algorithm for 2’s power-residue module 870 substantially improves the functionality of the apparatus 800.

[0115] While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

[0116] In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure.Atty. Docket No.4502-85800 (6000726PCT02) Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.

Claims

Atty. Docket No.4502-85800 (6000726PCT02) CLAIMS What is claimed is:

1. An apparatus comprising: a first set of modulo multipliers corresponding to a moduli set of a Residue Number System (RNS) number x, wherein the first set of modulo multipliers is configured to multiply a value xj ineach RNS channel of the RNS number x by a precomputed modular inverse ^ ^1^ = ^^ mod ^^to obtain a first set of first results;a second set of modulo multipliers corresponding to the moduli set, wherein each modulo multiplier of the second set of modulo multipliers is configured to multiply a first result in the first set of results by a precalculated modular constant corresponding to the first result to obtain a second set of second results; a first adder configured to add the second results in the second set of second results to obtain a sum; a first subtractor configured to calculate a partial residue correction value based on a redundant residue and a lower k bits of the sum ^^, wherein k is selected such that 2kis a smallestpositive integer satisfying 2^ ≥ ^, and wherein ^ is a number of moduli in the moduli set;a first multiplier configured to multiply the partial residue correction value and a multiplicative inverse of a dynamic range with respect to the redundant residue to obtain a third result; a second multiplier configured to multiply the third result and a residue of the dynamic range with respect to 2mto obtain correction value, wherein m is a predetermined power-of-2 scaling exponent; a second subtractor configured to calculate a fourth result by subtracting the correction value from the sum; a second adder configured to add 2^to the fourth result to obtain a fifth result when the fourth result is negative; andAtty. Docket No.4502-85800 (6000726PCT02) a multiplexer configured to select either the fourth result or the fifth result as an outputresult based on a sign bit of the fourth result, wherein the output result is a scaled value ^ |^|^^^computed fully in modular arithmetic and is used in scaling operations in a neural2. The apparatus of claim 1, wherein the precalculated modular constant is a lower m bits of a cofactor ^^.

3. The apparatus according to any of claims 1-2, further comprising a residue extractorconfigured to compute ^^ = |^^|^^.

4. The apparatus according to any of claims 1-3, wherein the multiplicative inverse of adynamic range with respect to the redundant residue is ^ ^1^ = ^ ^^^ ^^.

5. The apparatus according to any of claims 1-4, wherein the multiplexer outputs the fifth result as the output result when the sign bit equals 1, or outputs the fourth result as the output result when the sign bit equals 0.

6. The apparatus according to any of claims 1-5, wherein each modulo multiplier in the first set of modulo multiplier or the second set of modulo multiplier is implemented as a carry-free, parallel hardware unit operating independently for each modulus.

7. An apparatus comprising: a first Residue Number System (RNS) encoder configured to encode binary input values into RNS format using a set of moduli to obtain first RNS encoded values; a multi-channel RNS algorithms component configured to perform RNS-based computations on the first RNS encoded values to obtain first RNS result values;Atty. Docket No.4502-85800 (6000726PCT02) a second RNS encoder configured to encode the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; a single-channel RNS algorithms component configured to perform RNS-based computations on the second RNS encoded values to obtain second RNS result values; a calculation component configured to receive the first RNS result values and the second RNS result values; and obtain a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; a third RNS encoder configured to encode the modulus value into RNS format to obtain third RNS encoded values; and a least significant bit (LSB) removal component configured to perform LSB removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.

8. The apparatus according to any of claims 7-8, wherein the redundant modulus is a power of 2 value.

9. The apparatus according to any of claims 7-9, wherein the LSB removal component is configured to subtract the third RNS encoded values from the first RNS result values in a carry- free manner.

10. The apparatus according to any of claims 7-10, further comprising memory configured to store precomputed constants comprising Cofactor of Modulus ^^, Multiplicative Inverse of ^^with respect to ^^, and . lower m bits of ^^.

11. A method for performing scaling operations in a neural network, the method comprising:Atty. Docket No.4502-85800 (6000726PCT02) selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; selecting a k factor such that 2k is a smallest positive integer satisfying 2^ ≥ ^, wherein ^is a number of moduli in the moduli set; precomputing and storing parameters in memory; and computing a scaled version of a Residue Number System (RNS) number by removing m least significant bits using the parameters and the k factor without binary reconstruction of the RNS number.

12. The method according to claim 11, wherein selecting the power of 2 scaling factor m is based on a number of LSBs to remove.

13. The method according to any of claims 11-12, wherein the parameters comprise modular inverses and cofactors.

14. The method according to any of claims 11-13, wherein computing the scaled version the RNS number is implemented by a pipelined hardware scaling engine.

15. The method according to any of claims 11-16, wherein the set of prime moduli are pairwise relatively prime.

16. An apparatus comprising: a memory storing instructions; and one or more processors in communication with the memory, the one or more processors configured to execute the instructions to cause the apparatus to:Atty. Docket No.4502-85800 (6000726PCT02) select a dynamic range; select a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; select a power of 2 scaling factor m; select a k factor such that 2k is a smallest positive integer satisfying 2^ ≥ ^ ,wherein ^ is a number of moduli in the moduli set; precompute and store parameters in memory; and compute a scaled version of a Residue Number System (RNS) number by removing m least significant bits using the parameters and the k factor without binary reconstruction of the RNS number for performing scaling operations in a neural network.

17. The apparatus according to claim 16, wherein the power of 2 scaling factor m is based on a number of LSBs to remove.

18. The apparatus according to any of claims 16-17, wherein the parameters comprise modular inverses and cofactors.

19. The apparatus according to any of claims 16-18, wherein the apparatus comprises a pipelined hardware scaling engine.

20. The apparatus according to any of claims 16-19, wherein the set of prime moduli are pairwise relatively prime.

21. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by a processor of an apparatus, cause the apparatus to perform a method according to any of claims 11-15.

Images