LSB removal in RNS with 2's power residue

Abstract

A method for scaling a Residue Number System (RNS) number. The method includes selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; setting an extended modulus to be equal to 2 m ; selecting a redundant modulus; calculating and storing off-line parameters the off-line parameters in memory; and performing scaling of the RNS number using the off-line parameters and redundant modulus to obtain a binary value of the RNS number scaled by 2 m .

Description

Atty. Docket No.4502-85700 (6000720PCT02) LSB REMOVAL IN RNS WITH 2’S POWER RESIDUE CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 680,394 filed August 7, 2024 by Futurewei Technologies, Inc. and titled “LSB Removal in RNS with 2’s Power Residue |x|2m,” which is hereby incorporated by reference. TECHNICAL FIELD

[0002] The present disclosure is generally related to improving processing efficiency, and in particular to various embodiments of systems and methods for least significant bits (LSB) removal in the residue number system (RNS) with 2’s power residue. BACKGROUND

[0003] RNS is a non-weighted number system that provides efficient arithmetic operations, especially addition, subtraction, and multiplication. RNS represents numbers as a set of remainders (residues) with respect to a set of pairwise coprime moduli. RNS is widely used in digital signal processing and machine learning applications due to its carry-free, parallel nature. However, significant challenges are encountered for algorithms that involves consecutive multiplications, such as in multi-layer convolutional neural networks, where each layer involves a set of dot-products results in a dynamic range that necessitates LSB removal (scaling) to maintain efficiency. Traditional scaling approaches use sign detection and cascading operations, which are latency and hardware intensive. SUMMARY

[0004] A first aspect relates to an apparatus comprising a first set of N multipliers configured to multiply a value xj in each Residue Number System (RNS) channel of an RNS number x by a first precomputed constant ^^^̂^to generate a first set of results, wherein ^^^̂^=Atty. Docket No.4502-85700 (6000720PCT02)^^^^^^^^ ^^^^ |^^|^^, where ^^ is a redundant modulus, M is a dynamic range, ^^^^^^ is a firstresidue of cofactor of modulus ^^with respect to the redundant modulus, a firstmultiplicative inverse of M with respect to redundant modulus ^^^; a first adder configured to add the first set of results to obtain a first residue sum ^^^^with respect to the redundant modulus; a second set of N multipliers configured to multiply the value xjby a second precomputed constant^^^̂^ to generate a second set of results, wherein ^^^^̂ = ^^^^^^^ ^ ^^^^ |^^|^^ , where ^^ is anextended modulus and is a power of 2 scaling factor,of cofactor of modulus ^^with respect to the extended modulus, and^^^^^^^^is a second multiplicative inverse of M with respect to extended modulus ^^^; a second adder configured to add the second set of results to obtain a second residue sum ^^^^with respect to the extended modulus; a third adder configured to calculate a first temporary residue with respect to the redundant modulus using the first residue sum ^^^^; a first multiplier configured to calculate a multiplicative factor k with respect to the redundant modulus using the first temporary residue; a second multiplier configured to calculate a second temporary residue with respect to the extended modulus using the multiplicative factor k; and a fourth adder configured to calculate a third residue of the RNS number x with respect to the extended modulus xpe using the second residue sum and the second temporary residue, wherein the third residue xpeis used to perform least significant bits (LSB) removal in RNS of the value xj to enable scaling operations in a neural network.

[0005] Optionally, in a first implementation according to the first aspect, the third adder is configured to calculate the first temporary residue with respect to the redundant modulus by subtracting a fourth residue of the RNS number x with respect to the redundant modulus (xpr) from the first residue sum ^^^^.

[0006] Optionally, in a second implementation according to the first aspect or any implementation thereof, the first multiplier is configured to calculate the multiplicative factor kAtty. Docket No.4502-85700 (6000720PCT02) with respect to the redundant modulus by multiplying a multiplicative inverse of M with respect to the redundant modulus and the first temporary residue.

[0007] Optionally, in a third implementation according to the first aspect or any implementation thereof, the second multiplier is configured to calculate the second temporary residue with respect to the extended modulus by multiplying the multiplicative factor k and a fifth residue of the dynamic range with respect to the extended modulus (^^^).

[0008] Optionally, in a fourth implementation according to the first aspect or any implementation thereof, the redundant modulus is a power of 2 value.

[0009] Optionally, in a fifth implementation according to the first aspect or any implementation thereof, the redundant modulus is a smallest non-failing modulus value for the extended modulus for all values in the dynamic range based on simulation analysis.

[0010] A second aspect relates to an apparatus comprising a first Residue Number System (RNS) encoder configured to encode binary input values into RNS format using a set of moduli to obtain first RNS encoded values; a multi-channel RNS algorithms component configured to perform RNS-based computations on the first RNS encoded values to obtain first RNS result values; a second RNS encoder configured to encode the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; a single-channel RNS algorithms component configured to perform RNS-based computations on the second RNS encoded values to obtain second RNS result values; a calculations component configured to receive the first RNS result values and the second RNS result values; and obtain a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; a third RNS encoder configured to encode the modulus value into RNS format to obtain third RNS encoded values; and an LBS removal component configured to perform LBS removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.Atty. Docket No.4502-85700 (6000720PCT02)

[0011] Optionally, in a first implementation according to the second aspect, the redundant modulus is greater than a threshold sufficient to prevent failure conditions based on simulations.

[0012] Optionally, in a second implementation according to the second aspect or any implementation thereof, the redundant modulus is a power of 2 value.

[0013] Optionally, in a third implementation according to the second aspect or any implementation thereof, the LBS removal component is configured to subtract the third RNS encoded values from the first RNS result values in a carry-free manner.

[0014] Optionally, in a fourth implementation according to the second aspect or any implementation thereof, the apparatus further comprises memory configured to store precomputed constants comprising partial moduli, modular inverses, total modulus, and redundancy parameters.

[0015] A third aspect relates to a method for performing scaling operations in a neural network implemented by a scaling engine. The method includes selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; setting an extended modulus to be equal to 2m; selecting a redundant modulus; calculating and storing off-line parameters in memory; obtaining a value of the RNS number scaled by 2musing the off-line parameters and the redundant modulus; and performing LBS removal in RNS on the RNS number using the value to obtain scaled RNS values without binary reconstruction of the RNS number.

[0016] Optionally, in a first implementation according to the third aspect, the redundant modulus is greater than a threshold sufficient to prevent failure conditions.

[0017] Optionally, in a second implementation according to the third aspect or any implementation thereof, selecting the redundant modulus comprises ensuring that the redundant modulus prevent the failure conditions based on simulations.

[0018] Optionally, in a third implementation according to the third aspect or any implementation thereof, the scaling engine is a pipelined hardware scaling engine.

[0019] Optionally, in a fourth implementation according to the third aspect or any implementation thereof, the redundant modulus is a power of 2 value.Atty. Docket No.4502-85700 (6000720PCT02)

[0020] Optionally, in a fifth implementation according to the third aspect or any implementation thereof, the prime moduli are pairwise relatively prime.

[0021] Optionally, in a sixth implementation according to the third aspect or any implementation thereof, wherein the off-line parameters comprise: residue of the RNS number with respect to the redundant modulus; multiplicative inverse of the dynamic range with respect to the redundant modulus; and residue of the dynamic range with respect to the extended modulus.

[0022] A fourth aspect relates to a method comprising encoding binary input values into RNS format using a set of moduli to obtain first RNS encoded values; performing RNS-based computations on the first RNS encoded values to obtain first RNS result values; encoding the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; performing RNS-based computations on the second RNS encoded values to obtain second RNS result values; obtaining a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; encoding the modulus value into RNS format to obtain third RNS encoded values; and performing LBS removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.

[0023] Optionally, in a first implementation according to the fourth aspect, the redundant modulus is greater than a threshold sufficient to prevent failure conditions based on simulations.

[0024] Optionally, in a second implementation according to the fourth aspect or any implementation thereof, the redundant modulus is a power of 2 value.

[0025] Optionally, in a third implementation according to the fourth aspect or any implementation thereof, performing LBS removal in RNS comprises subtracting the third RNS encoded values from the first RNS result values in a carry-free manner.

[0026] A fifth aspect relates to an apparatus comprising a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute theAtty. Docket No.4502-85700 (6000720PCT02) instructions to cause the apparatus to perform the method according to any of the preceding aspects or any implementation thereof.

[0027] A sixth aspect relates to a computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer- executable instructions when executed by a processor of an apparatus, cause the apparatus to perform a method according to any of the preceding aspects or any implementation thereof.

[0028] For clarity, any one of the foregoing aspects may be combined with any one or more of the other foregoing aspects to create a new embodiment within the scope of the present disclosure.

[0029] These and other features, and the advantages thereof, will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF DRAWINGS

[0030] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0031] FIG. 1 is a schematic diagram illustrating a RNS operations using a set of moduli in accordance with an embodiment of the present disclosure.

[0032] FIG.2 is a schematic diagram illustrating a convolutional neural network in accordance with an embodiment of the present disclosure.

[0033] FIG.3 is a flowchart illustrating a process for performing a flooring operation in RNS with a 2’s power residue in accordance with an embodiment of the present disclosure.

[0034] FIG. 4 is a flow diagram illustrating a flooring operation in RNS with a 2’s power residue in accordance with an embodiment of the present disclosure.

[0035] FIG. 5 is a flow diagram illustrating a rounding process in RNS after flooring in accordance with an embodiment of the present disclosure.Atty. Docket No.4502-85700 (6000720PCT02)

[0036] FIG. 6 is a flow diagram illustrating a ceiling process in RNS after flooring in accordance with an embodiment of the present disclosure.

[0037] FIG. 7A is a block diagram illustrating a calculation module configured to perform efficient base extension in RNS in accordance with an embodiment of the present disclosure.

[0038] FIG. 7B is a schematic diagram illustrating a scaling engine configured to perform efficient base extension in RNS in accordance with an embodiment of the present disclosure.

[0039] FIGS.8A and 8B illustrate the failure rate of a redundant modulus for a fixed extended modulus.

[0040] FIGS.9A and 9B illustrate the failure rate of a redundant modulus for a fixed extended modulus.

[0041] FIGS. 10A and 10B illustrate the failure rate of a redundant modulus for a fixed extended modulus.

[0042] FIG. 11 is a block diagram illustrating LSB removal in multi-layer neural network in accordance with an embodiment of the present disclosure.

[0043] FIG. 12 is a flowchart illustrating a scaling process in accordance with a disclosed embodiment.

[0044] FIG.13A is a flowchart illustrating a process for scaling an RNS number.

[0045] FIG.13B is a flowchart illustrating a process for scaling an RNS number in accordance with the disclosed embodiments.

[0046] FIG. 14 is a diagram illustrating an apparatus according to an embodiment of the present disclosure. DESCRIPTION OF EMBODIMENTS

[0047] A problem addressed by the disclosed systems and methods below is the efficiency challenge encountered in RNS operations, particularly for algorithms involving consecutive multiplications, such as those found in multi-layer convolutional neural networks. Generally, computational requirements are resulting in ever increasing needs for computational power andAtty. Docket No.4502-85700 (6000720PCT02) memory to support the computational requirements. RNS may be used to reduce the computational resource requirements. While RNS provides significant advantages for arithmetic operations like addition, subtraction, and multiplication due to its carry-free and parallel nature, these benefits are quickly diminished in multi-layered applications where each layer involves dot- products and results in an exponential increase of the dynamic range. This uncontrolled growth in dynamic range necessitates LSB removal, or scaling, to maintain efficiency.

[0048] Traditional LSB removal approaches are problematic because they often rely on sign detection, cascading operations, or converting the RNS number back to a binary format, all of which are latency and hardware intensive. RNS inherently lacks direct magnitude or sign information, making these operations difficult without converting out of the RNS domain. The disclosed embodiments prevent the exponential increase of required dynamic range that would otherwise quickly erode the RNS benefits in applications like convolutional neural network. In particular, the disclosed embodiments overcome these limitations by enabling efficient, direct LSB removal within RNS, specifically using 2's power residue, without requiring binary reconstruction, sign detection, or even / odd detection, thereby preserving RNS's computational advantages for applications like neural networks and digital signal processing. As described herein, operations involving powers of 2 can be performed efficiently in the disclosed hardware implementations because they align with binary positions. This direct operation within RNS, combined with the bitwise simplicity of power-of-2 moduli and the parallel nature of RNS, leads to substantial improvements in hardware efficiency, including reduced latency, power consumption, and area.

[0049] It should be understood at the outset that although an illustrative implementation of one or more embodiments is provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.Atty. Docket No.4502-85700 (6000720PCT02)

[0050] The present disclosure relates generally to efficient systems and methods for LSB removal in RNS using 2’s power residue without sign detection, parity evaluation, or RNS to binary conversion. In an embodiment, a method for performing LSB removal directly in RNS includes computing the 2’s power residue from an RNS representation of x (written as ρ = x mod2m or ρ = |^|^^). In some embodiments, the method supports flooring, rounding, and ceilingoperations without reconstructing x in binary. The present disclosure describes three efficient algorithms for computing the 2’s power residue, including one based on base extension with a redundant modulus. The process for LSB removal in RNS using 2’s power residue as described in the present disclosure reduce latency and power consumption, and is particularly suited to neural networks and digital signal processing.

[0051] The RNS represents numbers using remainders with respect to a set prime moduli. The RNS provides advantages in performing addition, subtraction, and multiplication primarily due to no carry propagation between RNS channels. As an example, FIG. 1 is a schematic diagram illustrating a RNS operations using a set of moduli. In the depicted example, the set of moduli includes prime numbers [47, 37, 29, 23, 19, 13, 11, 7, 5, 3] and is used to perform a multiplication operation (x ∙ y) in RNS where: x=50, y=40, and (x ∙ y) or xy=2000. Each number is represented by its residues (i.e., remainder) with respect to the set of moduli. For instance, the residues of x mod 29 is 21 (i.e., 50 divided by 29 has a remainder of 21) and y mod 29 is 11. Then (21 × 11) mod 29 is 28. Each operation is performed independently in a separate channel allowing for parallel and carry-free computation (i.e., without having to carry values between digit positions, as in conventional number systems like decimal or binary).

[0052] FIG. 2 is a schematic diagram illustrating a convolutional neural network 200 in accordance with an embodiment of the present disclosure. The convolutional neural network 200 includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, and an output layer. Each of the layers in the convolutional neural network 200 involves a set of dot- products. Without loss of generality, assume the inputs are normalized with their magnitude less than 1. Each point at a layer’s output can be written as the following dot product:Atty. Docket No.4502-85700 (6000720PCT02) ^^ = ^ ^1 2^^^

[0053] where ^^^can be of the layer, and ^^^can be considered asthe neural net weights of that layer. both the inputs and the weights are to be quantized, for example into 16-bit integers (16-bit quantization). Without LSB removal or rescaling, the dynamic range will be 22∙16(2∙16 bit-width) at the end of layer 1, 23∙16(3∙16 bit- width) at the end of layer 2, 24∙16(4∙16 bit-width) at the end of layer 3, and 25∙16(5∙16 bit-width) at the end of layer 4. This exponential increase of dynamic range will quickly erode the RNS benefits. For continued efficient operation, at the end of each layer’s dot product operation, a) the RNS base is extended (add more RNS digits) to expand the dynamic range or b) the LSBs are removed (i.e., rescaling) to maintain the dynamic range. Expanding the dynamic range is suited to limited applications, while the LSB removal to maintain the dynamic range is suitable for most engineering applications.

[0054] Several rounding modes can be applied to the scaling, such as flooring, rounding, or ceiling. In the convolutional neural network 200 shown in FIG. 2, the LSB removal maintains the 16-bit precision at the input of each layer by removing the 16 least-significant bits to provide the output of the successive layer. The present disclosure describes a scaling solution which does not convert the RNS number to binary. Example embodiments described herein rescale (divide) by a factor of 216(16-bit LSB removal utilizing flooring, rounding, or ceiling). However, the disclosed embodiments may be applied to other rescaling factors.

[0055] Some LSB removal techniques use cascading ^ scaling-by-2 blocks to achieve scaling by 2^. In these techniques, each scaling-by-2 block uses sign detection and even / odd detection as well. The challenge of these techniques includes the complexity of the sign detection, even / odd detection, and the latency due to the cascading nature of the operations.

[0056] The present disclosure describes an LSB removal solution in RNS with 2’s power residue|^|^^, which is not cascading in nature. Instead, LSB removal is achieved in one stepAtty. Docket No.4502-85700 (6000720PCT02) and without using sign detection or even / order detection. The present disclosure also presents three different embodiments for calculating the 2’s power residue |^|^^.

[0057] In the present disclosure, let the RNS used in the dot product operation be defined by^ pairwise relatively co-prime moduli ^^ , ! = 1,2, … , ^. This RNS can represent any variable^ ∈ %0, ^' , where ^ = ^^^^ … , ^( , called therange of the RNS. The RNSrepresentation of a variable in this RNS is %^( , ^(^^, … , ^^] , where ^^, ! = 1,2, … , ^, is theresidue of ^ with respect to the j-th co-prime modulus ^^, where ^ is thebinary or decimal value. ^^is defined as follows. |^|^ , ,- ^ ≥ 0

[0058] ^^ = * +^ Equation (1)^ −

[0059] As referenced herein, the Division Remainder Zero Theorem is defined as follows:

[0060] Given the RNS representation %^( , … , ^^, ^^] of ^ with moduli %^( , … , ^^, ^^] ,and 6 a divider of ^, then

[0061] |^ / 6|8^9 = ^^(|6^^|^:^^: , ^^(^^|6^^|^:;<^^:;< , … , ^^^|6^^|^<^^< Equation (2)

[0062] divisor (GCD)( 6, ^^' = 1 for ! ∈ {1,2, .. , ^} , meaning 6 is a co-prime with any of the moduli ^^ , ! =1,2, … , ^ , where |6^^|^+ is the multiplicative inverse of 6 with respect to ^^ , which is aninteger less than^6|6^^|^+^^ = 1.

[0063] Asin convolutional neural networks will exponentially increase of dynamic range (i.e., the total range of integers that can be uniquely represented using the given set of moduli) at each layer of the convolutional neural network and will quickly erode the benefits of RNS. For continued efficient operation, at the end of each layer’s dot product operation, either the RNS base is extended to expand the dynamic range or the LSBs are removed (i.e., rescaling) to maintain the dynamic range.Atty. Docket No.4502-85700 (6000720PCT02)

[0064] The purpose of base extension is to extend the RNS base by introducing an additionalco-prime ^(@^, thus expanding the dynamic range to ^ = ^^^^ … , ^(^(@^, where ^(@^ is in thesame order of ^^, ^^, … , ^( . Thus ^(@^ is normally less than 100 for most engineeringapplications. The RNS representation of ^ in the extended RNS will be with one additional digit ^(@^, defined as follows: |^|^:A , ,- ^ ≥ 0

[0065] ^(@^ = * <^ − , 01ℎ345,63 Equation (3)

[0066] because it is added without disturbing orchanging the existing digits %^( , ^(^^, … , ^^], ^(@^.

[0067] For LSBof the present disclosure do not expand the dynamic range for the RNS. Instead, the disclosed embodiments scale down the dynamic range of variables in the RNS by using the 2’s power residue |^|^^, so that the next multiplication operation will remain in-range. The 2’s power residue|^|^^is defined as follows.

[0068] |^| | | ^^^ = B ^ ^ , ,- ^ ≥ 0− ^ Equation (4)^

[0069] the convolutional neural network 100 in FIG. 2.The left of Equation (4) is the named representation of the 2’s power residue. The right of Equation (4) is the definition of the 2’s power residue. The 2’s power residue|^|^^is very similar to base extension ^(@^with respect to an additional modulus ^(@^that is pairwiserelatively co-prime with ^^ , ! = 1,2, … , ^. The difference is that the added co-prime ^(@^ forbase extension is in the same order of magnitude as the moduli ^^, ! = 1,2, … , ^, which is <100,while the 2^for LSB removal is significantly larger. In the given example, m=16, then 2^= 65536, which is a few orders larger than the modulus ^(@^for base extension.

[0070] In the binary number system, there are three types of LSB removal, including flooring, rounding, and ceiling. Flooring is rounding to the nearest integer towards −∞, denoted as⌊∙⌋. Rounding is rounding to the nearest integer, normally denoted as ^,^1(∙'. Ceiling is rounding to the nearest integer towards +∞, denoted as⌈∙⌉. In the binary system, the operations of⌊^⌋, ^,^1(^' , and⌈^⌉are easy to achieve. However, in RNS, they are fundamentally difficultAtty. Docket No.4502-85700 (6000720PCT02) because the RNS representation of a variable ^ does not provide magnitude (i.e., does not indicate how big x is because different numbers may have the same modulus value (remainder) and sign information (because a negative number may provide the same modulus value as a positive number).

[0071] The disclosed embodiments include solutions for flooring in the RNS by using the 2’spower residue (N O P), flooring in the RNS by using the O^^ 2’s power residue (^,^1(^^'', and roundingin the RNS by using the 2’s power residue (Q O^^R).

[0072] In the disclosed embodiments, for ^ ≥ 0, the binary representation of x is as follows.

[0073] ^ = S 2(@^ + (@^^^ ^ ^(@^ S(@^^^2 + ⋯ + S^2 + ⋯ + S^2 + SU Equation (5)

[0074] where S^ = 0 / 1; ! = 0, … , ^ + ^.^ with respect to the modulus 2^is as follows.

[0076] |^|^^ = S^^^2^^^ + ⋯ + S^2^ + SU Equation (6)

[0077] and

[0078] ^ − |^| ^ = S 2(@^ + S (@^^^ ^^ (@^ (@^^^2 + ⋯ + S^2 Equation (7)

[0079] Zero Theorem canapplied to ^ − |^|^^ , and

[0080] ^ = O^|O|W^= ( (^^ U^^ S(@^2 + S(@^^^2 + ⋯ + S^2 Equation (8)

[0081] the flooring operation ⌊^ / 2^⌋ , or truncating ^ least significant bits in binary system. The division on the left side of Equation (8)can be achieved with multiplicative inverse of 2^ with respect to ^^ , ! = ^, ^ − 1, … ,1.

[0082] FIG. 3 is a flowchart illustrating a process 300 for performing ain RNS with the 2’s power residue|^|^^in accordance with an embodiment of the present disclosure. In the depicted embodiment, at step 302, the process 300 is configured to determine2’s power residue|^| ^ , given the RNS representation %^ , ^ , … , ^ ] o ^^ ( (^^ ^ f ^ , where 2 ispairwise relatively co-prime with the RNS’s moduli ^^ , ! = 1,2, … , ^ . |^|^^ is an integerAtty. Docket No.4502-85700 (6000720PCT02)between 0~2^ − 1. Solutions to obtaining the |^|^^ from the RNS representation%^(, ^(^^, … , ^^] are described further below.

[0083] At step 304, the process 300 is configured to encode the 2’s power residue|^|^^to theRNS base ^^, ! = 1,2, … , ^ as follows:

[0084] ^^ = %^^ ^ ^(, ^(^^ , … , ^^] = Y||^|^^|^: , ||^|^^|^:;< , … , ||^|^^|^<Z Equation (9)

[0085] residue|^|^^from ^ in RNS as follows:

[0086] ^[ = ^ − |^|^^ = %^[ [ [ ^ ^( , ^(^^ , … , ^^ ] = Y|^( − ^(|^: , |^(^^ − ^(^^ |^:;<, … , |^^ −(10)

[0087] At step 308, the process 300 is configured to multiple ^[by 1 / 2^in RNS with themultiplicative inverse ^^_](^^ = ^ ^^^^^ , ! = 1,2, … , ^, yielding the result of ⌊^ / 2^⌋ as follows:+

[0088] ^ = N O^^P = _^^[( ^^_](^( ^^: , ^^[(^^ ^^_](^(^^ ^^:;< , … , ^^[^ ^^_](^^ ^^<` Equation (11)

[0089] calculated.

[0090] FIG. 4 is a flow diagram illustrating a flooring operation 400 in RNS with 2’s power residue |^|^^as described in FIG.3 in accordance with an embodiment of the present disclosure. In particular, the flooring operation 400 shows that a binary equivalent truncation uses a modulo subtraction at step 402 and a modulo multiplication at step 404.

[0091] FIG. 5 is a flow diagram illustrating a rounding process 500 in RNS after flooring in accordance with an embodiment of the present disclosure. Similar to the binary system, roundingto nearest integer ^,^1( O^^' in RNS is achieved by adding 1 or 0 to ^ = N O^^P, depending onthe most significant bit S (i.e., the leftmost / highest value bit)power|^| ^ = S 2^^^ + ⋯ + S 2^ + O^ ^^^ ^ SU. ^^ results in the binary point being shifted to the left bythe fractional part. S^^^is the most significant bit of the fractional part and indicates whether that fraction is less than or greater than 0.5. If S^^^Atty. Docket No.4502-85700 (6000720PCT02) is 0, then the fraction is < 0.5. If S^^^is 1, then the fraction is ≥0.5. Thus, S^^^can be used to perform rounding as shown in

[0092] As shown in FIG. 5, at step 502, in an embodiment, the rounding process 500 determines whether S^^^equals 1. If so, the rounding process 500 rounds up at step 504. If not, the rounding 500 rounds down at step 506. For example, given x=27 and m=3, then2m=8. N O^^P is N^ab P =3. The binary representation of 3 (3 bits) is 011, so b2=0, b1=1, b0=1.Thus, b step 502, so the rounding process 500 rounds down at step 506. Theaddition of 1 to ^ is simple modulo addition of 1 to every channel of ^ in RNS. Thus, the rounding in RNS can be described as follows.

[0093] c^ = ^^^ + S^^^^^+ , ! = 1, … , ^ Equation (12)

[0094] a ceiling process 600 in RNS after flooring in accordance with an embodiment of the present disclosure. As stated above, ceiling is rounding to the nearest integer towards +∞ . As shown in FIG. 6, the ceiling process 600 includesdetermining, at step 602, whether the least significant bits of x (specifically, whether QO ^^R indicatea nonzero remainder. If at step 602, the least significant bits of x are all zero, the division is exact, and no rounding is needed. Thus, if no rounding is needed, at step 604, the ceiling process 600outputs N O^^P unchanged. Otherwise, at step 606, the ceiling process 600 rounds up. In anfor the ceiling operation, rounding up can be achieved by adding 1 to N O^^P. Thus,the ceiling operation can be represented as follows.1+ N O^^P , ,- |^|^^ ≠ 0

[0095] Equation (13)

[0096] both rounding and ceiling in RNS after the truncation or flooring has less latency than similar process performed in binary, because there is no carry across the RNS channels.

[0097] As shown in the following examples, the operation of flooring, rounding, and ceilingwith the 2’s power residue |^|^^ that are identical for both ^ < 0 and ^ ≥ 0. Thus, theAtty. Docket No.4502-85700 (6000720PCT02) disclosed embodiments do not use sign detection (i.e., determining whether x is negative orpositive). Given, an RNS with modulus set g = %^^, ! = 1,2, … ,9] =%3, 5, 7, 11, 13, 19, 23, 29, 37] , then the dynamic range of the RNS is ^ = ^^^^ … ^j =7040548515. Assume ^ = 3. Thus, 2^ = 8. Described herein are 6 examples where^ = +10, −10, +9, −9, +8, −8 to explain the flooring, rounding, and ceiling operations. These6 examples are selected to provide a detailed comparison of the flooring, rounding and ceiling, and between positive ^ and negative ^, are not intended to limit the present disclosure. Table 1 provides the intermediate values in the calculation of flooring, rounding and ceiling for the 6 examples. Flooring, Rounding, and Ceiling with Base extension, Example of x = + / -10, + / -9, + / -8 RNS (P = 37, 29, 23, 19, 13, 11, 7 ,5, 3), Base = 2^3 ng al)

[0098] In Table 1, column 1 contains the 6 example values in binary (i.e., ^= +10, -10, +9, - 9, +8, -8). In column 2, RNS-Integer represents the integer before RNS modulo encoding. In an embodiment, a negative number x is first converted to M-|x|, where M is the dynamic range of the RNS. Column 3 indicates the RNS encoded results of each of the x values based on the modulus set g. Column 4 contains the 2’s power residue (i.e., the residue of ^ with respect to 2mor |^|^n) in binary representation and decimal equivalent (value in parenthesis in column 4). The|^|^ncan be derived from various algorithms such as, but not limited to, one of the three algorithms further described below. In an embodiment, the binary representation of|^|^nisused to determine whether 1 is added to flooring N O^^P to achieve rounding ^,^1 o O^^p and ceilingQ O R as described herein. Column 5floori N O P^^ ng value ^^ representation.in accordance with the disclosed embodiments, in actual implementation the flooringAtty. Docket No.4502-85700 (6000720PCT02)value N O^^P, rounding, and ceiling operations are calculated in RNS. The decimal equivalent inTable decoded from the RNS representation for illustration purposes only. Column 6indicates the rounding value in decimal representation. In an embodiment, ^,^, o O^np iscalculated in RNS, as shown in FIG. 5, by using the result of flooring N O^^P in column 5 and thebinary representation of |^| n in column 4 for both ^ ≥ 0 and ^ Column 7 indicatesceiling value Q O^^R in decimal representation. In an embodiment, the Q O^nR calculated in RNS,as shown in by using the result of flooring NOP in column 5binaryof |^| n in column 4 for both ^ ≥ 0 and0 . In accordance with theembodiments, the operations of flooring, rounding, and ceiling in RNS for both x≥0 and x<0 are identical. Thus, no knowledge of the sign of x is needed and no additional processing is needed for x<0 to perform these operations. This provides an efficiency advantage. Additionally, the latency of the rounding and ceiling operation with 2’s power residue in the disclosed embodiments is lower than the latency in regular binary system because adding 1 in RNS is performed in parallel RNS channels as shown in FIG.1 and there is no carry-over across the RNS channels.

[0099] As described above, the disclosed embodiments scale down the dynamic range of variables in the RNS by using the 2’s power residue|^|^^. In the binary or decimal system, calculating the 2’s power residue |^|^^is trivial using bitwise operations. But this is not the case in RNS because a number x is not stored or processed as a single integer. Instead, as described in FIG.1, a number is represented as a set of residues modulo. For example, in FIG.1, with the given set of moduli [47, 37, 29, 23, 19, 13, 11, 7, 5, 3], x=50 is represented as [3, 13, 21, 4, 12, 11, 6, 1, 0, 2] in RNS. Thus, calculating the 2’s power residue|^|^^is not trivial because RNS does not store the original x value, but instead just stores the set of residues modulo. Thus, the disclosed embodiments describe three algorithms to calculate the 2’s power residue|^|^^in RNS. The present disclosure is not limited by the described embodiments. Other algorithms for calculating the 2’s power residue|^|^^in RNS may be applied.Atty. Docket No.4502-85700 (6000720PCT02)

[0100] In a first algorithm, 2^is used as a companion modulus with the original RNS modulus set ^(, ^(^^,…, ^^throughout the RNS modulo operation journey (i.e., use 2^as anextra modulus). This embodiment produces both the RNS result %^( , ^(^^,…, ^^] based on theoriginal RNS modulus set ^(, ^(^^, …, ^^as well as the 2’s power residue|^|^^based on the 2^as an extra / companion modulus. This is the simplest and most direct method to calculate|^|^^. Relative to the original, RNS modulus set ^(, ^(^^,…, ^^, 2^is several orders larger than ^(, ^(^^,…, ^^for practical applications. The modulo operations with respect to 2^is a 2’s power modulo operation, simply retaining the m least significant bits, in which no calculation is needed for any bits beyond the m least significant bits. However, the latency of the modulo operation with respect to 2^is still significantly larger that the modulo operations with respectto ^( , ^(^^,…, ^^, thus forming a latency bottleneck over the course of producing %^(, ^(^^,…,^^].

[0101] In a second algorithm, to calculate the 2’s power residue|^|^^, a weight-based RNS decoder is used to first recover ^ in binary, which is then used to calculate the 2’s power residue|^|^^in binary. In an embodiment, the weight based RNS decoding can be described as follows.

[0102] Assume %^(, ^(^^,…, ^^] is the RNS representation of a variable ^, and the RNSmoduli set is %^(, ^(^^,…, ^^]. The weight-based RNS decoding is used to recover the RNS integer of the signed integer ^ . In an embodiment, the RNS integer is related to the signed integer ^ as follows.

[0103] ^8^9_~(^ = B ^, ,- ^ ≥ 0^ − |^| 01ℎ345,63 Equation (14)

[0104] The RNS integer ^8^9_~(^can be decoded from the RNS representation %^^, ^^,…,^(] as follows.

[0105] ^8^9_~(^ = ^∑(^^^ ^^^5^^^ ^^ Equation (15)

[0106] precalculated weights stored in registers. The weights5^ are calculated by 5^ = ^^^^ , where ^^ = M / ^^ with M = ^( ^(^^ … ^^ , and ^^ =Atty. Docket No.4502-85700 (6000720PCT02)^^^^^ ^^+ , which is the multiplicative inverse of ^^ with respect to ^^, or ^^ ∈ %1, ^^) such that^^^^^^^+ = 1. Therefore, 5^ < ^ and 5^ is a big number.Using the definition of |^|^^in Equation (4) and the definition of ^8^9_~(^inEquation (14), results in the following. 8^9_~0108] ^ |^ (^| ^ 8^9_~(^[ | |^^ = B ^ , ,- ^ < ^ / 22^ − |^ − ^8^9^:^| Equation (16)^^ . 01ℎ345,63

[0109] is simply taking the m least significantbits.

[0110] In the second algorithm, although there is no extra operation over the course ofproducing %^( , ^(^^,…, ^^], the second algorithm uses operations after producing %^( , ^(^^,…,^^], and thus incurring additional latency. However, the latency caused in the first algorithm is afunction of the algorithm producing the %^(, ^(^^,…, ^^], while the latency caused in secondalgorithm is independent the algorithm.

[0111] Alternatively, in another embodiment, a third algorithm is disclosed that is based on an efficient base extension algorithm. In an embodiment, the third algorithm uses a redundantmodulus ^^ to generate a redundant residue ^^ , then ^^ and %^(, ^(^^ ,…, ^^] are used togenerate the silent digit ^^, the extended residue with respect^^to be extended (referred to as an extended modulus). A redundant modulus ^^is a modulus that is used during base extension, but is not part of the RNS representation (i.e., not needed to represent a number in RNS.) The redundant modulus ^^helps compute the residue for new moduli without reconstructing the original number. Whereas an extended modulus ^^is used to expand the original RNS base and is part of the new RNS base that is used to represent a number in RNS.

[0112] However, the third algorithm is not a theoretically derived solution and may not work for all pairs of ^^and ^^. In an embodiment, simulation is used to identify the smallest ^^for a given ^^. In an embodiment, the third algorithm includes both off-line (setup or front) calculation and on-line calculation as described below.

[0113] Front calculationAtty. Docket No.4502-85700 (6000720PCT02)

[0114] Given the original moduli ^^ , ! = 1,2, … , ^ , the redundant modulus ^^ and themodulus to be extended, ^^, calculate the following in front.

[0115] Given dynamic range ^ = ^(^(^^ … ^^ of the original RNS, the multiplicativeinverse ^^^^^of ^ with respect to the redundant modulus ^^is defined as:

[0116] ^^^^^^ ^^ ^^ = 1 Equation (17)

[0117] where ^^^^^ < ^^.

[0118] Residue ^^^of M with respect to the extended modulus ^^is defined as:

[0119] ^^^ = |^|^^ Equation (18)

[0120] Given ^^ = ^^+ , ! = 1,2, .. , ^ , calculate the multiplicative inverse ^^^^ of ^^ withrespect to ^, ! = 1,2, .. , ^, which is defined by:

[0121] ^^^^^ ^^^^ = 1, ! = 1,2, … , ^ Equation (19)

[0122] 1,2, … , ^, defined as follows:

[0123] ^^^^ , ! = 1,2, … , ^ Equation (20)^^

[0124] … , ^ Equation (21)

[0125]

[0126] Given ^^ and %^( , ^(^^,…, ^^], the calculation of the base extension ^^^ = |^|^^can be summarized in FIG. 7A, which uses the stored values calculated in the front calculation,^^̂^ , ! = 1,2, … , ^, ^^̂^ , ! = 1 ^^^ ^ ,2, … , ^, ^^^ and ^^^.

[0127] module 700 configured to perform efficient base extension in RNS in accordance with an embodiment of the present disclosure. The calculation module 700 is configured to convert a number represented in one set of moduli (base) into a new set of moduli (new base) without reconstructing the original number.Atty. Docket No.4502-85700 (6000720PCT02)

[0128] Table 2 provides the definition for each of the symbols of the calculation module 700 in FIG.7A. Definition Symbol Conditions Defining Equation Variable TypeAtty. Docket No.4502-85700 (6000720PCT02) Cofactor of Mjj=1,…,n^ ^^ =Precalculated ^Atty. Docket No.4502-85700 (6000720PCT02) Constants ^^̂^4j=1,…,n ^^^̂^ = ^^^^^^^^^^^^ ^^^^^^ Precalculatedt

[0129] As shown in Table 2, ^^̂^, ^^̂^ , ^^^, ^ are pre ^^^ ^ ^^ ^^ -calculated parameters. ^^̂ is aprecomputed constant used to extend the base to include to the redundant modulus ^^. ^^^̂^is a precomputed constant used in the base extension process to extend a number’s representation from its current base to include the new modulus ^^, which is used to capture the 2’s power residue. In the context of RNS and as referenced in Table 2, a cofactor of a modulus refers to the product of all the other moduli except / excluding that modulus. For example, as shown in Table 2, Mj isAtty. Docket No.4502-85700 (6000720PCT02)the cofactor of modulus pj. This means ^^ = ^^+ , where M is the total dynamic range of the RNSmoduli.

[0130] As shown in FIG.7A, the module 700 takes as input xj, which is the number to be scaled in RNS format. At block 702, the calculation module 700 calculates the residue sum with respect to pr(^^^^) based on xjand ^^^̂^. In an embodiment, the block 702 can be implemented using ten 6-bit x 6-bit multipliersinput 6-bit adder as labeled in FIG.7A.

[0131] At block 704, the calculation module 700 calculates (e.g., using one 6-bit x 6-bit adder) the residue difference with respect to pr (Temppr) based on ^^^^and the residue of input number to be scaled with respect to pr (xpr). The calculation module 700 calculates the multiplicative factor (k) (e.g., using one 6-bit x 6-bit multiplier) with respect to prbased on the residue difference with respect to pr (Temp) and the multiplicative inverse of M with respect to the redundant modulus Mpr.

[0132] Similarly, at block 706, the calculation module 700 calculates (e.g., using a set of multipliers and one adder as depicted in FIG.7A) the sum with respect to pe (^^^^) based on xj and ^^^̂^. At block 708, the calculation module 700 calculates (e.g., using one 6-bit x 16-bit multiplier) the residue difference with respect to pe(Temppe) based on the residue of M with respect to pe(^^^) and the multiplicative factor with respect to pr(k). The calculation module 700 calculates (e.g., using a 16-bit adder) and outputs the residue of input number to be scaled x with respect to pe (xpe) based on the residue sum with respect to pe(^^^^) and the residue difference with respect to pe(Temppe).

[0133] FIG.7B is a schematic diagram illustrating a scaling engine 800 configured to perform efficient base extension in RNS in accordance with an embodiment of the present disclosure. For example, the scaling engine 800 may be used to implement the calculation module 700 in FIG.7. In the depicted embodiment, the scaling engine 800 uses the following data and produces the following results:Atty. Docket No.4502-85700 (6000720PCT02)[ibed above, the scaling engine 800 is configured to perform a RNS base extension operation for recovering a binary value from its RNS representation using both a standard residue set, a redundant modulus pr=64, and an extension modulus pe=216=65536. The values at the top (x1 through x10) represent the input residues of a number x in the moduli set {3,5,7,11,13,17,19,23,29,47}.

[0135] The left half of the scaling engine 800 (corresponds to first / top branch in FIG. 7) is focused on computation under the auxiliary modulus pr=64. Each input residue xiis multiplied by a precomputed constant Mi-1mod prusing a modulo multiplier corresponding to its modulus. The results of these multiplications (shown as output) are then summed by a 10:1 adder to compute the residue sum with respect to pr (^^^^= 2). A Temppr Subtractor is then configured to calculateAtty. Docket No.4502-85700 (6000720PCT02) Temppr=|^^^^-xpr|pr =7. A k multiplier calculates k=| ^^^^^Temp|pr=29. A Temppe Multiplier calculates Temppe=|kMpe|pe=32217, which is used in an XpeSubtractor described below.

[0136] On the right side of the scaling engine 800 (corresponds to second / bottom branch in FIG. 7), a similar operation is performed using extension modulus pe=216=65536. Each input residue is multiplied by its corresponding Mi-1mod pe, and then summed by a 10:1 adder to compute ^^^^, which equals 33217. The XpeSubtractor calculates xpe=|^^^^- Temp|pe=1000, which is the value x scaled by 2m.

[0137] Thus, the scaling engine 800 computes the residue of a number x under a new modulus pe, where pe=216(a 2’s power modulus), without reconstructing x. The 2’s power modulus makes hardware implementation easy (e.g., using a bitwise AND) because no division is needed. Instead, because the 2’s powers align exactly with binary positions,|^|^^can determined quickly by simply keeping the last m bits of the binary value x and ignoring the rest (i.e., shifting right by m bits is the same as dividing by 2m). Thus, the disclosed embodiments can be efficiently implemented in hardware and is ideal for neural networks or other applications.

[0138] As stated above, in some embodiments, the base extension algorithm may not be applicable for all redundant modulus pair ^^and ^^. In an embodiment, simulation is used to analyze the behavior of the base extension algorithm and find the smallest non-failing ^^for a needed ^^. In an embodiment, the simulation calculates the base extension|^|^^as implemented in FIG. 7A and performs direct calculation of the base extension|^|^^using Matlab). A failure case is counted if the calculations are inconsistent. ^^^]^^^^^^^^is the totalnumber of failure cases for all ^ ∈ %0, ^' for a given set of (^^ , ^^'. The failure rate ^(^^,^^'is defined by

[0139] ^ ^^^^^^^^(^^,^^' = ^^^^^ Equation (22)

[0140] simulation uses a first moduli set ^^ = %29,23,19] with amodulus ^^ = 13 to be extended. In a second example, the simulation uses a second moduli set^^ = %67,51,59] with a modulus ^^ = 13 to be extended. The simulation searches for theAtty. Docket No.4502-85700 (6000720PCT02)smallest valid redundant modulus ^^ that gives no failure case for all values of ^ ∈ %0, ^'. Avalid redundant modulus ^^should be pairwise relatively co-prime with the RNS’s moduli set. A valid modulus ^^to be extended should be also pairwise relatively co-prime with the RNS’s moduli set. However, ^^and ^^do not need to be co-prime.

[0141] FIGS.8A and 8B illustrate the failure rate of the first example (FIG.8A) and the second example (FIG. 8B) vs. the valid redundant modulus ^^over the search range 0~99. The blue dots in the figures are to highlight the ^^that gives absolute zero-failure rate points (no failed cases). In an embodiment, the base extension algorithm is configured to determine the smallest value that provides a zero-failure rate.

[0142] As shown in FIGS. 8A and 8B, when ^^ = ^^^, ^ = 1,2, …, the algorithm providesthe correct base extension. Other than these special points, the smallest the ^^is always largerthan ^^ = 13. This means that the calculation of the ^^ using the base extension algorithm isalways more expensive that the direct calculation of ^^^. However, the goal is not base extension. Instead, the disclosed embodiments are looking to calculate the residue with respectto 2^, which is significantly larger than n ^^ = 13.

[0143] For instance, for m=8, FIGS. 9A and 9B illustrate the failure rate with ^^ for fixed^^ = 2b = 128 for the two cases ^^ = %29,23,19] and ^^ = %67,61,59] . As depicted inFIG.9A, for a fixed ^^ = 128,^^ = 45 for ^^ = %29,23,19]. As shown in FIG.9B, for a fixed ^^ = 128 , the smallest ^^ = 60 for= %67,61,59] . FIGS. 9A and 9Bindicate that in both cases, the smallest ^^ is smaller than ^^ = 128. Therefore, the disclosedbase extension algorithm can be an efficient solution to 2’s power residue|^|^^, even if not an efficient solution for base extension.

[0144] For example, FIGS. 10A and 10B illustrate the failure rate curves for ^ b^ = 2 (FIG.10A) and ^^ = 2^^(FIG. 10B) when applying the base extension algorithm to a practical exampleRNS system having moduli set (37, 29, 23, 19, 13, 11, 7, 5, 3) and M = 7040548515 ~ 1.4x2m^. As shown in FIGS.10A and 10B, the smallest ^^= 64 for both ^^= 2b, and ^^= 2^^. ^^=64 is a good choice in that it is the same number of bits as the largest modulus 37 of the originalAtty. Docket No.4502-85700 (6000720PCT02) RNS, and it is also a power-of-2. Therefore, the calculation of ^^will not introduce latencyover the course of producing %^( , ^(^^ ,…, ^^] . Additionally, after obtaining ^^ and%^(, ^(^^,…, ^^], using ^^ = 2^^ and ^^ = 2^ = 64, the modulo operations in FIG. 7A are allexact powers-of-2 and thus can be calculated with efficiently. Further, the modulo operations of ^^and ^^in FIG.7A can be performed in parallel. Therefore, the latency of the base extensionalgorithm is determined by ^^ since ^^ > ^^. Thus, in some embodiments, the base extensionalgorithm is more efficient than the second algorithm.

[0145] FIG. 11 is a block diagram illustrating LSB removal in multi-layer neural network in accordance with an embodiment of the present disclosure. In particular, FIG.11 illustrates how LSB removal can be done entirely within the RNS without converting back to binary, which significantly improves performance and hardware efficiency. In the depicted embodiment, in a first branch, binary inputs ^1, ^2,…, ^Nare encoded into RNS format at RNS encoding block 1102 using a set of relatively prime moduli ^(, ^(^1,…, ^1(e.g., ^(= 47 (47 is the largest prime modulus channel (modulo 47) used to reach 232dynamic range). Two integers are relatively prime (also called coprime) if their GCD is 1. The two integers do not both need to be prime numbers so long as the two integers do not have any prime factor in common. The RNS encoding allows each ^^^value to be processed in parallel across multiple RNS channels for each (i.e., one) layer of the neural network at RNS algorithms (n-channel) block 1104. RNS algorithms (n- channel) block 1104 performs RNS-based computations such as dot products or weighted sums in parallel across multiple RNS channels. The results ^^of these operations are then passed from block 1104 to an online calculation block 1112 and to an LSB removal in RNS with⌊^ / 2^⌋block 1106.

[0146] In a second branch, the binary inputs ^1, ^2,…, ^Nare encoded at RNS encodingblock 1108 using a single modulus pr (e.g., pr = 64 = 26 (6 bits & 2’s power modulo channel)).The encoded values ^^^are passed to a RNS algorithms (1-channel pr) block 1110, which is configured to perform similar RNS-based computations (performed in block 1104) using pr (i.e., only using one power-of-2 modulus). As described above, this operation can be performedAtty. Docket No.4502-85700 (6000720PCT02) efficiently due to shift-and-mask based modulo arithmetic. The result ^^of block 1110 is passed to the online calculation block 1112.

[0147] The online calculation block 1112 takes as input ^^calculated at block 1104 and ^^calculated at block 1110, and is configured to extract the LSB part of the result using modulo 2m(|^|^^' (i.e., keep lowest m bits). In an embodiment, the online calculation block 1112 is implemented using the scaling engine 800 of FIG. 8. The result|^|^^is passed to a RNS Encoding block 1114. RNS Encoding block 1114 is configured to encode the result back into multi-channel RNS representation using moduli ^(, ^(^1,…, ^1to produce computed value 6^, which is passed to the LSB removal in RNS withblock 1106.

[0148] The LSB removal in RNS with⌊^ / 2^⌋block 1106 is configured to perform rescaling in accordance with the disclosed embodiments. As described above, instead of converting the values back to binary to drop precision (rescale), block 1106 performs flooring in the RNS byusing the 2’s power residue (N O^^P ) to rescale directly in RNS. An example of the flooringoperation is described in FIG.3. For each channel, block 1106 is configured to shift ^^right by m bits (logical division by 2m) directly in RNS domain using the computed value 6^. This operation effectively reduces the bit-width of the values (removing lower bits) while in the RNS format, which maintains efficiency and enables further computation. In some embodiments, the block 1106 may also be configured to perform rounding (Equation (12)) or ceiling (Equation (13)) operations after flooring.

[0149] The block 1106 outputs ^^after rescaling in RNS representation, which provides enough dynamic range for further RNS operations (e.g., processing in a next layer of the neural network). The disclosed embodiments are significantly more efficient in terms of power, delay, and area than the process of decoding back to binary, LSB removal in binary, then encoding to RNS.

[0150] FIG.12 is a flowchart illustrating a scaling process 1200 in accordance with a disclosed embodiment. The scaling process 1200 is configured to convert a number in RNS form into a scaled binary value (scaled by a power of 2). The scaling process 1200 begins at step 1202 byAtty. Docket No.4502-85700 (6000720PCT02) selecting a desired dynamic range Md. The dynamic range is the total range of integers that can be uniquely represented using a given set of moduli. The dynamic range defines how large the final or intermediate number (after scaling) should be.

[0151] At step 1204, the scaling process 1200 selects n prime moduli pj to achieve ^ =^(^^ >= Md. As defined in Table 2, each ^^ is an RNS modulus used in the RNS. Thus,^^^this step is selecting a set of moduli where the product of all chosen moduli is greater than the dynamic range to ensure that all numbers within the dynamic range can be represented.

[0152] The scaling process 1200 at step 1206, selects power of 2 scaling factor m (scaling factor will therefore be 2m). For example, if m=16, the scaling factor is 216. This sets the extended modulus pe=2m. As described above, a scaling factor is used to shift or rescale values during base extension in RNS. By choosing the scaling factor to be a power of 2, the disclosed embodiments can perform scaling directly in RNS. To provide the RNS equivalent operation of right-shifting a value in binary. As described above, to extend the RNS representation of a number from one set of moduli to another, requires calculating the residue of the number with respect to a new modulus (e.g., pe) without fully reconstructing the number in binary. To do this efficiently, the redundant modulus pr is used to compute intermediate values (like weighted sums or scaled versions of the number), which then can be used to derive the residues in the extended base.

[0153] The scaling process 1200 at step 1208, selects the redundant modulus pr. pr must be large enough so that no failure cases occur. As described above, to extend the RNS representation of a number from one set of moduli to another, the residue of the number is calculated with respect to a new modulus (e.g., pe) without fully reconstructing the number in binary. To do this efficiently, the redundant modulus pr is used to compute intermediate values (like weighted sums or scaled versions of the number), which then can be used to derive the residues in the extended base.

[0154] At step 1210, the scaling process 1200 pre-calculates the off-line parameters and store as hardware constants or in registers (refer to Table 2 for list of pre-calculated parameters).Atty. Docket No.4502-85700 (6000720PCT02)

[0155] The scaling process 1200 at step 1212, issues primary inputs (online inputs) and xjto a (pipelined or non-pipelined) hardware scaling engine (e.g., FIG. 7B). Output of the hardware scaling engine will be xpe =|^|^^, which is the binary value of x scaled by 2m. In some embodiments, this binary value of xpeis then converted to equivalent RNS values.

[0156] FIG.13A is a flowchart illustrating a process 1300A for scaling an RNS number. In FIG. 13A, Xj is an RNS number (i.e., a number represented by its residues modulo) to be scaled (i.e., multiplied by another number). As shown in FIG. 13A, Xjis first converted from RNS format to binary at block 1302. The number in binary format is then truncated / scaled at block 1304, and then converted from binary back to RNS at block 1306 to produce scaled RNS number Xj / 2m.

[0157] FIG. 13B is a flowchart illustrating a process 1300B for scaling an RNS number in accordance with the disclosed embodiments. In contrast to FIG.13A, to scale RNS number Xj, the disclosed embodiments do not have to first convert RNS number Xj to binary. Instead, using LSB removal in RNS with 2’s power residue as described herein, the RNS number Xjcan be scaled in RNS at block 1308. Block 1308 then output the binary value of x scaled by 2m. This binary value is then converted to RNS at block 1310 to produce scaled RNS number Xj / 2m. Thus, the disclosed embodiments eliminate the need to perform RNS to binary conversion before truncation. This vastly improves efficiency when performing scaling such as in neural networks.

[0158] In summary, RNS native operations such as integer multiplication, addition, and subtraction are fast and efficient, but encounter significant challenges for algorithms that involve consecutive multiplications, such as in multi-layer convolutional neural networks. RNS representation of a variable provides no visibility to its magnitude. Because of this fundamental barrier, the LSB removal operations in RNS, such as flooring ⌊∙⌋, round (∙), and ceiling ⌈∙⌉, are tremendously difficult, thus limiting its value for practical use. The present disclosure describes the use of the 2’s power residue|^|^^as a bridge between RNS and binary system. The 2’s power residue|^|^^enables efficient LSB removal in RNS, an essential RNS operation for RNS practical use, without need of sign detection or even / odd detection. However, as described above,Atty. Docket No.4502-85700 (6000720PCT02) in some embodiments, simulation may be used to identify the smallest ^^for a given ^^because the base extension may not work for all pairs of ^^and ^^.

[0159] FIG.14 is a diagram illustrating an apparatus 1400 according to an embodiment of the present disclosure. The apparatus 1400 can be used to implement embodiments of the present disclosure. The apparatus 1400 includes receiver units (RX) 1420 for receiving data via ingress ports 1410. The apparatus 1400 also includes transmitter units (TX) 1440 for transmitting via data egress ports 1450.

[0160] The apparatus 1400 includes a memory 1460 for storing the instructions and various data. The memory 1460 can be any type of, or combination of, memory components capable of storing data and / or instructions. For example, the memory 1460 can include volatile and / or non- volatile memory such as read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM). The memory 1460 can also include one or more disks, tape drives, and solid-state drives. In some embodiments, the memory 1460 can be used as an over-flow data storage device to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. In some embodiments, the memory 1460 can be memory that is integrated with the processor 1430.

[0161] The apparatus 1400 has one or more processors 1430 (e.g., central processing unit (CPU)) to process instructions. The one or more processors 1430 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The one or more processors 1430 are communicatively coupled via a system bus with the ingress ports 1410, RX 1420, TX 1440, egress ports 1450, Input / Output (I / O) 1480, and memory 1460. I / O 1480 provide the communication interfaces for enabling the apparatus 1400 to receive input (e.g., from a keyboard, mouse, or touchscreen) and output information (e.g., to a display or printer).

[0162] The one or more processors 1430 can be configured to execute instructions stored in the memory 1460. For instance, the one or more processors 1430 can be used to perform anyAtty. Docket No.4502-85700 (6000720PCT02) computational, comparison, determination, initiation, configuration, or any other action corresponding to the claims when the appropriate instruction is executed by the processor 1430. As an example, in one embodiment, the memory 1460 stores a LSB removal in RNS with 2’s power residue module 1470. The LSB removal in RNS with 2’s power residue module 1470 includes data, executable instructions, and / or one more sub-modules for implementing the disclosed embodiments. Alternatively, in some embodiments, the LSB removal in RNS with 2’s power residue module 1470 may be implemented in specialized hardware (e.g., a hardware scaling engine comprising adders and multipliers as described in FIG. 7B) as opposed to be stored the memory 1460. Thus, the inclusion of the LSB removal in RNS with 2’s power residue module 1470 substantially improves the functionality of the apparatus 1400.

[0163] While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

[0164] In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.

Claims

Atty. Docket No.4502-85700 (6000720PCT02) CLAIMS What is claimed is:

1. An apparatus comprising: a first set of N multipliers configured to multiply a value xjin each Residue Number System (RNS) channel of an RNS number x by a first precomputed constant ^^^̂^to generate a first set ofresults, wherein ^^^̂^ = ^^^^^^^^ ^^^^ |^^|^^ , where ^^ is amodulus, M is a dynamicrange, ^^^^^^is a first^^with respect to the redundant modulus, anda first multiplicative inverse of M with respect to redundant modulus ^^^; adder configured to add the first se^^^t of results to obtain a first residue sum ^ with respect to the redundant modulus; a second set of N multipliers configured to multiply the value xjby a second precomputedconstant ^^^̂^ to generate a second set of results, wherein ^^^^̂ = ^^^^^^^^ ^^^^ |^^|^^, where ^^ isan extended modulus and is a power of 2 scaling factor,of cofactor of modulus ^^with respect to the extended modulus, and^^^^^^^^is a second multiplicative inverse of M with respect to extended modulus ^^^; a second adder configured to add the second set of results to obtain a second residue sum ^^^^with respect to the extended modulus; a third adder configured to calculate a first temporary residue with respect to the redundant modulus using the first residue sum ^^^^; a first multiplier configured to calculate a multiplicative factor k with respect to the redundant modulus using the first temporary residue; a second multiplier configured to calculate a second temporary residue with respect to the extended modulus using the multiplicative factor k; and a fourth adder configured to calculate a third residue of the RNS number x with respect to the extended modulus xpeusing the second residue sum and the second temporary residue, whereinAtty. Docket No.4502-85700 (6000720PCT02) the third residue xpeis used to perform least significant bits (LSB) removal in RNS of the value xjto enable scaling operations in a neural network.

2. The apparatus of claim 1, wherein the third adder is configured to calculate the first temporary residue with respect to the redundant modulus by subtracting a fourth residue of the RNS number x with respect to the redundant modulus (xpr) from the first residue sum ^^^^.

3. The apparatus according to any of claims 1-2, wherein the first multiplier is configured to calculate the multiplicative factor k with respect to the redundant modulus by multiplying a multiplicative inverse of M with respect to the redundant modulus and the first temporary residue.

4. The apparatus according to any of claims 1-3, wherein the second multiplier is configured to calculate the second temporary residue with respect to the extended modulus by multiplying the multiplicative factor k and a fifth residue of the dynamic range with respect to the extended modulus (^^^).

5. The apparatus according to any of claims 1-4, wherein the redundant modulus is a power of 2 value.

6. The apparatus according to any of claims 1-5, wherein the redundant modulus is a smallest non-failing modulus value for the extended modulus for all values in the dynamic range based on simulation analysis.

7. An apparatus comprising: a first Residue Number System (RNS) encoder configured to encode binary input values into RNS format using a set of moduli to obtain first RNS encoded values;Atty. Docket No.4502-85700 (6000720PCT02) a multi-channel RNS algorithms component configured to perform RNS-based computations on the first RNS encoded values to obtain first RNS result values; a second RNS encoder configured to encode the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; a single-channel RNS algorithms component configured to perform RNS-based computations on the second RNS encoded values to obtain second RNS result values; a calculations component configured to receive the first RNS result values and the second RNS result values; and obtain a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; a third RNS encoder configured to encode the modulus value into RNS format to obtain third RNS encoded values; and an LBS removal component configured to perform LBS removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.

8. The apparatus according to claim 7, wherein the redundant modulus is greater than a threshold sufficient to prevent failure conditions based on simulations.

9. The apparatus according to any of claims 7-8, wherein the redundant modulus is a power of 2 value.

10. The apparatus according to any of claims 7-9, wherein the LBS removal component is configured to subtract the third RNS encoded values from the first RNS result values in a carry- free manner.Atty. Docket No.4502-85700 (6000720PCT02) 11. The apparatus according to any of claims 7-10, further comprising memory configured to store precomputed constants comprising partial moduli, modular inverses, total modulus, and redundancy parameters.

12. A method for performing scaling operations in a neural network, the method comprising: selecting a dynamic range; selecting a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; selecting a power of 2 scaling factor m; setting an extended modulus to be equal to 2m; selecting a redundant modulus; calculating and storing off-line parameters in memory; obtaining a value of the RNS number scaled by 2musing the off-line parameters and the redundant modulus; and performing LBS removal in RNS on the RNS number using the value to obtain scaled RNS values without binary reconstruction of the RNS number.

13. The method according to claim 12, wherein the redundant modulus is greater than a threshold sufficient to prevent failure conditions.

14. The method according to any of claims 12-13, wherein selecting the redundant modulus comprises ensuring that the redundant modulus prevent the failure conditions based on simulations.

15. The method according to any of claims 12-14, wherein performing scaling of the RNS number is implemented by a pipelined hardware scaling engine.Atty. Docket No.4502-85700 (6000720PCT02) 16. The method according to any of claims 12-15, wherein the redundant modulus is a power of 2 value.

17. The method according to any of claims 12-16, wherein the prime moduli are pairwise relatively prime.

18. The method according to any of claims 12-16, wherein the off-line parameters comprise: residue of the RNS number with respect to the redundant modulus; multiplicative inverse of the dynamic range with respect to the redundant modulus; and residue of the dynamic range with respect to the extended modulus.

19. An apparatus comprising: a memory storing instructions; and a processor in communication with the memory, the processor configured to execute the instructions to cause the apparatus to: select a dynamic range; select a set of prime moduli such that a product of moduli in the set of prime moduli is greater than or equal to the dynamic range; select a power of 2 scaling factor m; set an extended modulus to be equal to 2m; select a redundant modulus; calculate and storing off-line parameters in memory; and obtain a value of the RNS number scaled by 2musing the off-line parameters and the redundant modulus; and perform LBS removal in RNS on the RNS number using the value to obtain scaled RNS values without binary reconstruction of the RNS number.Atty. Docket No.4502-85700 (6000720PCT02) 20. The apparatus according to claim 19, wherein the redundant modulus is greater than a threshold sufficient to prevent failure conditions based on simulations.

21. The apparatus according to any of claims 19-20, wherein the redundant modulus is a power of 2 value.

22. The apparatus according to any of claims 19-21, wherein the prime moduli are pairwise relatively prime.

23. The apparatus according to any of claims 19-22, wherein the off-line parameters comprise: residue of the RNS number with respect to the redundant modulus; multiplicative inverse of the dynamic range with respect to the redundant modulus; and residue of the dynamic range with respect to the extended modulus.

24. A method comprising: encoding binary input values into RNS format using a set of moduli to obtain first RNS encoded values; performing RNS-based computations on the first RNS encoded values to obtain first RNS result values; encoding the binary input values into RNS format using a redundant modulus to obtain second RNS encoded values; performing RNS-based computations on the second RNS encoded values to obtain second RNS result values; obtaining a modulus value for each of the binary input values with respect to a power of 2 scaling factor without reconstructing the binary input values, wherein the modulus value is in binary; encoding the modulus value into RNS format to obtain third RNS encoded values; andAtty. Docket No.4502-85700 (6000720PCT02) performing LBS removal in RNS on the first RNS result values using the third RNS encoded values to obtain scaled first RNS result values without needing binary reconstruction to enable scaling operations in a neural network.

25. The method according to claim 24, wherein the redundant modulus is greater than a threshold sufficient to prevent failure conditions based on simulations.

26. The method according to any of claims 24-25, wherein the redundant modulus is a power of 2 value.

27. The method according to any of claims 24-26, wherein performing LBS removal in RNS comprises subtracting the third RNS encoded values from the first RNS result values in a carry- free manner.

28. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by a processor of an apparatus, cause the apparatus to perform a method according to any of claims 12-18 or 24-27.

Images