Signed multiword multiplier

Abstract

Methods, systems, and apparatus for a hardware circuit configured as a signed multiword multiplier, including a computer program encoded on a computer storage medium. The circuit includes processing circuitry that receives inputs, each input having a corresponding bit width. The processing circuitry is capable of representing at least one input as a signed multiword input based on a first input having a bit width that exceeds a fixed bit width of the hardware circuit. The circuit includes signed multipliers, each of which is configured to multiply the signed inputs. Each signed multiplier includes multiplication circuitry configured to: receive a signed multiword input; receive a signed second input; and generate a signed output in response to multiplying the signed multiword input by the signed second input.

Description

Technical Field

[0001] This specification relates to hardware circuits for performing mathematical calculations. Background Art

[0002] Computational circuits can include multiplication circuits with hardware multipliers for multiplying digital inputs such as integers and floating-point numbers. Acquiring and integrating multiplication circuits into existing computational circuits can be expensive, and the size of some circuits is not suitable for certain applications. For example, some multiplication circuits can include both signed and unsigned multipliers, which consume a considerable area of the circuit die but, despite their large size, offer no advantage in computational throughput. For some computational applications, overly large multiplier circuits can lead to inefficient power consumption and utilization.

[0003] Hardware circuits can be used to implement neural networks. In particular, a neural network with multiple layers can be implemented using computational circuits that include several hardware multipliers. The computational circuits of the hardware circuits can also represent the computational units used to perform neural network computations at a given layer. For example, given an input, the circuit can use the neural network to compute an inference of the input by performing a dot product operation using one or more multipliers in the computational units of the hardware circuits. Summary of the Invention

[0004] This document describes a dedicated hardware circuit for multiplying inputs. The hardware circuit includes processing circuitry that receives inputs, each input having a corresponding bit width. The processing circuitry is capable of representing at least one input as a signed multiword input based on a first input having a bit width that exceeds a fixed bit width of the hardware circuit. The hardware circuit is configured as a signed multiword multiplier and includes signed multipliers each configured to multiply the signed inputs. Each signed multiplier includes multiplication circuitry configured to: receive the signed multiword input; receive a signed second input; and generate a signed output in response to multiplying the signed multiword input by the signed second input.

[0005] One aspect of the subject matter described in this specification can be embodied in a hardware circuit for multiplying a set of inputs. The hardware circuit includes: a processing circuit that receives a first input and a second input, each of the first input and the second input having a respective bit width, wherein the processing circuit is configured to represent at least the first input as a signed multi-word input based on the first input having a bit width that exceeds a fixed bit width of the hardware circuit; and a plurality of signed multipliers, each of the plurality of signed multipliers configured to multiply two or more signed inputs, each signed multiplier including a multiplication circuit configured to: receive a signed multi-word input representing the first input; receive a signed second input corresponding to the second input; and generate a signed output in response to multiplying the signed multi-word input by the signed second input.

[0006] These and other implementations can each optionally include one or more of the following features. For example, in some implementations, the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; and N is an integer greater than 1, and B is an integer greater than 1. In some implementations, the value of the shifted signed number is based on a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} In some implementations, the representable range of values of the shifted signed number is based on [-2 (N*B-1) -S,2 (N*B-1) -1-S] defined. In some implementations, S is based on 2 (B-1) *(1+2 B +…+2 {(N-2)B} In some implementations, the processing circuit is configured to represent the first input as a signed multi-word input, the signed multi-word input comprising: a signed high word portion; and a signed low word portion.

[0007] In some implementations, representing the first input as a signed multi-word input includes: modifying a data format of the first input using a quantization scheme based on a fixed bit width of the hardware circuit. In some implementations, the quantization scheme is configured to modify the data format of the first input by generating corresponding word portions to represent the first input as a signed multi-word input; and the total bit width including each corresponding word portion is equal to the fixed bit width of the hardware circuit. In some implementations, the signed multi-word input includes a plurality of corresponding words; and the multiplication circuit is configured to generate a signed output by multiplying each word of the signed multi-word input with each word of the signed second input. In some implementations, the signed second input includes a plurality of corresponding signed words; and the multiplication circuit is configured to generate the signed output as the sum of respective products calculated by multiplying each word of the signed multi-word input with each signed word of the signed second input.

[0008] One aspect of the subject matter described herein can be embodied in a method for multiplying a set of inputs using hardware circuitry. The method includes receiving, by processing circuitry of the hardware circuitry, a first input and a second input, each of the first input and the second input having a respective bit width, wherein at least the first input has a bit width that exceeds a fixed bit width of multiplication hardware included in the hardware circuitry, the multiplication hardware being operable to multiply the first input and the second input; generating, from at least the first input, a signed multi-word input comprising a plurality of signed words, each signed word having a plurality of bits, wherein the bit width of the signed multi-word input is less than the fixed bit width of the multiplication hardware; providing the signed multi-word input and a signed second input to the multiplication hardware for multiplication, wherein the signed second input corresponds to the second input and has a bit width that is within the fixed bit width of the multiplication hardware; and generating a signed output from the multiplication hardware using at least the first input and the second input.

[0009] These and other implementations can each optionally include one or more of the following features. For example, in some implementations, the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; and N is an integer greater than 1, and B is an integer greater than 1. In some implementations, the value of the shifted signed number is based on a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} In some implementations, the representable range of values of the shifted signed number is based on [-2 (N*B-1) -S,2 (N*B-1) -1-S] defined. In some implementations, S is based on 2 (B-1) *(1+2 B +…+2 {(N-2)B}In some implementations, generating the signed multi-word input includes representing the first input as a signed multi-word input, the signed multi-word input including: a signed high word portion; and a signed low word portion.

[0010] In some implementations, representing the first input as a signed multi-word input includes: modifying a data format of the first input using a quantization scheme based on a fixed bit width of the hardware circuit. In some implementations, the method further includes: modifying the data format of the first input by generating corresponding word portions based on the quantization scheme to represent the first input as a signed multi-word input, wherein the total bit width including each corresponding word portion is equal to the fixed bit width of the hardware circuit. In some implementations, the signed second input includes a plurality of corresponding words; and the method further includes: generating a signed output as a sum of corresponding products of each word of the signed multi-word input multiplied by each word of the signed second input using a signed multiplier of the multiplication hardware.

[0011] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs configured to perform the actions of the method encoded on a computer storage device (e.g., a non-transitory machine-readable storage medium). A computing system of one or more computers or hardware circuits can be configured by software, firmware, hardware, or a combination thereof installed on the system, which, in operation, causes the system to perform the actions. One or more computer programs can be configured by having instructions that, when executed by a data processing device, cause the device to perform the actions.

[0012] The subject matter described in this specification can be implemented in specific embodiments to achieve one or more of the following advantages. The described techniques can be used to implement a dedicated hardware circuit for multiplying two or more inputs while requiring less power than conventional circuits for multiplying the inputs. The components of the hardware circuit described in this document form a signed multi-word multiplier circuit having a signed multiplier configured to multiply signed inputs to generate a signed output. The multi-word multiplier can be a low-power hardware multiplication circuit that efficiently multiplies several inputs (e.g., floating-point inputs) based on a unique digital format for representing signed numbers.

[0013] The multiplication circuit can be configured to have multiplication hardware that only includes a signed hardware multiplier for performing input multiplication. The circuit includes a processing circuit for generating a shifted signed multi-word number in response to a processing input having a traditional numbering format such as a binary complement format. The signed multi-word number is multiplied by a signed hardware multiplier to generate a signed output. Compared to traditional circuits that multiply inputs, these characteristics of the multiplication circuit reduce the power consumption of the circuit. This is because the multiplication is performed using only a signed multiplier, rather than using both a signed multiplier and an unsigned multiplier. In addition, the circuit including a hardware multiplier for supporting multiple modes (e.g., signed and unsigned modes) also increases the chip area consumed by the circuit, thereby increasing the manufacturing cost of the circuit. Therefore, the proposed technology not only reduces power consumption, but also reduces manufacturing cost.

[0014] When the multiplication hardware of the circuit is configured to include only signed hardware multipliers, the entire hardware circuit consumes less power than a conventional circuit that must include additional multiplication hardware to support both signed and unsigned computation modes. Thus, this low-power hardware multiplier circuit can be optimized to multiply digital inputs while reducing power consumption requirements based on at least a signed multiplier configuration that affects only a signed mode to generate a product of multiplying two or more signed multi-word inputs.

[0015] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Diagram showing an example dedicated hardware circuit for multiplying inputs.

[0017] Figure 2 A flow chart is shown for generating a signed multi-word input that is provided to a signed hardware multiplier to generate a signed output.

[0018] Figure 3 A flow chart illustrating an example process for multiplying inputs in the described hardware multiplier circuit.

[0019] The same reference numbers and names in different drawings represent the same elements. DETAILED DESCRIPTION

[0020] Conventional computer architectures provide multiplication hardware with a fixed bit width, B. When these architectures need to multiply inputs with a number of bits exceeding the bit width, they break the input numbers into segments ("words"), where each word has a length or bit width, B. To produce the computational output, these architectures multiply each word of the first input with each word of the second input. However, to produce a signed (e.g., positive, negative, or zero) output, the architecture must be configurable in both a signed mode and an unsigned mode (e.g., where the inputs are only positive or zero). Conventional circuits that must be configurable in both signed and unsigned modes require additional hardware components, which results in increased power consumption.

[0021] In an example implementation, hardware circuitry can be used to implement a multi-layer neural network and perform computations (e.g., neural network computations) by processing inputs passing through each layer of the neural network. Specifically, each layer of the neural network can each have a corresponding parameter set. Each layer receives input and processes the input according to the parameter set for that layer to generate an output based on computations performed using multiplication circuitry of an example computation unit. For example, a neural network layer computes multiple products when performing matrix multiplication of an input array and a parameter array or as part of computing a convolution between an input array and a parameter kernel array.

[0022] Generally speaking, processing inputs through the layers of a neural network is accomplished using circuits for performing mathematical operations, such as multiplication and addition. An example hardware circuit can include a hardware multiplier for multiplying two or more inputs. The multiplier circuit can be grouped with a hardware adder to form a computational unit of the hardware circuit, such as a matrix or vector processing unit. The computational unit is used to add and multiply digital inputs, such as integers and floating-point numbers. For example, when the hardware circuit is used to perform neural network calculations, addition and multiplication occur, such as matrix-vector multiplication for processing inputs through the layers of the neural network.

[0023] With the above scenario in mind, this document describes techniques for implementing dedicated hardware circuitry for multiplying two or more inputs represented as signed multiword inputs. These techniques can be used to represent signed or unsigned inputs as "shifted signed multiword numbers." These shifted signed multiword numbers use a unique numeric format to represent the received input as a signed number. The received input can be a single word of a multiword number, or can include both single-word and multiword inputs. By representing the input as a signed number, the dedicated hardware circuitry does not need to support unsigned mode. Therefore, the described hardware circuitry uses a more streamlined architecture that includes multiplication circuitry for signed mode operation rather than for both signed and unsigned mode operation. Because the described hardware circuitry is configured for signed mode operation only, the circuitry requires fewer components, which translates into improved power efficiency when compared to traditional architectures.

[0024] Figure 1 A diagram of an example dedicated hardware circuit 100 for multiplying inputs 102 is shown. In an example implementation, inputs 102A ("input A") and 102B ("input B") are respective floating-point numbers or two's complement numbers that can be represented in software using a binary data structure. The binary data structure can have a specific number of bits, such as a 16-bit, 24-bit, or 32-bit data structure. For example, each of inputs A or B can be a respective signed floating-point number, and the sign bit(s) of each input can indicate the sign of the input (e.g., positive or negative).

[0025] The data structure of each digital input can be associated with a specific data format. The data format can indicate a limited range of values that can be represented using the data format. In some implementations, the 16-bit data structure of input A can include a binary input (e.g., 0010) representing a two's complement data format for input A. Regarding the range of numbers, a common two's complement number can have the following limited representable range of values: [-32, 768, 32, 767]. In addition, each digital input has one or more bits in its data structure indicating whether the number is a signed number or an unsigned number.

[0026] As described herein, a data structure representing a signed numeric input (e.g., an integer) can hold both positive values (e.g., integer values) and negative values, while a data structure representing an unsigned numeric input can hold a wider range of positive values without negative values. Generally speaking, a processor circuit, such as a GPU or a neural network processor, typically includes an arithmetic logic unit (ALU) or computational unit for performing computations involving different types of inputs, such as integer or floating-point inputs.

[0027] Computations involving signed inputs correspond to signed mode operations, while computations involving unsigned inputs correspond to unsigned mode operations. ALUs and computational units used to perform computations involving signed and unsigned numeric inputs require different sets of hardware components to support their respective signed and unsigned mode operations. For example, as described above, some computer architectures provide multiplication hardware with a fixed bit width B. When these architectures need to multiply inputs with a number of bits exceeding the bit width, the architecture divides the input numbers into multiple segments ("words"), where each word has a length or bit width of B. To produce the computational output, the architecture multiplies each word of the first input with each word of the second input.

[0028] However, as previously mentioned, in order to produce a signed (e.g., positive, negative, or zero) output, the architecture must be configurable in both signed and unsigned modes (e.g., where the input is only positive). An architecture that must be configurable for both signed and unsigned operations requires additional hardware components, which results in increased power consumption. In this context, techniques are described for implementing a dedicated hardware circuit 100 that is configured to perform multiplication operations on signed inputs having a unique data format while consuming less power relative to conventional hardware circuits. The dedicated circuit 100 includes multiplication circuitry that only supports signed mode operation. The circuit achieves certain power savings when the inputs are represented only as signed numbers. For example, by multiplying only signed inputs to generate a computational output, the circuit 100 can include fewer hardware components and a smaller instruction set with a reduced number of software instructions to multiply the inputs.

[0029] Circuit 100 includes an input processor 104 configured to generate signed multi-word inputs. A portion of hardware circuit 100 can include a computation unit 103 having multiplication circuitry that provides a hardware multiplier for multiplying inputs 102. Input processor 104 can be configured to generate signed multi-word inputs based on a fixed bit width of the multiplication circuitry in computation unit 103 of circuit 100. More specifically, input processor 104 is configured to generate shifted signed multi-word numbers from input 102. For example, input processor 104 can generate shifted signed multi-word numbers 106 and 108. Shifted signed multi-word number 106 can include respective signed word inputs C and D, each generated from input A, while shifted signed multi-word number 108 can include respective signed word inputs E and F, each generated from input b.

[0030] Hardware circuit 100 includes signed hardware multipliers 110 and 112. In some implementations, circuit 100 is configured to include low-power signed integer or floating-point multiplication circuitry. In some examples, multipliers 110, 112 can be connected via optional connection 113 to form a single, large-scale signed multiplication circuit of hardware circuit 100. In some other examples, multipliers 110 and 112 can represent different hardware multipliers of larger multiplication circuit 114, and circuit 100 can include one or more multiplication circuits 114. Although two multipliers are shown in the example of Figure 1 , circuit 100 (or circuit 114) can be configured to include more or fewer multipliers. For example, circuit 100 can include a single multiplier that is configured to be used for multiple purposes over time to achieve the same (or similar) computational effect as multiple separate multipliers. In this way, circuit 100 can be optimized to multiply certain digital inputs by, for example, including only signed multipliers or other hardware components required to support signed-mode operation, and having a reduced power requirement. In some cases, dedicated hardware circuit 100 uses multiplication circuitry to perform computations for processing inputs through a neural network layer. The computations can include multiplying an input and a parameter to generate an accumulated value, which is further processed to generate a layer output of the neural network layer.

[0031] In an example operation, given an input set that includes corresponding signed word inputs C and D (each generated from input A) and corresponding signed word inputs E and F (each generated from input B), circuit 100 is configured to multiply input C and E (C*E), multiply input C and F (C*F), multiply input D and E (D*E), and multiply input D and F (D*F). Computational unit 103 includes adder circuit 120 (“adder 120”), which is configured to perform appropriate addition operations between the products generated by one or more of multipliers 110, 112 of multiplication circuit 114. Computational unit 103 is configured to perform the addition operation after shifting one or more product values by a necessary bit width. For example, computational unit 103 can use adder 120 to perform shift operations (e.g., <<2*B, <<B, etc.) before performing the following addition operation (C*E<<(2*B))+((C*F+D*E)<<B)+D*F.

[0032] Adder 120 receives signed products 116 and 118 as inputs and adds signed products 116 and 118 to generate a signed output 122 for computation unit 103. In some implementations, a two's complement version of the negative signed product 118 is used to perform an addition operation that includes adding signed product 116 to the two's complement version of signed product 118 to generate signed output 122. In some cases, adding the inputs can include performing a rounding operation on the preliminary sum using rounding logic before generating signed output 122. For example, rounding logic can be used to round the preliminary sum to the nearest decimal or integer value before generating signed output 122. In some implementations, signed output 122 represents an accumulated value used to generate a layer output of a neural network layer in response to processing numerical input 102 by the neural network layer.

[0033] Figure 2 A process diagram 200 is shown for generating a signed multi-word input that is provided to a signed hardware multiplier of circuit 100 to generate a signed output 122. As described in more detail below, process diagram 200 includes a plurality of logic blocks, each representing a respective logic function of input processor 104. In general, one or more respective logic functions may be used to generate a shifted signed multi-word number.

[0034] Referring to process diagram 200, hardware circuit 100 is configured as a signed mode circuit and includes input processing circuit 104 for generating a signed multi-word number 106. Input processor 104 generates a shifted signed multi-word number from input 102 based at least on a determination that the input has a bit width that exceeds a fixed bit width of a hardware multiplier included in the hardware circuit (204). For example, input processor 104 can analyze the binary data structure of input 102 to determine whether each corresponding input exceeds the fixed bit width of multiplication circuit 114 included in computational unit 103.

[0035] Generating the signed multi-word number 106 includes generating the number 106 based on the input processor 104 determining that the input 102 is within a predetermined numeric range of the data format used to represent the shifted signed multi-word number 106 (206). For example, the input processor 104 generates the signed multi-word number 106 in response to determining that the numeric value of the input 102, such as a two's complement number, fits within the available numeric range of the data format used to represent the shifted signed multi-word number 106. For a given input 102, if the input processor 104 determines that the numeric value of the input 102 does not fit within the available numeric range of the data format, the input processor 104 ends the process 200 (208).

[0036] If the input processor 104 determines that the input 102 is within the predetermined numerical range of the data format, the input processor 204 causes the one or more inputs to be represented as corresponding signed multi-word inputs based on at least the first input having a bit width that exceeds the fixed bit width of the hardware circuit 100. For example, to represent the input as a signed multi-word input, the input processor 104 generates corresponding signed N words, each word having B bits (210). The input processor 104 then uses each of the signed N words, each having B bits, to generate a shifted signed number (212). In some implementations, N is an integer greater than 1, and B is an integer greater than 1. The signed multi-word input is provided to the signed hardware multiplier of the multiplication circuit 114 to ultimately produce a signed output.

[0037] In some cases, input processor 104 determines that the bit width of input 102 does not exceed the fixed bit width of hardware multiplier 110 included in the hardware circuit (205). In this scenario, input processor 104 provides input 214 to a signed multiplier of multiplication circuit 114. For example, input processor 104 can provide input 214 to a particular hardware multiplier based on matching the sign of the input with the sign of the particular hardware multiplier. In this implementation, because the bit width of input 214 is not greater than the fixed bit width of multiplication circuit 114, input 214 is not a suitable input for generating a signed multi-word input.

[0038] For the example multiplication operation, determining whether to generate a shifted signed multiword from input 102, and subsequently generating the signed multiword input, can occur relatively early in the computation cycle. For example, the determination can be made off-chip using an external host controller in communication with circuit 100 to obtain input for processing by a neural network layer. In some implementations, the determination and subsequent generation occur when the input is obtained from a memory of an example neural network processor, such as an activation memory storing activations generated by a neural network layer implemented on a neural network processor including hardware circuit 100.

[0039] In other implementations, determining whether to generate a signed multi-word input and subsequently generating the signed multi-word input can occur at a previous pipeline stage, such as at a preceding multiplier, ALU, or bypass circuit of the computational unit 103. In some cases, the interface of each signed hardware multiplier 110, 112 can be modified or augmented to include a corresponding input processor 104. In this case, the input 102 received at the input of each multiplier 110, 112 can be processed to generate an appropriate number of shifted multi-word inputs for multiplication at the corresponding hardware multiplier 110, 112.

[0040] Figure 3A flow chart of an example process 300 for multiplying inputs using the described hardware multiplier circuit 100 is shown. As described above, the inputs can be digital inputs, such as floating point numbers represented as a data structure of bits, e.g., 16 bits or 32 bits. Process 300 can be performed using at least circuit 100 in conjunction with other circuits, components, and systems described herein.

[0041] Referring now to process 300, circuit 100 receives a first input and a second input, each of the first input and the second input having a corresponding bit width (302). Processing circuitry is configured to represent at least the first input as a signed multi-word input based on the first input having a bit width that exceeds a fixed bit width of a hardware circuit. For example, the fixed bit width of the hardware circuit can be 16 bits, while the bit width of an example data structure for the first input is 32 bits.

[0042] Circuit 100 generates a signed multi-word input from at least a first input, comprising a plurality of signed words, each signed word having a plurality of bits (304). The signed multi-word input / number is a signed number comprising a shift of N words, each of the N words comprising B bits. In general, N can be an integer greater than 1, and B can be an integer greater than 1. For example, in response to analyzing the data structure of the first input, input processor 104 can determine that the first input consists of 32 bits. Input processor 104 can determine or calculate the difference between the number of bits in the first input and the number of bits of a fixed bit width of the hardware circuit.

[0043] The input processor 104 can generate a signed multi-word number based on the calculated difference. In some implementations, a portion of the bits of the 32-bit data structure forming the first input 102 is used to generate each word of the signed multi-word number. For example, the signed multi-word number can be composed of four 8-bit numbers or two 16-bit numbers. These numbers can correspond to the above-mentioned signed multi-word numbers 106 and 108. In some cases, each word of the signed multi-word number is a signed word, which includes a portion of the bits from the first input and a corresponding sign bit representing the sign of the signed word forming the signed multi-word number.

[0044] In some implementations, when a shifted signed multi-word number is formed by four 8-bit numbers, the shifted signed number comprises N=4 words, where each of the N words comprises B=8 bits. This "shifted signed N-word B-bit number" is represented by N ordinary signed numbers, each of which has a bit width of B. For example, let a0, a1, ..., a{N-1} be those ordinary signed numbers, and let a be the shifted signed number that each number collectively represents. The value u of the shifted signed number is defined as: a=a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B}, where a represents the corresponding signed word of the signed multi-word input. The individual words a0, a1, ..., a{N-1} are each signed numbers. In some other implementations, the original input number is zero-extended (e.g., a "0" bit is added to the most significant end) or sign-extended (e.g., the most significant bit of the original input number is copied to the extra bits) until the bit width is a multiple of B.

[0045] As described above, a data format may have a limited range of values that can be represented using the data format. In some implementations, a shifted signed multiword number has a representable range of values that is defined based on an example known expression for representing the range of values for ordinary two's complement numbers, but includes an additional parameter S. The range of values for the shifted signed multiword number is defined using [-2 (N*B-1) -S,2 (N*B-1) -1-S]. The parameter S introduces a shift function into the known expression to represent the range of values of two's complement numbers. For example, when B=8 and N=2, the representable range of ordinary two's complement numbers is: [-32, 768, 32, 767]. This range of ordinary two's complement numbers is expressed using the known expression [-2 (N*B-1) ,2 (N*B-1) -1]. With respect to the only data format described in this document, the parameter S is used to shift the known expression to the left (e.g., toward negative infinity) by a distance S relative to the representable range of two's complement of the ordinary N word*B bits. In some implementations, S and the corresponding shift are based on 2 (B-1) *(1+2 B +...+2 {(N-2)B} ) defined.

[0046] In some implementations, the hardware circuit 100 and the input processor 104 use a quantization scheme to modify the data format of the first input based on a fixed bit width of the hardware circuit. The quantization scheme is configured to modify the data format of the first input by generating corresponding word portions to represent the first input as a signed multi-word input. For example, the data format for generating signed multi-word numbers from parameters or kernel weight values of a neural network layer can be modified based on a particular quantization scheme so that the output of the layer can be calculated using these parameters appropriately. For the generated signed multi-word input, the total bit width including each corresponding word portion can be equal to the fixed bit width of the hardware circuit. In some implementations, the input processor 104 is configured to adjust certain software schemes to requantize or change the way parameters and weights are obtained and processed at the circuit 100.

[0047] Circuit 100 provides a signed multiword input and a signed second input to multiplication hardware for multiplication (306). The signed second input corresponds to the received second input. In some implementations, the second input can correspond to a signed input that does not exceed the bit width of the hardware circuit or another shifted signed multiword. In some other implementations, the second input corresponds to a signed input that exceeds the bit width of the hardware circuit, such that circuit 100 generates a signed multiword from the second input.

[0048] The circuit 100 generates a signed product (308) from the multiplication hardware using at least the first input and the second input. For example, the circuit 100 generates a signed product 116 or 118 in response to multiplying a shifted signed multi-word number of the first input with a shifted signed multi-word number of the second input. These shifted signed multi-word inputs include a plurality of corresponding words, and the multiplication circuit 114 is configured to generate a signed product by multiplying each word of the signed multi-word first input with each word of the signed multi-word second input. One advantage of shifted signed multi-word numbers is that they can be multiplied without requiring an unsigned hardware multiplier. For example, to calculate the signed product 116 of two such numbers a and b:

[0049] a=a0+a1*2 B +a2*2 (2B) +...+a{N-1}*2 {(N-1)B}

[0050] b=b0+b1*2 B +b2*2 (2B) +…+b{N-1}*2 {(N-1)B}

[0051] The hardware circuit 100 calculates a i *b j These products can all be calculated using the signed hardware multiplier of circuit 100.

[0052] A number of embodiments have been described. However, it will be appreciated that various modifications may be made without departing from the scope of the present invention. For example, various forms of the processes shown above may be used, with steps reordered, added, or removed. Therefore, other embodiments are within the scope of the following claims. Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of what is claimed, but rather as descriptions of features unique to particular embodiments. Certain features described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment.

[0053] Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed as such, in some cases one or more features from a claimed combination can be deleted from that combination, and a claimed combination may be directed to a subcombination or variations of the subcombinations.

[0054] Similarly, although operations are described in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order or sequence shown, or that all illustrated operations be performed, in order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0055] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order or sequential sequence shown to achieve the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

Claims

1. A hardware circuit for multiplying a set of inputs, the hardware circuit comprising: a processing circuit that receives a first input and a second input, each of the first input and the second input having a respective bit width, wherein the processing circuit is configured to represent at least the first input as a signed multi-word input based on the first input having a bit width that exceeds a fixed bit width of the hardware circuit; and one or more signed multipliers, each of the one or more signed multipliers configured to multiply two or more signed inputs, each signed multiplier comprising multiplication circuitry configured to: receiving a signed multi-word input representing a first input; receiving a signed second input corresponding to the second input; and generating a signed output in response to multiplying a signed multi-word input by a signed second input, wherein the signed multi-word input is a shifted signed number comprising N words, each of the N words comprising B bits; and N is an integer greater than 1, and B is an integer greater than 1.

2. The hardware circuit according to claim 1, wherein: The value of the shifted signed number is based on a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} where a represents the corresponding signed word of the signed multiword input.

3. The hardware circuit according to claim 2, wherein: The representable range of the shifted signed number is based on [-2 (N*B-1) -S,2 (N*B-1) -1-S] defined.

4. The hardware circuit according to claim 2, wherein: S is based on 2 (B-1) *(1+2 B +…+2 {(N-2)B} ) defined.

5. The hardware circuit according to claim 1, wherein: The processing circuit is configured to represent the first input as a signed multi-word input, the signed multi-word input comprising: the signed high word portion; and Signed low word part.

6. The hardware circuit according to claim 5, wherein: Representing the first input as a signed multiword input includes: Based on a fixed bit width of the hardware circuit, a quantization scheme is used to modify the data format of the first input.

7. The hardware circuit according to claim 6, wherein: a quantization scheme configured to modify a data format of the first input by generating corresponding word portions to represent the first input as a signed multi-word input; as well as The total bit width including each corresponding word portion is equal to the fixed bit width of the hardware circuit.

8. The hardware circuit according to claim 1, wherein: The multiplication circuit is configured to generate a signed output by multiplying each word of the signed multi-word input with each word of the signed second input.

9. The hardware circuit according to claim 1, wherein: the second input is a signed multi-word input, such that the signed second input comprises a plurality of corresponding signed words; and The multiplication circuit is configured to generate a signed output as a sum of respective products computed by multiplying each word of the signed multi-word input with each signed word of the signed second input.

10. A method for multiplying a set of inputs using a hardware circuit, the method comprising: receiving, by processing circuitry of the hardware circuitry, a first input and a second input, each of the first input and the second input having a respective bit width, wherein at least the first input has a bit width that exceeds a fixed bit width of multiplication hardware included in the hardware circuitry, the multiplication hardware being configured to multiply the first input and the second input; generating a signed multi-word input comprising a plurality of signed words from at least a first input, each signed word having a plurality of bits, wherein a bit width of the signed multi-word input is less than a fixed bit width of the multiplication hardware; providing the signed multi-word input and a signed second input to the multiplication hardware for multiplication, wherein the signed second input corresponds to the second input and has a bit width within a fixed bit width of the multiplication hardware; and generating a signed output from multiplication hardware using at least a first input and a second input, wherein the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; and N is an integer greater than 1, and B is an integer greater than 1.

11. The method according to claim 10, wherein: The value of the shifted signed number is based on a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} Defined as , where a represents the corresponding signed word of the signed multi-word input.

12. The method according to claim 11, wherein The representable range of the shifted signed number is based on [-2 (N*B-1) -S,2 (N*B-1) -1-S] defined.

13. The method according to claim 11, wherein S is based on 2 (B-1) *(1+2 B +…+2 {(N-2)B} ) defined.

14. The method according to claim 10, wherein: Generating a signed multi-word input includes representing the first input as a signed multi-word input, the signed multi-word input including: the signed high word portion; and Signed low word part.

15. The method according to claim 14, wherein Representing the first input as a signed multiword input includes: Based on a fixed bit width of the hardware circuit, a quantization scheme is used to modify the data format of the first input.

16. The method according to claim 15, further comprising: modifying a data format of the first input by generating corresponding word portions based on a quantization scheme to represent the first input as a signed multi-word input, The total bit width including each corresponding word portion is equal to the fixed bit width of the hardware circuit.

17. The method according to claim 10, wherein The second input is a signed multi-word input such that the signed second input includes a plurality of corresponding words, and the method further comprises: A single signed multiplier using the multiplication hardware generates a signed output as the sum of the products of each word of a signed multi-word input multiplied by each word of a signed second input.

18. One or more non-transitory machine-readable storage devices of hardware circuitry for storing instructions executable by one or more processing devices to cause operations to be performed, the operations comprising: receiving, by processing circuitry of the hardware circuitry, a first input and a second input, each of the first input and the second input having a respective bit width, wherein at least the first input has a bit width that exceeds a fixed bit width of multiplication hardware included in the hardware circuitry, the multiplication hardware being configured to multiply the first input and the second input; generating a signed multi-word input comprising a plurality of signed words from at least a first input, each signed word having a plurality of bits, wherein a bit width of the signed multi-word input is less than a fixed bit width of the multiplication hardware; providing the signed multi-word input and a signed second input to the multiplication hardware for multiplication, wherein the signed second input corresponds to the second input and has a bit width that is less than a fixed bit width of the multiplication hardware; and generating a signed output from multiplication hardware using at least a first input and a second input, wherein the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; and N is an integer greater than 1, and B is an integer greater than 1.

Images