Signed multiword multiplier

Abstract

To provide a signed multiword multiplier.SOLUTION: Methods, systems, and apparatus include computer programs encoded on a computer storage medium for a hardware circuit configured as a signed multiword multiplier. The circuit includes a processing circuit that receives pieces of input that each have a bit width. The processing circuit can represent at least one input as signed multiword input based on first input having the bit width that exceeds a fixed bit width of the hardware circuit. The circuit includes signed multipliers that are each configured to multiply the signed input. Each signed multiplier includes a multiplication circuit configured to receive the signed multiword input, receive signed second input, and generate signed output in response to multiplying the signed multiword input with the signed second input.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This specification relates to hardware circuits for performing numerical computations. [Background technology]

[0002] The computation circuit may include a multiplication circuit having a hardware multiplier used to multiply numeric inputs such as integers and floating-point numbers. Multiplication circuits may be expensive to procure and integrate into existing computation circuits, and some circuits are not sized efficiently for certain applications. For example, some multiplication circuits may include both signed and unsigned multipliers that consume significant area of ​​the circuit die, but do not provide benefits in computation throughput despite their large size. Multiplier circuits that are too large for certain computation applications may result in inefficiencies in power consumption and utilization.

[0003] Hardware circuits can be used to implement neural networks. In particular, neural networks with multiple layers can be implemented in a computation circuit that includes several hardware multipliers. The computation circuit of the hardware circuit may represent a computation unit used to perform neural network computations for a given layer. For example, given an input, the circuit may compute an inference about the input using a neural network by performing a dot product operation using one or more of the multipliers in the computation unit of the hardware circuit. Summary of the Invention [Means for solving the problem]

[0004] This document describes a dedicated hardware circuit for multiplying inputs. The hardware circuit includes processing circuitry that receives inputs each having a respective bit-width. The processing circuitry can represent at least one input as a signed multiword input based on a first input having a bit-width that exceeds a fixed bit-width of the hardware circuit. The hardware circuit is configured as a signed multiword multiplier and includes signed multipliers each configured to multiply signed inputs. Each signed multiplier includes multiplication circuitry configured to receive the signed multiword input, receive a signed second input, and generate a signed output responsive to multiplying the signed multiword input with the signed second input.

[0005] One aspect of the subject matter described herein may be embodied in a hardware circuit for multiplying a set of inputs, the hardware circuit including: a processing circuit that receives a first input and a second input, each of the first and second inputs having a respective bit-width, the processing circuit configured to represent at least the first input as a signed multiword input based on the first input having a bit-width that exceeds a fixed bit-width of the hardware circuit; and a plurality of signed multipliers, each signed multiplier of the plurality of signed multipliers configured to multiply two or more signed inputs, each signed multiplier including a multiplication circuit configured to receive a signed multiword input representing the first input, receive a signed second input corresponding to the second input, and generate a signed output responsive to multiplying the signed multiword input with the signed second input.

[0006] In these and other implementations, each may optionally include one or more of the following features. For example, in some implementations, the signed multiword input is a shifted signed number that includes N words, each of the N words including B bits, where N is an integer greater than 1 and B is an integer greater than 1. In some implementations, the numeric value of the shifted signed number is a0+a1*2 B +a2*2(2B) +…+a{N-1}*2 {(N-1)B} where a represents each signed word of the signed multiword input. In some implementations, a representative range of values ​​for the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) In some implementations, S is defined based on 2 (B-1) *(1+2 B +…+2 {(N-2)B} In some implementations, the processing circuit is configured to represent the first input as a signed multi-word input including a signed high order word portion and a signed low order word portion.

[0007] In some implementations, representing the first input as a signed multiword input includes using a quantization scheme to modify a data format of the first input based on a fixed bit-width of the hardware circuit. In some implementations, the quantization scheme is configured to modify a data format of the first input by generating respective word portions to represent the first input as a signed multiword input, the total bit-width including each respective word portion being equal to the fixed bit-width of the hardware circuit. In some implementations, the signed multiword input includes a plurality of respective words, and the multiplication circuit is configured to generate the signed output by multiplying each word of the signed multiword input with each word of the signed second input. In some embodiments, the signed second input includes a plurality of respective signed words, and the multiplication circuit is configured to generate the signed output as a sum of respective products calculated from multiplying each word of the signed multiword input with each signed word of the signed second input.

[0008] One aspect of the subject matter described herein may be embodied in a method for multiplying a set of inputs using a hardware circuit, the method including: receiving, by a processing circuit of the hardware circuit, a first input and a second input, each of the first and second inputs having a respective bit-width, at least the first input having a bit-width that exceeds a fixed bit-width of multiplication hardware included in the hardware circuit, the multiplication hardware being used to multiply the first input and the second input; generating from the at least the first input a signed multiword input including a plurality of signed words each having a plurality of bits, the bit-width of the signed multiword input being smaller than the fixed bit-width of the multiplication hardware; providing the signed multiword input and a signed second input to the multiplication hardware for multiplication, the signed second input corresponding to the second input and having a bit-width that is within the fixed bit-width of the multiplication hardware; and generating a signed output from the multiplication hardware using at least the first and second inputs.

[0009] In these and other implementations, each may optionally include one or more of the following features. For example, in some implementations, the signed multiword input is a shifted signed number that includes N words, each of the N words including B bits, where N is an integer greater than 1 and B is an integer greater than 1. In some implementations, the numeric value of the shifted signed number is a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} where a represents each signed word of the signed multiword input. In some implementations, a representative range of values ​​for the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) In some implementations, S is defined based on 2 (B-1) *(1+2 B +…+2 {(N-2)B}In some implementations, generating the signed multi-word input includes representing the first input as a signed multi-word input including a signed high-order word portion and a signed low-order word portion.

[0010] In some implementations, representing the first input as a signed multiword input includes using a quantization scheme to modify a data format of the first input based on a fixed bit-width of the hardware circuit. In some implementations, the method further includes modifying a data format of the first input by generating respective word portions to represent the first input as a signed multiword input based on the quantization scheme, the total bit-width including each respective word portion being equal to the fixed bit-width of the hardware circuit. In some embodiments, the signed second input includes a plurality of respective words, and the method further includes generating the signed output as a sum of respective products of multiplications of each word of the signed multiword input with each word of the signed second input using a signed multiplier of the multiplication hardware.

[0011] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs configured to perform the actions of the methods encoded on a computer storage device (e.g., a non-transitory machine-readable storage medium). A computing system of one or more computers or hardware circuits may be configured by software, firmware, hardware, or a combination thereof installed on the system, such that, when operated, the system performs actions. One or more computer programs may be configured by having instructions that, when executed by a data processing device, may cause the device to perform actions.

[0012] The subject matter described herein can be implemented in certain embodiments to achieve one or more of the following advantages: The techniques described can be used to implement a dedicated hardware circuit that multiplies two or more inputs while requiring less power than conventional circuits used to multiply the inputs. The hardware circuit components described in this document form a signed multiword multiplier circuit having a signed multiplier configured to multiply signed inputs to produce a signed output. The multiword multiplier can be a low-power hardware multiplication circuit that efficiently multiplies several inputs (e.g., floating-point inputs) based on a unique numeric format for representing the signed numbers.

[0013] The multiplication circuit can be configured to have multiplication hardware that includes only signed hardware multipliers for performing multiplication of inputs. The circuit includes processing circuitry used to generate shifted signed multi-word numbers in response to processing inputs having a conventional number system, such as two's complement format. The signed multi-word numbers are multiplied using the signed hardware multipliers to generate signed outputs. These features of the multiplication circuit result in lower power consumption in the circuit relative to conventional circuits that multiply inputs. This is because the multiplication is completed using only signed multipliers, rather than both signed and unsigned multipliers. Furthermore, a circuit that includes hardware multipliers to support multiple modes (e.g., signed and unsigned modes) still increases the chip area consumed by the circuit, thereby increasing the manufacturing cost of the circuit. Thus, the proposed technique achieves a reduction in not only power consumption but also manufacturing cost.

[0014] When the multiplication hardware of a circuit is configured to include only signed hardware multipliers, the overall hardware circuit consumes much less power than conventional circuits that must include additional multiplication hardware to support both signed and unsigned computation modes. Thus, the low power hardware multiplier circuit can be optimized for multiplying numeric inputs with reduced power requirements based at least on a signed multiplier configuration that exploits a signed-only mode to generate a product that multiplies two or more signed multiword inputs.

[0015] The details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0016] [Figure 1] FIG. 2 illustrates an exemplary dedicated hardware circuit for multiplying inputs. [Diagram 2] 1 is a flow diagram for generating a signed multi-word input that is provided to a signed hardware multiplier to generate a signed output. [Diagram 3] 4 is a flowchart illustrating an exemplary process for multiplying inputs in the described hardware multiplier circuit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Like reference numbers and designations in the various drawings indicate like elements.

[0018] Conventional computer architectures provide multiplication hardware with a fixed bit-width B. When these architectures need to multiply inputs with a number of bits that exceeds the bit-width, the architectures divide the number of inputs into multiple pieces ("words"), where each word has a length, or bit-width, B. To create a computational output, these architectures multiply every word of the first input with every word of the second input. However, to create a signed (e.g., positive, negative, or zero) output, the architecture must be configurable in both signed and unsigned modes (e.g., where the inputs are simply positive or zero). Conventional circuits that must be configurable in both signed and unsigned modes require additional hardware components that would increase power consumption.

[0019] In an exemplary implementation, a hardware circuit can be used to implement a multi-layer neural network and perform a computation (e.g., a neural network computation) by processing an input through each of the layers of the neural network. In particular, each individual layer of the neural network can have a respective set of parameters. Each layer receives an input, processes the input according to the set of parameters for the layer, and generates an output based on a computation performed using multiplication circuitry of the exemplary computation unit. For example, a neural network layer computes multiple products when performing a matrix multiplication of an input array and a parameter array, or as part of computing a convolution between an input array and a parameter kernel array.

[0020] Generally, processing inputs through layers of a neural network is accomplished using circuits for performing arithmetic operations, such as, for example, multiplication and addition. An exemplary hardware circuit may include a hardware multiplier for multiplying two or more inputs. The multiplier circuit may be grouped with a hardware adder to form a computation unit, such as, for example, a matrix or vector processing unit, of the hardware circuit. The computation unit is used to add and multiply numeric inputs, such as integers and floating-point numbers. For example, addition and multiplication occur when a hardware circuit is used to perform neural network calculations, such as matrix-vector multiplication, to process inputs through layers of a neural network.

[0021] With the above context in mind, this document describes techniques for implementing dedicated hardware circuits for multiplying two or more inputs represented as signed multiword inputs. Using this technique, signed or unsigned inputs can be represented as "shifted signed multiword numbers". These shifted signed multiword numbers use a unique numeric format to represent the received inputs as signed numbers. The received inputs can be individual words of a multiword number and can also include single word and multiword inputs. By representing the inputs as signed numbers, there is no need for the dedicated hardware circuit to support the unsigned mode. Thus, the described hardware circuit uses a more simplified architecture that includes multiplication circuitry for signed mode operations rather than operations for both signed and unsigned modes. Because the described hardware circuit is configured for signed mode operations only, the circuit requires fewer components, which improves power efficiency when compared to conventional architectures.

[0022] 1 shows a diagram of an exemplary dedicated hardware circuit 100 for multiplying inputs 102. In an exemplary implementation, inputs 102A ("Input A") and 102B ("Input B") are each floating point or two's complement numbers that can be represented in software using a binary data structure. The binary data structure can have a particular number of bits, such as, for example, a 16-bit, 24-bit, or 32-bit data structure. For example, each of inputs A or B may each be a signed floating point number and a sign bit, such that each input can indicate the sign of the input (e.g., positive or negative).

[0023] The data structure for each numeric input can be associated with a particular data format. The data format can indicate a finite range of numeric values ​​that can be represented using the data format. In some implementations, a 16-bit data structure for input A can include a binary input (e.g., 0010) that represents a two's complement data format for input A. In terms of numeric range, a normal two's complement number can have a finite representable range of numeric values: [-32.768, 32.767]. Additionally, each numeric input has one or more bits in its data structure that indicate whether the number is a signed or unsigned number.

[0024] As described herein, a data structure representing a signed numeric input (e.g., an integer) can hold both positive numeric values ​​(e.g., integer values) and negative numeric values, while a data structure representing an unsigned numeric input can hold a larger range of positive numeric values ​​and cannot hold negative numeric values. In general, processor circuits such as GPUs or neural network processors often include arithmetic logic units (ALUs) or computation units for performing calculations involving different types of inputs, such as integer or floating point inputs.

[0025] Calculations involving signed inputs correspond to signed mode operations, while calculations involving unsigned inputs correspond to unsigned mode operations. ALUs and computation units for performing calculations involving signed and unsigned numeric inputs require separate sets of hardware components to support the respective signed and unsigned mode operations. For example, as described above, some computer architectures implement multiplication hardware with a fixed bit width B. When these architectures need to multiply inputs with a number of bits that exceeds the bit width, the architectures divide the number of inputs into multiple pieces ("words"), where each word has a length, or bit width, B. To create a computation output, the architectures multiply every word of the first input with every word of the second input.

[0026] However, as previously discussed, to create signed (e.g., positive, negative, or zero) outputs, the architecture must be configurable in both signed and unsigned modes (e.g., where the inputs are only positive). An architecture that must be configurable in both signed and unsigned operations requires additional hardware components that result in increased power consumption. In this context, techniques are described for implementing a dedicated hardware circuit 100 configured to multiply signed inputs having a unique data format, while consuming less power than conventional hardware circuits. The dedicated circuit 100 includes multiplication circuitry to support only signed mode operations. When the inputs are represented only as signed numbers, the circuit achieves some power savings. For example, by generating a computational output from multiplying only signed inputs, the circuit 100 can multiply the inputs with fewer hardware components and a smaller set of instructions that reduce the number of software instructions.

[0027] The circuit 100 includes an input processor 104 configured to generate a signed multiword input. A portion of the hardware circuit 100 may include a computation unit 103 having a multiplication circuit implementing a hardware multiplier for multiplying the input 102. The input processor 104 may be configured to generate the signed multiword input based on a fixed bit-width of the multiplication circuit in the computation unit 103 of the circuit 100. More specifically, the input processor 104 may be configured to generate a shifted signed multiword number from the input 102. For example, the input processor 104 may generate shifted signed multiword numbers 106 and 108. The shifted signed multiword number 106 may include respective signed word inputs C and D, each generated from input A, while the shifted signed multiword number 108 may include respective signed word inputs E and F, each generated from input B.

[0028] The hardware circuit 100 includes signed hardware multipliers 110 and 112. In some implementations, the circuit 100 is configured to include a low-power signed integer or floating-point multiplication circuit. In some examples, the multipliers 110, 112 can be connected via an optional connection 113 to form the hardware circuit 100 of a single, large, signed multiplication circuit. In some other examples, the multipliers 110 and 112 can represent different hardware multipliers of a large multiplication circuit 114, and the circuit 100 can include one or more multiplication circuits 114. While two multipliers are shown in the example of FIG. 1, the circuit 100 (or the circuit 114) can be configured to include more or fewer multipliers. For example, the circuit 100 can include a single multiplier configured to be used for multiple purposes over time to achieve the same (or similar) computational effect as multiple individual multipliers. In this manner, circuit 100 can be optimized for multiplying certain numeric inputs with reduced power requirements, for example, by including only signed multipliers or other hardware components necessary to support only signed mode arithmetic. In some cases, dedicated hardware circuit 100 uses multiplication circuitry to perform calculations to process inputs through layers of a neural network. The calculations may include multiplication of inputs and parameters to generate accumulated values ​​that are further processed to generate layer outputs of the neural network layer.

[0029] In an exemplary operation, if there is a set of inputs including signed word inputs C and D (each generated from input A) and signed word inputs E and F (each generated from input B), circuit 100 is configured to multiply input C by E (C*E), multiply input C by F (C*F), multiply input D by E (D*E), and multiply input D by F (D*F). Calculation unit 103 includes an addition circuit 120 (referred to as "adder 120") configured to perform appropriate addition operations among the products generated by one or more multipliers 110, 112 of multiplication circuit 114. Calculation unit 103 is configured to perform an addition operation after shifting the values of one or more products by the required bit width. For example, calculation unit 103 can perform shift operations (such as <<2*B, <<B, etc.) before using adder 120 to perform the following addition operation, i.e., (C*E<<(2*B))+((C*F+D*E)<<B)+D*F.

[0030] Adder 120 receives signed products 116 and 118 as inputs, adds signed products 116 and 118, and generates a signed output 122 of calculation unit 103. In some implementations, a two's complement version of the negative signed product 118 is used to perform an addition operation including adding signed product 116 and the two's complement version of signed product 118 to generate signed output 122. In some cases, adding the inputs may include using rounding logic to perform a rounding operation on the provisional sum before generating signed output 122. For example, the rounding logic can be used to round the provisional sum to the nearest decimal or integer value before generating signed output 122. In some implementations, signed output 122 represents an accumulated value for generating a layer output of a neural network layer in response to processing numerical input 102 through the neural network layer.

[0031] 2 illustrates a process diagram 200 for generating a signed multi-word input that is provided to a signed hardware multiplier of circuit 100 to generate a signed output 122. As described in more detail below, process diagram 200 includes a number of logic blocks, each representing a respective logic function of input processor 104. In general, one or more of the respective logic functions may be used to generate a shifted signed multi-word number.

[0032] With reference to process diagram 200, hardware circuit 100 is configured as a signed mode circuit and includes an input processing circuit 104 for generating a signed multi-word number 106. The input processor 104 generates a shifted signed multi-word number from input 102 based at least on determining (204) that the input has a bit-width that exceeds a fixed bit-width of a hardware multiplier included in the hardware circuit. For example, input processor 104 may analyze the binary data structure of input 102 to determine whether each respective input exceeds a fixed bit-width of a multiplier circuit 114 included in computation unit 103.

[0033] Generating the signed multi-word number 106 includes generating the number 106 based on the input processor 104 determining (206) that the input 102 is within a predefined numeric range of a data format used to represent the shifted signed multi-word number 106. For example, the input processor 104 generates the signed multi-word number 106 in response to determining whether the numeric value, e.g., two's complement, of the input 102 matches within an available numeric range of the data format representing the shifted signed multi-word number 106. If, for a given input 102, the input processor 104 determines (208) that the numeric value of the input 102 does not match within the available numeric range of the data format, the input processor 104 terminates the process 200.

[0034] If the input processor 104 determines that the input 102 is within the predefined numeric range of the data format, the input processor 104 causes one or more inputs to be represented as respective signed multi-word inputs based on at least a first input having a bit width that exceeds the fixed bit width of the hardware circuit 100. For example, to represent the inputs as signed multi-word inputs, the input processor 104 generates N signed words, each having B bits (210). The input processor 104 then generates a shifted signed number using each of the N signed words, each having B bits (212). In some implementations, N is an integer greater than 1 and B is an integer greater than 1. The signed multi-word inputs are provided to a signed hardware multiplier of the multiplication circuit 114, which ultimately generates a signed output.

[0035] In some cases, the input processor 104 determines (205) that the input 102 has a bit-width that does not exceed the fixed bit-width of a hardware multiplier 110 included in the hardware circuit. In this scenario, the input processor 104 provides the input 214 to a signed multiplier of the multiplication circuit 114. For example, the input processor 104 may provide the input 214 to a particular hardware multiplier based on the sign of the input matching the sign of the particular hardware multiplier. In this implementation, because the input 214 does not have a bit-width that is greater than the fixed bit-width of the multiplication circuit 114, the input 214 would not be a suitable input for generating a signed multiword input.

[0036] In an exemplary multiplication operation, the decision to generate a signed multi-word number shifted from the input 102, and the subsequent generation of the signed multi-word input, may occur relatively early in the computation cycle. For example, the decision may be made off-chip using an external host controller in communication with the circuit 100 to obtain the input for processing through the neural network layers. In some implementations, the decision and subsequent generation occurs once the input is obtained from a memory of the exemplary neural network processor, such as an activation memory that stores activations generated by a neural network layer implemented on a neural network processor that includes the hardware circuit 100.

[0037] In other implementations, the decision to generate a signed multiword input, as well as the subsequent generation of the signed multiword input, may occur in an earlier pipeline stage, for example, a previous multiplier, an ALU, or a bypass circuit of the computation unit 103. In some cases, the interface of each signed hardware multiplier 110, 112 may be modified or expanded to include a respective input processor 104. In such cases, the input 102 received at the input of each multiplier 110, 112 may be processed to generate a suitable number of shifted multiword inputs for multiplication by the respective hardware multiplier 110, 112.

[0038] 3 shows a flow chart of an exemplary process 300 for multiplying inputs using the described hardware multiplier circuit 100. As indicated above, the inputs may be numeric inputs such as floating point numbers represented as bits of a data structure, e.g., 16-bit or 32-bit. The process 300 may be implemented using at least the circuit 100 in combination with other circuits, components, and systems described herein.

[0039] Now referring to process 300, circuit 100 receives a first input and a second input, each having a respective bit-width (302). The processing circuit is configured to represent at least the first input as a signed multi-word input based on the first input having a bit-width that exceeds a fixed bit-width of the hardware circuit. For example, the fixed bit-width of the hardware circuit may be 16 bits, while the bit-width for an exemplary data structure of the first input is 32 bits.

[0040] The circuit 100 generates (304) a signed multiword input including multiple signed words each having multiple bits from at least the first input. The signed multiword input / number is a shifted signed number including N words, each of the N words including B bits. In general, N may be an integer greater than 1, and B is an integer greater than 1. For example, in response to analyzing the data structure of the first input, the input processor 104 may determine that the first input consists of 32 bits. The input processor 104 may determine or calculate a difference between the number of bits of the first input and the number of bits for the fixed bit width of the hardware circuit.

[0041] The input processor 104 may generate a signed multi-word number based on the calculated difference. In some implementations, each word of the signed multi-word number is generated using a portion of the bits that form the 32-bit data structure of the first input 102. For example, the signed multi-word number may be formed from four 8-bit numbers or two 16-bit numbers. These numbers may correspond to the signed multi-word numbers 106 and 108 described above. In some cases, each word of the signed multi-word number is a signed word that includes a portion of the bits from the first input and a corresponding sign bit that indicates the sign of the signed word that forms the signed multi-word number.

[0042] In some implementations, when a shifted signed multi-word number is formed from four 8-bit numbers, the shifted signed number contains N=4 words, where each of the N words contains B=8 bits. This "shifted signed N-word 8-bit number" is represented by N regular signed numbers, each of bit width B. As an example, let a0, a1, ..., a{N-1} be the regular signed numbers, and let a be the shifted signed number that each number represents together. The numerical value u of the shifted signed number is defined as follows: a=a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} where a represents each signed word of the signed multiword input. The individual words a0, a1, ..., a{N-1} are each signed numbers. In some other implementations, the original input numbers are zero-extended (e.g., "0" bits are added to the most significant end) or sign-extended (e.g., the most significant bit of the original input number is copied into the excess bit) until the bit width is a multiple of B.

[0043] As discussed above, the data format may have a finite range of numeric values ​​that can be represented using the data format. In some implementations, the shifted signed multi-word number has a representable numeric range that is defined based on an exemplary known representation for representing the numeric range of regular two's complement numbers, but includes an additional parameter S. The numeric range of the shifted signed multi-word number is [-2 (N*B-1) -S, 2 (N*B-1) -1-S]. The parameter S provides a shift function for the known representation to represent the range of values ​​for two's complement numbers. For example, when B=8 and N=2, ordinary two's complement numbers have a representation range that is [-32.768, 32.767]. This range for ordinary two's complement numbers is obtained using the known representation, [-2 (N*B-1) , 2 (N*B-1)-1]. For the unique data format described in this document, a parameter S is used to shift the known representation to the left (e.g., toward negative infinity) a distance S relative to the normal N word*B bit two's complement representation range. In some implementations, S and the corresponding shift are 2 (B-1) *(1+2 B +…+2 {(N-2)B} ) is prescribed.

[0044] In some implementations, the hardware circuit 100 and the input processor 104 use a quantization scheme to modify the data format of the first input based on the fixed bit width of the hardware circuit. The quantization scheme is configured to modify the data format of the first input by generating respective word parts to represent the first input as a signed multi-word input. For example, the data format for generating a signed multi-word number from parameters or kernel weight values ​​for a neural network layer can be modified based on a particular quantization scheme, so that the parameters can be suitably used to calculate an output for the layer. The generated signed multi-word input may have a total bit width including each respective word part equal to the fixed bit width of the hardware circuit. In some implementations, the input processor 104 is configured to adjust certain software schemes to requantize or modify the way in which parameters and weights are obtained and processed in the circuit 100.

[0045] The circuit 100 provides the signed multiword input and a signed second input to the multiplication hardware for the multiplication (306). The signed second input corresponds to the received second input. In some implementations, the second input can correspond to a signed input that does not exceed the bit width of the hardware circuit or another shifted signed multiword number. In some other implementations, the second input corresponds to a signed input that exceeds the bit width of the hardware circuit, such that the circuit 100 generates the signed multiword number from the second input.

[0046] The circuit 100 generates 308 a signed product from the multiplication hardware using at least the first and second inputs. For example, the circuit 100 generates a signed product 116 or 118 in response to multiplying a shifted signed multiword number of the first input with a shifted signed multiword number of the second input. These shifted signed multiword inputs include a plurality of respective words, and the multiplication circuit 114 is configured to generate the signed product by multiplying each word of the signed multiword first input with each word of the signed multiword second input. An advantage of the shifted signed multiword numbers is that they can be multiplied without the need for an unsigned hardware multiplier. For example, to calculate the signed product 116 of two such numbers a and b: a=a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} b=b0+b1*2 B +b2*2 (2B) +…+b{N-1}*2 {(N-1)B} The hardware circuit 100 includes a i *b j This can all be calculated using the signed hardware multiplier of circuit 100.

[0047] Several embodiments have been described. Nevertheless, it will be understood that various modifications can be made without departing from the scope of the present invention. For example, various forms of the flows shown above can be used with steps rearranged, added, or removed. Accordingly, other embodiments are within the scope of the following claims. While this specification contains many specific implementation details, these should not be considered as limitations on the scope of what can be claimed, but rather as descriptions of features that may be specific to a particular embodiment. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment.

[0048] Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Further, although features may be described above as working in certain combinations, and may even be initially claimed as such, one or more features from the claimed combination may in some cases be removed from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0049] Similarly, while operations are depicted in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations be performed, to achieve desired results. In certain environments, multitasking and parallel processing may be advantageous. Furthermore, the division of the various system modules and components in the embodiments described above should not be understood as requiring such division in all embodiments, and it should be understood that the program components and systems described may generally be integrated together in a single software product or packaged into multiple software products.

[0050] Particular embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. [Explanation of symbols]

[0051] 100 Dedicated Hardware Circuits 102 Input 102A Input 102B Input 103 Computational Units 104 Input processor, processing circuit, signed multiword generator 106 Shifted Signed Multiword Number 108 Shifted Signed Multiword Number 110 Signed Hardware Multiplier_1 112 Signed Hardware Multiplier_2 113 Connection 114 Multiplication circuit, signed hardware multiplier 116 Signed Product_1 118 Signed Product_2 120 Adder circuit, adder 122 Signed Output 200 Process Diagram 214 Input 300 processes

Claims

1. 1. A hardware circuit for multiplying a set of inputs, comprising: a processing circuit that receives a first input and a second input, each of the first and second inputs having a respective bit-width, the processing circuit being configured to represent at least the first input as a signed multi-word input based on the first input having a bit-width that exceeds a fixed bit-width of the hardware circuit; one or more signed multipliers, each of the one or more signed multipliers configured to multiply two or more signed inputs, each signed multiplier comprising: receiving the signed multiword input representing the first input; receiving a signed second input corresponding to the second input; one or more signed multipliers comprising a multiplication circuit configured to generate a signed output responsive to multiplying the signed multiword input with the signed second input; A hardware circuit comprising:

2. the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; 2. The hardware circuit of claim 1, wherein N is an integer greater than 1 and B is an integer greater than 1.

3. The value of the shifted signed number is a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} It is prescribed based on 3. The hardware circuit of claim 2, wherein a represents each signed word of the signed multiword input.

4. A representative range of the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) 4. The hardware circuit of claim 3, wherein the hardware circuit is defined based on [.-1-S].

5. S is 2 (B-1) *(1+2 B +…+2 {(N-2)B} 4. The hardware circuit of claim 3, wherein the hardware circuit is defined based on

6. The processing circuitry comprises: Signed high word part and Signed lower word part and 2. The hardware circuit of claim 1 , configured to represent the first input as a signed multi-word input comprising:

7. Representing the first input as the signed multiword input, using a quantization scheme to modify a data format of the first input based on the fixed bit-width of the hardware circuit.

7. The hardware circuit of claim 6, comprising:

8. the quantization scheme is configured to modify the data format of the first input by generating respective word portions to represent the first input as the signed multi-word input; 8. The hardware circuit of claim 7, wherein a total bit-width including each respective word portion is equal to the fixed bit-width of the hardware circuit.

9. the signed multi-word input comprises a plurality of respective words; 2. The hardware circuit of claim 1, wherein the multiplication circuitry is configured to generate the signed output by multiplying each word of the signed multiword input with each word of the signed second input.

10. the second input is a signed multi-word input, such that the signed second input comprises a plurality of respective signed words; 2. The hardware circuit of claim 1, wherein the multiplication circuitry is configured to generate the signed output as a sum of respective products calculated from multiplying each word of the signed multiword input with each signed word of the signed second input.

11. 1. A method for multiplying a set of inputs using hardware circuitry, comprising: receiving, by a processing circuit of the hardware circuit, a first input and a second input, each of the first and second inputs having a respective bit-width, at least the first input having a bit-width that exceeds a fixed bit-width of multiplication hardware included in the hardware circuit, the multiplication hardware being used to multiply the first input and the second input; generating a signed multiword input from at least the first input, the signed multiword input including a plurality of signed words each having a plurality of bits, the bit-width of the signed multiword input being smaller than the fixed bit-width of the multiplication hardware; providing the signed multiword input and a signed second input to the multiplication hardware for multiplication, the signed second input corresponding to the second input and having a bit width that is within the fixed bit width of the multiplication hardware; generating a signed output from said multiplication hardware using at least said first and second inputs; A method comprising:

12. the signed multiword input is a shifted signed number comprising N words, each of the N words comprising B bits; 12. The method of claim 11, wherein N is an integer greater than 1 and B is an integer greater than 1.

13. The value of the shifted signed number is a0+a1*2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} 13. The method of claim 12, wherein a represents each signed word of the signed multi-word input.

14. A representative range of the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) The method of claim 13, wherein the method is defined based on [-1-S].

15. S is 2 (B-1) *(1+2 B +…+2 {(N-2)B} The method of claim 13, wherein the method is based on

16. The step of generating a signed multiword input comprises: Signed high word part and Signed lower word part and 12. The method of claim 11, comprising representing the first input as a signed multi-word input comprising:

17. 17. The method of claim 16, wherein representing the first input as the signed multi-word input comprises using a quantization scheme to change a data format of the first input based on the fixed bit-width of the hardware circuit.

18. further comprising modifying the data format of the first input by generating respective word portions to represent the first input as the signed multi-word input based on the quantization scheme; 20. The method of claim 17, wherein a total bit-width including each respective word portion is equal to the fixed bit-width of the hardware circuit.

19. The second input is a signed multi-word input, such that the signed second input includes a plurality of respective words, and the method comprises: generating said signed output as a sum of respective products of multiplications of each word of said signed multiword input with each word of said signed second input using a single signed multiplier of said multiplication hardware; 12. The method of claim 11, further comprising:

20. One or more non-transitory machine-readable storage devices of hardware circuitry, provided to one or more processing devices: receiving, by a processing circuit of the hardware circuit, a first input and a second input, each of the first and second inputs having a respective bit-width, at least the first input having a bit-width that exceeds a fixed bit-width of multiplication hardware included in the hardware circuit, the multiplication hardware being configured to multiply the first and second inputs; generating a signed multiword input from at least the first input, the signed multiword input including a plurality of signed words each having a plurality of bits, the bit-width of the signed multiword input being smaller than the fixed bit-width of the multiplication hardware; providing the signed multiword input and a signed second input to the multiplication hardware for multiplication, the signed second input corresponding to the second input and having a bit-width smaller than the fixed bit-width of the multiplication hardware; generating a signed output from said multiplication hardware using at least said first and second inputs; One or more non-transitory machine-readable storage devices for storing instructions capable of causing the execution of operations including:

Images