Signed multiword multiplier

Abstract

To provide a signed multiword multiplier.SOLUTION: Methods, systems, and apparatus include computer programs encoded on a computer storage medium for a hardware circuit configured as a signed multiword multiplier. The circuit includes a processing circuit that receives pieces of input that each have a bit width. The processing circuit can represent at least one input as signed multiword input based on first input having the bit width that exceeds a fixed bit width of the hardware circuit. The circuit includes signed multipliers that are each configured to multiply the signed input. Each signed multiplier includes a multiplication circuit configured to receive the signed multiword input, receive signed second input, and generate signed output in response to multiplying the signed multiword input with the signed second input.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This specification relates to hardware circuits for performing numerical calculations. [Background technology]

[0002] Computational circuits may include multiplication circuits that have hardware multipliers used to multiply numerical inputs such as integers and floating-point numbers. Multiplication circuits can be costly to procure and integrate into existing computational circuits, and some circuits are not efficiently sized for certain applications. For example, some multiplication circuits may include both signed and unsigned multipliers that consume a considerable area of ​​the circuit die, but despite their large size, they do not provide any benefit in terms of computational throughput. Multiplier circuits that are too large for certain computational applications may result in inefficiencies in terms of power consumption and utilization.

[0003] Hardware circuits can be used to implement neural networks. In particular, a neural network with multiple layers can be implemented in a computing circuit that includes several hardware multipliers. The computing circuit of a hardware circuit may also represent a computing unit used to perform neural network computations for a given layer. For example, given an input, the circuit can compute inference about the input using a neural network by performing a dot product operation using one or more multipliers in the computing unit of the hardware circuit. [Overview of the project] [Means for solving the problem]

[0004] This document describes a dedicated hardware circuit for multiplying inputs. The hardware circuit includes a processing circuit that receives inputs, each having a bit width. The processing circuit can represent at least one input as a signed multiword input, based on a first input having a bit width exceeding the fixed bit width of the hardware circuit. The hardware circuit is configured as a signed multiword multiplier and includes signed multipliers, each configured to multiply a signed input. Each signed multiplier includes a multiplier circuit configured to receive a signed multiword input, receive a signed second input, and produce a signed output corresponding to multiplying the signed multiword input by the signed second input.

[0005] One aspect of the subject matter described herein can be embodied in a hardware circuit for multiplying a set of inputs. The hardware circuit includes a processing circuit that receives a first input and a second input, each of the first and second inputs having its own bit width, and the processing circuit is configured to represent at least the first input as a signed multiword input based on the first input having a bit width exceeding the fixed bit width of the hardware circuit; and a plurality of signed multipliers, each of the plurality of signed multipliers is configured to multiply two or more signed inputs, and each signed multiplier includes a multiplier circuit configured to receive a signed multiword input representing the first input, receive a signed second input corresponding to the second input, and produce a signed output corresponding to multiplying the signed multiword input with the signed second input.

[0006] These and other implementations may each optionally include one or more of the following features. For example, in some implementations, the signed multiword input is a shifted signed number containing N words, each of the N words containing B bits, where N is an integer greater than 1 and B is an integer greater than 1. In some implementations, the numerical value of the shifted signed number is a0 + a1 * 2 B +a2*2(2B) +…+a{N-1}*2 {(N-1)B} It is defined based on, where a represents each signed word in a signed multiword input. In some implementations, the typical numerical range of the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) It is defined based on -1-S]. In some implementations, S is 2 (B-1) *(1+2 B +...+2 {(N-2)B} It is defined based on ). In some implementations, the processing circuit is configured to represent the first input as a signed multiword input including a signed high-level word portion and a signed low-level word portion.

[0007] In some implementations, representing the first input as a signed multiword input involves using a quantization scheme to change the data format of the first input based on a fixed bit width of the hardware circuit. In some implementations, the quantization scheme is configured to change the data format of the first input by generating each word portion to represent the first input as a signed multiword input, and the total bit width including each respective word portion is equal to the fixed bit width of the hardware circuit. In some implementations, the signed multiword input includes a plurality of respective words, and a multiplier circuit is configured to produce a signed output by multiplying each word of the signed multiword input with each word of the signed second input. In some embodiments, the signed second input includes a plurality of respective signed words, and the multiplier circuit is configured to produce a signed output as the sum of the respective products calculated from multiplying each word of the signed multiword input with each signed word of the signed second input.

[0008] One aspect of the subject matter described in this specification can be embodied as a method for multiplying a set of inputs using a hardware circuit. The method includes steps of receiving, by a processing circuit of the hardware circuit, a first input and a second input, wherein each of the first and second inputs has a respective bit width, at least the first input has a bit width that exceeds a fixed bit width of a multiplication hardware included in the hardware circuit, and the multiplication hardware is used to multiply the first input and the second input; generating, from at least the first input, a signed multi-word input including a plurality of signed words each having a plurality of bits, wherein a bit width of the signed multi-word input is less than the fixed bit width of the multiplication hardware; providing, to the multiplication hardware for multiplication, the signed multi-word input and a signed second input, wherein the signed second input corresponds to the second input and has a bit width within the fixed bit width of the multiplication hardware; and generating a signed output from the multiplication hardware using at least the first and second inputs.

[0009] In these and other implementations, each can optionally include one or more of the following features. For example, in some implementations, the signed multi-word input is a shifted signed number that includes N words, each of the N words includes B bits, N is an integer greater than 1, and B is an integer greater than 1. In some implementations, the numerical value of the shifted signed number is defined based on a0 + a1*2 B + a2*2 (2B) + … + a{N - 1}*2 {(N-1)B} , where a represents each signed word of the signed multi-word input. In some implementations, a representative numerical range of the shifted signed number is defined based on [-2 (N*B-1) - S, 2 (N*B-1) - 1 - S]. In some implementations, S is 2 (B-1) *(1 + 2 B + … + 2 {(N-2)B}This is defined based on ). In some implementations, the step of generating a signed multiword input includes representing a first input as a signed multiword input comprising a signed high-level word portion and a signed low-level word portion.

[0010] In some implementations, the step of representing a first input as a signed multiword input includes the step of using a quantization scheme to change the data format of the first input based on a fixed bit width of hardware circuitry. In some implementations, the method further includes the step of changing the data format of the first input by generating each word portion to represent the first input as a signed multiword input based on a quantization scheme, such that the total bit width including each respective word portion is equal to the fixed bit width of hardware circuitry. In some embodiments, the signed second input includes a plurality of respective words, and the method further includes the step of using a signed multiplier of multiplication hardware to generate a signed output as the sum of each product of the multiplication of each word of the signed multiword input and each word of the signed second input.

[0011] Other implementations of this and other embodiments include computer programs configured to perform the actions of this method, encoded on corresponding systems, devices, and computer storage devices (e.g., non-temporary machine-readable storage media). A computing system of one or more computers or hardware circuits consists of software, firmware, hardware, or a combination thereof installed on the system, so as to cause the system to perform actions when operating. One or more computer programs consist of instructions, so as to cause a data processing device to perform an action when executed by the device.

[0012] The subject matter described herein can be implemented in specific embodiments to achieve one or more of the following advantages: Using the techniques described, dedicated hardware circuits can be implemented that multiply two or more inputs while requiring less power than conventional circuits used to multiply inputs. The components of the hardware circuits described herein form a signed multiword multiplier circuit having a signed multiplier configured to multiply signed inputs to produce a signed output. The multiword multiplier may be a low-power hardware multiplier circuit that efficiently multiplies several inputs (e.g., floating-point inputs) based on a specific numerical format for representing signed numbers.

[0013] A multiplication circuit can be configured to have multiplication hardware that includes only a signed hardware multiplier for performing multiplication of inputs. The circuit includes a processing circuit used to generate a shifted signed multiword number in response to an input having a conventional number system such as two's complement form. The signed multiword number is multiplied using the signed hardware multiplier to produce a signed output. These features of the multiplication circuit result in reduced power consumption compared to conventional circuits that multiply inputs. This is because the multiplication is completed using only a signed multiplier, rather than both a signed and an unsigned multiplier. Furthermore, a circuit that includes a hardware multiplier to support multiple modes (e.g., signed and unsigned modes) also increases the chip area consumed by the circuit, thereby increasing the manufacturing cost of the circuit. Therefore, the proposed technique achieves reductions not only in power consumption but also in manufacturing costs.

[0014] When the multiplication hardware of the circuit is configured to include only a signed hardware multiplier, the overall hardware circuit consumes far less power than conventional circuits that must include additional multiplication hardware to support both signed and unsigned calculation modes. Thus, this low-power hardware multiplier circuit can be optimized to reduce power requirements and multiply numerical inputs based on at least a signed multiplier configuration that utilizes only the signed mode to generate a product that multiplies two or more signed multi-word inputs.

[0015] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

Brief Description of the Drawings

[0016] [Figure 1] FIG. is a diagram showing an exemplary dedicated hardware circuit for multiplying inputs. [Figure 2] FIG. is a flowchart for generating a signed multi-word input provided to a signed hardware multiplier to generate a signed output. [Figure 3] FIG. is a flowchart showing an exemplary process for multiplying inputs in the described hardware multiplier circuit.

Modes for Carrying Out the Invention

[0017] Like reference numerals and symbols in the various drawings indicate like elements.

[0018] <一 Traditional computer architectures provide multiplication hardware with a fixed bit width B. When these architectures need to multiply an input with more bits than the bit width, the architecture divides the number of bits in the input into multiple parts ("words"), where each word has a length, or bit width B. To produce a computational output, these architectures multiply every word of the first input by every word of the second input. However, to produce a signed (e.g., positive, negative, or zero) output, the architecture must be configurable in both signed and unsigned modes (e.g., where the input is simply positive or zero). Traditional circuits that must be configurable in both signed and unsigned modes require additional hardware components, which increases power consumption.

[0019] In an exemplary implementation, a multilayer neural network can be implemented using hardware circuitry, and computations (e.g., neural network computations) can be performed by processing inputs through each layer of the neural network. In particular, each individual layer of the neural network may have its own set of parameters. Each layer receives an input, processes the input according to the set of parameters for the layer, and produces an output based on computations performed using the multiplication circuits of the exemplary computation unit. For example, a neural network layer computes multiple products, such as when performing matrix multiplication of an input array and a parameter array, or as a part that computes a convolution between an input array and a parameter kernel array.

[0020] Generally, processing inputs through layers of a neural network is achieved using circuits to perform arithmetic operations such as multiplication and addition. An exemplary hardware circuit may include a hardware multiplier for multiplying two or more inputs. Multiplier circuits can be grouped together with hardware adders to form computational units of the hardware circuit, such as matrix or vector processing units. These computational units are used to add and multiply numerical inputs, such as integers and floating-point numbers. Addition and multiplication occur, for example, when hardware circuits are used to perform neural network computations, such as matrix-vector multiplication, in order to process inputs through layers of a neural network.

[0021] Considering the above context, this document describes a technique for implementing dedicated hardware circuits for multiplying two or more inputs represented as signed multiword inputs. Using this technique, signed or unsigned inputs can be represented as "shifted signed multiword numbers." These shifted signed multiword numbers represent the received inputs as signed numbers using a unique numerical format. The received inputs may be individual words of the multiword number and may also include single-word and multiword inputs. By representing the inputs as signed numbers, the dedicated hardware circuit does not need to support unsigned mode. Therefore, the hardware circuit described uses a simpler architecture that includes a multiplication circuit for signed-mode operations rather than operations for both signed and unsigned modes. Because the hardware circuit described is configured only for signed-mode operations, the circuit requires fewer components, which improves power efficiency compared to conventional architectures.

[0022] Figure 1 shows a diagram of an exemplary dedicated hardware circuit 100 for multiplying input 102. In the exemplary implementation, inputs 102A ("Input A") and 102B ("Input B") are each a floating-point number or two's complement, which can be represented in software using a binary data structure. The binary data structure can have a specific number of bits, such as a 16-bit, 24-bit, or 32-bit data structure. For example, each of input A or B may be a signed floating-point number and a sign bit, so that each input can indicate the sign of the input (e.g., positive or negative).

[0023] The data structure for each numeric input can be associated with a specific data format. The data format can represent a finite range of numbers that can be represented using that format. In some implementations, a 16-bit data structure for input A may include a binary input (e.g., 0010) representing the two's complement data format of input A. Regarding the numeric range, a typical two's complement can have the following finite representable range of numbers: [-32.768, 32.767]. Furthermore, each numeric input may have one or more bits in its data structure indicating whether the number is signed or unsigned.

[0024] As described in this document, data structures representing signed numeric inputs (e.g., integers) can hold both positive numbers (e.g., integer values) and negative numbers, while data structures representing unsigned numeric inputs can hold a wider range of positive numbers but cannot hold negative numbers. Generally, processor circuits such as GPUs or neural network processors often include logic operation units (ALUs) or computation units for performing calculations involving different types of inputs, such as integer or floating-point inputs.

[0025] Calculations involving signed inputs correspond to signed-mode arithmetic, while calculations involving unsigned inputs correspond to unsigned-mode arithmetic. ALUs and computing units for performing calculations involving signed and unsigned numeric inputs require separate sets of hardware components to support the respective signed-mode and unsigned-mode arithms. For example, as described above, some computer architectures implement multiplication hardware with a fixed bit width B. When these architectures need to multiply an input with a number of bits exceeding the bit width, the architecture divides the number of inputs into multiple parts ("words"), where each word has a length, or bit width B. To produce a computational output, this architecture multiplies every word of the first input with every word of the second input.

[0026] However, as previously discussed, in order to produce a signed (e.g., positive, negative, or zero) output, the architecture must be configurable in both signed mode and unsigned mode (e.g., where the input is only positive). An architecture that must be configurable in both signed and unsigned operations requires additional hardware components, which will increase power consumption. In this context, a technique is described for implementing a dedicated hardware circuit 100 configured to multiply signed inputs having a unique data format, while consuming less power than conventional hardware circuits. The dedicated circuit 100 includes a multiplication circuit to support only signed-mode operations. When the input is represented only as a signed number, the circuit achieves some power savings. For example, by generating a computed output from multiplying only signed inputs, the circuit 100 can multiply the inputs with fewer hardware components and a smaller set of instructions, resulting in a reduced number of software instructions.

[0027] The circuit 100 includes an input processor 104 configured to generate a signed multiword input. A portion of the hardware circuit 100 may include a computing unit 103 having a multiplication circuit that implements a hardware multiplier for multiplying the input 102. The input processor 104 may be configured to generate a signed multiword input based on a fixed bit width of the multiplication circuit in the computing unit 103 of the circuit 100. More specifically, the input processor 104 is configured to generate a shifted signed multiword number from the input 102. For example, the input processor 104 can generate shifted signed multiword numbers 106 and 108. The shifted signed multiword number 106 may include the respective signed word inputs C and D, each generated from input A, while the shifted signed multiword number 108 may include the respective signed word inputs E and F, each generated from input B.

[0028] Hardware circuit 100 includes signed hardware multipliers 110 and 112. In some implementations, circuit 100 is configured to include low-power signed integer or floating-point multipliers. In some examples, multipliers 110 and 112 can be connected via an optional connection 113 to form a single, large, signed multiplier hardware circuit 100. In some other examples, multipliers 110 and 112 may represent different hardware multipliers of a larger multiplier circuit 114, and circuit 100 may include one or more multiplier circuits 114. While two multipliers are shown in the example in Figure 1, circuit 100 (or circuit 114) can be configured to include more or fewer multipliers. For example, circuit 100 may include a single multiplier configured to be used over time for multiple purposes to achieve the same (or similar) computational effect as multiple individual multipliers. In this configuration, the circuit 100 can be optimized to multiply certain numerical inputs with reduced power requirements by including only the signed multiplier or other hardware components necessary to support only signed-mode arithmetic. In some cases, a dedicated hardware circuit 100 uses the multiplication circuit to perform calculations to process the input through the layers of a neural network. The calculations may include multiplication of the input and parameters to generate cumulative values ​​that are further processed to produce the layer outputs of the neural network layers.

[0029] In an exemplary operation, given a set of inputs that includes signed word inputs C and D (each generated from input A) and signed word inputs E and F (each generated from input B), circuit 100 is configured to multiply input C and E (C*E), multiply input C and F (C*F), multiply input D and E (D*E), and multiply input D and F (D*F). Calculation unit 103 includes an addition circuit 120 (the "adder 120") configured to perform appropriate addition operations among the products generated by one or more multipliers 110, 112 of multiplication circuit 114. Calculation unit 103 is configured to perform an addition operation after shifting the values of one or more products by a required bit width. For example, calculation unit 103 can perform shift operations (such as <<2*B, <<B, etc.) before using adder 120 to perform the following addition operation: (C*E<<(2*B))+((C*F+D*E)<<B)+D*F.

[0030] Adder 120 receives signed products 116 and 118 as inputs, adds signed products 116 and 118, and generates a signed output 122 of calculation unit 103. In some implementations, a two's complement version of negative signed product 118 is used to perform an addition operation that includes adding signed product 116 and the two's complement version of signed product 118 to generate signed output 122. In some cases, adding the inputs may include using rounding logic to perform a rounding operation on a provisional sum before generating signed output 122. For example, the rounding logic can be used to round the provisional sum to the nearest decimal or integer value before generating signed output 122. In some implementations, signed output 122 represents a cumulative value for generating a layer output of a neural network layer in response to processing numerical input 102 through the neural network layer.

[0031] Figure 2 shows a process diagram 200 for generating a signed multiword input, which is provided to the signed hardware multiplier of circuit 100 to produce a signed output 122. As described in more detail below, the process diagram 200 includes multiple logic blocks, each representing a different logic function of the input processor 104. Generally, one or more of each logic functions can be used to generate a shifted signed multiword number.

[0032] Referring to process diagram 200, the hardware circuit 100 is configured as a signed-mode circuit and includes an input processing circuit 104 for generating a signed multiword number 106. The input processor 104 generates a signed multiword number shifted from input 102, at least based on the determination that the input has a bit width that exceeds the fixed bit width of the hardware multiplier included in the hardware circuit (204). For example, the input processor 104 can analyze the binary data structure of input 102 to determine whether each input exceeds the fixed bit width of the multiplication circuit 114 included in the computing unit 103.

[0033] Generating a signed multiword number 106 involves generating the number 106 based on the input processor 104 determining that the input 102 falls within a predetermined numerical range of the data format used to represent the shifted signed multiword number 106 (206). For example, the input processor 104 generates the signed multiword number 106 in accordance with determining whether the numerical value of the input 102, for example, two's complement, matches within the available numerical range of the data format that represents the shifted signed multiword number 106. If the input processor 104 determines that, for a given input 102, the numerical value of the input 102 does not match within the available numerical range of the data format, the input processor 104 terminates process 200 (208).

[0034] If the input processor 104 determines that the input 102 is within a predetermined numerical range of the data format, the input processor 104 causes one or more inputs to be represented as signed multiword inputs, based on at least one input having a bit width exceeding the fixed bit width of the hardware circuit 100. For example, to represent an input as a signed multiword input, the input processor 104 generates N signed words, each having B bits (210). The input processor 104 then uses each of the N signed words, each having B bits, to generate a shifted signed number (212). In some implementations, N is an integer greater than 1 and B is an integer greater than 1. The signed multiword inputs are provided to the signed hardware multiplier of the multiplication circuit 114, which ultimately produces a signed output.

[0035] In some cases, the input processor 104 determines that input 102 has a bit width that does not exceed the fixed bit width of the hardware multiplier 110 included in the hardware circuit (205). In this scenario, the input processor 104 provides input 214 to the signed multiplier of the multiplication circuit 114. For example, the input processor 104 may provide input 214 to a specific hardware multiplier based on the fact that the sign of the input matches the sign of that particular hardware multiplier. In this implementation, input 214 is not a suitable input for generating a signed multiword input because it does not have a bit width greater than the fixed bit width of the multiplication circuit 114.

[0036] In an exemplary multiplication operation, the decision to generate a shifted signed multiword number from input 102, and the subsequent generation of the signed multiword input, may occur relatively early in the computation cycle. For example, the decision can be made off-chip using an external host controller communicating with circuit 100 to obtain the input for processing through the neural network layer. In some implementations, the decision and subsequent generation occur once the input is obtained from the memory of the exemplary neural network processor, such as an activation memory that stores activations generated by the neural network layer implemented on the neural network processor, including the hardware circuit 100.

[0037] In other implementations, the decision to generate a signed multiword input, and the subsequent generation of the signed multiword input, may be performed in an earlier pipeline stage, such as a previous multiplier, ALU, or bypass circuit for the compute unit 103. In some cases, the interface of each signed hardware multiplier 110, 112 can be modified or extended to include their respective input processors 104. In such cases, the input 102 received at the inputs of each multiplier 110, 112 can be processed to generate a suitable number of shifted multiword inputs for multiplication by the respective hardware multipliers 110, 112.

[0038] Figure 3 shows a flowchart of an exemplary process 300 for multiplying an input using the hardware multiplier circuit 100 described. As shown above, the input may be a numerical input, such as a floating-point number represented as bits of a data structure, such as 16-bit or 32-bit. Process 300 can be carried out by using at least the circuit 100 in combination with other circuits, components, and systems described in this document.

[0039] Referring here to process 300, circuit 100 receives a first input and a second input, each having a bit width (302). The processing circuit is configured to represent at least the first input as a signed multiword input, based on the first input having a bit width exceeding the fixed bit width of the hardware circuit. For example, the fixed bit width of the hardware circuit may be 16 bits, while the bit width for an exemplary data structure of the first input is 32 bits.

[0040] Circuit 100 generates a signed multiword input (304) from at least a first input, which includes multiple signed words, each having multiple bits. The signed multiword input / number is a shifted signed number containing N words, each of which contains B bits. Generally, N may be an integer greater than 1, and B may be an integer greater than 1. For example, by analyzing the data structure of the first input, input processor 104 may determine that the first input consists of 32 bits. Input processor 104 may determine or calculate the difference between the number of bits in the first input and the number of bits for the fixed bit width of the hardware circuit.

[0041] The input processor 104 can generate signed multiword numbers based on the calculated difference. In some implementations, each word of the signed multiword number is generated using a portion of the bits that form a 32-bit data structure of the first input 102. For example, a signed multiword number can be formed from four 8-bit numbers or two 16-bit numbers. These numbers can correspond to the signed multiword numbers 106 and 108 described above. In some cases, each word of the signed multiword number is a signed word that includes a portion of the bits from the first input and a corresponding sign bit indicating the sign of the signed word forming the signed multiword number.

[0042] In some implementations, when a shifted signed multiword number is formed from four 8-bit numbers, this shifted signed number contains N=4 words, where each N word contains B=8 bits. This "shifted signed N-word 8-bit number" is represented by N ordinary signed numbers, each with a bit width B. For example, let a0, a1, ..., a{N-1} be their ordinary signed numbers, and let a be the shifted signed number that each of these numbers represents together. The numerical value u of the shifted signed number is defined as follows: a = a0 + a1 * 2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} Here, a represents each signed word of the signed multiword input. The individual words a0, a1, ..., a{N-1} are each signed numbers. In some other implementations, the original input number is zero-extended (e.g., a "0" bit is added to the most significant bit) or sign-extended (e.g., the most significant bit of the original input number is copied to the excess bits) until the bit width is a multiple of B.

[0043] As discussed above, a data format can have a finite range of numbers that can be represented using the data format. In some implementations, a shifted signed multiword number is defined based on an exemplary known representation for representing the usual two's complement numerical range, but with an additional parameter S, and has a representable numerical range. The numerical range of a shifted signed multiword number is [-2 (N*B-1) -S, 2 (N*B-1) This is obtained using -1-S]. The parameter S provides a shift function to the known representation for representing the numerical range for two's complement. For example, when B=8 and N=2, the usual two's complement has a representation range of [-32.768, 32.767]. This range for the usual two's complement is obtained using the known representation, [-2 (N*B-1) , 2 (N*B-1)-1] is used to obtain the known representation by a distance S to the left (e.g., toward negative infinity) for the typical N-word * B-bit two's complement representation range, with respect to the specific data format described in this document. The parameter S is used to shift the known representation by a distance S to the left (e.g., toward negative infinity) for the usual N-word * B-bit two's complement representation range. In some implementations, S and the corresponding shift are 2 (B-1) *(1+2 B +...+2 {(N-2)B} It is defined based on the following.

[0044] In some implementations, the hardware circuit 100 and input processor 104 modify the data format of the first input based on a fixed bit width of the hardware circuit using a quantization scheme. The quantization scheme is configured to modify the data format of the first input by generating each word portion to represent the first input as a signed multiword input. For example, the data format for generating signed multiword numbers from parameters or kernel weight values ​​for a neural network layer can be modified based on a particular quantization scheme, so that the parameters can be suitably used to compute the output for the layer. In the generated signed multiword input, the total bit width including each word portion may be equal to the fixed bit width of the hardware circuit. In some implementations, the input processor 104 is configured to tune some kind of software scheme to requantize or modify how the parameters and weights are obtained and processed in the circuit 100.

[0045] Circuit 100 provides multiplication hardware for multiplication with a signed multiword input and a signed second input (306). The signed second input corresponds to the received second input. In some implementations, the second input can correspond to a signed input that does not exceed the bit width of the hardware circuit or another shifted signed multiword number. In some other implementations, the second input corresponds to a signed input that exceeds the bit width of the hardware circuit, and so circuit 100 generates a signed multiword number from the second input.

[0046] Circuit 100 generates a signed product from multiplication hardware using at least a first and a second input (308). For example, circuit 100 generates a signed product 116 or 118 depending on the number of shifted signed multiword numbers in the first input multiplied by the number of shifted signed multiword numbers in the second input. These shifted signed multiword inputs contain multiple respective words, and the multiplication circuit 114 is configured to generate the signed product by multiplying each word of the first input of the signed multiword by each word of the second input of the signed multiword. The advantage of shifted signed multiword numbers is that they can be multiplied without the need for an unsigned hardware multiplier. For example, to calculate the signed product 116 of two such numbers a and b, the following equation is obtained: a = a0 + a1 * 2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} b = b0 + b1 * 2 B +b2*2 (2B) +…+b{N-1}*2 {(N-1)B} Hardware circuit 100 is a i *b j Calculate the product of [the two values]. This can all be calculated using the signed hardware multiplier of circuit 100.

[0047] Several embodiments are described. Nevertheless, it will be understood that various modifications can be made without departing from the scope of the invention. For example, the various forms of the flow shown above can be used by rearranging, adding, or removing steps. Thus, other embodiments are within the scope of the following claims. While this specification includes many specific implementation details, these should not be considered as limitations on the scope of the claims, but rather as descriptions of features that may be specific to particular embodiments. Certain features described herein in the context of individual embodiments can also be implemented in combination within a single embodiment.

[0048] Conversely, various features described in the context of a single embodiment can be implemented separately in multiple embodiments or in any preferred sub-combination. Furthermore, features may be described above to operate in certain combinations, and may even be initially claimed as such, but one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may cover a sub-combination or a variation of a sub-combination.

[0049] Similarly, while operations are depicted in a specific order, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all depicted operations be performed, in order to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Furthermore, the demarcations of various system modules and components in the embodiments described above should not be understood as requiring such demarcations in all embodiments, and the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0050] Specific embodiments of this subject matter have been described. Other embodiments fall within the scope of the following claims. For example, the actions mentioned in the claims can still be performed in a different order to achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require a specific order or sequential sequence to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. [Explanation of symbols]

[0051] 100 Dedicated Hardware Circuit 102 inputs 102A Input 102B input 103 Computing Units 104 Input processor, processing circuit, signed multiword generator 106 Shifted signed multiword count 108 Shifted signed multiword count 110 Signed Hardware Multiplier_1 112 Signed Hardware Multiplier_2 113 Connection 114 Multiplier circuits, signed hardware multipliers 116 Signed product_1 118 Signed product_2 120 Adder circuit, adder 122 Signed output 200 Process Diagrams 214 inputs 300 processes

Claims

1. A hardware circuit for multiplying a set of inputs, wherein the hardware circuit is A processing circuit that receives a first input and a second input, wherein each of the first and second inputs has a corresponding bit width and is either a signed input or an unsigned input. The processing circuit is configured to generate a first shifted signed multiword number based on the determination that the first input exceeds a fixed bit width. The processing circuit is configured to generate a second shifted signed multiword number from the second input based on the determination that the second input exceeds a fixed bit width. One or more signed hardware multipliers, wherein the one or more signed hardware multipliers are Receiving the first shifted signed multiword number, Receiving the second shifted signed multiword number, One or more signed hardware multipliers configured to generate a signed output corresponding to multiplying the first shifted signed multiword number by the second shifted signed multiword number, A hardware circuit equipped with this feature.

2. The hardware circuit according to claim 1, wherein each of the one or more signed hardware multipliers includes a low-power signed integer multiplier or a floating-point multiplier.

3. The hardware circuit according to claim 1, comprising two or more signed hardware multipliers, wherein the two or more signed hardware multipliers are connected to form a single, large multiplication circuit.

4. The hardware circuit according to claim 1, comprising an adder circuit configured to perform an addition operation between products generated by one or more signed hardware multipliers.

5. The hardware circuit according to claim 4, wherein the addition operation includes adding a first signed product from one or more signed hardware multipliers to a two's complement version of a second signed product from one or more signed hardware multipliers.

6. The hardware circuit according to claim 4, wherein the addition operation includes a rounding operation on the provisional sum from the addition circuit.

7. The first shifted signed multiword number contains N words, each of which contains B bits. The hardware circuit according to claim 1, wherein N is an integer greater than 1 and B is an integer greater than 1.

8. The shifted signed number is a0 + a1 * 2 B +a2*2 (2B) +…+a{N-1}*2 {(N-1)B} It is stipulated based on, The hardware circuit according to claim 7, wherein a represents each of the signed words of the signed multiword input.

9. The typical numerical range of the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) The hardware circuit according to claim 8, defined based on [-1-S].

10. S, 2 (B-1) *(1+2 B +...+2 {(N-2)B} The hardware circuit according to claim 8, as defined based on ).

11. The first shifted signed multiword number is Signed high-level word part and Signed low-level word portion and The hardware circuit according to claim 1, including the following:

12. A method for multiplying a set of inputs using hardware circuitry, A step of receiving a first input and a second input, wherein each of the first and second inputs has its own bit width and is either a signed input or an unsigned input, The steps include generating a first shifted signed multiword number based on the determination that the first input exceeds a fixed bit width, A step of generating a second shifted signed multiword number from the second input based on the determination that the second input exceeds a fixed bit width, The steps include receiving the first shifted signed multiword number in one or more signed hardware multipliers, The steps include receiving the second shifted signed multiword number in one or more signed hardware multipliers, The steps of the one or more signed hardware multipliers, which generate a signed output corresponding to multiplying the first shifted signed multiword number by the second shifted signed multiword number, A method that includes [a certain feature].

13. The method according to claim 12, further comprising the step of performing an addition operation between products generated by the one or more signed hardware multipliers.

14. The method according to claim 13, wherein the addition operation includes adding a first signed product from one or more signed hardware multipliers to the two's complement version of a second signed product from one or more signed hardware multipliers.

15. The method according to claim 14, wherein the addition operation includes a rounding operation on the provisional sum.

16. The first shifted signed multiword number contains N words, each of which contains B bits. The method according to claim 12, wherein N is an integer greater than 1 and B is an integer greater than 1.

17. The numerical value of the shifted signed number is a0 + a1 * 2 B + a2 * 2 (2B) + … + a{N - 1} * 2 {(N-1)B} is defined based on The method according to claim 16, wherein a represents each of the signed words in the signed multiword input.

18. The typical numerical range of the shifted signed number is [-2 (N*B-1) -S, 2 (N*B-1) The method according to claim 17, as defined by -1-S.

19. S, 2 (B-1) *(1+2 B +...+2 {(N-2)B} The method according to claim 18, as defined based on ).

Images