Device and method for accelerating matrix multiplication

Abstract

The invention relates to an apparatus and method for accelerating matrix multiplication. Methods and apparatus relating to performing vector multiplication are described. Hardware accelerators that perform vector multiplication are also described. In one embodiment, a combined fixed-point and floating-point vector multiplication circuit includes at least one switch to switch the circuit between a first mode in which the fixed-point and floating-point vector multiplication is performed and a second mode in which the fixed-point and floating-point vector multiplication is performed. Each multiplier in a set of multipliers multiplies mantissas from the same element positions of the first floating point vector and the second floating point vector to produce a corresponding product, which is shifted with a set of shift registers based on a maximum exponent of the exponents of the corresponding product determined by the maximum exponent determiner to produce a shifted product, a numerical conversion operation is performed on the shifted product using a set of numerical conversion circuits based on sign bits from the same element positions of the first floating point vector and the second floating point vector to produce a signed representation of the shifted product.

Description

[0001] Division Case Explanation

[0002] This application is a divisional application of the invention patent application with the application date of September 27, 2018, application number 201880096937.X, and title "Apparatus and Method for Accelerating Matrix Multiplication". Technical Field

[0003] The present disclosure generally relates to electronic products, and more particularly, embodiments of the present disclosure relate to a hardware accelerator for performing vector multiplication. Background Art

[0004] A processor or a collection of processors executes instructions from an instruction set, such as an instruction set architecture (ISA). An instruction set is a part of a computer architecture related to programming and generally includes native data types, instructions, register architectures, addressing modes, memory architectures, interrupt and exception handling, and external input and output (I / O). It should be noted that the term instruction here can refer to a macro instruction, such as an instruction provided to the processor for execution, or a micro instruction, such as an instruction generated by decoding a macro instruction by a decoder of the processor. Summary of the Invention

[0005] According to an embodiment of the present disclosure, there is provided an apparatus including: a plurality of multipliers for calculating a product by multiplying integer values corresponding to input data elements of a neural network operation; a plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in a floating-point mode to shift the product based on the maximum exponent of the product, and when the input data element is an integer data element, the shifter operates in an integer mode to bypass shifting the product; one or more adders for generating a sum from the output of the shifter; and an accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums.

[0006] According to an embodiment of the present disclosure, there is provided an apparatus, including: a data storage unit for storing input data elements for neural network operations; a plurality of multipliers for calculating products by multiplying integer values corresponding to the input data elements, the integer values being represented by bits obtained from the data storage unit; a plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in a floating-point mode to shift the product based on the maximum exponent of the product, and when the input data element is an integer data element, the shifter operates in an integer mode to bypass shifting the product; one or more adders for generating a sum from the output of the shifter; and an accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums.

[0007] According to an embodiment of the present disclosure, there is provided an apparatus, including: a plurality of multipliers for calculating products by multiplying integer values corresponding to input data elements of a neural network operation; an exponent unit for operating in different modes for different data precisions, wherein the exponent unit determines the maximum exponent of the product in floating-point mode and is bypassed in integer mode; a plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in a floating-point mode to shift the product based on the maximum exponent of the product, and when the input data element is an integer data element, the shifter operates in an integer mode to bypass shifting the product; one or more adders for generating a sum from the output of the shifter; and an accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure is illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals indicate like elements and in which:

[0009] Figure 1 Embodiments according to the present disclosure illustrate a hardware processor having a hardware accelerator.

[0010] Figure 2 Embodiments according to the present disclosure illustrate a hardware processor and a hardware accelerator.

[0011] Figure 3 Embodiments according to the present disclosure illustrate a fixed-point multiplier-accumulator (MAC) tree circuit.

[0012] Figure 4Embodiments in accordance with the present disclosure illustrate a fixed-point accumulator circuit having a plurality of fixed-point multiplier-accumulator (MAC) tree circuits.

[0013] Figure 5 Embodiments in accordance with the present disclosure illustrate a floating-point adder circuit.

[0014] Figure 6 Embodiments in accordance with the present disclosure illustrate a floating-point multiplier circuit.

[0015] Figure 7 Embodiments in accordance with the present disclosure illustrate a floating-point multiplier-accumulator (MAC) tree circuit.

[0016] Figure 8 Embodiments in accordance with the present disclosure illustrate a floating-point accumulator circuit having a plurality of floating-point multiplier-accumulator (MAC) tree circuits.

[0017] Figure 9 Embodiments in accordance with the present disclosure illustrate a floating-point matrix multiplication circuit.

[0018] Figure 10 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit having a fixed-point accumulator circuit and a floating-point accumulator circuit.

[0019] Figure 11 Embodiments in accordance with the present disclosure illustrate a four-by-four fixed-point and floating-point vector multiplication circuit having a fixed-point accumulator circuit and a floating-point accumulator circuit.

[0020] Figure 12 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit having a fixed-point accumulator circuit and a floating-point conversion circuit to convert fixed-point to floating-point at the end of accumulation.

[0021] Figures 13A - 13C Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit subdivided into a plurality of segments, where each segment has its own maximum exponent.

[0022] Figures 14A - 14C Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit subdivided into a plurality of segments, where each segment has its own maximum exponent and the segments are sorted by the sum of the exponents.

[0023] Figure 15 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit including a rounding circuit having a fixed-point accumulator circuit and a floating-point accumulator circuit.

[0024] Figure 16 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point matrix multiplication circuit.

[0025] Figure 17 A flowchart is illustrated according to an embodiment of the present disclosure.

[0026] Figure 18A is a block diagram illustrating a general vector friendly instruction format and its class A instruction templates according to an embodiment of the present disclosure.

[0027] Figure 18B is a block diagram illustrating a general vector friendly instruction format and its class B instruction templates according to an embodiment of the present disclosure.

[0028] Figure 19A is illustrated according to an embodiment of the present disclosure Figure 18A and Figure 18B a block diagram of fields of a general vector friendly instruction format in.

[0029] Figure 19B is a block diagram illustrating the Figure 19A fields of a specific vector friendly instruction format in that make up a complete opcode field according to an embodiment of the present disclosure.

[0030] Figure 19C is a block diagram illustrating the Figure 19A fields of a specific vector friendly instruction format in that make up a register index field according to an embodiment of the present disclosure.

[0031] Figure 19D is a block diagram illustrating the Figure 19A fields of a specific vector friendly instruction format in that make up an enhanced operation field 1850 according to an embodiment of the present disclosure.

[0032] Figure 20 is a block diagram of a register architecture according to an embodiment of the present disclosure.

[0033] Figure 21A is a block diagram illustrating both a exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline according to an embodiment of the present disclosure.

[0034] Figure 21B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core to be included in a processor and an exemplary register renaming, out-of-order issue / execution architecture core according to an embodiment of the present disclosure.

[0035] Figure 22A is a block diagram of a single processor core and its connection to an on-chip interconnect network and its local subset of a level 2 (L2) cache according to an embodiment of the present disclosure.

[0036] Figure 22B is according to an embodiment of the present disclosure Figure 22AAn expanded view of a portion of a processor core therein.

[0037] Figure 23 is a block diagram of a processor according to an embodiment of the present disclosure that may have more than one core, may have an integrated memory controller, and may have integrated graphics.

[0038] Figure 24 is a block diagram of a system according to an embodiment of the present disclosure.

[0039] Figure 25 is a block diagram of a more specific exemplary system according to an embodiment of the present disclosure.

[0040] Figure 26 illustrates a block diagram of a second more specific exemplary system according to an embodiment of the present disclosure.

[0041] Figure 27 illustrates a block diagram of a system on chip (SoC) according to an embodiment of the present disclosure.

[0042] Figure 28 is a block diagram contrasting an embodiment of the present disclosure with the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set. Detailed Description

[0043] In the following description, numerous specific details are set forth. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0044] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but not necessarily every embodiment includes that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is considered within the knowledge of one of ordinary skill in the art to implement such feature, structure, or characteristic in connection with other embodiments (whether or not explicitly described).

[0045] (For example, a hardware) processor (e.g., having one or more cores) can execute instructions (e.g., threads of instructions) to operate on data, e.g., to perform arithmetic, logical, or other functions. For example, software can request an operation and the hardware processor (e.g., its one or more cores) can execute the operation in response to the request. In one embodiment, the processor is coupled to (e.g., on-chip or off-chip) an accelerator (e.g., a load transfer engine) to perform one or more (e.g., load transfer) operations, e.g., instead of performing these operations only on the processor. In one embodiment, the processor includes (e.g., on-chip or off-chip) an accelerator (e.g., a load transfer engine) to perform one or more operations, e.g., instead of performing these operations only on the processor. Two non-limiting examples of operations are vector multiplication operations and matrix multiplication operations. In one embodiment, the accelerator executes the operation in response to a request for the processor (e.g., a central processing unit (CPU)) to perform the operation. The accelerator can be a hardware accelerator (e.g., an accelerator circuit). The accelerator can be coupled to a data storage device (e.g., on-chip or off-chip with the accelerator) to store data, e.g., input data and / or output data. In one embodiment, the accelerator is coupled to an input data storage device (e.g., an input buffer) to load input therefrom. In one embodiment, the accelerator is coupled to an output data storage device (e.g., an output buffer) to store output thereon. The processor can execute instructions to transfer a load of one or more operations (e.g., instructions, threads of instructions, or other work) to the accelerator. The operations can be performed on numerical data having different formats (e.g., representations) in a computing system (e.g., the accelerator and / or the processor). In certain embodiments, the numbers take a fixed-point format or a floating-point format. Integers can be represented in binary format. Signed integers can be represented in binary (2's complement) format (e.g., where a leading zero indicates a positive integer and a leading one indicates a negative integer). (e.g., real) numbers can be represented in floating-point format to represent numbers of different orders of magnitude, e.g., with a fixed number of digits.

[0046] An example of a numerical format is where numbers are generally approximated to a fixed number of significant digits (significand) and are scaled using an exponent with a certain fixed base (e.g., a base of 2, 10, or 16). An example of a numerical format where S represents the sign bit, M represents the mantissa, and E represents the exponent is as follows:

[0047] x = significand x base exponent (1)

[0048] Examples of floating-point formats are as follows:

[0049] x = (-1)s x1.Mx2 E-bias (2)

[0050] According to the IEEE 754 standard for binary floating-point (FP) arithmetic, the mantissa is an unsigned number (e.g., a binary fraction) and a normalized floating-point number has a single one in the most significant bit (MSB) position. In some embodiments, this bit (e.g., this bit to the left of the decimal point) is implicit and thus the mantissa does not need to store it. In some embodiments, the exponent is represented here as a non-negative integer from which a constant bias is subtracted. Examples of floating-point formats are floating-point 16 (e.g., binary16), floating-point 32 (e.g., binary32), floating-point 64 (e.g., binary64), floating-point 128 (e.g., binary128), and floating-point 256 (e.g., binary256), although any number of sign, significant digit (e.g., its mantissa), or exponent bits may be used in some embodiments. In one embodiment, the binary16 format has one bit for the sign bit, 5 bits for the exponent, and 11 bits implicit (10 bits stored explicitly) for the significant digit. In one embodiment, the binary32 format has one bit for the sign bit, 8 bits for the exponent, and 24 bits implicit (23 bits stored explicitly) for the significant digit. In one embodiment, the binary64 format has one bit for the sign bit, 11 bits for the exponent, and 53 bits implicit (52 bits stored explicitly) for the significant digit. In one embodiment, the binary128 format has one bit for the sign bit, 15 bits for the exponent, and 113 bits implicit (112 bits stored explicitly) for the significant digit. In one embodiment, the binary256 format has one bit for the sign bit, 19 bits for the exponent, and 237 bits implicit (236 bits stored explicitly) for the significant digit.

[0051] Another digital format is Flexpoint (Flex). Flexpoint is based on numbers that have an N-bit mantissa storing the integer value in two's complement form, and an M-bit exponent e shared among all elements of the tensor. This format is denoted as FlexN+M. As an example, a Flexpoint format with a 16-bit mantissa and a 5-bit exponent can be called flex16+5, in contrast to the floating-point (FP) format. Flex number representation has a shared exponent for a given tensor. In some embodiments, the vector (and / or matrix) multiplication accelerator for Flexpoint is based on integer circuitry (e.g., pipelined) (e.g., rather than a pure floating-point pipeline), since Flexpoint is substantially an integer-based system.

[0052] In some embodiments, a vector (and / or matrix) multiplication accelerator is included to perform operations on numbers in more than one format (e.g., fixed-point, inflection point, or floating-point). In some embodiments, a vector (and / or matrix) multiplication accelerator is included to perform operations on numbers in any one (e.g., all) of the fixed-point, inflection point, or floating-point formats.

[0053] Figure 1 Embodiments in accordance with the present disclosure illustrate a hardware processor 100 with a hardware accelerator 104. The hardware processor 100 (e.g., core 102) may receive a request (e.g., from software) to execute a vector multiplication (or matrix multiplication) thread (e.g., operation) and may offload the vector multiplication (or matrix multiplication) thread (e.g., operation) (e.g., at least a portion thereof) to the hardware accelerator (e.g., hardware accelerator 104). The hardware processor 100 may include one or more cores (0 to N). In one embodiment, each core may communicate (e.g., be coupled to) the hardware accelerator 104. In one embodiment, each core may communicate (e.g., be coupled to) one of a plurality of hardware decompression accelerators. The (one or more) cores, the (one or more) accelerators, the data storage device 106, and the registers 110 may communicate (e.g., be coupled to) with each other. The arrows indicate two-way communication (e.g., to and from components), but one-way communication may be used. In one embodiment, the (e.g., each) core may communicate (e.g., be coupled to) the data storage device, e.g., to store and / or output the data stream 108. The hardware accelerator may include any hardware (e.g., circuitry or circuit system) discussed herein. In one embodiment, the (e.g., each) accelerator may communicate (e.g., be coupled to) the data storage device, e.g., to receive a stream of vector data (e.g., its elements).

[0054] Figure 2Embodiments in accordance with the present disclosure illustrate a hardware processor 201 and a hardware accelerator 204. In one embodiment, the hardware accelerator is on-chip with the hardware processor. In one embodiment, the hardware accelerator is off-chip from the hardware processor. In one embodiment, a system 200 including at least the hardware processor 201 and the hardware accelerator 204 is a SoC. The hardware processor 201 (e.g., core 202) may receive a request (e.g., from software) to execute a vector (and / or matrix) thread (e.g., operation) and may offload at least a portion of the thread (e.g., operation) to the hardware accelerator (e.g., hardware accelerator 204). The hardware processor 201 may include one or more cores (0 to N). In one embodiment, each core may communicate (e.g., be coupled to) the hardware decompression accelerator 204. In one embodiment, each core may communicate (e.g., be coupled to) one of a plurality of hardware accelerators. The (one or more) cores, the (one or more) accelerators, the data storage device 206, and the registers 210 may communicate (e.g., be coupled) with each other. Arrows indicate two-way communication (e.g., to and from components), but one-way communication may be used. In one embodiment, (e.g., each) core may communicate (e.g., be coupled to) the data storage device, e.g., to store and / or output a data stream 208. The hardware accelerator may include any hardware (e.g., circuitry or circuit system) discussed herein. In one embodiment, (e.g., each) accelerator may communicate (e.g., be coupled to) the data storage device, e.g., to receive a stream of vector data (e.g., its elements). The data stream 208 may be pre-loaded into the data storage device 206, e.g., by a hardware compression accelerator or the hardware processor.

[0055] The hardware accelerator may include one or more fixed-point (e.g., only fixed-point) adder circuits and / or fixed-point (e.g., only fixed-point) multiplier circuits. The hardware accelerator may include one or more floating-point (e.g., only floating-point) adder circuits and / or floating-point (e.g., only floating-point) multiplier circuits.

[0056] Figure 3Embodiments in accordance with the present disclosure illustrate a fixed-point multiplier and accumulator (MAC) tree circuit. In one embodiment, the MAC tree circuit 300 only takes in fixed-point numbers as inputs 302 (shown as 16 inputs). In one embodiment, the left input in each pair of inputs of the multiplier 304 is the corresponding element of the first input vector and the right input in each pair of inputs of the multiplier 304 is the corresponding element of the second input vector, e.g., two input vectors each having eight fixed-point elements (e.g., indexed 0-7). The depicted MAC tree circuit 300 will multiply the input pairs 302 together and then add these products together using the adder 306 to generate a single result 308.

[0057] Figure 4 A fixed-point accumulator circuit 400 (e.g., an accelerator) having a plurality of fixed-point (e.g., integer) multiplier and accumulator (MAC) tree circuits 402 is illustrated in embodiments in accordance with the present disclosure. In one embodiment, each MAC tree circuit 402 is an instance of the fixed-point (e.g., integer) MAC tree circuit 300 from Figure 3 In the depicted embodiment, each output from the MAC tree circuits 402 is added together using the adder 404 to generate a single result 408, which can be accumulated with a previous result using the accumulator circuit 410.

[0058] Figure 5Embodiments in accordance with the present disclosure illustrate a floating-point adder circuit 500. The floating-point adder circuit 500 will add a first floating-point value (first input) 502 and a second floating-point value (second input) 504 to produce a resulting floating-point value (output) 506. The values can take any floating-point format, e.g., depicted with (one or more) sign fields (e.g., SignA and Sign B for inputs 502 and 504 respectively), exponent fields (exp) (e.g., ExpA and ExpB for inputs 502 and 504 respectively), and mantissa fields (mantissa). A comparator (comp) 508 will compare the exponents of the first floating-point value (first input) 502 and the second floating-point value (second input) 504, and if the exponent of the second input value 506 is greater than the exponent of the first input value 504, swap the mantissas of the first input value 502 and the second input value 504 in a swap register 510. A shift register 512 can shift the mantissa to align its exponent based on the exponent of the other mantissa (e.g., such that the mantissa of the lower number is aligned to the same magnitude as the larger number). An adder 514 (e.g., a two's complement adder or a fixed-point adder) then adds the mantissa values of the first floating-point value (first input) 502 and the second floating-point value (second input) 504, and a leading zero detection, normalization, and rounding circuit 516 generates the resulting floating-point value (output) 506. In some embodiments, the most significant bit of the significant digits that require non-zero digits is non-zero (e.g., except when the corresponding exponent will be less than a minimum value of one). This process is generally referred to as normalization. For a binary format (e.g., which only uses the digits 0 and 1), this non-zero digit will necessarily be 1. Thus, it does not need to be explicitly represented and allows the format to have one extra bit of implicit precision. This can also be referred to as a leading bit convention, an implicit bit convention, a hidden bit convention, or a presumed bit convention.

[0059] Figure 6 Embodiments in accordance with the present disclosure illustrate a floating-point multiplier circuit 600. The floating-point multiplier circuit 600. The floating-point multiplier circuit 600 will multiply a first floating-point value (first input) 602 and a second floating-point value (second input) 604 to produce a resulting floating-point value (output) 606. An adder circuit 614 will add together the exponents from the first floating-point value (first input) 602 and the second floating-point value (second input) 604. A multiplier 618 (e.g., a two's complement multiplier or a fixed-point multiplier) will multiply together the mantissas from the first floating-point value (first input) 602 and the second floating-point value (second input) 604. A leading zero detection, normalization, and rounding circuit 616 generates the resulting floating-point value (output) 606.

[0060] Figure 7Embodiments in accordance with the present disclosure illustrate a floating-point multiplier-accumulator (MAC) tree circuit 700. In one embodiment, the MAC tree circuit 700 takes only floating-point numbers as inputs 702 (shown as 16 inputs). In one embodiment, the left input in each pair of inputs of the multiplier 704 is the corresponding element of a first input vector and the right input in each pair of inputs of the multiplier 704 is the corresponding element of a second input vector, e.g., two input vectors each having eight floating-point elements (e.g., indexed 0 - 7). Note that each element of the floating-point number may include multiple fields (e.g., a sign field, an exponent field, and a mantissa field for each element). The depicted MAC tree circuit 700 will multiply pairs of inputs 702 together and then add these products together using adders 706 to generate a single result 708. In one embodiment, each multiplier 704 is Figure 6 an instance of the floating-point multiplier circuit 600 of Figure 5 and / or each adder 706 is an instance of the floating-point adder circuit 500 of

[0061] Figure 8 Embodiments in accordance with the present disclosure illustrate a floating-point accumulator circuit 800 (e.g., an accelerator) having a plurality of floating-point multiplier-accumulator (MAC) tree circuits 802. In one embodiment, each MAC tree circuit 802 is an instance of the (e.g., floating-point) MAC tree circuit 700 of Figure 7 and / or each adder 804 is an instance of the floating-point adder circuit 500 of Figure 5 In the depicted embodiment, each output from the MAC tree circuit 802 is added together using adder 804 to generate a single result 804, which can be accumulated with a previous result using accumulator circuit 810.

[0062] Figure 9 Embodiments in accordance with the present disclosure illustrate a floating-point matrix multiplication circuit 900. The floating-point matrix multiplication circuit 900 includes a plurality (e.g., 32, indexed 0 - 31) of floating-point accumulator circuits 902 and may include a set of adders 904 to allow summing of the accumulated outputs of the floating-point accumulator circuits 902. In one embodiment, each floating-point accumulator circuit 902 is Figure 8 an instance of the floating-point accumulator circuit 800 of Figure 7 e.g., where each of four MAC trees is an instance of the MAC tree circuit 700 of

[0063] In some embodiments, the circuit includes multiple occurrences of exponent comparison, shifting, leading zero detection, and / or normalization (e.g., for each addition within a MAC tree and / or accumulator circuit). It may be desirable to minimize the exponent comparison, shifting, leading zero detection, rounding, and / or normalization performed by the circuit, e.g., to save the circuit area (e.g., gate count) and power (e.g., energy) consumption of the circuit. Some embodiments herein save circuit area and power (e.g., energy) consumption by having a single circuit (e.g., accelerator) for both fixed-point and floating-point operations, e.g., without a separate fixed-point (e.g., integer) circuit and a separate floating-point circuit. In some embodiments, a single circuit (e.g., accelerator) for both fixed-point and floating-point operations saves circuit area and power consumption compared to having separate fixed-point (e.g., integer) and floating-point circuits. Utilizing such a single circuit effectively translates into higher operations per second because it increases the number of overall (e.g., vector and / or matrix multiplication) circuits that can be achieved for a given area and power budget. In one embodiment, the circuit is part of a processor (e.g., computer) and improves the functionality of the processor (e.g., computer) itself. Some embodiments herein have a single circuit (e.g., accelerator) for any one (e.g., all) of fixed-point, inflection point, and floating-point operations.

[0064] Referring Figure 7 to one embodiment of, there are seven instances of floating-point addition including exponent comparison, shifting, leading zero detection, rounding, and / or normalization, e.g., Figure 5 seven instances of the floating-point adder circuit 500 of, and thus seven instances of components therein (e.g., including comparator 508, swap register 510, shift register 512, and / or leading zero detection, normalization, and rounding circuit 516). Further referring Figure 7 to this embodiment of, there are eight instances of floating-point multiplication including exponent comparison, shifting, leading zero detection, rounding, and / or normalization, e.g., Figure 6 eight instances of the floating-point multiplier circuit 600 of, and thus eight instances of components therein (e.g., including leading zero detection, normalization, and rounding circuit 616). Referring Figure 8 to the embodiment of, wherein each MAC tree circuit 802 is an instance of the floating-point MAC tree circuit 700 from Figure 7 and each adder 804 is an instance of the floating-point adder circuit 500 from Figure 5 , the number of components (e.g., comparator 508, swap register 510, shift register 512, leading zero detection, normalization, and rounding circuit 516, and / or leading zero detection, normalization, and rounding circuit 616) is even greater. Referring Figure 9 to the embodiment of, wherein each floating-point accumulator circuit 902 is Figure 8An example of the floating-point accumulator circuit 800 has an even greater number of components (e.g., comparator 508, swap register 510, shift register 512, leading zero detection, normalization, and rounding circuit 516, and / or leading zero detection, normalization, and rounding circuit 616). Some embodiments herein minimize the number of repetitive components (e.g., exponent comparison, shift, leading zero detection, rounding, and / or normalization components) used to perform floating-point operations, for example.

[0065] Some embodiments herein provide a circuit for floating-point operations that minimizes exponent comparison, shift, leading zero detection, rounding, and / or normalization for floating-point operations in a circuit (e.g., an accelerator) by performing a single exponent comparison, shift, leading zero detection, rounding, and / or normalization (e.g., once in a pipeline) for (e.g., each) vector multiplication, e.g., as opposed to performing exponent comparison, shift, leading zero detection, rounding, and / or normalization for each addition in a vector multiplication.

[0066] Some embodiments herein provide a (e.g., fixed-point and floating-point) vector multiplication circuit that uses a fixed-point multiplier (e.g., a multiplier circuit) and a fixed-point adder (e.g., an adder circuit) to perform floating-point vector multiplication by quantifying the (one or more) fixed-point values produced by the fixed-point multiplier and the fixed-point adder into a floating-point result using a maximum exponent.

[0067] Figure 10Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit 1000 having a fixed-point accumulator circuit 1002 and a floating-point accumulator circuit 1004. The circuit 1000 includes a mode controller 1006 (e.g., a circuit) to switch (e.g., select) between a fixed-point mode and a floating-point mode. In one embodiment, the mode controller 1006 will switch between a fixed-point mode, an inflection point mode, and a floating-point mode (e.g., any one of them). In one embodiment, the mode controller issues a plurality of control values (e.g., 0 to M) to control circuit components (e.g., switches). The circuit 1000 includes a sum-of-products circuit (e.g., a pipeline) 1001 and an accumulator 1010 (e.g., a fixed-point accumulator circuit 1002 and a floating-point accumulator circuit 1004). In the depicted embodiment, the floating-point (and / or inflection point) inputs (e.g., input registers) and the fixed-point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed-point data or floating-point data (e.g., or inflection point data). For example, a first register stores fixed-point data or floating-point data (e.g., or inflection point data) for a first vector (e.g., vector A) and a second register stores fixed-point data or floating-point data (e.g., or inflection point data) for a second vector (e.g., vector B). In one embodiment, separate registers are used for fixed-point data and floating-point data (e.g., and inflection point data). For example, a first register stores fixed-point data for a first fixed-point vector (e.g., fixed-point vector A), a second register stores fixed-point data for a second fixed-point vector (e.g., fixed-point vector B), a third register stores floating-point data for a third floating-point vector (e.g., floating-point vector A), and a fourth register stores floating-point data for a fourth floating-point vector (e.g., floating-point vector B). In one embodiment, inflection point data is stored in its own (one or more) registers.In one embodiment, input 1012 includes an input for each element of the fixed-point vector (e.g., for the first element [0] of the first (e.g., integer) vector A includes input 1020 (e.g., where the integer vector A is separate or distinct from the floating-point vector A) and / or for the first element [0] of the second (e.g., integer) vector B includes input 1022 (e.g., where the integer vector B is separate or distinct from the floating-point vector B), and input 1012 includes an input for each element of the floating-point vector (e.g., the sign bit, exponent, and mantissa of each element) (e.g., for the sign bit of the first element [0] of the first vector A includes input 1026, for the exponent bit of the first element [0] of the first vector A includes input 1014, and for the mantissa bit of the first element [0] of the first vector A includes input 1008, and / or for the sign bit of the first element [0] of the second vector B includes input 1026, for the exponent bit of the first element [0] of the second vector B includes input 1014, and for the mantissa bit of the first element [0] of the second vector B includes input 1008). In some embodiments, input 1012 includes an input for each element of the inflection-point vector (e.g., the exponent and mantissa of each element) (e.g., for the exponent bit of the first element [0] of the first vector A includes input 1014 and for the mantissa bit of the first element [0] of the first vector A includes input 1008, and / or for the exponent bit of the first element [0] of the second vector B includes input 1014, and for the mantissa bit of the first element [0] of the second vector B includes input 1008). In one embodiment, the vectors (e.g., A[N+1] and B[N+1]) to be operated on by the combined (e.g., single) fixed-point and floating-point vector multiplication circuit 1000 have the same number of elements equal to N+1. N can be any positive integer in some embodiments.

[0068] The labels for other inputs (e.g., for elements 1 through N) are not drawn in this figure so as not to obscure the data flow / line. Additionally, the control inputs / lines (e.g., for switches, shifters, numeric conversion circuits (e.g., two's complement circuits)) are not drawn so as not to obscure the data flow / line. The control inputs / lines to the components can extend from a mode controller.

[0069] In some embodiments, each element position of the vector to be processed (e.g., 0 through N) includes a corresponding multiplier, shift register, and numeric conversion circuit (e.g., unsigned to signed conversion circuit) to process the mantissa in that element position, including an exclusive OR logic gate, and / or includes an adder to process the exponent in that element position (e.g., add together). This set of components can be referred to as an element block (e.g., there are N+1 element blocks in Figure 10 ).

[0070] First Mode

[0071] In one embodiment, a mode controller (e.g., via a request to perform a sum-of-products operation) is in a first (e.g., floating-point) mode to perform operations on floating-point input data.

[0072] As an example, in the first (e.g., floating-point) mode, each of these floating-point numbers is divided into its sign bit, exponent bit, and mantissa bit. At this initial division, depending on the exponent, leading hidden bits that can be 1 or 0 are concatenated to the mantissa. Subsequently, pairwise multiplications are performed between the corresponding mantissas of each vector, the exponents are added together, and a logical exclusive OR is performed on the corresponding sign bits. At the output of the multiplication stage, the mantissa can be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissa is converted to a fixed-point (e.g., two's complement) number representation. In an embodiment of the next stage, sum reduction is performed on the mantissa, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed-point number is re-converted to sign and magnitude, aligned to remove leading ones, and the exponent is corrected. From these values, a standard-compliant floating-point number is generated and output in this example. In one embodiment, the standard is the IEEE floating-point arithmetic standard (e.g., 754-2008 – the IEEE Computer Society's floating-point arithmetic standard released on August 29, 2008).

[0073] As another example, in the first mode, input source switches (e.g., multiplexers such as multiplexers 1030 and 1032) are sent control values to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of two elements with index 0 of floating-point vectors A and B as a pair, source the mantissas of two elements with index 1 of floating-point vectors A and B as a second pair, source the mantissas of two elements with index 2 of floating-point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the mantissa A[0]1008 of the element with index 0 from vector A and the mantissa B[0]1024 of the element with index 0 from vector B are multiplied by a (e.g., fixed-point) multiplier 1034 to produce a product. The products from each multiplier in the depicted embodiment are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of two elements with index 0 of floating-point vectors A and B as a pair, source the exponents of two elements with index 1 of floating-point vectors A and B as a second pair, source the exponents of two elements with index 2 of floating-point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vector A and vector B). In this example, each of the sums of exponents (e.g., n + 1 sums in this figure) is input into a maximum exponent determiner 1040 (e.g., comparator circuit). For example, the exponent A[0]1014 of the element with index 0 from vector A and the exponent B[0]1016 of the element with index 0 from vector B are added together by a (e.g., fixed-point) (e.g., integer) adder 1038 to produce the sum of exponent A[0]1014 and exponent B[0]1016. The maximum exponent determiner 1040 (e.g., circuit) then compares the sums of exponents with each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one place (e.g., one element) (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1041 to a set of shift registers (e.g., including shift register 1036) (e.g., shown as horizontal lines extending between each shift register starting from shift register 1036) and normalization circuit 1050. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit) (e.g., unsigned to signed conversion circuit). For example, numeric conversion circuit 1042 (e.g., two's complement circuit) receives the unsigned (e.g., and shifted) mantissa product output from shift register 1036. Each corresponding numeric conversion circuit (e.g., two's complement circuit) then converts the (e.g., shifted) mantissa product from unsigned format (e.g., value) to signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from unsigned (e.g., binary) format to two's complement format) performed on the (e.g., shifted) mantissa product based on the output from each corresponding exclusive OR logic gate (e.g., the output indicating a positive or negative mantissa product).

[0074] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1042 and shift register 1036 are used for index position 0 of the input vector.

[0075] In the depicted embodiment, each sign bit of a pair of sign bits from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., taking the sign bits of the two elements with index 0 of floating point vectors A and B as a first pair, the sign bits of the two elements with index 1 of floating point vectors A and B as a second pair, the sign bits of the two elements with index 2 of floating point vectors A and B as a third pair, etc.) are logically exclusive OR (XOR)ed together by corresponding (e.g., fixed point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, exclusive OR logic gate 1044 receives the sign bit from input 1026 for the first element [0] of the first vector A and the sign bit from input 1028 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of inputs 1026, 1028 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0076] The converted (e.g., signed) values of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1046 to form a single product. The normalization circuit 1050 is then used in this first mode to convert the single product into a floating-point representation based on the maximum exponent. In one embodiment, the circuit 1050 also performs rounding or other operations to convert the (e.g., integer) format of the single product into a floating-point format to create the result of the multiply-accumulate circuit 1001. In one embodiment, the normalization circuit 1050 shifts the most significant bit of the significand (e.g., the single product) of the non-zero digits to be non-zero (e.g., and correspondingly shifts the maximum exponent value according to the shift of the single product).

[0077] Second Mode

[0078] In one embodiment, the mode controller (e.g., via a request to perform a multiply-accumulate operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0079] As an example, in the second (e.g., fixed-point) mode, the multiply-accumulate circuit will multiply each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to N) of the input vector to form products, and sum these products to produce a single fixed-point result.

[0080] As another example, in a second (e.g., fixed-point) mode, the input source muxes (e.g., muxes 1030 and 1032) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the values of two elements having index 0 of fixed-point vectors A and B as a pair, source the values of two elements having index 1 of fixed-point vectors A and B as a second pair, source the values of two elements having index 2 of fixed-point vectors A and B as a third pair, etc.). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the value A[0]1020 of the element having index 0 from vector A and the value B[0]1022 of the element having index 0 from vector B are multiplied (e.g., by a fixed-point) multiplier 1034 to produce a product. In the depicted embodiment the products from each multiplier are output to a corresponding shift register (e.g., shifter), but in the second mode the products pass through the shift register unchanged, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input into a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the numeric conversion circuit unchanged, e.g., the products are already in two's complement representation, so they are not further converted. The (e.g., signed) products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1046 to form a single fixed-point result. In one embodiment, this single fixed-point result passes through a normalization circuit 1050 without any modification to the single fixed-point result.

[0081] Third Mode

[0082] In one embodiment, the mode controller (e.g., via a request to perform a sum of products operation) is in a third (e.g., inflection point) mode to perform operations on inflection point input data.

[0083] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. A pairwise multiplication is then performed between the corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissas may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation as they are already in that format. In one embodiment of the next stage, a sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., in the FlexN+M format described above).

[0084] As another example, in the third mode, the input source muxes (e.g., muxes 1030 and 1032) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements of inflection point vectors A and B having index 0 as a pair, source the mantissas of the two elements of inflection point vectors A and B having index 1 as a second pair, source the mantissas of the two elements of inflection point vectors A and B having index 2 as a third pair, etc.). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the mantissa A[0]1008 of the element of vector A having index 0 and the mantissa B[0]1024 of the element of vector B having index 0 are multiplied (e.g., fixed-point) by multiplier 1034 to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements of inflection point vectors A and B having index 0 as a pair, source the exponents of the two elements of inflection point vectors A and B having index 1 as a second pair, source the exponents of the two elements of inflection point vectors A and B having index 2 as a third pair, etc.) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., n + 1 sums in this figure) is input into a maximum exponent determiner 1040 (e.g., comparator circuit). For example, the exponent A[0]1014 of the element of vector A having index 0 and the exponent B[0]1016 of the element of vector B having index 0 are added (e.g., fixed-point) (e.g., integer) by adder 1038 to produce the sum of exponent A[0]1014 and exponent B[0]1016. The maximum exponent determiner 1040 (e.g., circuit) then compares the sums of exponents with each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other products of the mantissas accordingly, e.g., to align the other products of the mantissas to the maximum exponent. In one embodiment, the maximum exponent is 6 (e.g., the sum of two exponents each having a value of 3), and another exponent is 4, so the product of the mantissa of the other exponent is shifted two places (e.g., two elements) (e.g., shifted left) to align the product of the mantissa to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1041 to a set of shift registers (e.g., including shift register 1036) and the normalization circuit 1050. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted.

[0085] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where the numeric conversion circuit 1042 and the shift register 1036 are used for index position 0 of the input vector.

[0086] In the depicted embodiment, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1046 to form a single product. The normalization circuit 1050 can then be used in this third mode to convert the single product into an inflection point representation based on the maximum exponent. In one embodiment, instead of a full normalization operation in this mode, the circuit 1050 will perform rounding to reduce the bit width of the sum from the adder 1046 to the desired bit width of the inflection point format to create the result of the product sum circuit 1001 (e.g., where the exponent has no effect on this reduction and the sign has been taken into account since the fixed-point representation is two's complement). In one embodiment, the circuit 1050 shifts the most significant bit of the significant digits of the non-zero number (e.g., the single product) to be non-zero (e.g., and correspondingly shifts the maximum exponent value according to the shift of the single product).

[0087] Accumulator

[0088] Optionally, circuit 1000 includes an accumulator 1010 to accumulate the results from the sum-of-products circuit 1001. In one embodiment, a separate fixed-point accumulator circuit 1002 and a floating-point accumulator circuit 1004 are used in accumulator 1010. In some embodiments, the mode controller 1006 in a first mode causes the results from the sum-of-products circuit 1001 to be accumulated (e.g., only) using the floating-point accumulator circuit 1004 and / or the mode controller 1006 in a second mode causes the results from the sum-of-products circuit 1001 to be accumulated (e.g., only) using the fixed-point accumulator circuit 1002. The depicted fixed-point accumulator circuit 1002 includes a (e.g., fixed-point) (e.g., integer) adder 1056 and a fixed-point accumulator store 1058. The depicted floating-point accumulator circuit 1004 includes a floating-point adder 1052 and a floating-point accumulator store 1054. In one embodiment, the floating-point adder 1052 is an instance of the floating-point adder circuit 500 from Figure 5 of the floating-point adder circuit 500.

[0089] In some embodiments, circuit 1000 receives a value (e.g., from a core transferring a load to an accelerator embodiment of circuit 1000) indicating the number of iterations for the sum-of-products circuit 1001 to perform, e.g., to decouple the width of a vector (e.g., having a number of elements greater than N + 1) to be processed by circuit 1000 from the hardware width of the sum-of-products circuit 1001 (e.g., a vector having at most N + 1 elements). In one embodiment, circuit 1000 will notify the requesting entity (e.g., the core) of completion by performing a notification action (e.g., raising a flag such that the appropriate accumulator store 1054 or accumulator store 1058 is read by the requesting entity). In one embodiment, a floating-point accumulator (e.g., adder) or a fixed-point accumulator (e.g., adder) is used to accumulate the inflection results from circuit 1001, e.g., no sign field is used in the floating-point accumulator (e.g., adder).

[0090] Figure 11 Embodiments in accordance with the present disclosure illustrate a four-by-four fixed-point and floating-point vector multiplication circuit 1100 having a fixed-point accumulator circuit 1102 and a floating-point accumulator circuit 1104. In this embodiment, each input vector has at most 4 elements (e.g., each having the same bit width of elements).

[0091] In Figure 11In [the figure], each of the four element positions of the vector to be processed (e.g., 0, 1, 2, and 3, where N = 3) includes a corresponding multiplier (1134A, 1134B, 1134C, 1134D), a shift register (1136A, 1136B, 1136C, 1136D), and a numerical conversion circuit (1142A, 1142B, 1142C, 1142D) to process the mantissa in that element position, including exclusive-OR logic gates (1144A, 1144B, 1144C, 1144D), and / or includes adders (1138A, 1138B, 1138C, 1138D) to process the exponent in that element position (e.g., add them together). This set of components may be referred to as an element block (e.g., in Figure 11 there are four element blocks). Adder 1146 includes three adders (1146A, 1146B, and 1146C), each adder having two inputs. In another embodiment, adder 1146 is a single adder having N + 1 inputs (e.g., in Figure 11 there are four inputs).

[0092] Circuit 1100 includes a mode controller 1106 (e.g., circuitry) to switch (e.g., select) between a fixed-point mode and a floating-point mode. In one embodiment, the mode controller 1106 will switch between a fixed-point mode, an inflection point mode, and a floating-point mode (e.g., any one of them). Circuit 1100 includes a sum-of-products circuit (e.g., pipeline) 1101 and an accumulator 1110 (e.g., a fixed-point accumulator circuit 1102 and a floating-point accumulator circuit 1104). In the depicted embodiment, the floating-point (and / or inflection point) inputs (e.g., input registers) and the fixed-point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed-point data or floating-point data (e.g., or inflection point data). For example, a first register stores fixed-point data or floating-point data (e.g., or inflection point data) for a first vector (e.g., vector A) and a second register stores fixed-point data or floating-point data (e.g., or inflection point data) for a second vector (e.g., vector B). In one embodiment, separate registers are used for fixed-point data and floating-point data (e.g., and inflection point data). For example, a first register stores fixed-point data for a first fixed-point vector (e.g., fixed-point vector A), a second register stores fixed-point data for a second fixed-point vector (e.g., fixed-point vector B), a third register stores floating-point data for a third floating-point vector (e.g., floating-point vector A), and a fourth register stores floating-point data for a fourth floating-point vector (e.g., floating-point vector B). In one embodiment, the inflection point data is stored in its own register(s). In one embodiment, input 1112 includes an input for each element of the fixed-point vector (e.g., for the first element [0] of the first vector A includes input 1120 and / or for the first element [0] of the second vector B includes input 1122) and an input for each element of the floating-point vector (e.g., the sign bit 1112(sign), exponent 1112(exp), and mantissa of each element) (e.g., for the sign bit of the first element [0] of the first vector A includes input 1126, for the exponent bit of the first element [0] of the first vector A includes input 1114, and for the mantissa bit of the first element [0] of the first vector A includes input 1108, and / or for the sign bit of the first element [0] of the second vector B includes input 1126, for the exponent bit of the first element [0] of the second vector B includes input 1114, and for the mantissa bit of the first element [0] of the second vector B includes input 1108).In some embodiments, the input 1112 includes an input for each element of the inflection point vector (e.g., for the exponent 1112 (exp) and mantissa of each element), such as an input for the exponent bits of the first element [0] of the first vector A includes the input 1114 and the input for the mantissa bits of the first element [0] of the first vector A includes the input 1108, and / or the input for the exponent bits of the first element [0] of the second vector B includes the input 1114, and the input for the mantissa bits of the first element [0] of the second vector B includes the input 1108). In one embodiment, the vectors (e.g., A[N + 1] and B[N + 1]) to be operated on by the fixed-point and floating-point vector multiplication circuit 1100 have the same number of elements equal to N + 1 (e.g., four in. Figure 11 ). N can be any positive integer in some embodiments.

[0093] The labels for other inputs (e.g., for elements 1 to 3) are not shown in this figure so as not to obscure the data flow / line. Additionally, the control inputs / lines (e.g., for switches, shifters, numeric conversion circuits (e.g., two's complement circuits)) are not shown so as not to obscure the data flow / line. The control inputs / lines to the components can extend from the mode controller.

[0094] In some embodiments, each element position of the vector to be processed (e.g., 0 to 3) includes a corresponding multiplier, shift register, and numeric conversion circuit to process the mantissa in that element position, and includes an adder to process the exponent in that element position (e.g., add them together). This set of components can be referred to as an element block (e.g., there are four element blocks in. Figure 11 ).

[0095] First Mode

[0096] In one embodiment, the mode controller (e.g., via a request to perform a sum of products operation) is in a first (e.g., floating-point) mode to perform operations on floating-point input data.

[0097] As an example, in a first (e.g., floating point) mode, each of these floating point numbers is partitioned into its sign bit, exponent bit, and mantissa bit. At this initial partitioning, depending on the exponent, leading hidden bits that can be 1 or 0 are concatenated to the mantissa. Subsequently, pairwise multiplications are performed between the corresponding mantissas of each vector, the exponents are added together, and the corresponding sign bits are logically XORed. At the output of the multiplication stage, the mantissa may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissa is converted to a fixed point (e.g., two's complement) number representation. In one embodiment of the next stage, a sum reduction is performed on the mantissa, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed point number is re-converted to sign and magnitude, aligned to remove a leading one, and the exponent is corrected. From these values, standard compliant floating point numbers are generated and output in this example. In one embodiment, the standard is the IEEE floating point arithmetic standard (e.g., 754 - 2008 – the IEEE Computer Society's standard for floating point arithmetic published on August 29, 2008).

[0098] As another example, in the first mode, the input source multiplexers (e.g., multiplexers 1130 and 1132) are sent control values to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements with index 0 of floating-point vectors A and B as a pair, source the mantissas of the two elements with index 1 of floating-point vectors A and B as a second pair, source the mantissas of the two elements with index 2 of floating-point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1108 of the element with index 0 from vector A and the mantissa B[0]1124 of the element with index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1134A to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements with index 0 of floating-point vectors A and B as a pair, source the exponents of the two elements with index 1 of floating-point vectors A and B as a second pair, source the exponents of the two elements with index 2 of floating-point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1140 (e.g., comparator circuit). For example, the exponent A[0]1114 of the element with index 0 from vector A and the exponent B[0]1116 of the element with index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1338A to produce the sum of exponent A[0]1114 and exponent B[0]1116. The maximum exponent determiner 1140 (e.g., circuit) then compares the sums of exponents with each other to determine the largest (e.g., maximum) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one place (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1141 to a set of shift registers (e.g., including shift register 1136A) and to normalization circuit 1150. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit). For example, numeric conversion circuit 1142A (e.g., two's complement circuit) receives the unsigned (e.g., and shifted) mantissa product output from shift register 1136A. Each corresponding numeric conversion circuit (e.g., two's complement circuit) then converts the (e.g., shifted) mantissa product from unsigned format (e.g., value) to signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from unsigned (e.g., binary) format to two's complement format) performed on the (e.g., shifted) mantissa product based on the output from each corresponding exclusive OR logic gate.

[0099] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1142A and shift register 1136A are used for index position 0 of the input vector.

[0100] In the depicted embodiment, each sign bit of a pair of sign bits from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., taking the sign bits of the two elements with index 0 of floating point vectors A and B as a first pair, the sign bits of the two elements with index 1 of floating point vectors A and B as a second pair, the sign bits of the two elements with index 2 of floating point vectors A and B as a third pair, etc.) are logically exclusive ORed (XORed) together by corresponding (e.g., fixed point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, exclusive OR logic gate 1144A receives the sign bit from input 1126 for the first element [0] of the first vector A and receives the sign bit from input 1128 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of inputs 1126, 1128 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0101] The converted (e.g., signed) values of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1146 to form a single product. The normalization circuit 1150 is then used in this first mode to convert the single product into a floating-point representation based on the maximum exponent. In one embodiment, the circuit 1150 also performs rounding or other operations to convert the (e.g., integer) format of the single product into a floating-point format to create the result of the multiply-accumulate circuit 1101. In one embodiment, the normalization circuit 1150 shifts the most significant bit of the significant digits (e.g., the single product) of the non-zero number to be non-zero (e.g., and correspondingly shifts the maximum exponent value according to the shift of the single product).

[0102] Second Mode

[0103] In one embodiment, the mode controller (e.g., via a request to perform a multiply-accumulate operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0104] As an example, in the second (e.g., fixed-point) mode, the multiply-accumulate circuit will multiply each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to 3) of the input vector to form products, and sum these products to produce a single fixed-point result.

[0105] As another example, in a second (e.g., fixed-point) mode, the input source muxes (e.g., muxes 1130 and 1132) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the values of the two elements with index 0 of fixed-point vectors A and B as a pair, source the values of the two elements with index 1 of fixed-point vectors A and B as a second pair, source the values of the two elements with index 2 of fixed-point vectors A and B as a third pair, and so on). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the value A[0]1120 of the element with index 0 from vector A and the value B[0]1122 of the element with index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1134A to produce a product. In the depicted embodiment, the products from each multiplier are output to corresponding shift registers (e.g., shifters), but in the second mode the products pass through the shift registers without being changed, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input into a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted. The (e.g., signed) products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1146 to form a single fixed-point result. In one embodiment, the single fixed-point result passes through a normalization circuit 1150 without any modification to the single fixed-point result.

[0106] Third Mode

[0107] In one embodiment, the mode controller (e.g., via a request to perform a sum of products operation) is in a third (e.g., inflection point) mode to perform operations on inflection point input data.

[0108] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissa may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation because they are already in that format. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., taking the FlexN+M format described above).

[0109] As another example, in the third mode, the input source muxes (e.g., muxes 1130 and 1132) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements having index 0 of inflection point vectors A and B as a pair, source the mantissas of the two elements having index 1 of inflection point vectors A and B as a second pair, source the mantissas of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1108 of the element having index 0 from vector A and the mantissa B[0]1124 of the element having index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1134A to produce a product. In the depicted embodiment the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements having index 0 of inflection point vectors A and B as a pair, source the exponents of the two elements having index 1 of inflection point vectors A and B as a second pair, source the exponents of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1140 (e.g., comparator circuit). For example, the exponent A[0]1114 of the element having index 0 from vector A and the exponent B[0]1116 of the element having index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1138 to produce the sum of exponent A[0]1114 and exponent B[0]1116. The maximum exponent determiner 1140 (e.g., circuit) then compares the sums of exponents to each other to determine the largest of the sums of exponents (e.g., the maximum value), e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one place (e.g., left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1141 to a set of shift registers (e.g., including shift register 1136A) and a normalization circuit 1150. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation and thus are not further converted.

[0110] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1142A and shift register 1136A are used for index position 0 of the input vector.

[0111] In the depicted embodiment, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1146 to form a single product. The normalization circuit 1150 can then be used in this third mode to convert the single product into an inflection point representation based on the maximum exponent. In one embodiment, instead of a full normalization operation in this mode, circuit 1150 will perform rounding to reduce the bit width of the sum from adder 1146 to the desired bit width of the inflection point format to create the result of product sum circuit 1101 (e.g., where the exponent has no effect on this reduction and the sign has been taken into account since the fixed-point representation is two's complement). In one embodiment, circuit 1150 shifts the most significant bit of the significant digits of the non-zero number (e.g., the single product) to be non-zero (e.g., and correspondingly shifts the maximum exponent value according to the shift of the single product).

[0112] Accumulator

[0113] Optionally, circuit 1100 includes an accumulator 1110 to accumulate the results from the sum-of-products circuit 1101. In one embodiment, a separate fixed-point accumulator circuit 1102 and a floating-point accumulator circuit 1104 are used in accumulator 1100. In certain embodiments, mode controller 1106 in a first mode causes the results from sum-of-products circuit 1101 to be accumulated (e.g., only) using floating-point accumulator circuit 1104 and / or mode controller 1106 in a second mode causes the results from sum-of-products circuit 1101 to be accumulated (e.g., only) using fixed-point accumulator circuit 1102. The depicted fixed-point accumulator circuit 1102 includes (e.g., fixed-point) (e.g., integer) adder 1156 and fixed-point accumulator storage 1158. The depicted floating-point accumulator circuit 1104 includes floating-point adder 1152 and floating-point accumulator storage 1154. In one embodiment, floating-point adder 1152 is an instance of floating-point adder circuit 500 from Figure 5 is an instance of floating-point adder circuit 500 of

[0114] In certain embodiments, circuit 1100 receives (e.g., from a core transferring a load to an accelerator embodiment of circuit 1100) a value indicating the number of iterations that sum-of-products circuit 1101 is to perform, e.g., to decouple the width of a vector (e.g., having a number of elements greater than four) to be processed by circuit 1100 from the hardware width of sum-of-products circuit 1101 (e.g., a vector having at most four elements). In one embodiment, circuit 1100 will notify the requesting entity (e.g., the core) of completion by performing a notification action (e.g., raising a flag such that appropriate accumulator storage 1154 or accumulator storage 1158 is read by the requesting entity). In one embodiment, a floating-point accumulator (e.g., adder) or a fixed-point accumulator (e.g., adder) is used to accumulate the inflection results from circuit 1101, e.g., no sign field is used in the floating-point accumulator (e.g., adder).

[0115] Figure 12 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit 1200 having a fixed-point accumulator circuit 1210 and a floating-point conversion circuit 1270 to convert fixed-point to floating-point at the end of accumulation.

[0116] In one embodiment, accumulation for floating-point inputs is performed on floating-point format values (e.g., for each result from the sum-of-products circuit) and each result is performed with a fixed-point (e.g., integer) to floating-point conversion. However, in certain embodiments, the integer length of the sum-of-products circuit (e.g., pipelined) is greater than the number of mantissa bits of the floating-point accumulator, resulting in some loss of accuracy due to this conversion. In Figure 12In one embodiment, a fixed-point (e.g., integer) accumulator 1210 is employed and the conversion from fixed-point format to floating-point format is performed only at the end of the overall calculation (e.g., when the final accumulation is completed).

[0117] Circuit 1200 includes a mode controller 1206 (e.g., circuitry) to switch (e.g., select) between a fixed-point mode and a floating-point mode. In one embodiment, mode controller 1206 will switch between a fixed-point mode, an inflection-point mode, and a floating-point mode (e.g., any one of them). Circuit 1200 includes a sum-of-products circuit (e.g., pipeline) 1210 and a fixed-point accumulator 1210. In the depicted embodiment, floating-point (and / or inflection-point) inputs (e.g., input registers) and fixed-point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed-point data or floating-point data (e.g., or inflection-point data). For example, a first register stores fixed-point data or floating-point data (e.g., or inflection-point data) for a first vector (e.g., vector A) and a second register stores fixed-point data or floating-point data (e.g., or inflection-point data) for a second vector (e.g., vector B). In one embodiment, separate registers are used for fixed-point data and floating-point data (e.g., and inflection-point data). For example, a first register stores fixed-point data for a first fixed-point vector (e.g., fixed-point vector A), a second register stores fixed-point data for a second fixed-point vector (e.g., fixed-point vector B), a third register stores floating-point data for a third floating-point vector (e.g., floating-point vector A), and a fourth register stores floating-point data for a fourth floating-point vector (e.g., floating-point vector B). In one embodiment, inflection-point data is stored in its own register(s). In one embodiment, input 1212 includes an input for each element of a fixed-point vector (e.g., for the first element [0] of the first vector A includes input 1220 and / or for the first element [0] of the second vector B includes input 1222) and an input for each element of a floating-point vector (e.g., the sign bit 1212(sign), exponent 1212(exp), and mantissa of each element) (e.g., for the sign bit of the first element [0] of the first vector A includes input 1226, for the exponent bit of the first element [0] of the first vector A includes input 1214, and for the mantissa bit of the first element [0] of the first vector A includes input 1208, and / or for the sign bit of the first element [0] of the second vector B includes input 1226, for the exponent bit of the first element [0] of the second vector B includes input 1214, and for the mantissa bit of the first element [0] of the second vector B includes input 1208). In certain embodiments, input 1212 includes an input for each element of an inflection-point vector (e.g., the exponent 1212(exp) and mantissa of each element) (e.g., for the exponent bit of the first element [0] of the first vector A includes input 1214 and for the mantissa bit of the first element [0] of the first vector A includes input 1208, and / or for the exponent bit of the first element [0] of the second vector B includes input 1214, and for the mantissa bit of the first element [0] of the second vector B includes input 1208).In one embodiment, the vectors to be operated on by the fixed-point and floating-point vector multiplication circuit 1200 (e.g., A[N+1] and B[N+1]) have the same number of elements equal to N+1. N can be any positive integer in some embodiments.

[0118] The labels for other inputs (e.g., for elements 1 through N) are not drawn in this figure so as not to obscure the data flow / line. Additionally, control inputs / lines (e.g., for switches, shifters, numeric conversion circuits (e.g., two's complement circuits)) are not drawn so as not to obscure the data flow / line. The control inputs / lines to the components can extend from a mode controller.

[0119] In some embodiments, each element position of the vector to be processed (e.g., 0 through N) includes a corresponding multiplier, shift register, and numeric conversion circuit to process the mantissa in that element position, including an exclusive-OR logic gate, and / or includes an adder to process the exponent in that element position (e.g., add together). This set of components can be referred to as an element block (e.g., there are N+1 element blocks in Figure 12 ).

[0120] First Mode

[0121] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a first (e.g., floating-point) mode to perform operations on floating-point input data.

[0122] As an example, in the first (e.g., floating-point) mode, each of these floating-point numbers is partitioned into its sign bit, exponent bit, and mantissa bit. At this initial partitioning, depending on the exponent, leading hidden bits that can be 1 or 0 are concatenated to the mantissa. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, the exponents are added together, and a logical exclusive-OR is performed on the corresponding sign bits. At the output of the multiplication stage, the mantissa can be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissas are converted to fixed-point (e.g., two's complement) number representation. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed-point number is re-converted to sign and magnitude, aligned to remove the leading one, and the exponent is corrected. From these values, a standard floating-point number is generated and output in this example. In one embodiment, the standard is the IEEE floating-point arithmetic standard (e.g., 754-2008 – the IEEE Computer Society's floating-point arithmetic standard published on August 29, 2008).

[0123] As another example, in the first mode, the input source muxes (e.g., muxes 1230 and 1232) are sent control values to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements with index 0 of floating point vectors A and B as a pair, source the mantissas of the two elements with index 1 of floating point vectors A and B as a second pair, source the mantissas of the two elements with index 2 of floating point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1208 of the element with index 0 from vector A and the mantissa B[0]1224 of the element with index 0 from vector B are multiplied (e.g., fixed-point) by multiplier 1234 to produce a product. In the depicted embodiment the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements with index 0 of floating point vectors A and B as a pair, source the exponents of the two elements with index 1 of floating point vectors A and B as a second pair, source the exponents of the two elements with index 2 of floating point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1240 (e.g., comparator circuit). For example, the exponent A[0]1214 of the element with index 0 from vector A and the exponent B[0]1216 of the element with index 0 from vector B are added (e.g., fixed-point) (e.g., integer) by adder 1238 to produce the sum of exponent A[0]1214 and exponent B[0]1216. The maximum exponent determiner 1240 (e.g., circuit) then compares the sums of exponents to each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one bit (e.g., left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1241 to a set of shift registers (e.g., including shift register 1236) and a second maximum exponent determiner circuit 1264. In this example, each shifted mantissa product is output to a corresponding numerical conversion circuit (e.g., two's complement circuit). For example, numerical conversion circuit 1242 (e.g., two's complement circuit) receives the unsigned (e.g., and shifted) mantissa product output from shift register 1236. Each corresponding numerical conversion circuit (e.g., two's complement circuit) then converts the (e.g., shifted) mantissa product from unsigned format (e.g., value) to signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from unsigned (e.g., binary) format to two's complement format) performed on the (e.g., shifted) mantissa product based on the output from each corresponding exclusive - OR logic gate.

[0124] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numerical conversion circuit 1242 and shift register 1236 are used for index position 0 of the input vector.

[0125] In the depicted embodiment, each sign bit of a pair of sign bits of corresponding (e.g., having the same element position in the vector) elements from the input vector (e.g., taking the sign bits of the two elements with index 0 of floating - point vectors A and B as a first pair, the sign bits of the two elements with index 1 of floating - point vectors A and B as a second pair, the sign bits of the two elements with index 2 of floating - point vectors A and B as a third pair, etc.) are logically exclusive - ORed (XORed) together by corresponding (e.g., fixed - point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, exclusive - OR logic gate 1244 receives a sign bit from input 1226 for the first element [0] of the first vector A and a sign bit from input 1228 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of inputs 1226, 1228 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0126] The converted (e.g., signed) values of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed - point) (e.g., integer) adders 1246 to form a single product. In one embodiment, the single product is sent from output 1247 to accumulator 1210, e.g., as described below.

[0127] Second Mode

[0128] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0129] As an example, in the second (e.g., fixed-point) mode, the sum-of-products circuit will multiply each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to N) of the input vector into products, and sum these products to produce a single fixed-point result.

[0130] As another example, in the second (e.g., fixed-point) mode, the input source muxes (e.g., muxes 1230 and 1232) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) element sources of the input vector (e.g., source the values of two elements with index 0 of fixed-point vectors A and B as a pair, source the values of two elements with index 1 of fixed-point vectors A and B as a second pair, source the values of two elements with index 2 of fixed-point vectors A and B as a third pair, and so on). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., n+1 corresponding products in this figure), e.g., the value A[0]1220 of the element with index 0 from vector A and the value B[0]1222 of the element with index 0 from vector B are multiplied (e.g., by a fixed-point) multiplier 1234 to produce a product. In the depicted embodiment, the products from each multiplier are output to a corresponding shift register (e.g., shifter), but in the second mode the products pass through the shift register without being changed, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input into a corresponding numerical conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the numerical conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted. The (e.g., signed) products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1246 to become a single fixed-point result. In one embodiment, the single fixed-point result (e.g., product) is sent from output 1247 to accumulator 1210, e.g., as described below.

[0131] Third Mode

[0132] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a third (e.g., inflection point) mode to perform operations on inflection point input data.

[0133] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. Subsequently, pairwise multiplication is performed between corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissas may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation because they are already in that format. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., taking the FlexN+M format described above).

[0134] As another example, in the third mode, the input source muxes (e.g., muxes 1230 and 1232) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements with index 0 of inflection point vectors A and B as a pair, source the mantissas of the two elements with index 1 of inflection point vectors A and B as a second pair, source the mantissas of the two elements with index 2 of inflection point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1208 of the element with index 0 from vector A and the mantissa B[0]1224 of the element with index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1234 to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements with index 0 of inflection point vectors A and B as a pair, source the exponents of the two elements with index 1 of inflection point vectors A and B as a second pair, source the exponents of the two elements with index 2 of inflection point vectors A and B as a third pair, and so on) are added together by a corresponding (e.g., fixed-point) (e.g., integer) adder to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vector A and vector B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1240 (e.g., comparator circuit). For example, the exponent A[0]1214 of the element with index 0 from vector A and the exponent B[0]1216 of the element with index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1238 to produce the sum of exponent A[0]1214 and exponent B[0]1216. The maximum exponent determiner 1240 (e.g., circuit) then compares the sums of exponents with each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other products of the mantissas accordingly, e.g., to align the other products of the mantissas to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the product of the mantissa of the other exponent is shifted one place (e.g., left) to align the product of the mantissa to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1241 to a set of shift registers (e.g., including shift register 1236) and normalization circuit 1250. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted.

[0135] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1242 and shift register 1236 are used for index position 0 of the input vector.

[0136] In the depicted embodiment, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1246 to form a single product. In one embodiment, the single product is sent from output 1247 to accumulator 1210, e.g., as described below.

[0137] Accumulator

[0138] Optionally, circuit 1200 includes accumulator 1210 to accumulate the results from product sum circuit 1201. In one embodiment, accumulator 1210 includes a fixed-point (e.g., integer) adder 1256, e.g., and does not include a (one or more) floating-point adder.

[0139] In certain embodiments, mode controller 1206 in the second mode causes the results sent from output 1247 of product sum circuit 1201 to be accumulated using accumulator circuit 1210 (e.g., added by fixed-point adder 1256 to the previously accumulated result from (e.g., fixed-point) accumulator storage 1258). In one embodiment, other circuits (e.g., shift registers 1266, 1268, and fixed-point to floating-point conversion circuit 1270) pass through without modifying the data (e.g., the results) in this mode.

[0140] In some embodiments, the mode controller 1206 in the first mode causes the result sent from the output 1247 of the sum-of-products circuit 1201 to be accumulated (e.g., only) by using the maximum exponent determined for the current result from path 1241. In one embodiment, the mode controller 1206 in the first mode causes the current result (e.g., the sum of products of mantissa pairs) to be sent from the output 1247 of the sum-of-products circuit 1201 to the first shift register 1266 and any accumulated result (e.g., from the accumulator store 1258) to be sent to the second shift register 1268 such that the shift register 1266 and / or the shift register 1268 shift (e.g., align) the current result and / or the accumulated result respectively based on the maximum exponent stored in the exponent accumulator store 1262 for the accumulation operation (e.g., for a plurality of floating-point results accumulated from the output 1247 over a period of time) by the second maximum exponent determiner circuit 1264 and the current maximum exponent determined for the current result from path 1241. After the results are shifted (e.g., aligned), in one embodiment they are added together by a (e.g., fixed-point) (e.g., integer) adder 1256 and that updated result (and its exponent) can then be sent to the normalization circuit 1250, e.g., to shift the most significant bit of the significand of the non-zero digits (e.g., a single product) to be non-zero (e.g., and accordingly shift the maximum exponent value based on the shift of the single product). In some embodiments, the (e.g., unsigned) version of the accumulated sum of products of mantissas is stored in the accumulator store 1258. The current exponent of the accumulated sum of products of mantissas can then be adjusted by the accumulator exponent circuit 1260 to be compared with the previous exponent of the accumulated sum of products of mantissas in the accumulator store 1262, e.g., and if the current exponent is greater than the previous exponent, the value is updated. In some embodiments, at the end of the accumulation, the accumulated sum of products of mantissas from the accumulator store 1258 and the exponent of the accumulated sum of products of mantissas from the accumulator store 1262 are sent to the fixed-point to floating-point conversion circuit 1270 (e.g., including a two's complement circuit) to convert the accumulated sum and its exponent into floating-point format.

[0141] In some embodiments, the circuit 1200 receives (e.g., from a core transferring a load to the accelerator embodiment of the circuit 1200) a value indicating the number of iterations to be performed by the sum-of-products circuit 1201, e.g., to decouple the width of a vector (e.g., having a number of elements greater than N + 1) to be processed by the circuit 1200 from the hardware width of the sum-of-products circuit 1201 (e.g., a vector having at most N + 1 elements). In one embodiment, the circuit 1200 will notify the requesting entity (e.g., the core) of completion by performing a notification action (e.g., raising a flag such that appropriate output data is read by the requesting entity).

[0142] Figures 13A - 13C Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit 1300 that is subdivided into a plurality of segments 1301A - 1301Z, where each segment has its own maximum exponent. Z in 1301Z can be any number or letter indicating the desired number of instances of circuit segments. For example, two circuit segments can be referred to as 1301A and 1301B, thereby finding two maximum exponents. In certain embodiments, statistically, the error is based on the total vector hardware (e.g., pipeline) size (e.g., Figures 10 - 12 N + 1 in), e.g., the larger the size, the higher the probability of error. Some embodiments herein reduce the error by dividing the sum-of-products circuit (e.g., pipeline) into smaller blocks of size R + 1, e.g., by employing (N + 1) / (R + 1) such sum-of-products circuits (e.g., pipelines) to create a total size of N + 1. For example, a 32-element [0 - 31 element positions] vector operation can include four sum-of-products circuits, where each sum-of-products circuit processes a proper subset of the 32-element vector, e.g., element positions 0 - 7 input to the first circuit segment (e.g., circuit 1301A), element positions 8 - 15 input to the second circuit segment (e.g., circuit 1301B according to this naming convention), element positions 16 - 23 input to the third circuit segment (e.g., circuit 1301C according to this naming convention), and element positions 24 - 31 input to the fourth circuit segment (e.g., circuit 1301D according to this naming convention).

[0143] However, in certain embodiments, when merging between these sum-of-products circuits (e.g., pipelines), the leading-zero detection, re-adjustment of the accumulator, and / or the shift of each final adder output (e.g., output 1247 from Figure 12 as an example) will be redone. Some embodiments herein re-adjust the exponents from each final adder output (e.g., output 1247 from Figure 12 as an example), e.g., using a realignment circuit 1303 to re-adjust in such a way that by detecting the leading zero and then calculating the maximum exponent among the sum-of-products circuits (e.g., “barrels”). Each adder output can then be added based on the new maximum exponent among the sum-of-products circuits 1301A - 1301Z. In certain embodiments, this scheme provides statistically more error-resistant results, e.g., and has a lower gate count and area compared to a floating-point-only vector multiplication circuit.

[0144] In one embodiment, the input exponent value (e.g., the exponent value of a floating-point number) is only allowed to take certain values, e.g., the new exponent can only be a multiple of 4. In one embodiment, the shift register utilized is not a full barrel shifter but provides a limited combination of shifts. In some embodiments, this also simplifies the leading zero detection (LZD) circuitry as zeros can be detected in groups of two or more rather than bit by bit.

[0145] Circuit 1300 includes a mode controller 1306 (e.g., circuitry) to switch (e.g., select) between fixed-point mode and floating-point mode. In one embodiment, mode controller 1306 will switch between fixed-point mode, inflection point mode, and floating-point mode (e.g., any one of them). Circuit 1300 includes a plurality (e.g., two or more) of sum-of-products circuits (e.g., pipelined) 1301A (in Figure 13A ), - 1301Z (in Figure 13B ), a realignment circuit 1303 (in Figure 13C ), and a fixed-point accumulator 1310 (in Figure 13C ). In one embodiment, use Figure 10The accumulator in [description] instead of accumulator 1310. In the depicted embodiment, the floating point (and / or inflection point) inputs (e.g., input registers) and the fixed point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed point data or floating point data (e.g., or inflection point data). For example, the first register stores fixed point data or floating point data (e.g., or inflection point data) for the first vector (e.g., vector A) and the second register stores fixed point data or floating point data (e.g., or inflection point data) for the second vector (e.g., vector B). In one embodiment, separate registers are used for fixed point data and floating point data (e.g., and inflection point data). For example, the first register stores fixed point data for the first fixed point vector (e.g., fixed point vector A), the second register stores fixed point data for the second fixed point vector (e.g., fixed point vector B), the third register stores floating point data for the third floating point vector (e.g., floating point vector A), and the fourth register stores floating point data for the fourth floating point vector (e.g., floating point vector B). In one embodiment, the inflection point data is stored in its own register(s). In one embodiment, input 1312 includes an input for each element of the fixed point vector (e.g., for the first element [0] of the first vector A includes input 1320 and / or for the first element [0] of the second vector B includes input 1322) and an input for each element of the floating point vector (e.g., the sign bit 1312(sign), exponent 1312(exp), and mantissa of each element) (e.g., for the sign bit of the first element [0] of the first vector A includes input 1326, for the exponent bit of the first element [0] of the first vector A includes input 1314, and for the mantissa bit of the first element [0] of the first vector A includes input 1308, and / or for the sign bit of the first element [0] of the second vector B includes input 1326, for the exponent bit of the first element [0] of the second vector B includes input 1314, and for the mantissa bit of the first element [0] of the second vector B includes input 1308). In certain embodiments, input 1312 includes an input for each element of the inflection point vector (e.g., the exponent 1312(exp) and mantissa of each element) (e.g., for the exponent bit of the first element [0] of the first vector A includes input 1314 and for the mantissa bit of the first element [0] of the first vector A includes input 1308, and / or for the exponent bit of the first element [0] of the second vector B includes input 1314, and for the mantissa bit of the first element [0] of the second vector B includes input 1308). In one embodiment, the vectors (e.g., A[N + 1] and B[N + 1]) to be operated on by the fixed point and floating point vector multiplication circuit 1300 have the same number of elements equal to N + 1. N can be any positive integer in certain embodiments.

[0146] The reference numerals for other inputs (e.g., for elements 1 through N) are not drawn in this figure so as not to obscure the data flow / line. Additionally, the control inputs / lines (e.g., for switches, shifters, numeric conversion circuits (e.g., two's complement circuits)) are not drawn so as not to obscure the data flow / line. The control inputs / lines to the components can extend from the mode controller.

[0147] In some embodiments, each element position of the vector to be processed (e.g., 0 through N) includes a corresponding multiplier, shift register, and numeric conversion circuit to process the mantissa in that element position, includes an exclusive OR logic gate, and / or includes an adder to process the exponent in that element position (e.g., add them together). This collection of components can be referred to as an element block (e.g., in Figure 13C there are a total of N + 1 element blocks in the (N + 1) / (R + 1) product-sum circuit segments). The following description refers to product-sum circuit segment 1301A, but is also applicable to other product-sum circuit segments, e.g., all product-sum circuit segments 1301A - 1301Z are the same (e.g., and are controlled by a single mode controller 1306).

[0148] First Mode

[0149] In one embodiment, the mode controller (e.g., via a request to perform a product-sum operation) is in a first (e.g., floating-point) mode to perform operations on floating-point input data.

[0150] As an example, in the first (e.g., floating-point) mode, each of these floating-point numbers is divided into its sign bit, exponent bit, and mantissa bit. At this initial division, depending on the exponent, leading hidden bits that can be 1 or 0 are concatenated to the mantissa. Subsequently, pairwise multiplications are performed between the corresponding mantissas of each vector, the exponents are added together, and a logical exclusive OR is performed on the corresponding sign bits. At the output of the multiplication stage, the mantissa can be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissa is converted to a fixed-point (e.g., two's complement) number representation. In one embodiment of the next stage, a sum reduction is performed on the mantissa, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed-point number is re-converted to sign and magnitude, aligned to remove leading ones, and the exponent is corrected. From these values, a standard floating-point number is generated and output in this example. In one embodiment, the standard is the IEEE floating-point arithmetic standard (e.g., 754 - 2008 – the IEEE Computer Society's floating-point arithmetic standard released on August 29, 2008).

[0151] As another example, in the first mode, the input source muxes (e.g., muxes 1330 and 1332) are sent control values to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of two elements with index 0 of floating-point vectors A and B as a pair, source the mantissas of two elements with index 1 of floating-point vectors A and B as a second pair, source the mantissas of two elements with index 2 of floating-point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the mantissa A[0]1308 of the element with index 0 from vector A and the mantissa B[0]1324 of the element with index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1334 to produce a product. In the depicted embodiment, the product from each multiplier is output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of two elements with index 0 of floating-point vectors A and B as a pair, source the exponents of two elements with index 1 of floating-point vectors A and B as a second pair, source the exponents of two elements with index 2 of floating-point vectors A and B as a third pair, and so on) are added together by a corresponding (e.g., fixed-point) (e.g., integer) adder to produce the sum of the exponents (e.g., the exponent of the product of a pair of elements from vector A and vector B). In this example, each of the sums of the exponents (e.g., n + 1 sums in this figure) is input into a maximum exponent determiner 1340 (e.g., comparator circuit). For example, the exponent A[0]1314 of the element with index 0 from vector A and the exponent B[0]1316 of the element with index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1338 to produce the sum of exponent A[0]1314 and exponent B[0]1316. The maximum exponent determiner 1340 (e.g., circuit) then compares the sums of the exponents to each other to determine the largest (e.g., maximum) of the sums of the exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of the exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one bit (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1341 to a set of shift registers (e.g., including shift register 1336) and a second maximum exponent determiner circuit 1364. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., a two's complement circuit). For example, numeric conversion circuit 1342 (e.g., a two's complement circuit) receives the unsigned (e.g., and shifted) mantissa product output from shift register 1336. Each corresponding numeric conversion circuit (e.g., a two's complement circuit) then converts the (e.g., shifted) mantissa product from an unsigned format (e.g., value) to a signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from an unsigned (e.g., binary) format to a two's complement format) performed on the (e.g., shifted) mantissa product based on the output from each corresponding exclusive OR logic gate.

[0152] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1342 and shift register 1336 are used for index position 0 of the input vector.

[0153] In the depicted embodiment, each sign bit of a pair of sign bits from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., taking the sign bits of the two elements with index 0 of floating point vectors A and B as a first pair, the sign bits of the two elements with index 1 of floating point vectors A and B as a second pair, the sign bits of the two elements with index 2 of floating point vectors A and B as a third pair, etc.) are logically exclusive ORed (XORed) together by corresponding (e.g., fixed point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, exclusive OR logic gate 1344 receives a sign bit from input 1326 for the first element [0] of the first vector A and a sign bit from input 1328 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of inputs 1326, 1328 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0154] In the depicted embodiment, a leading zero detection (LDZ) circuit (e.g., circuit 1351 for the sum-of-products circuit 1301A) detects any leading zeros of the output (e.g., output 1347A for the sum-of-products circuit 1301A and output 1347Z for the sum-of-products circuit 1301Z), and the realignment circuit 1303 will adjust the maximum exponent (e.g., from path 1341 having the adjusted maximum exponent circuit 1353 for the sum-of-products circuit 1301A) to an adjusted maximum exponent (e.g., adjusted maximum exponent 1349A for the sum-of-products circuit 1301A and adjusted maximum exponent 1349Z for the sum-of-products circuit 1301Z) that reflects the number of leading zeros to be removed by the respective shift registers 1355A - 1355Z (e.g., 1355A for the sum-of-products circuit 1301A and 1355Z for the sum-of-products circuit 1301Z) to align the maximum exponents determined (e.g., determined by 1340 for the sum-of-products circuit 130A) for each maximum exponent from the sum-of-products circuits 1301A - 1301Z. The shifted (e.g., aligned) individual results from the output (e.g., output 1347A for the sum-of-products circuit 1301A and output 1347Z for the sum-of-products circuit 1301Z) are then added together by the second set of adders 1357 to produce a single result at output 1359. In one embodiment, for each output of the set of sum-of-products circuits (e.g., output 1347A of the sum-of-products circuit 1301A and output 1347Z of the sum-of-products circuit 1301Z) (e.g., a single adjusted maximum exponent from output 1361, and the single product from output 1359 corresponding to that single adjusted maximum exponent), the adjusted maximum exponent (e.g., output 1361 from another maximum exponent circuit 1363 of the realignment circuit 1303) and / or the single product (e.g., output 1359 of the realignment circuit 1303) are sent to the accumulator 1310, e.g., as described below.

[0155] Second Mode

[0156] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0157] As an example, in the second (e.g., fixed-point) mode, the sum-of-products circuit will multiply each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to N) of the input vector into products, and sum these products to produce a single fixed-point result at output 1359.

[0158] As another example, in a second (e.g., fixed-point) mode, for each of the product-sum circuits 1301A - 1301Z, the input source muxes (e.g., muxes 1330 and 1332) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the values of two elements of the fixed-point vectors A and B having index 0 as a pair, source the values of two elements of the fixed-point vectors A and B having index 1 as a second pair, source the values of two elements of the fixed-point vectors A and B having index 2 as a third pair, and so on). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the value A[0]1320 of the element of vector A having index 0 and the value B[0]1322 of the element of vector B having index 0 are multiplied (e.g., by a fixed-point) multiplier 1334 to produce a product. In the depicted embodiment, the products from each multiplier are output to a corresponding shift register (e.g., shifter), but in the second mode the products pass through the shift register unchanged, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input into a corresponding numerical conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the numerical conversion circuit unchanged, e.g., the products are already in two's complement representation, so they are not further converted. The (e.g., signed) products of each pair of elements (e.g., at the same element position) of a single product-sum circuit are then all added together by a set of (e.g., fixed-point) (e.g., integer) adders (e.g., adder 1346 for product-sum circuit 1301A) to become a single fixed-point result. These single results from each of the product-sum circuits 1301A - 1301Z are then added together by a set of (e.g., fixed-point) (e.g., integer) adders 1357 to become a single fixed-point result (e.g., without any shifting by shift register 1355A or shift register 1355Z). In one embodiment, the single fixed-point result (e.g., product) is sent from output 1359 to accumulator 1310, e.g., as described below.

[0159] Third Mode

[0160] In one embodiment, the mode controller (e.g., via a request to perform a product-sum operation) is in a third (e.g., inflection point) mode to operate on inflection point input data.

[0161] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissas may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation since they are already in that format. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., taking the FlexN+M format described above).

[0162] As another example, in the third mode, the input source muxes (e.g., muxes 1330 and 1332) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements having index 0 of inflection point vectors A and B as a pair, source the mantissas of the two elements having index 1 of inflection point vectors A and B as a second pair, source the mantissas of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1308 of the element having index 0 from vector A and the mantissa B[0]1324 of the element having index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1334 to produce a product. In the depicted embodiment the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements having index 0 of inflection point vectors A and B as a pair, source the exponents of the two elements having index 1 of inflection point vectors A and B as a second pair, source the exponents of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vector A and vector B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1340 (e.g., comparator circuit). For example, the exponent A[0]1314 of the element having index 0 from vector A and the exponent B[0]1316 of the element having index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1338 to produce the sum of exponent A[0]1314 and exponent B[0]1316. The maximum exponent determiner 1340 (e.g., circuit) then compares the sums of exponents to each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having value 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one place (e.g., left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1341 to a set of shift registers (e.g., including shift register 1336). In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted.

[0163] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1342 and shift register 1336 are used for index position 0 of the input vector.

[0164] In the depicted embodiment, for each sum-of-products circuit 1301A - 1301Z, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders (e.g., adder 1346 for sum-of-products circuit 1301A) to form a single product (e.g., a single product from output 1347A for sum-of-products circuit 1301A and a single product from output 1347Z for sum-of-products circuit 1301Z). In the depicted embodiment, a leading zero detection (LDZ) circuit (e.g., circuit 1351 for sum-of-products circuit 1301A) is included to detect any leading zeros in the output (e.g., output 1347A for sum-of-products circuit 1301A), and to adjust the maximum exponent (e.g., from path 1341 for sum-of-products circuit 1301A) to an adjusted maximum exponent (e.g., adjusted maximum exponent 1349A for sum-of-products circuit 1301A and adjusted maximum exponent 1349Z for sum-of-products circuit 1301Z) that reflects the number of leading zeros to be removed by the corresponding shift registers 1355A - 1355Z of the realignment circuit 1303 (e.g., 1355A for sum-of-products circuit 1301A and 1355Z for sum-of-products circuit 1301Z) to align the maximum exponents determined (e.g., determined by 1340 for sum-of-products circuit 130A) for each maximum exponent from sum-of-products circuits 1301A - 1301Z. The shifted (e.g., aligned) single results from the output (e.g., output 1347A for sum-of-products circuit 1301A and output 1347Z for sum-of-products circuit 1301Z) are then added together by a second set of adders 1357 to produce a single result at output 1359. In one embodiment, for each output of the set of sum-of-products circuits (e.g., output 1347A of sum-of-products circuit 1301A and output 1347Z of sum-of-products circuit 1301Z) (e.g., a single adjusted maximum exponent from output 1361, and the single product from output 1359 corresponding to that single adjusted maximum exponent), the adjusted maximum exponent (e.g., output 1361 from another maximum exponent circuit 1363 of the realignment circuit 1303) and / or the single product (e.g., for output 1359 of the realignment circuit 1303) are sent to the accumulator 1310, e.g., as described below.

[0165] Accumulator

[0166] Optionally, circuit 1300 includes an accumulator 1310 to accumulate the results from the sum-of-products circuits 1301A - 1301Z. In one embodiment, accumulator 1310 includes a fixed-point (e.g., integer) adder 1356, for example, and does not include a (one or more) floating-point adder.

[0167] In some embodiments, the mode controller 1306 in the second mode causes the results sent from the outputs 1359 of the sum-of-products circuits 1301A - 1301Z to be accumulated using the accumulator circuit 1310 (e.g., added by the fixed-point adder 1356 to a previously accumulated result from the (e.g., fixed-point) accumulator storage 1358). In one embodiment, other circuits (e.g., shift register 1366, shift register 1368, and fixed-point to floating-point conversion circuit 1370) are allowed to pass through in this mode without modifying the data (e.g., the results).

[0168] In some embodiments, the mode controller 1306 in the first mode causes the results sent from the output 1359 of the sum-of-products circuits 1301A - 1301Z to be accumulated (e.g., only) by using the maximum exponent determined for the current result from the output 1361. In one embodiment, the mode controller 1306 in the first mode causes the current result (e.g., the sum of products of mantissa pairs) to be sent from the output 1359 of the re-alignment circuit 1303 of the sum-of-products circuits 1301A - 1301Z to the first shift register 1366 and any accumulated results (e.g., from the accumulator store 1358) to be sent to the second shift register 1368, such that the shift register 1366 and / or the shift register 1368 shift (e.g., align) the current result and / or the accumulated result respectively based on the maximum exponent stored in the exponent accumulator store 1362 for the accumulation operation (e.g., for multiple floating-point results accumulated from the output 1359 over a period of time) by the second maximum exponent determiner circuit 1364 and the current maximum exponent determined for the current result from the path 1341. After the results are shifted (e.g., aligned), in one embodiment they are added together by a (e.g., fixed-point) (e.g., integer) adder 1356 and that updated result (and its exponent) can then be sent to the normalization circuit 1350, e.g., to shift the most significant bit of the significant digits of the non-zero numbers (e.g., a single product) to be non-zero (e.g., and to correspondingly shift the maximum exponent value according to the shift of the single product). In some embodiments, the (e.g., unsigned) version of the accumulated sum of products of mantissas is stored in the accumulator store 1358. The current exponent of the accumulated sum of products of mantissas can then be adjusted by the accumulator exponent circuit 1360 to be compared with the previous exponent of the accumulated sum of products of mantissas in the accumulator store 1362, e.g., and if the current exponent is greater than the previous exponent, the value is updated. In some embodiments, at the end of the accumulation, the accumulated sum of products of mantissas from the accumulator store 1358 and the exponent of the accumulated sum of products of mantissas from the accumulator store 1362 are sent to the fixed-point to floating-point conversion circuit 1370 (e.g., including a two's complement circuit) to convert the accumulated sum and its exponent into floating-point format.

[0169] In some embodiments, circuit 1300 receives a value (e.g., from a core that offloads a load to an accelerator embodiment of circuit 1300) indicative of the number of iterations for product-sum circuits 1301A - 1301Z to perform, e.g., to decouple the width of a vector (e.g., having a number of elements greater than N + 1) to be processed by circuit 1300 from the hardware width of product-sum circuits 1301A - 1301Z (e.g., a vector having at most N + 1 elements). In one embodiment, circuit 1300 will notify the requesting entity (e.g., the core) of completion by performing a notification action (e.g., raising a flag such that appropriate output data is read by the requesting entity).

[0170] Figures 14A - 14C Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit 1400 that is subdivided into a plurality of segments 1401A - 1401Z, where each segment has its own maximum exponent and the segments are sorted by the sum of exponents.

[0171] Z in 1401Z can be any number or letter indicative of the desired number of instances of circuit segments. For example, two circuit segments can be referred to as 1401A and 1401B, thereby finding two maximum exponents. Some embodiments herein re-adjust the exponents from each final adder output (e.g., from output 1247 in the example), e.g., using a realignment circuit 1403 by detecting leading zeros and then calculating the maximum exponent among product-sum circuits (e.g., “barrels”). In one embodiment, each adder output can then be added based on the new maximum exponent among product-sum circuits 1401A - 1401Z. In some embodiments, this scheme provides statistically less error-prone results, e.g., and has a lower gate count and area compared to a floating-point-only vector multiplication circuit. Compared to Figure 12 one embodiment in Figures 13A - 13C and Figures 14A - 14CThe embodiments depicted include a sorter circuit 1480 (e.g., shared among each of the sum-of-products circuits 1401A - 1401Z). In one embodiment, the sorter circuit 1480 sorts input elements (e.g., in ascending or descending order) by the sum of exponents (e.g., the exponents of the products of a pair of elements from vector A and vector B). In one embodiment, the sorting does change the physical sorting in the input registers. In one embodiment, the sorting does not change the physical sorting in the input registers, but changes the connections (e.g., switches) in the respective sum-of-products circuits 1401A - 1401Z. In one embodiment, sorting by the sum of exponents causes the corresponding mantissas (or the mantissa products from each multiplier) to be sorted in that same order, e.g., the associated values of the sorted exponents (e.g., the corresponding mantissa products and exclusive-OR logic gate outputs (e.g., input sign bits) from each multiplier) are sorted into the sum-of-products circuits 1401A - 1401Z. In certain embodiments, the magnitudes of each group of elements of the vectors operated on by the respective sum-of-products circuits 1401A - 1401Z will be closer than in unsorted embodiments, thus providing statistically more error-free results. Although the sorter circuit is shown between certain components, it should be understood that it can be placed elsewhere (e.g., between a multiplexer (e.g., 1430) and a multiplier (e.g., 1434)). In one embodiment, the sorter circuit 1480 is used only in a single mode (e.g., the first mode).

[0172] Circuit 1400 includes a mode controller 1406 (e.g., a circuit) to switch (e.g., select) between fixed-point mode and floating-point mode. In one embodiment, the mode controller 1406 will switch between fixed-point mode, inflection point mode, and floating-point mode (e.g., any one of them). Circuit 1400 includes a plurality (e.g., 2 or more) of sum-of-products circuits (e.g., pipelined) 1401A (in Figure 14A - 1401Z (in Figure 14B ), a realignment circuit 1403 (in Figure 14C ), and a fixed-point accumulator 1410 (in Figure 14C ). In one embodiment, Figure 10the accumulator in it, rather than accumulator 1410. In the depicted embodiment, floating point (and / or inflection point) inputs (e.g., input registers) and fixed point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed point data or floating point data (e.g., or inflection point data). For example, the first register stores fixed point data or floating point data (e.g., or inflection point data) for the first vector (e.g., vector A) and the second register stores fixed point data or floating point data (e.g., or inflection point data) for the second vector (e.g., vector B). In one embodiment, separate registers are used for fixed point data and floating point data (e.g., and inflection point data). For example, the first register stores fixed point data for the first fixed point vector (e.g., fixed point vector A), the second register stores fixed point data for the second fixed point vector (e.g., fixed point vector B), the third register stores floating point data for the third floating point vector (e.g., floating point vector A), and the fourth register stores floating point data for the fourth floating point vector (e.g., floating point vector B). In one embodiment, inflection point data is stored in its own register(s). In one embodiment, input 1412 includes an input for each element of the fixed point vector (e.g., includes input 1420 for the first element [0] of the first vector A and / or includes input 1422 for the first element [0] of the second vector B) and includes an input for each element of the floating point vector (e.g., the sign bit 1412(sign), exponent 1412(exp), and mantissa of each element) (e.g., includes input 1426 for the sign bit of the first element [0] of the first vector A, includes input 1414 for the exponent bit of the first element [0] of the first vector A, and includes input 1408 for the mantissa bit of the first element [0] of the first vector A, and / or includes input 1426 for the sign bit of the first element [0] of the second vector B, includes input 1414 for the exponent bit of the first element [0] of the second vector B, and includes input 1408 for the mantissa bit of the first element [0] of the second vector B). In certain embodiments, input 1412 includes an input for each element of the inflection point vector (e.g., the exponent 1412(exp) and mantissa of each element) (e.g., includes input 1414 for the exponent bit of the first element [0] of the first vector A and includes input 1408 for the mantissa bit of the first element [0] of the first vector A, and / or includes input 1414 for the exponent bit of the first element [0] of the second vector B, and includes input 1408 for the mantissa bit of the first element [0] of the second vector B). In one embodiment, the vectors to be operated on by the fixed point and floating point vector multiplication circuit 1400 (e.g., A[N+1] and B[N+1]) have the same number of elements equal to N+1. N can be any positive integer in certain embodiments.

[0173] The reference numerals for other inputs (e.g., for elements 1 through N) are not drawn in this figure so as not to obscure the data flow / line. Additionally, the control inputs / lines (e.g., for switches, shifters, numeric conversion circuits (e.g., two's complement circuits)) are not drawn so as not to obscure the data flow / line. The control inputs / lines to the components can extend from the mode controller.

[0174] In some embodiments, each element position of the vector to be processed (e.g., 0 through N) includes a respective multiplier, shift register, and numeric conversion circuit to process the mantissa in that element position, including an exclusive OR logic gate, and / or includes an adder to process the exponent in that element position (e.g., add together). This collection of components can be referred to as an element block (e.g., in FIG. 14 there are a total of N + 1 element blocks in the (N + 1) / (R + 1) product-sum circuit segments). The following description refers to product-sum circuit segment 1401A, but also applies to other product-sum circuit segments, e.g., where all of the product-sum circuit segments 1401A - 1401Z are the same (e.g., and are controlled by the same mode controller 1406). Note that an element with a certain index (e.g., index 0) below can refer to the index in its sorted format (e.g., sorted by sorter circuit 1480) rather than the index when the element is input into the respective product-sum circuits 1401A - 1401Z.

[0175] First Mode

[0176] In one embodiment, the mode controller (e.g., via a request to perform a product-sum operation) is in a first (e.g., floating point) mode to perform operations on floating point input data.

[0177] As an example, in a first (e.g., floating point) mode, each of these floating point numbers is divided into its sign bit, exponent bit, and mantissa bit. At this initial division, depending on the exponent, leading hidden bits that can be 1 or 0 are concatenated onto the mantissa. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, the exponents are added together, and a logical exclusive OR is performed on the corresponding sign bits. At the output of the multiplication stage, the mantissa can be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissa is converted to a fixed point (e.g., two's complement) number representation. In one embodiment of the next stage, a sum reduction is performed on the mantissa, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed point number is reconverted to sign and magnitude, aligned to remove leading ones, and the exponent is corrected. From these values, a standard compliant floating point number is generated and output in this example. In one embodiment, the standard is the IEEE floating point arithmetic standard (e.g., 754 - 2008 – The IEEE Computer Society's floating point arithmetic standard released on August 29, 2008).

[0178] In one embodiment, sorting circuit 1480 sorts the sorted input data (e.g., mantissa and its corresponding exponent), e.g., into true subsets based on the value of the sum of a pair of exponents.

[0179] As another example, in the first mode, the input source muxes (e.g., muxes 1430 and 1432) are sent control values (e.g., sent from the sorting circuit 1480) to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the mantissas of two elements with index 0 of floating-point vectors A and B as a pair, source the mantissas of two elements with index 1 of floating-point vectors A and B as a second pair, source the mantissas of two elements with index 2 of floating-point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the mantissa A[0]1408 of the element with index 0 from vector A and the mantissa B[0]1424 of the element with index 0 from vector B are multiplied (e.g., fixed-point) by multiplier 1434 to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the exponents of two elements with index 0 of floating-point vectors A and B as a pair, source the exponents of two elements with index 1 of floating-point vectors A and B as a second pair, source the exponents of two elements with index 2 of floating-point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., n + 1 sums in this figure) is input into a maximum exponent determiner 1440 (e.g., comparator circuit). For example, the exponent A[0]1414 of the element with index 0 from vector A and the exponent B[0]1416 of the element with index 0 from vector B are added (e.g., fixed-point) (e.g., integer) by adder 1438 to produce the sum of exponent A[0]1414 and exponent B[0]1416. The maximum exponent determiner 1440 (e.g., circuit) then compares the sums of exponents with each other to determine the largest (e.g., maximum value) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponent is shifted one place (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1441 to a set of shift registers (e.g., including shift register 1436) and a second maximum exponent determiner circuit 1464. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit). For example, numeric conversion circuit 1442 (e.g., two's complement circuit) receives the unsigned (e.g., and shifted) mantissa product output from shift register 1436. Each corresponding numeric conversion circuit (e.g., two's complement circuit) then converts the (e.g., shifted) mantissa product from an unsigned format (e.g., value) to a signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from an unsigned (e.g., binary) format to a two's complement format) performed on the (e.g., shifted) mantissa product based on the output from each corresponding exclusive OR logic gate.

[0180] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1442 and shift register 1436 are used for index position 0 of the input vector.

[0181] In the depicted embodiment, each sign bit of a pair of sign bits from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., taking the sign bits of the two elements with index 0 of floating point vectors A and B as a first pair, the sign bits of the two elements with index 1 of floating point vectors A and B as a second pair, the sign bits of the two elements with index 2 of floating point vectors A and B as a third pair, etc.) are logically exclusive OR (XOR)ed together by corresponding (e.g., fixed point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, exclusive OR logic gate 1444 receives a sign bit from input 1426 for the first element [0] of the first vector A and a sign bit from input 1428 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of inputs 1426, 1428 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0182] In the depicted embodiment, a leading zero detection (LDZ) circuit (e.g., circuit 1451 for the sum-of-products circuit 1401A) detects any leading zeros of the output (e.g., output 1447A for the sum-of-products circuit 1401A and output 1447Z for the sum-of-products circuit 1401Z), and the realignment circuit 1403 adjusts the maximum exponent (e.g., from path 1441 for the sum-of-products circuit 1401A) to an adjusted maximum exponent (e.g., adjusted maximum exponent 1449A for the sum-of-products circuit 1401A and adjusted maximum exponent 1449Z for the sum-of-products circuit 1401Z) that reflects the number of leading zeros to be removed by the respective shift registers 1455A - 1455Z (e.g., 1455A for the sum-of-products circuit 1401A and 1455Z for the sum-of-products circuit 1401Z) to align the maximum exponents determined (e.g., determined by 1440 for the sum-of-products circuit 1401A) for each maximum exponent from the sum-of-products circuits 1401A - 1401Z. The shifted (e.g., aligned) individual results from the output (e.g., output 1447A for the sum-of-products circuit 1401A and output 1447Z for the sum-of-products circuit 1401Z) are then added together by the second set of adders 1457 to produce a single result at output 1459. In one embodiment, for each output of the set of sum-of-products circuits (e.g., output 1447A of the sum-of-products circuit 1401A and output 1447Z of the sum-of-products circuit 1401Z), the adjusted maximum exponent (e.g., output 1461 from another maximum exponent circuit 1463 of the realignment circuit 1403) and / or the individual product (e.g., for output 1459 of the realignment circuit 1403) is sent to the accumulator 1410, e.g., as described below.

[0183] Second Mode

[0184] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0185] As an example, in the second (e.g., fixed-point) mode, the sum-of-products circuit multiplies each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to N) of the input vector to produce products, and sums these products to produce a single fixed-point result at output 1459.

[0186] As another example, in a second (e.g., fixed-point) mode, for each of the sum-of-products circuits 1401A - 1401Z, the input source muxes (e.g., muxes 1430 and 1432) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the values of two elements with index 0 of the fixed-point vectors A and B as a pair, source the values of two elements with index 1 of the fixed-point vectors A and B as a second pair, source the values of two elements with index 2 of the fixed-point vectors A and B as a third pair, and so on). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., n + 1 corresponding products in this figure), e.g., the value A[0]1420 of the element with index 0 from vector A and the value B[0]1422 of the element with index 0 from vector B are multiplied (e.g., by a fixed-point) multiplier 1434 to produce a product. In the depicted embodiment, the products from each multiplier are output to a corresponding shift register (e.g., shifter), but in the second mode the products pass through the shift register unchanged, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input into a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the numeric conversion circuit unchanged, e.g., the products are already in two's complement representation, so they are not further converted. The (e.g., signed) products of each pair of elements (e.g., at the same element position) of a single sum-of-products circuit are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders (e.g., adder 1446 for sum-of-products circuit 1401A) to form a single fixed-point result (e.g., without any shifting by shift register 1355A or shift register 1355Z). These single results from each of the sum-of-products circuits 1401A - 1401Z are then added together using a set of (e.g., fixed-point) (e.g., integer) adders 1457 to form a single fixed-point result. In one embodiment, the single fixed-point result (e.g., product) is sent from output 1459 to accumulator 1410, e.g., as described below.

[0187] Third Mode

[0188] In one embodiment, the mode controller (e.g., via a request to perform a sum-of-products operation) is in a third (e.g., inflection point) mode to perform an operation on inflection point input data.

[0189] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissa may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation as they are already in that format. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., taking the FlexN+M format described above).

[0190] As another example, in the third mode, the input source multiplexers (e.g., multiplexers 1430 and 1432) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the mantissas of the two elements having index 0 of inflection point vectors A and B as a pair, source the mantissas of the two elements having index 1 of inflection point vectors A and B as a second pair, source the mantissas of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1408 of the element having index 0 from vector A and the mantissa B[0]1424 of the element having index 0 from vector B are multiplied (e.g., fixed-point) by multiplier 1434 to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the exponents of the two elements having index 0 of inflection point vectors A and B as a pair, source the exponents of the two elements having index 1 of inflection point vectors A and B as a second pair, source the exponents of the two elements having index 2 of inflection point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1440 (e.g., comparator circuit). For example, the exponent A[0]1414 of the element having index 0 from vector A and the exponent B[0]1416 of the element having index 0 from vector B are added (e.g., fixed-point) (e.g., integer) by adder 1438 to produce the sum of exponent A[0]1414 and exponent B[0]1416. The maximum exponent determiner 1440 (e.g., circuit) then compares the sums of exponents to each other to determine the largest (e.g., maximum) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other products of the mantissas accordingly, e.g., to align the other products of the mantissas to the maximum exponent. In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the product of the mantissa of the other exponent is shifted one place (e.g., left) to align the product of the mantissa to have the same exponent as the maximum exponent.In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1441 to a set of shift registers (e.g., including shift register 1436) and normalization circuit 1450. In this example, each shifted mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation and thus are not further converted.

[0191] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent at that index position, e.g., where numeric conversion circuit 1442 and shift register 1436 are used for index position 0 of the input vector.

[0192] In the depicted embodiment, for each sum-of-products circuit 1401A - 1401Z, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders (e.g., adder 1446 for sum-of-products circuit 1401A) to form a single product (e.g., a single product from output 1447A for sum-of-products circuit 1401A and a single product from output 1447Z for sum-of-products circuit 1401Z). In the depicted embodiment, a leading zero detection (LDZ) circuit (e.g., circuit 1451 for sum-of-products circuit 1401A) is included to detect any leading zeros in the output (e.g., output 1447A for sum-of-products circuit 1401A), and will adjust the maximum exponent (e.g., from path 1441 for sum-of-products circuit 1401A) to an adjusted maximum exponent (e.g., adjusted maximum exponent 1455A for sum-of-products circuit 1401A and adjusted maximum exponent 1449Z for sum-of-products circuit 1401Z) that reflects the number of leading zeros to be removed by the corresponding shift registers 1455A - 1455Z of realignment circuit 1403 (e.g., 1455A for sum-of-products circuit 1401A and 1455Z for sum-of-products circuit 1401Z) to align the maximum exponents determined (e.g., determined by 1440 for sum-of-products circuit 1401A) for each maximum exponent from sum-of-products circuits 1401A - 1401Z. The shifted (e.g., aligned) single results from the output (e.g., output 1447A for sum-of-products circuit 1401A and output 1447Z for sum-of-products circuit 1401Z) are then added together by a second set of adders 1457 to produce a single result at output 1459. In one embodiment, for each output of the set of sum-of-products circuits (e.g., output 1447A of sum-of-products circuit 1401A and output 1447Z of sum-of-products circuit 1401Z) (e.g., a single adjusted maximum exponent from output 1461, and the single product from output 1459 corresponding to that single adjusted maximum exponent), the adjusted maximum exponent (e.g., output 1461 from another maximum exponent circuit 1463 of realignment circuit 1403) and / or the single product (e.g., for output 1459 of realignment circuit 1403) are sent to accumulator 1410, e.g., as described below.

[0193] Accumulator

[0194] Optionally, circuit 1400 includes an accumulator 1410 to accumulate the results from the sum-of-products circuits 1401A - 1401Z. In one embodiment, accumulator 1410 includes a fixed-point (e.g., integer) adder 1456, for example, and does not include a (one or more) floating-point adder.

[0195] In certain embodiments, mode controller 1406 in the second mode causes the results sent from the outputs 1459 of the sum-of-products circuits 1401A - 1401Z to be accumulated using accumulator circuit 1410 (e.g., added by fixed-point adder 1456 to a previously accumulated result from (e.g., fixed-point) accumulator storage 1458). In one embodiment, other circuits (e.g., shift register 1466, shift register 1468, and fixed-point to floating-point conversion circuit 1470) are allowed to pass through in this mode without modifying the data (e.g., the results).

[0196] In some embodiments, the mode controller 1406 in the first mode causes the result sent from the output 1459 of the sum-of-products circuits 1401A - 1401Z to be accumulated (e.g., only) by using the maximum exponent determined for the current result from the output 1461. In one embodiment, the mode controller 1406 in the first mode causes the current result (e.g., the sum-of-products of mantissa pairs) to be sent from the output 1459 of the re-alignment circuit 1403 of the sum-of-products circuits 1401A - 1401Z to the first shift register 1466 and any accumulated result (e.g., from the accumulator store 1458) to be sent to the second shift register 1468, such that the shift register 1466 and / or the shift register 1468 shift (e.g., align) the current result and / or the accumulated result respectively based on the maximum exponent stored in the exponent accumulator store 1462 for the accumulation operation (e.g., for multiple floating-point results accumulated from the output 1459 over a period of time) by the second maximum exponent determiner circuit 1464 and the current maximum exponent determined for the current result from the path 1441. After the results are shifted (e.g., aligned), in one embodiment they are added together by a (e.g., fixed-point) (e.g., integer) adder 1456 and that updated result (along with its exponent) can then be sent to the normalization circuit 1450, e.g., to shift the most significant bit of the significant digits (e.g., a single product) of the non-zero number to be non-zero (e.g., and to shift the maximum exponent value accordingly based on the shift of the single product). In some embodiments, the (e.g., unsigned) version of the accumulated sum-of-products of mantissas is stored in the accumulator store 1458. The current exponent of the accumulated sum-of-products of mantissas can then be adjusted by the accumulator exponent circuit 1460 to be compared with the previous exponent of the accumulated sum-of-products of mantissas in the accumulator store 1462, e.g., and if the current exponent is greater than the previous exponent, the value is updated. In some embodiments, at the end of the accumulation, the accumulated sum-of-products of mantissas from the accumulator store 1458 and the exponent of the accumulated sum-of-products of mantissas from the accumulator store 1462 are sent to the fixed-point to floating-point conversion circuit 1470 (e.g., including a two's complement circuit) to convert the accumulated sum and its exponent into floating-point format.

[0197] In some embodiments, circuit 1400 receives, e.g., from a core offloading a load to an accelerator embodiment of circuit 1400, a value indicative of the number of iterations for multiply-accumulate circuits 1401A - 1401Z to perform, e.g., to decouple the width of a vector (e.g., having a number of elements greater than N+1) to be processed by circuit 1400 from the hardware width of multiply-accumulate circuits 1401A - 1401Z (e.g., a vector having at most N+1 elements). In one embodiment, circuit 1400 will notify a requesting entity (e.g., a core) of completion by performing a notification action (e.g., raising a flag such that appropriate output data is read by the requesting entity).

[0198] Figure 15 Embodiments in accordance with the present disclosure illustrate a fixed-point and floating-point vector multiplication circuit 1500 including a rounding circuit (e.g., rounding circuit 1537) having a fixed-point accumulator circuit 1002 and a floating-point accumulator circuit 1004.

[0199] Compared with Figure 10 an embodiment in Figure 15 the depicted embodiment in includes a rounding circuit. In one embodiment, each rounding circuit (e.g., rounding circuit 1537) will round the output from a shift register (e.g., shift register 1036) (e.g., round up, round down, round towards zero, or round to nearest, e.g., round to nearest integer).

[0200] Circuit 1500 includes a mode controller 1506 (e.g., circuitry) to switch (e.g., select) between a fixed-point mode and a floating-point mode. In one embodiment, mode controller 1506 will switch between a fixed-point mode, an inflection point mode, and a floating-point mode (e.g., any one of them). Circuit 1500 includes a sum-of-products circuit (e.g., pipeline) 1501 and an accumulator 1510 (e.g., a fixed-point accumulator circuit 1502 and a floating-point accumulator circuit 1504). In the depicted embodiment, floating-point (and / or inflection point) inputs (e.g., input registers) and fixed-point inputs are shown as separate inputs. In one embodiment, the same register is used for fixed-point data or floating-point data (e.g., or inflection point data). For example, a first register stores fixed-point data or floating-point data (e.g., or inflection point data) for a first vector (e.g., vector A) and a second register stores fixed-point data or floating-point data (e.g., or inflection point data) for a second vector (e.g., vector B). In one embodiment, separate registers are used for fixed-point data and floating-point data (e.g., and inflection point data). For example, a first register stores fixed-point data for a first fixed-point vector (e.g., fixed-point vector A), a second register stores fixed-point data for a second fixed-point vector (e.g., fixed-point vector B), a third register stores floating-point data for a third floating-point vector (e.g., floating-point vector A), and a fourth register stores floating-point data for a fourth floating-point vector (e.g., floating-point vector B). In one embodiment, inflection point data is stored in its own register(s). In one embodiment, input 1512 includes an input for each element of the fixed-point vector (e.g., for the first element [0] of the first vector A includes input 1520 and / or for the first element [0] of the second vector B includes input 1522) and an input for each element of the floating-point vector (e.g., the sign bit 1512(sign), exponent 1512(exp), and mantissa of each element) (e.g., for the sign bit of the first element [0] of the first vector A includes input 1526, for the exponent bit of the first element [0] of the first vector A includes input 1514, and for the mantissa bit of the first element [0] of the first vector A includes input 1508, and / or for the sign bit of the first element [0] of the second vector B includes input 1526, for the exponent bit of the first element [0] of the second vector B includes input 1514, and for the mantissa bit of the first element [0] of the second vector B includes input 1508).In some embodiments, input 1512 includes an input for each element of the inflection point vector (e.g., exponent 1512 (exp) and mantissa for each element), such as for the exponent bits of the first element [0] of the first vector A including input 1514 and for the mantissa bits of the first element [0] of the first vector A including input 1508, and / or for the exponent bits of the first element [0] of the second vector B including input 1514 and for the mantissa bits of the first element [0] of the second vector B including input 1508). In one embodiment, the vectors to be operated on by the fixed-point and floating-point vector multiplication circuit 1500 (e.g., A[N + 1] and B[N + 1]) have the same number of elements equal to N + 1. N can be any positive integer in some embodiments.

[0201] The labels for other inputs (e.g., for elements 1 through N) are not drawn in this figure so as not to obscure the data flow / lines. Additionally, control inputs / lines (e.g., for switches, shifters, rounding circuits, numeric conversion circuits (e.g., two's complement circuits)) are not drawn so as not to obscure the data flow / lines. Control inputs / lines to components can extend from the mode controller.

[0202] In some embodiments, each element position of the vector to be processed (e.g., 0 through N) includes a corresponding multiplier, shift register, rounding circuit, and numeric conversion circuit to process the mantissa in that element position, including an exclusive OR logic gate, and / or includes an adder to process the exponent in that element position (e.g., add together). This set of components can be referred to as an element block (e.g., there are N + 1 element blocks in Figure 15 .

[0203] First Mode

[0204] In one embodiment, the mode controller (e.g., via a request to perform a sum of products operation) is in a first (e.g., floating-point) mode to perform operations on floating-point input data.

[0205] As an example, in a first (e.g., floating point) mode, each of these floating point numbers is divided into its sign bit, exponent bit, and mantissa bit. At this initial division, depending on the exponent, leading hidden bits which can be 1 or 0 are concatenated onto the mantissa. Subsequently, pairwise multiplication is performed between the corresponding mantissas of each vector, the exponents are added together, and a logical exclusive OR is performed on the corresponding sign bits. At the output of the multiplication stage, the mantissa can be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment, the mantissa is converted to a fixed point (e.g., two's complement) number representation. In an embodiment of the next stage, sum reduction is performed on the mantissa and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, the fixed point number is re-converted to sign and magnitude, aligned to remove leading ones, and the exponent is corrected. From these values, a standard compliant floating point number is generated and output in this example. In one embodiment, the standard is the IEEE floating point arithmetic standard (e.g., 754 - 2008 – the IEEE Standard for Floating-Point Arithmetic, published by the IEEE Computer Society on August 29, 2008).

[0206] As another example, in the first mode, the input source muxes (e.g., muxes 1530 and 1532) are sent control values to source a pair of mantissas from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the mantissas of two elements having (sorted) index 0 of floating-point vectors A and B as a pair, source the mantissas of two elements having (sorted) index 1 of floating-point vectors A and B as a second pair, source the mantissas of two elements having (sorted) index 2 of floating-point vectors A and B as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1508 of the element having (sorted) index 0 from vector A and the mantissa B[0]1524 of the element having (sorted) index 0 from vector B are multiplied by (e.g., fixed-point) multiplier 1534A to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vectors (e.g., source the exponents of two elements having (sorted) index 0 of floating-point vectors A and B as a pair, source the exponents of two elements having (sorted) index 1 of floating-point vectors A and B as a second pair, source the exponents of two elements having (sorted) index 2 of floating-point vectors A and B as a third pair, and so on) are added together by corresponding (e.g., fixed-point) (e.g., integer) adders to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vectors A and B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1540 (e.g., comparator circuit). For example, the exponent A[0]1514 of the element having (sorted) index 0 from vector A and the exponent B[0]1516 of the element having (sorted) index 0 from vector B are added together by (e.g., fixed-point) (e.g., integer) adder 1538 to produce the sum of exponent A[0]1514 and exponent B[0]1516. The maximum exponent determiner 1540 (e.g., circuit) then compares the sums of exponents to each other to determine the largest (e.g., maximum) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent.In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponents is shifted by one bit (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent. In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1541 to a set of shift registers (e.g., including shift registers 1536A - D) and the normalization circuit 1550. In this example, each shifted mantissa product is output to a corresponding rounding circuit, and the rounded output is sent to a numerical conversion circuit (e.g., a two's complement circuit). For example, the numerical conversion circuit 1542 (e.g., a two's complement circuit) receives the unsigned and rounded (e.g., and shifted) mantissa product output from the rounding circuit 1537. Each corresponding numerical conversion circuit (e.g., a two's complement circuit) then converts the (e.g., shifted) rounded mantissa product from an unsigned format (e.g., value) to a signed format (e.g., value). In one embodiment, this conversion is a two's complement operation (e.g., from an unsigned (e.g., binary) format to a two's complement format) performed on the (e.g., shifted) rounded mantissa product based on the output from each corresponding exclusive - OR logic gate.

[0207] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent of the (sorted) index position, e.g., where the numerical conversion circuit 1542, the rounding circuit 1537, and the shift register 1536 are used for the (sorted) index position 0 of the input vector.

[0208] In the depicted embodiment, each sign bit of a pair of sign bits of corresponding (e.g., having the same element position in the vector) elements from the input vectors (e.g., taking the sign bits of the two elements with (sorted) index 0 of floating point vectors A and B as a first pair, the sign bits of the two elements with (sorted) index 1 of floating point vectors A and B as a second pair, the sign bits of the two elements with (sorted) index 2 of floating point vectors A and B as a third pair, and so on) are logically exclusive-OR (XOR)ed together by corresponding (e.g., fixed-point) logic gates that implement XOR to produce a value indicating whether the mantissa product is negative or positive, e.g., the value is 0 for positive and 1 for negative. In the depicted embodiment, the XOR logic gate 1544A receives the sign bit from input 1526 for the first element [0] of the first vector A and receives the sign bit from input 1528 for the first element [0] of the second vector B and outputs a first value (e.g., 1 indicating the product is negative) to indicate false (e.g., negative) when only one of the inputs 1526, 1528 is 1, and otherwise outputs a second value (e.g., 0 indicating the product is positive).

[0209] The converted (e.g., signed) values of each pair of elements (e.g., at the same element position) are then all added together by a set of (e.g., fixed-point) (e.g., integer) adders 1546 to form a single product. The normalization circuit 1550 is then used in this first mode to convert the single product into a floating-point representation based on the maximum exponent. In one embodiment, the circuit 1550 also performs rounding or other operations to convert the (e.g., integer) format of the single product into a floating-point format to create the result of the multiply-accumulate circuit 1501. In one embodiment, the normalization circuit 1550 shifts the most significant bit of the significand (e.g., the single product) of the non-zero digits to be non-zero (e.g., and correspondingly shifts the maximum exponent value according to the shift of the single product).

[0210] Second Mode

[0211] In one embodiment, the mode controller (e.g., via a request to perform a multiply-accumulate operation) is in a second (e.g., fixed-point) mode to perform operations on fixed-point input data.

[0212] As an example, in the second (e.g., fixed-point) mode, the multiply-accumulate circuit will multiply each pair of fixed-point (e.g., integer) values from the same element position (e.g., 0 to N) of the input vectors to form products, and sum these products to produce a single fixed-point result.

[0213] As another example, in a second (e.g., fixed-point) mode, the input source muxes (e.g., muxes 1530 and 1532) are sent control values to source a pair of values from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the values of the two elements of fixed-point vectors A and B having (sorted) index 0 as a pair, source the values of the two elements of fixed-point vectors A and B having (sorted) index 1 as a second pair, source the values of the two elements of fixed-point vectors A and B having (sorted) index 2 as a third pair, and so on). In this example, each pair of values is multiplied together by a multiplier to produce a corresponding product (e.g., the n+1 corresponding products in this figure), e.g., the value A[0]1520 of the element of vector A having (sorted) index 0 and the value B[0]1522 of the element of vector B having (sorted) index 0 are multiplied (e.g., by a fixed-point) multiplier 1534 to produce a product. In the depicted embodiment the products from each multiplier are output to a corresponding shift register (e.g., shifter), but in the second mode the products pass through the shift register without being changed, e.g., the products are not shifted (e.g., moved). In this example, each unshifted product output from each shift register is input to a rounding circuit, and the output from the rounding circuit is input to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the second mode, the products pass through the rounding circuit and / or the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted and / or rounded. The (e.g., signed) products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1546 to form a single fixed-point result. In one embodiment, the single fixed-point result passes through a normalization circuit 1550 without any modification to the single fixed-point result.

[0214] Third Mode

[0215] In one embodiment, the mode controller (e.g., via a request to perform a sum of products operation) is in a third (e.g., inflection point) mode to perform operations on inflection point input data.

[0216] As an example, in a third (e.g., inflection point) mode, each of these inflection point numbers is partitioned into exponent bits, and integer (two's complement) mantissa bits, e.g., rather than binary mantissa bits as in a floating point format. Pairwise multiplication is then performed between the corresponding mantissas of each vector, and the exponents are added together. At the output of the multiplication stage, the mantissas may be truncated or extended to a desired internal bit length (e.g., word length) to trade off resource utilization with solution accuracy. In parallel, all exponents are compared in a tree with log2 n levels. The highest exponent becomes the reference and all mantissas are aligned to this reference. After this alignment in one embodiment of the third mode, the mantissas are not converted to fixed point (e.g., two's complement) number representation because they are already in that format. In one embodiment of the next stage, sum reduction is performed on the mantissas, and at each level of this reduction tree, the bit length (e.g., word length) is increased by one bit to prevent overflow. After this summation, in one embodiment, the result (e.g., mantissa and exponent) is shifted to a desired magnitude (e.g., taking the FlexN+M format described above).

[0217] As another example, in the third mode, the input source muxes (e.g., muxes 1530 and 1532) are sent control values to source a pair of inflection points (two's complement) from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the mantissas of the two elements of inflection point vectors A and B having (sorted) index 0 as a pair, source the mantissas of the two elements of inflection point vectors A and B having (sorted) index 1 as a second pair, source the mantissas of the two elements of inflection point vectors A and B having (sorted) index 2 as a third pair, and so on). In this example, each pair of mantissas is multiplied together by a multiplier to produce a corresponding product (e.g., the n + 1 corresponding products in this figure), e.g., the mantissa A[0]1508 of the element of vector A having (sorted) index 0 and the mantissa B[0]1524 of the element of vector B having (sorted) index 0 are multiplied by (e.g., fixed-point) multiplier 1534 to produce a product. In the depicted embodiment, the products from each multiplier are output to a shift register (e.g., shifter) to shift (e.g., move) each product based on its determined exponent (e.g., the exponent of the product) and the maximum exponent (e.g., the maximum exponent of the product). In this example, a pair of exponents from corresponding (e.g., having the same element position in the vector) elements of the input vector (e.g., source the exponents of the two elements of inflection point vectors A and B having (sorted) index 0 as a pair, source the exponents of the two elements of inflection point vectors A and B having (sorted) index 1 as a second pair, source the exponents of the two elements of inflection point vectors A and B having (sorted) index 2 as a third pair, and so on) are added together by a corresponding (e.g., fixed-point) (e.g., integer) adder to produce a sum of exponents (e.g., the exponent of the product of a pair of elements from vector A and vector B). In this example, each of the sums of exponents (e.g., the n + 1 sums in this figure) is input into a maximum exponent determiner 1540 (e.g., a comparator circuit). For example, the exponent A[0]1514 of the element of vector A having (sorted) index 0 and the exponent B[0]1516 of the element of vector B having (sorted) index 0 are added together by (e.g., fixed-point) (e.g., integer) adder 1538A to produce the sum of exponent A[0]1514 and exponent B[0]1516. The maximum exponent determiner 1540 (e.g., a circuit) then compares the sums of exponents to each other to determine the largest (e.g., maximum) of the sums of exponents, e.g., the maximum exponent among the exponents of the corresponding products of the mantissas. In one embodiment, the largest of the sums of exponents is the maximum exponent of the product of the mantissas. The maximum exponent in the depicted embodiment is then used to shift the other mantissa products accordingly, e.g., to align the other mantissa products to the maximum exponent.In one embodiment, the maximum exponent is 4 (e.g., the sum of two exponents each having a value of 2), and another exponent is 3, so the mantissa product of the other exponents is shifted one place (e.g., to the left) to align the mantissa product to have the same exponent as the maximum exponent. In the depicted embodiment, the maximum exponent (e.g., the value indicating the maximum exponent) is output along path 1541 to a set of shift registers (e.g., including shift register 1536) and normalization circuit 1550. In this example, each shifted mantissa product is output to a corresponding rounding circuit, and the rounded mantissa product is output to a corresponding numeric conversion circuit (e.g., two's complement circuit), but in the third mode, the products pass through the numeric conversion circuit without being changed, e.g., the products are already in two's complement representation, so they are not further converted.

[0218] In one embodiment, the shift register passes the value without shifting, e.g., when the maximum exponent (e.g., the sum of two exponents) is the same as the exponent of the (sorted) index position, e.g., where numeric conversion circuit 1542, rounding circuit 1537, and shift register 1536 are used for the (sorted) index position 0 of the input vector.

[0219] In the depicted embodiment, the shifted mantissa products of each pair of elements (e.g., at the same element position) are then all added together using a set of (e.g., fixed-point) (e.g., integer) adders 1546 to form a single product. Normalization circuit 1550 can then be used in this third mode to convert the single product into an inflection point representation based on the maximum exponent. In one embodiment, instead of a full normalization operation in this mode, circuit 1550 will perform rounding to reduce the bit width of the sum from adder 1546 to the desired bit width of the inflection point format to create the result of product sum circuit 1501 (e.g., where the exponent has no effect on this reduction and the sign has been taken into account since the fixed-point representation is two's complement). In one embodiment, circuit 1550 moves the most significant bit of the significant digits of the non-zero number (e.g., the single product) to be non-zero (e.g., and moves the maximum exponent value accordingly based on the movement of the single product).

[0220] Accumulator

[0221] Optionally, circuit 1500 includes an accumulator 1510 to accumulate the results from the sum-of-products circuit 1501. In one embodiment, a separate fixed-point accumulator circuit 1502 and a floating-point accumulator circuit 1504 are used in accumulator 1510. In certain embodiments, the mode controller 1506 in the first mode causes the results from the sum-of-products circuit 1501 to be accumulated (e.g., only) using the floating-point accumulator circuit 1504 and / or the mode controller 1506 in the second mode causes the results from the sum-of-products circuit 1501 to be accumulated (e.g., only) using the fixed-point accumulator circuit 1502. The depicted fixed-point accumulator circuit 1502 includes (e.g., fixed-point) (e.g., integer) adder 1556 and fixed-point accumulator storage 1558. The depicted floating-point accumulator circuit 1504 includes a floating-point adder 1552 and a floating-point accumulator storage 1554. In one embodiment, the floating-point adder 1552 is an instance of the floating-point adder circuit 500 from Figure 5 of.

[0222] In certain embodiments, circuit 1500 receives (e.g., from a core transferring a load to an accelerator embodiment of circuit 1500) a value indicating the number of iterations for the sum-of-products circuit 1501 to perform, e.g., to decouple the width of a vector (e.g., having a number of elements greater than N + 1) to be processed by circuit 1500 from the hardware width of the sum-of-products circuit 1501 (e.g., a vector having at most N + 1 elements). In one embodiment, circuit 1500 will notify the requesting entity (e.g., the core) of completion by performing a notification action (e.g., raising a flag such that the appropriate accumulator storage 1554 or accumulator storage 1558 is read by the requesting entity). In one embodiment, a floating-point accumulator (e.g., adder) or a fixed-point accumulator (e.g., adder) is used to accumulate the inflection results from circuit 1501, e.g., a sign field is not used in the floating-point accumulator (e.g., adder).

[0223] Figure 16 Embodiments in accordance with the present disclosure illustrate fixed-point and floating-point matrix multiplication circuit 1600. Circuit 1600 includes a plurality (e.g., 32, indexed 0 - 31) of fixed-point and floating-point vector multiplication circuits 1602 and may include a set of (e.g., fixed-point) adders 1604 to allow summing of the accumulated outputs of the fixed-point and floating-point vector multiplication circuits. In one embodiment, each fixed-point and floating-point vector multiplication circuit is Figure 10 the fixed-point and floating-point vector multiplication circuit 1000 of Figure 11 the fixed-point and floating-point vector multiplication 1100 of Figure 12 the fixed-point and floating-point vector multiplication 1200 of Figures 13A - 13C the fixed-point and floating-point vector multiplication 1300 of Figures 14A - 14C the fixed-point and floating-point vector multiplication 1400 of orFigure 15 Examples of fixed-point and floating-point vector multiplication 1500. In one embodiment, circuit 1600 is utilized to perform matrix multiplication, for example, by broadcasting columns of matrix data and unicasting rows of matrix data.

[0224] Figure 17 Embodiments in accordance with the present disclosure are illustrated by flowchart 1700. The depicted process 1700 includes: at 1702, using at least one switch to cause a circuit to change between a first mode and a second mode, the circuit including a set of multipliers and including a maximum exponent determiner, the set of multipliers being coupled to a set of shift registers, the set of shift registers being coupled to a set of numerical conversion circuits, the set of numerical conversion circuits being coupled to a set of adders, the maximum exponent determiner being coupled to the set of shift registers and a normalization circuit; at 1704, in the first mode, each multiplier in the set of multipliers multiplies mantissas at the same element positions from a first floating-point vector and a second floating-point vector to produce a corresponding product, shifts the corresponding product using the set of shift registers based on a maximum exponent among exponents of the corresponding product determined by the maximum exponent determiner to produce a shifted product, performs a numerical conversion operation on the shifted product using the set of numerical conversion circuits based on sign bits at the same element positions from the first floating-point vector and the second floating-point vector to produce a signed representation of the shifted product, adds the signed representations of the shifted product using the set of adders to produce a single product, and normalizes the single product to a single floating-point result using the normalization circuit based on the maximum exponent; and at 1706, in the second mode, each multiplier in the set of multipliers multiplies values at the same element positions from a first fixed-point vector and a second fixed-point vector to produce a corresponding product, and adds each corresponding product using the set of adders to produce a single fixed-point result.

[0225] Certain embodiments herein allow for the use of deep learning operations, where any small errors imparted by the fixed hardware width do not affect the deep learning operations. Certain embodiments herein decouple the input vector width of an operation from the hardware width, for example, by including one or more accumulators such that an entire vector (e.g., greater than 32 elements) iterated through a combined fixed-point and floating-point vector multiplication circuit is not limited to having the maximum width of the fixed-point and floating-point vector multiplication circuit. In certain embodiments, the final accumulation of the entire vector by the accumulator checks the exponent of each iteration against the maximum exponent of each previous iteration of the operation. Certain embodiments herein utilize a non-floating-point adder (e.g., an integer adder) in the accumulator circuit to add numbers in a non-floating-point format for each iteration and then perform a single conversion of the sum of the numbers to a floating-point format, rather than adding multiple floating-point numbers together for each iteration.

[0226] In one embodiment, a device (e.g., an accelerator) includes a circuit that includes a set of multipliers and includes a maximum exponent determiner for generating a result, the set of multipliers being coupled to a set of shift registers, the set of shift registers being coupled to a set of numerical conversion circuits, the set of numerical conversion circuits being coupled to a set of adders, and the maximum exponent determiner being coupled to the set of shift registers and a normalization circuit; and at least one switch for causing the circuit to change between a first mode and a second mode, wherein: in the first mode, each multiplier in the set of multipliers multiplies the mantissas of the same element positions from a first floating-point vector and a second floating-point vector to generate a corresponding product, shifts the corresponding product using the set of shift registers based on the maximum exponent among the exponents of the corresponding products determined by the maximum exponent determiner to generate a shifted product, performs a numerical conversion operation on the shifted product using the set of numerical conversion circuits based on the sign bits of the same element positions from the first floating-point vector and the second floating-point vector to generate a signed representation of the shifted product, adds the signed representations of the shifted products using the set of adders to generate a single product, and normalizes the single product to a single floating-point result using the normalization circuit based on the maximum exponent, and in the second mode, each multiplier in the set of multipliers multiplies the values of the same element positions from a first fixed-point vector and a second fixed-point vector to generate a corresponding product, and adds each corresponding product using the set of adders to generate a single fixed-point result. The device may include an accumulator circuit that includes: a floating-point adder circuit that adds the single floating-point result to a previously generated single floating-point result when the circuit is in the first mode; and a fixed-point adder circuit that adds the single fixed-point result to a previously generated single fixed-point result when the circuit is in the second mode. The normalization circuit may include a rounding circuit for rounding the single product. The device may include an accumulator circuit that includes a fixed-point adder circuit for: when the circuit is in the first mode, adding a single product from the set of adders to a previously generated single product from the set of adders to generate an accumulated product, and normalizing the accumulated product to the single floating-point result using the normalization circuit based on the maximum exponent. The device may include a second instance of the circuit, and the circuit is a first instance of the circuit, wherein when the circuit is in the first mode, the first instance of the circuit operates on a first proper subset of the first floating-point vector having a first maximum exponent and the second instance of the circuit operates on a second proper subset of the second floating-point vector having a second maximum exponent. The first maximum exponent and the second maximum exponent may both be multiples of an integer greater than three.The apparatus may include a sorter circuit for sorting corresponding products from each multiplier of the set of multipliers before shifting each corresponding product by the set of shift registers. The apparatus may include a set of rounding circuits between the set of shift registers and the set of numerical conversion circuits for rounding the shifted products.

[0227] In another embodiment, a method includes using at least one switch to switch a circuit between a first mode and a second mode, the circuit including a set of multipliers and including a maximum exponent determiner, the set of multipliers being coupled to a set of shift registers, the set of shift registers being coupled to a set of numerical conversion circuits, the set of numerical conversion circuits being coupled to a set of adders, and the maximum exponent determiner being coupled to the set of shift registers and a normalization circuit; in the first mode, each multiplier in the set of multipliers multiplies the mantissas at the same element positions of a first floating-point vector and a second floating-point vector to produce a corresponding product, uses the set of shift registers to shift the corresponding product based on the maximum exponent among the exponents of the corresponding product determined by the maximum exponent determiner to produce a shifted product, performs a numerical conversion operation on the shifted product using the set of numerical conversion circuits based on the sign bits at the same element positions of the first floating-point vector and the second floating-point vector to produce a signed representation of the shifted product, uses the set of adders to add the signed representations of the shifted products to produce a single product, and normalizes the single product to a single floating-point result using the normalization circuit based on the maximum exponent; and in the second mode, each multiplier in the set of multipliers multiplies the values at the same element positions of a first fixed-point vector and a second fixed-point vector to produce a corresponding product, and uses the set of adders to add each corresponding product to produce a single fixed-point result. The method may include: adding the single floating-point result to a previously generated single floating-point result using a floating-point adder circuit when the circuit is in the first mode; and adding the single fixed-point result to a previously generated single fixed-point result using a fixed-point adder circuit when the circuit is in the second mode. The method may include: using a rounding circuit to round the single product. The method may include: when the circuit is in the first mode, adding a single product from the set of adders to a previously generated single product from the set of adders using a fixed-point adder circuit to produce an accumulated product, and normalizing the accumulated product to the single floating-point result using the normalization circuit based on the maximum exponent. The method may include: when the circuit is in the first mode, operating the circuit on a first proper subset of the first floating-point vector having a first maximum exponent, and operating a second instance of the circuit on a second proper subset of the second floating-point vector having a second maximum exponent. The method may include wherein the first maximum exponent and the second maximum exponent are both multiples of an integer greater than three. The method may include sorting the corresponding products from each multiplier in the set of multipliers using a sorter circuit before shifting each corresponding product using the set of shift registers. The method may include rounding the shifted product using a set of rounding circuits between the set of shift registers and the set of numerical conversion circuits.

[0228] In another embodiment, a non-transitory machine-readable medium storing code that, when executed by a machine, causes the machine to perform a method that includes switching a circuit between a first mode and a second mode using at least one switch, the circuit including a set of multipliers and including a maximum exponent determiner, the set of multipliers being coupled to a set of shift registers, the set of shift registers being coupled to a set of numerical conversion circuits, the set of numerical conversion circuits being coupled to a set of adders, and the maximum exponent determiner being coupled to the set of shift registers and a normalization circuit; in the first mode, each multiplier in the set of multipliers multiplies the mantissas at the same element positions of a first floating-point vector and a second floating-point vector to produce a corresponding product, shifts the corresponding product using the set of shift registers based on the maximum exponent among the exponents of the corresponding product determined by the maximum exponent determiner to produce a shifted product, performs a numerical conversion operation on the shifted product using the set of numerical conversion circuits based on the sign bits at the same element positions of the first floating-point vector and the second floating-point vector to produce a signed representation of the shifted product, adds the signed representations of the shifted product using the set of adders to produce a single product, and normalizes the single product to a single floating-point result using the normalization circuit based on the maximum exponent; and in the second mode, each multiplier in the set of multipliers multiplies the values at the same element positions of a first fixed-point vector and a second fixed-point vector to produce a corresponding product, and adds each corresponding product using the set of adders to produce a single fixed-point result. The method may include: adding the single floating-point result to a previously generated single floating-point result using a floating-point adder circuit when the circuit is in the first mode; and adding the single fixed-point result to a previously generated single fixed-point result using a fixed-point adder circuit when the circuit is in the second mode. The method may include: rounding the single product using a rounding circuit. The method may include: when the circuit is in the first mode, adding a single product from the set of adders to a previously generated single product from the set of adders using a fixed-point adder circuit to produce an accumulated product, and normalizing the accumulated product to the single floating-point result using the normalization circuit based on the maximum exponent. The method may include: when the circuit is in the first mode, operating the circuit on a first proper subset of the first floating-point vector having a first maximum exponent, and operating a second instance of the circuit on a second proper subset of the second floating-point vector having a second maximum exponent. The method may include where the first maximum exponent and the second maximum exponent are both multiples of an integer greater than three. The method may include: sorting the corresponding products from each multiplier in the set of multipliers using a sorter circuit before shifting each corresponding product using the set of shift registers.The method may include: rounding the shifted product by a set of rounding circuits between the set of shift registers and the set of numerical conversion circuits.

[0229] In another embodiment, an apparatus (e.g., an accelerator) includes a circuit that includes a set of multipliers and includes a maximum exponent determiner for generating a result, the set of multipliers being coupled to a set of shift registers, the set of shift registers being coupled to a set of numerical conversion circuits, the set of numerical conversion circuits being coupled to a set of adders, and the maximum exponent determiner being coupled to the set of shift registers and a normalization circuit; and means for causing the circuit to change between a first mode and a second mode, wherein: in the first mode, each multiplier in the set of multipliers multiplies the mantissas of the same element positions from a first floating-point vector and a second floating-point vector to generate a corresponding product, the corresponding product is shifted using the set of shift registers based on the maximum exponent among the exponents of the corresponding product determined by the maximum exponent determiner to generate a shifted product, a numerical conversion operation is performed on the shifted product using the set of numerical conversion circuits based on the sign bits of the same element positions from the first floating-point vector and the second floating-point vector to generate a signed representation of the shifted product, the signed representations of the shifted product are added together using the set of adders to generate a single product, and the single product is normalized to a single floating-point result using the normalization circuit based on the maximum exponent, and in the second mode, each multiplier in the set of multipliers multiplies the values of the same element positions from a first fixed-point vector and a second fixed-point vector to generate a corresponding product, and the corresponding products are added together using the set of adders to generate a single fixed-point result.

[0230] In yet another embodiment, an apparatus includes a data storage device storing code that, when executed by a hardware processor, causes the hardware processor to perform any of the methods disclosed herein. An apparatus may be as described in the detailed description. A method may be as described in the detailed description.

[0231] An instruction set may include one or more instruction formats. A given instruction format may define various fields (e.g., number of bits, position of bits) to specify the operation to be performed (e.g., opcode) and the operand(s) on which the operation is to be performed and / or other data field(s) (e.g., mask), etc. Some instruction formats are further decomposed by the definition of instruction templates (or sub-formats). For example, an instruction template of a given instruction format may be defined to have different subsets of the fields of that instruction format (the included fields are typically in the same order, but at least some have different bit positions since fewer fields are included) and / or be defined to interpret the given fields differently. Thus, each instruction of the ISA is expressed using a given instruction format (and if defined, expressed using a given one of the instruction templates of that instruction format) and includes fields for specifying the operation and operands. For example, an exemplary ADD instruction has a specific opcode and an instruction format that includes an opcode field to specify the opcode and includes operand fields to select the operands (source1 / destination and source2); and the appearance of this ADD instruction in the instruction stream will have specific contents in the operand fields that select the specific operands. A set of SIMD extensions called Advanced Vector Extensions (AVX) (AVX1 and AVX2) and using the Vector Extensions (VEX) encoding scheme have been published and / or made available (e.g., see the 64 and IA-32 Architectures Software Developer's Guide published in May 2018 and see the Architecture Instruction Set Extension Programming Reference) published in May 2018).

[0232] Exemplary Instruction Formats

[0233] Embodiments of the instruction(s) described herein may be implemented in different formats. Additionally, exemplary systems, architectures, and pipelines are detailed below. Embodiments of the instruction(s) may be executed on such systems, architectures, and pipelines, but are not limited to those detailed.

[0234] General Vector-Friendly Instruction Formats

[0235] A vector-friendly instruction format is an instruction format suitable for vector instructions (e.g., having certain fields specific to vector operations). Although embodiments are described in which both vector and scalar operations are supported via a vector-friendly instruction format, alternative embodiments use only vector operations of the vector-friendly instruction format.

[0236] Figures 18A - 18B is a block diagram illustrating a general vector-friendly instruction format and its instruction templates in accordance with embodiments of the present disclosure.Figure 18A is a block diagram illustrating a general vector friendly instruction format and its class A instruction templates in accordance with an embodiment of the present disclosure; and Figure 18B is a block diagram illustrating a general vector friendly instruction format and its class B instruction templates in accordance with an embodiment of the present disclosure. Specifically, for the general vector friendly instruction format 1800, class A and class B instruction templates are defined, both of which include a memory access-free 1805 instruction template and a memory access 1820 instruction template. The term "general" in the context of the vector friendly instruction format means that the instruction format is not bound to any particular instruction set.

[0237] While embodiments of the present disclosure will be described in which the vector friendly instruction format supports the following: a 64-byte vector operand length (or size) with 32-bit (4-byte) or 64-bit (8-byte) data element widths (or sizes) (thus, a 64-byte vector is composed of 16 double-word sized elements or 8 quad-word sized elements); a 64-byte vector operand length (or size) with 16-bit (2-byte) or 8-bit (1-byte) data element widths (or sizes); a 32-byte vector operand length (or size) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or sizes); and a 16-byte vector operand length (or size) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or sizes); alternative embodiments may support more, fewer, and / or different vector operand sizes (e.g., a 256-byte vector operand) with more, fewer, or different data element widths (e.g., a 128-bit (16-byte) data element width).

[0238] Figure 18A The class A instruction templates in include: 1) within the memory access-free 1805 instruction template, a memory access-free, fully rounded control type operation 1810 instruction template and a memory access-free, data transformation type operation 1815 instruction template are shown; and 2) within the memory access 1820 instruction template, a memory access, transient 1825 instruction template and a memory access, non-transient 1830 instruction template are shown. Figure 18B The class B instruction templates in include: 1) within the memory access-free 1805 instruction template, a memory access-free, write mask control, partially rounded control type operation 1812 instruction template and a memory access-free, write mask control, vsize type operation 1817 instruction template are shown; and 2) within the memory access 1820 instruction template, a memory access, write mask control 1827 instruction template is shown.

[0239] The general vector-friendly instruction format 1800 includes the following fields listed in the order shown in Figures 18A - 18B as follows.

[0240] Format field 1840 – The specific value in this field (the instruction format identifier value) uniquely identifies the vector-friendly instruction format, and thus identifies the occurrence of an instruction in the instruction stream that takes the vector-friendly instruction format. Thus, this field is optional because it is not needed for an instruction set that has only the general vector-friendly instruction format.

[0241] Base operation field 1842 – Its content differentiates different base operations.

[0242] Register index field 1844 – Its content directly or through address generation specifies the locations of source and target operands, whether they are in registers or in memory. These include a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. Although in one embodiment N can be up to three sources and one target register, alternative embodiments may support more or fewer sources and target registers (e.g., may support up to two sources, where one of these sources also acts as a target, may support up to three sources, where one of these sources also acts as a target, may support up to two sources and one target).

[0243] Modifier field 1846 – Its content differentiates the occurrence of instructions that specify memory access in the general vector instruction format from those that do not; that is, differentiates the no-memory-access 1805 instruction template and the memory-access 1820 instruction template. Memory access operations read and / or write to the memory hierarchy (in some cases using values in registers to specify source and / or target addresses), while non-memory access operations do not read and / or write to the memory hierarchy (e.g., the sources and targets are registers). Although in one embodiment this field also selects among three different ways to perform memory address calculation, alternative embodiments may support more, fewer, or different ways to perform memory address calculation.

[0244] Enhanced operation field 1850 – Its content differentiates which of a variety of different additional operations to perform in addition to the base operation. This field is context-dependent. In one embodiment of the present disclosure, this field is divided into a category field 1868, an alpha field 1852, and a beta field 1854. The enhanced operation field 1850 allows a common group of operations to be performed in a single instruction rather than in 2, 3, or 4 instructions.

[0245] Scale factor field 1860 – Its content allows the content of the scaled index field to be scaled for memory address generation (e.g., for use with 2缩放比例 *(Address generation of index + base address).

[0246] Displacement field 1862A - whose content is used as part of memory address generation (e.g., for use 2 缩放比例 *(Address generation of index + base address + displacement).

[0247] Displacement factor field 1862B (note that juxtaposing displacement field 1862A directly above displacement factor field 1862B indicates that one or the other is used) - whose content is used as part of address generation; it specifies a displacement factor to be scaled by the size (N) of the memory access - where N is the number of bytes in the memory access (e.g., for use 2 缩放比例 *(Address generation of index + base address + scaled displacement). Redundant low-order bits are ignored, and thus, the content of the displacement factor field is multiplied by the total size (N) of the memory operand to generate the final displacement to be used in calculating the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 1874 (described later herein) and the data manipulation field 1854C. The displacement field 1862A and the displacement factor field 1862B are optional because they are not used in the no-memory-access 1805 instruction template, and / or different embodiments may implement only one of the two or neither.

[0248] Data element width field 1864 - whose content differentiates which of several data element widths will be used (in some embodiments for all instructions; in other embodiments only for some of the instructions). This field is optional because it is not needed if only one data element width is supported and / or the data element width is supported using some aspect of the opcode.

[0249] Write mask field 1870 - its content controls whether the data element position in the destination vector operand reflects the results of the primary operation and the enhancement operation based on each data element position. The Category A instruction templates support merge-write masking, while the Category B instruction templates support both merge-write masking and zero-write masking. When merged, the vector mask allows any set of elements in the destination to be protected from update during the execution of any operation (specified by the primary operation and the enhancement operation); in another embodiment, the old value of each element with the corresponding mask bit in the destination being 0 is retained. In contrast, the zero vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the primary operation and the enhancement operation); in one embodiment, the elements of the destination are set to 0 when the corresponding mask bit has a value of 0. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span of the elements being modified, from the first to the last); however, the elements being modified are not required to be contiguous. Thus, the write mask field 1870 allows partial vector operations, including loads, stores, arithmetic, logic, etc. Although embodiments of the present disclosure have been described in which the content of the write mask field 1870 selects which one of a number of write mask registers contains the write mask to be used (and thus the content of the write mask field 1870 indirectly identifies the masking to be performed), alternative embodiments allow, alternatively or additionally, the content of the mask write field 1870 to directly specify the masking to be performed.

[0250] Immediate field 1872 - its content allows the specification of an immediate. This field is optional as it does not exist in implementations of the general vector-friendly format that do not support immediates and does not exist in instructions that do not use immediates.

[0251] Category field 1868 - its content differentiates between different categories of instructions. Refer to Figures 18A - 18B , the content of this field selects between Category A and Category B instructions. In Figures 18A - 18B , rounded rectangles are used to indicate that a particular value exists in a field (e.g., Figures 18A - 18B Category A 1868A and Category B 1868B for the category field 1868, respectively).

[0252] Instruction templates for Category A

[0253] In the case of the non-memory access 1805 instruction template of category A, the alpha field 1852 is interpreted as the RS field 1852A, the content of which differentiates which one of the different enhanced operation types is to be performed (e.g., for the non-memory access rounding operation 1810 and the non-memory access data transformation operation 1815 instruction templates, rounding 1852A.1 and data transformation 1852A.2 are specified respectively), while the beta field 1854 differentiates which one of the operations of the specified type is to be performed. In the non-memory access 1805 instruction template, the scale factor field 1860, the displacement field 1862A, and the displacement scale factor field 1862B do not exist.

[0254] Non-memory access instruction template – fully rounded control operation

[0255] In the non-memory access fully rounded control operation 1810 instruction template, the beta field 1854 is interpreted as the rounding control field 1854A, the content of which provides static rounding. Although in the described embodiments of the present disclosure, the rounding control field 1854A includes a suppress all floating point exceptions (SAE) field 1856 and a rounding operation control field 1858, alternative embodiments may support encoding both concepts into the same field or may have only one or the other of these concepts / fields (e.g., may have only the rounding operation control field 1858).

[0256] SAE field 1856 – the content of which differentiates whether to disable the reporting of exception events; when the content of the SAE field 1856 indicates that suppression is enabled, a given instruction does not report any kind of floating point exception flag and does not trigger any floating point exception handler.

[0257] Rounding operation control field 1858 – the content of which differentiates which one of a set of rounding operations is to be performed (e.g., round up, round down, round towards zero, or round towards the nearest). Thus, the rounding operation control field 1858 allows the rounding mode to be changed on a per-instruction basis. In one embodiment of the present disclosure where the processor includes a control register for specifying the rounding mode, the content of the rounding operation control field 1850 overrides the register value.

[0258] Non-memory access instruction template – data transformation operation

[0259] In the non-memory access data transformation operation 1815 instruction template, the beta field 1854 is interpreted as the data transformation field 1854B, the content of which differentiates which one of several data transformations is to be performed (e.g., no data transformation, swizzle, broadcast).

[0260] In the case of the memory access 1820 instruction template of class A, the alpha field 1852 is interpreted as an eviction hint field 1852B, the content of which differentiates which one of the eviction hints is to be used (in Figure 18A for the memory access transient 1825 instruction template and the memory access non-transient 1830 instruction template, transient 1852B.1 and non-transient 1852B.2 are specified respectively), while the beta field 1854 is interpreted as a data manipulation field 1854C, the content of which differentiates which one of several data manipulation operations (also called primitives) is to be executed (e.g., no manipulation; broadcast; up-conversion of source; and down-conversion of destination). The memory access 1820 instruction template includes a scale factor field 1860 and optionally includes a displacement field 1862A or a displacement scale factor field 1862B.

[0261] Vector memory instructions perform vector loads from and vector stores to memory, with conversion support. Like conventional vector instructions, vector memory instructions transfer data to / from memory in a per-data-element manner, where the elements actually transferred are specified by the content of a vector mask that is selected as a write mask.

[0262] Memory access instruction template – transient

[0263] Transient data is data that may be reused soon enough to benefit from being cached. However, this is a hint, and different processors may implement it in different ways, including completely ignoring the hint.

[0264] Memory access instruction template – non-transient

[0265] Non-transient data is data that is not likely to be reused soon enough to benefit from caching in the level 1 cache and should be given eviction priority. However, this is a hint, and different processors may implement it in different ways, including completely ignoring the hint.

[0266] Instruction template of class B

[0267] In the case of the instruction template of class B, the alpha field 1852 is interpreted as a write mask control (Z) field 1852C, the content of which differentiates whether the write masking controlled by the write mask field 1870 should be merge or zero.

[0268] In the case of a class B non-memory access 1805 instruction template, a portion of the beta field 1854 is interpreted as an RL field 1857A, the content of which differentiates which one of different enhanced operation types is to be performed (e.g., specifying round 1857A.1 and vector length (VSIZE) 1857A.2 for the no-memory access, write mask control, partial rounding control type operation 1812 instruction template and the no-memory access, write mask control, VSIZE type operation 1817 instruction template respectively), while the remainder of the beta field 1854 differentiates which one of the operations of the specified type is to be performed. In the no-memory access 1805 instruction template, the scale factor field 1860, displacement field 1862A, and displacement scale factor field 1862B do not exist.

[0269] In the no-memory access, write mask control, partial rounding control type operation 1810 instruction template, the remainder of the beta field 1854 is interpreted as a rounding operation field 1859A and exception event reporting is disabled (the given instruction does not report any kind of floating point exception flag and does not trigger any floating point exception handler).

[0270] The rounding operation control field 1859A – like the rounding operation control field 1858, its content differentiates which one of a set of rounding operations is to be performed (e.g., round up, round down, round towards zero or round towards nearest). Thus, the rounding operation control field 1859A allows the rounding mode to be changed on a per-instruction basis. In one embodiment of the present disclosure where the processor includes a control register for specifying the rounding mode, the content of the rounding operation control field 1850 overrides the register value.

[0271] In the no-memory access, write mask control, VSIZE type operation 1817 instruction template, the remainder of the beta field 1854 is interpreted as a vector length field 1859B, the content of which differentiates on which one of several data vector lengths the operation is to be performed (e.g., 128, 256 or 512 bytes).

[0272] In the case of a class B memory access 1820 instruction template, a portion of the beta field 1854 is interpreted as a broadcast field 1857B, the content of which differentiates whether a broadcast type data manipulation operation is to be performed, while the remainder of the beta field 1854 is interpreted as a vector length field 1859B. The memory access 1820 instruction template includes a scale factor field 1860, and optionally includes a displacement field 1862A or a displacement scale factor field 1862B.

[0273] For the general vector friendly instruction format 1800, the full opcode field 1874 is shown as including a format field 1840, a base operation field 1842, and a data element width field 1864. Although one embodiment is shown in which the full opcode field 1874 includes all of these fields, the full opcode field 1874 may include only some of these fields in embodiments that do not support all of these fields. The full opcode field 1874 provides the operation code (opcode).

[0274] The enhanced operation field 1850, the data element width field 1864, and the write mask field 1870 allow these features to be specified on a per-instruction basis in the general vector friendly instruction format.

[0275] The combination of the write mask field and the data element width field creates typed instructions because they allow the mask to be applied based on different data element widths.

[0276] The various instruction templates found within categories A and B are beneficial in different scenarios. In some embodiments of the present disclosure, different processors or different cores within a processor may support only category A, only support category B, or support both categories. For example, a high-performance general out-of-order core intended for general computing may support only category B, a core intended primarily for graphics and / or scientific (throughput) computing may support only category A, and a core intended for both may support both (of course, a core having some mix of templates and instructions from both categories, but not all templates and instructions from both categories, is within the scope of the present disclosure). Additionally, a single processor may include multiple cores, all of which support the same category or where different cores support different categories. For example, in a processor having separate graphics and general cores, one of the graphics cores intended primarily for graphics and / or scientific computing may support only category A, while one or more of the general cores may be high-performance general cores with out-of-order execution and register renaming intended for general computing that support only category B. Another processor without a separate graphics core may include one or more general in-order or out-of-order cores that support both category A and category B. Of course, in different embodiments of the present disclosure, features from one category may also be implemented in the other category. Programs written in a high-level language will be placed (e.g., just-in-time compiled or statically compiled) into a variety of different executable forms, including: 1) a form having only the instructions of (one or more) categories supported by the target processor for execution; or 2) a form having replacement routines written using different combinations of instructions from all categories and having control flow code that selects which routine to execute based on the instructions supported by the processor currently executing the code.

[0277] Exemplary specific vector friendly instruction formats

[0278] FIG. 19 is a block diagram illustrating an exemplary specific vector-friendly instruction format in accordance with an embodiment of the present disclosure. FIG. 19 shows a specific vector-friendly instruction format 1900 that is specific in the sense that it specifies the positions, sizes, interpretations, and order of fields, as well as the values of some of these fields. The specific vector-friendly instruction format 1900 can be used to extend the x86 instruction set such that some of the fields are similar or identical to those used in the existing x86 instruction set and its extensions (e.g., AVX). This format is consistent with the prefix coding field, the real opcode byte field, the MOD R / M field, the SIB field, the displacement field, and the immediate field of the existing x86 instruction set with extensions. The figure shows the fields from FIG. 19 mapped to the fields from FIG. 18.

[0279] It should be understood that while embodiments of the present disclosure are described with reference to the specific vector-friendly instruction format 1900 in the context of the general vector-friendly instruction format 1800 for illustrative purposes, the present disclosure is not limited to the specific vector-friendly instruction format 1900 unless stated otherwise. For example, the general vector-friendly instruction format 1800 contemplates a variety of possible sizes for various fields, while the specific vector-friendly instruction format 1900 is shown as having fields of specific sizes. As a specific example, while the data element width field 1864 is shown as a one-bit field in the specific vector-friendly instruction format 1900, the present disclosure is not limited thereto (i.e., the general vector-friendly instruction format 1800 contemplates other sizes for the data element width field 1864).

[0280] The general vector-friendly instruction format 1800 includes the following fields listed below in the order shown Figure 19A in.

[0281] EVEX prefix (bytes 0-3) 1902 – Encoded in a four-byte form.

[0282] Format field 1840 (EVEX byte 0, bits [7:0]) - The first byte (EVEX byte 0) is the format field 1840 and it contains 0x62 (a unique value used to distinguish the vector-friendly instruction format in one embodiment of the present disclosure).

[0283] The second - fourth bytes (EVEX bytes 1-3) include a number of bit fields that provide specific capabilities.

[0284] REX Field 1905 (EVEX Byte 1, Bits [7-5]) - Consists of the EVEX.R bit field (EVEX Byte 1, Bit [7] – R), the EVEX.X bit field (EVEX Byte 1, Bit [6] – X), and 1857BEX Byte 1, Bit [5] – B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded in inverse form, i.e., ZMM0 is encoded as 1111B and ZMM15 is encoded as 0000B. Other fields of the instruction encode the lower three bits of the register index (rrr, xxx, and bbb) as known in the art, such that Rrrr, Xxxx, and Bbbb can be formed by adding EVEX.R, EVEX.X, and EVEX.B.

[0285] REX’ Field 1810 – This is the first part of the REX’ Field 1810 and is the EVEX.R’ bit field (EVEX Byte 1, Bit [4] - R’) used to encode the upper 16 or lower 16 of the extended 32 register set. In one embodiment of the present disclosure, this bit and other bits as shown below are stored in bit-reversed format to distinguish from the BOUND instruction (in the well-known x86 32-bit mode), whose true opcode byte is 62 but does not accept the value 11 in the MOD field in the MOD R / M field (described below); alternative embodiments of the present disclosure do not store this bit and other bits indicated below in reversed format. The value 1 is used to encode the lower 16 registers. In other words, R’Rrrr is formed by combining EVEX.R’, EVEX.R, and other RRRs from other fields.

[0286] Opcode Map Field 1915 (EVEX Byte 1, Bits [3:0] – mmmm) – Its content encodes the implied leading opcode byte (0F, 0F 38, or 0F 3).

[0287] Data Element Width Field 1864 (EVEX Byte 2, Bit [7] – W) - Represented by the symbol EVEX.W. EVEX.W is used to define the granularity (size) of the data type (32-bit data element or 64-bit data element).

[0288] EVEX.vvvv 1920 (EVEX byte 2, bits [6:3] - vvvv) - The role of EVEX.vvvv may include the following: 1) EVEX.vvvv encodes the first source register operand specified in inverted (complemented) form and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand specified in complemented form for certain vector shifts; or 3) EVEX.vvvv does not encode any operand, and this field is reserved and should contain 1111b. Thus, the EVEX.vvvv field 1920 encodes the 4 low-order bits of the first source register specifier stored in inverted (complemented) form. Depending on the instruction, an additional different EVEX bit field is used to extend the specifier size to 32 registers.

[0289] EVEX.U 1868 class field (EVEX byte 2, bit [2] - U) – If EVEX.U = 0, it indicates class A or EVEX.U0; if EVEX.U = 1, it indicates class B or EVEX.U1.

[0290] Prefix encoding field 1925 (EVEX byte 2, bits [1:0] - pp) – Provides additional bits for the basic operation field. In addition to supporting traditional SSE instructions that take the EVEX prefix format, this also has the benefit of compacting the SIMD prefix (the EVEX prefix only requires 2 bits instead of a byte to represent the SIMD prefix). In one embodiment, to support traditional SSE instructions that use SIMD prefixes (66H, F2H, F3H) in both traditional and EVEX prefix formats, these traditional SIMD prefixes are encoded into the SIMD prefix encoding field; and at runtime, they are extended into the traditional SIMD prefix before being provided to the PLA of the decoder (so the PLA can execute both the traditional and EVEX formats of these traditional instructions without modification). Although updated instructions can directly use the content of the EVEX prefix encoding field as an opcode extension, some embodiments extend it in a similar way for consistency but allow these traditional SIMD prefixes to specify different meanings. Alternative embodiments may redesign the PLA to support 2-bit SIMD prefix encoding, thus not requiring extension.

[0291] Alpha field 1852 (EVEX byte 3, bit [7] – EH; also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.write mask control, and EVEX.N; also illustrated with α) – As described previously, this field is context-dependent.

[0292] Beta field 1854 (EVEX byte 3, bits [6:4] – SSS; also known as EVEX.s2-0, EVEX.r2-0, EVEX.rr1, EVEX.LL0, EVEX.LLB; also illustrated by βββ) – As previously mentioned, this field is context-dependent.

[0293] REX’ field 1810 – This is the remaining part of the REX’ field and is the EVEX.V’ bit field (EVEX byte 3, bit [3] - V’) that can be used to encode the upper 16 or lower 16 of the extended 32 register set. This bit is stored in bit-reversed format. The value 1 is used to encode the lower 16 registers. In other words, V’VVVV is formed by combining EVEX.V’ and EVEX.vvvv.

[0294] Write mask field 1870 (EVEX byte 3, bits [2:0] - kkk) - Its content specifies the index of the register in the write mask register as previously mentioned. In one embodiment of the present disclosure, the specific value EVEX.kkk = 000 has a special behavior indicating that no write mask is used for a particular instruction (this can be achieved in various ways, including using a write mask hard-wired to all ones or hardware that bypasses the masking hardware).

[0295] True opcode field 1930 (byte 4) is also referred to as the opcode byte. Part of the opcode is specified in this field.

[0296] MOD R / M field 1940 (byte 5) includes MOD field 1942, Reg field 1944, and R / M field 1946. As previously mentioned, the content of the MOD field 1942 differentiates between memory access and non-memory access operations. The role of the Reg field 1944 can be summarized into two cases: encoding the destination register operand or the source register operand, or being treated as an opcode extension and not being used to encode any instruction operand. The role of the R / M field 1946 can include the following: encoding the instruction operand that references the memory address, or encoding the destination register operand or the source register operand.

[0297] Scale, Index, Base (SIB) byte (byte 6) - As previously mentioned, the content of the scale field 1850 is used for memory address generation. SIB.xxx1954 and SIB.bbb 1956 – The content of these fields has been previously mentioned for register indices Xxxx and Bbbb.

[0298] Displacement field 1862A (bytes 7 - 10) - When the MOD field 1942 contains 10, bytes 7 - 10 are the displacement field 1862A, and it operates the same as a traditional 32 - bit displacement (disp32) and works at the byte granularity.

[0299] Displacement factor field 1862B (byte 7) - When the MOD field 1942 contains 01, byte 7 is the displacement factor field 1862B. The position of this field is the same as that of the 8 - bit displacement (disp8) in the traditional x86 instruction set, and it works at the byte granularity. Since disp8 is sign - extended, it can only address between - 128 and 127 byte offsets; in terms of a 64 - byte cache line, disp8 uses 8 bits, and these 8 bits can be set to only four really useful values: - 128, - 64, 0, and 64; since a larger range is often needed, disp32 is used; however, disp32 requires 4 bytes. Different from disp8 and disp32, the displacement factor field 1862B is a reinterpretation of disp8; when the displacement factor field 1862B is used, the actual displacement is determined by multiplying the content of the displacement factor field by the size (N) of the memory operand access. This type of displacement is called disp8*N. This reduces the average instruction length (a single byte is used for the displacement but has a much larger range). This compressed displacement is based on the assumption that the effective displacement is a multiple of the memory access granularity, and thus, the redundant low - order bits of the address offset do not need to be encoded. In other words, the displacement factor field 1862B replaces the 8 - bit displacement of the traditional x86 instruction set. Thus, the displacement factor field 1862B is encoded in the same way as the 8 - bit displacement of the x86 instruction set (so there is no change in the ModRM / SIB encoding rules), with the only exception being that disp8 is overloaded to disp8*N. In other words, there is no change in the encoding rules or encoding length, only a change in the hardware's interpretation of the displacement value (the hardware needs to scale the displacement by the size of the memory operand to obtain the byte - address offset). The immediate field 1872 operates as described above.

[0300] Full opcode field

[0301] Figure 19B is a block diagram showing the fields that constitute a specific vector - friendly instruction format 1900 of the full opcode field 1874 according to an embodiment of the present disclosure. Specifically, the full opcode field 1874 includes a format field 1840, a basic operation field 1842, and a data element width (W) field 1864. The basic operation field 1842 includes a prefix encoding field 1925, an opcode mapping field 1915, and a true opcode field 1930.

[0302] Register index field

[0303] Figure 19C

[0303] is a block diagram showing the fields of a specific vector-friendly instruction format 1900 that constitutes the register index field 1844 according to an embodiment of the present disclosure. Specifically, the register index field 1844 includes a REX field 1905, a REX' field 1910, a MODR / M.reg field 1944, a MODR / M.r / m field 1946, a VVVV field 1920, an xxx field 1954, and a bbb field 1956.

[0304] Enhanced operation field

[0305] Figure 19D

[0305] is a block diagram showing the fields of a specific vector-friendly instruction format 1850 that constitutes the enhanced operation field 1900 according to an embodiment of the present disclosure. When the class (U) field 1868 contains 0, it represents EVEX.U0 (class A 1868A); when it contains 1, it represents EVEX.U1 (class B 1868B). When U = 0 and the MOD field 1942 contains 11 (indicating no memory access operation), the alpha field 1852 (EVEX byte 3, bit [7] – EH) is interpreted as the rs field 1852A. When the rs field 1852A contains 1 (rounding 1852A.1), the beta field 1854 (EVEX byte 3, bits [6:4] - SSS) is interpreted as the rounding control field 1854A. The rounding control field 1854A includes a one-bit SAE field 1856 and a two-bit rounding operation field 1858. When the rs field 1852A contains 0 (data transformation 1852A.2), the beta field 1854 (EVEX byte 3, bits [6:4] - SSS) is interpreted as a three-bit data transformation field 1854B. When U = 0 and the MOD field 1942 contains 00, 01, or 10 (indicating a memory access operation), the alpha field 1852 (EVEX byte 3, bit [7] – EH) is interpreted as the eviction hint (EH) field 1852B and the beta field 1854 (EVEX byte 3, bits [6:4] - SSS) is interpreted as a three-bit data manipulation field 1854C.

[0306] When U = 1, the alpha field 1852 (EVEX byte 3, bit [7] – EH) is interpreted as the write mask control (Z) field 1852C. When U = 1 and the MOD field 1942 contains 11 (indicating no memory access operation), a part of the beta field 1854 (EVEX byte 3, bit [4] – S0) is interpreted as the RL field 1857A; when it contains 1 (rounding 1857A.1), the rest of the beta field 1854 (EVEX byte 3, bits [6 - 5] - S 2-1) is interpreted as the rounding operation field 1859A, and when the RL field 1857A contains 0 (VSIZE 1857.A2), the remainder of the beta field 1854 (EVEX byte 3, bits [6-5]-S 2-1 ) is interpreted as the vector length field 1859B (EVEX byte 3, bits [6-5]-L 1-0 ). When U = 1 and the MOD field 1942 contains 00, 01, or 10 (indicating a memory access operation), the beta field 1854 (EVEX byte 3, bits [6:4]-SSS) is interpreted as the vector length field 1859B (EVEX byte 3, bits [6-5]-L 1-0 ) and the broadcast field 1857B (EVEX byte 3, bit [4]-B).

[0307] Exemplary Register Architecture

[0308] Figure 20 is a block diagram of a register architecture 2000 according to an embodiment of the present disclosure. In the illustrated embodiment, there are 32 vector registers 2010 that are 512 bits wide; these registers are referred to as zmm0 through zmm31. The lower 256 bits of the lower 16 zmm registers are overlaid on the registers ymm0-16. The lower 128 bits of the lower 16 zmm registers (the lower 128 bits of the ymm registers) are overlaid on the registers xmm0-15. Certain vector-friendly instruction formats 1900 operate on these overlaid register files as shown in the following table.

[0309]

[0310] In other words, the vector length field 1859B selects between a maximum length and one or more other shorter lengths, where each such shorter length is half of the previous length; and instruction templates without a vector length field 1859B operate on the maximum vector length. Additionally, in one embodiment, the class B instruction templates of the certain vector-friendly instruction format 1900 operate on packed or scalar single / double precision floating-point data and packed or scalar integer data. A scalar operation is an operation performed on the lowest-order data element position in a zmm / ymm / xmm register; higher-order data element positions are either left the same as they were before the instruction or are zeroed, depending on the embodiment.

[0311] Write Mask Registers 2015 – In the illustrated embodiment, there are eight write mask registers (k0 through k7), each 64 bits in size. In an alternative embodiment, the write mask registers 2015 are 16 bits in size. As previously mentioned, in one embodiment of the present disclosure, vector mask register k0 can be used as a write mask; when the encoding that would normally indicate k0 is used for a write mask, it selects the hard-wired write mask 0xFFFF, effectively disabling write masking for that instruction.

[0312] General-Purpose Registers 2025 – In the illustrated embodiment, there are sixteen 64-bit general-purpose registers that are used, along with existing x86 addressing modes, to address memory operands. These registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0313] Scalar Floating-Point Stack Register File (x87 Stack) 2045, on which is aliased the MMX Packed Integer Flat Register File 2050 – In the illustrated embodiment, the x87 stack is an eight-element stack used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using the x87 instruction set extensions; while the MMX registers are used to perform operations on 64-bit packed integer data and to hold operands for some operations performed between MMX and XMM registers.

[0314] Alternative embodiments of the present disclosure may use wider or narrower registers. Additionally, alternative embodiments of the present disclosure may use more, fewer, or different register files and registers.

[0315] Exemplary Core Architectures, Processors, and Computer Architectures

[0316] Processor cores may be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance general-purpose out-of-order core intended for general-purpose computing; 3) a special-purpose core intended primarily for graphics and / or scientific (throughput) computing. Different processor implementations may include: 1) a CPU including one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor including one or more special-purpose cores intended primarily for graphics and / or scientific (throughput). Such different processors lead to different computer system architectures, which may include: 1) coprocessors on a separate chip from the CPU; 2) coprocessors in the same package as the CPU, on a separate die; 3) coprocessors on the same die as the CPU (in which case such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as dedicated cores); and 4) systems on a chip, which may include the described CPUs (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality on the same die. An exemplary core architecture is described next, followed by a description of an exemplary processor and computer architecture.

[0317] Exemplary Core Architecture

[0318] In-order and out-of-order core block diagram

[0319] Figure 21A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline according to an embodiment of the present disclosure. Figure 21B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor according to an embodiment of the present disclosure. Figures 21A - 21B The solid line boxes in illustrate the in-order pipeline and in-order core, while the optional addition of dashed line boxes illustrates the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0320] exist Figure 21A , the processor pipeline 2100 includes a fetch stage 2102, a length decode stage 2104, a decode stage 2106, an allocation stage 2108, a rename stage 2110, a dispatch (also called a dispatch or issue) stage 2112, a register read / memory read stage 2114, an execute stage 2116, a write back / memory write stage 2118, an exception handling stage 2122, and a commit stage 2124.

[0321] Figure 21B It is shown that the processor core 2190 includes a front-end unit 2130 coupled to an execution engine unit 2150, and both are coupled to a memory unit 2170. The core 2190 can be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 2190 can be a dedicated core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, and so on.

[0322] The front-end unit 2130 includes a branch prediction unit 2132, which is coupled to an instruction cache unit 2134, the instruction cache unit 2134 is coupled to an instruction translation lookaside buffer (TLB) 2136, the TLB 2136 is coupled to an instruction fetch unit 2138, and the instruction fetch unit 2138 is coupled to a decoding unit 2140. The decoding unit 2140 (or decoder or decoder unit) can decode an instruction (e.g., a macro-instruction) and generate one or more micro-operations, micro-code entry points, micro-instructions, other instructions, or other control signals as output, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decoding unit 2140 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), micro-code read only memories (ROMs), and so on. In one embodiment, the core 2190 includes a micro-code ROM or other medium that stores micro-code for certain macro-instructions (e.g., in the decoding unit 2140 or otherwise within the front-end unit 2130). The decoding unit 2140 is coupled to a rename / allocator unit 2152 in the execution engine unit 2150.

[0323] The execution engine unit 2150 includes a rename / allocator unit 2156 coupled to a retirement unit 2152 and a set of one or more scheduler units 2154. The (one or more) scheduler units 2156 represent any number of different schedulers, including reservation stations, a central instruction window, and the like. The (one or more) scheduler units 2156 are coupled to the (one or more) physical register file units 2158. Each of the physical register file units 2158 represents one or more physical register files, where different physical register files in these physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer that is the address of the next instruction to be executed), and so on. In one embodiment, the physical register file unit 2158 includes a vector register unit, a write mask register unit, and a scalar register unit. These register units may provide architected vector registers, vector mask registers, and general-purpose registers. The (one or more) physical register file units 2158 overlap with the retirement unit 2154 to illustrate various ways that can be used to implement register renaming and out-of-order execution (e.g., using the (one or more) reorder buffers and the (one or more) retirement register files; using the (one or more) future files, the (one or more) history buffers, and the (one or more) retirement register files; using register mapping and a pool of registers; and so on). The retirement unit 2154 and the (one or more) physical register file units 2158 are coupled to the (one or more) execution clusters 2160. The (one or more) execution clusters 2160 include a set of one or more execution units 2162 and a set of one or more memory access units 2164. The execution units 2162 can perform various operations (e.g., shift, add, subtract, multiply) on various types of data (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some embodiments may include several execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The (one or more) scheduler units 2156, the (one or more) physical register file units 2158, and the (one or more) execution clusters 2160 are shown as potentially being multiple because some embodiments create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each having its own scheduler unit, physical register file unit, and / or execution cluster - and in the case of a separate memory access pipeline, some embodiments are implemented where only the execution cluster of this pipeline has the (one or more) memory access units 2164).It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be out-of-order issue / execution and the rest are in-order.

[0324] A set of memory access units 2164 is coupled to a memory unit 2170, which includes a data TLB unit 2172 that is coupled to a data cache unit 2174, and the data cache unit 2174 is coupled to a level 2 (L2) cache unit 2176. In one exemplary embodiment, the memory access units 2164 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 2172 in the memory unit 2170. The instruction cache unit 2134 is further coupled to the level 2 (L2) cache unit 2176 in the memory unit 2170. The L2 cache unit 2176 is coupled to one or more other levels of caches and ultimately to the main memory.

[0325] As an example, an exemplary register renaming, out-of-order issue / execution core architecture may implement pipeline 2100 as follows: 1) Instruction fetch 2138 performs the fetch and length decoding stages 2102 and 2104; 2) The decode unit 2140 performs the decode stage 2106; 3) The rename / allocator unit 2152 performs the allocation stage 2108 and the rename stage 2110; 4) The (one or more) scheduler units 2156 perform the schedule stage 2112; 5) The (one or more) physical register file units 2158 and the memory unit 2170 perform the register read / memory read stage 2114; The execution cluster 2160 performs the execution stage 2116; 6) The memory unit 2170 and the (one or more) physical register file units 2158 perform the write-back / memory write stage 2118; 7) Various units may be involved in the exception handling stage 2122; and 8) The retirement unit 2154 and the (one or more) physical register file units 2158 perform the commit stage 2124.

[0326] The core 2190 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, Calif.; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, Calif. (with optional additional extensions such as NEON)), including the (one or more) instructions described herein. In one embodiment, the core 2190 includes logic to support the Scalar Extensions for Multimedia Instruction Set (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using scalar data.

[0327] It should be understood that the core can support multi-threading (two or more parallel sets of operations or threads), and can support multi-threading in a variety of ways, including time-sliced multi-threading, simultaneous multi-threading (where a single physical core provides a logical core for each thread that the physical core is multi-threading simultaneously), or a combination of these (e.g., time-sliced fetching and decoding followed by simultaneous multi-threading, such as in the Hyper-threading technology).

[0328] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in in-order architectures. Although the illustrated embodiment of the processor also includes separate instruction and data cache units 2134 / 2174 and a shared L2 cache unit 2176, alternative embodiments can have a single internal cache for both instructions and data, such as a level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system can include a combination of internal caches and external caches outside the core and / or processor. Alternatively, all caches can be outside the core and / or processor.

[0329] Specific exemplary in-order core architecture

[0330] Figures 22A - 22B A block diagram illustrating a more specific exemplary in-order core architecture, which would be one of several logical blocks in a chip (including other cores of the same type and / or different types). The logical blocks communicate with certain fixed-function logic, memory I / O interfaces, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network), depending on the application.

[0331] Figure 22A is a block diagram of a single processor core according to an embodiment of the present disclosure and its connection to an on-chip interconnect network 2202 and its local subset of a level 2 (L2) cache 2204. In one embodiment, the instruction decoding unit 2200 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 2206 allows low-latency access to cache memory into scalar and vector units. Although in one embodiment (for simplicity of design), the scalar unit 2208 and the vector unit 2210 use separate register sets (scalar registers 2212 and vector registers 2214 respectively) and the data transferred between them is written to memory and then read back from the level 1 (L1) cache 2206, alternative embodiments of the present disclosure can use different schemes (e.g., using a single register set or including a communication path that allows data to be transferred between two register files without being written and read back).

[0332] The local subset 2204 of the L2 cache is part of the global L2 cache, which is partitioned into separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 2204 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 2204 and can be accessed quickly, in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 2204 and is flushed out of other subsets when necessary. A ring network ensures the coherence of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0333] Figure 22B is part of a processor core according to an embodiment of the present disclosure Figure 22A expanded view. Figure 22B The L1 data cache 2206A portion includes the L1 cache 2204, as well as more details regarding the vector unit 2210 and the vector registers 2214. Specifically, the vector unit 2210 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 2228) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. The VPU supports dispatching register inputs using the dispatching unit 2220, numerical conversion using the numerical conversion units 2222A-B, and copying of memory inputs using the copy unit 2224. The write mask register 2226 allows assertion of result vector writes.

[0334] Figure 23 is a block diagram of a processor 2300 according to an embodiment of the present disclosure that may have more than one core, may have an integrated memory controller, and may have integrated graphics. Figure 23 The solid box in illustrates a processor 2300 having a single core 2302A, a system agent 2310, and a group of one or more bus controller units 2316, while the optional addition of the dashed box illustrates an alternative processor 2300 having multiple cores 2302A-N, a group of one or more integrated memory control units 2314 within the system agent unit 2310, and dedicated logic 2308.

[0335] Thus, different implementations of the processor 2300 may include: 1) a CPU in which the dedicated logic 2308 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores) and the cores 2302A-N are one or more general-purpose cores (e.g., general-purpose in-order cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor in which the cores 2302A-N are a large number of dedicated cores primarily intended for graphics and / or scientific (throughput); and 3) a coprocessor in which the cores 2302A-N are a large number of general-purpose in-order cores. Thus, the processor 2300 may be a general-purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit), a high-throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, and so on. The processor may be implemented on one or more chips. The processor 2300 may be a part of one or more substrates and / or may be implemented on one or more substrates using any of several process technologies, such as BiCMOS, CMOS, or NMOS.

[0336] The memory hierarchy may include one or more levels of cache within the core, a group or one or more shared cache units 2306, and external memory (not shown) coupled to the group of integrated memory controller units 2314. The group of shared cache units 2306 may include one or more intermediate-level caches, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level of cache, a last level cache (LLC), and / or a combination thereof. Although in one embodiment the ring-based interconnect unit 2312 interconnects the integrated graphics logic 2308, the group of shared cache units 2306, and the system agent unit 2310 / (one or more) integrated memory controller units 2314, alternative embodiments may also use any number of well-known techniques to interconnect such units. In one embodiment, coherence is maintained between one or more cache units 2306 and the cores 2302A-N.

[0337] In some embodiments, one or more of the cores 2302A-N are capable of multithreading. The system agent 2310 includes those components that coordinate and operate the cores 2302A-N. The system agent unit 2310 may include, for example, a power control unit (PCU) and a display unit. The PCU may be or may include the logic and components required to regulate the power states of the cores 2302A-N and the integrated graphics logic 2308. The display unit is used to drive one or more externally connected displays.

[0338] The core 2302A-N can be homogeneous or heterogeneous in terms of the architecture instruction set; that is, two or more of the cores 2302A-N may be capable of executing the same instruction set, while others may only be capable of executing a subset of the instruction set or a different instruction set.

[0339] Exemplary computer architecture

[0340] Figures 24 - 27 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptop computers, desktop computers, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular telephones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, many types of systems or electronic devices capable of incorporating the processors and / or other execution logic disclosed herein are generally suitable.

[0341] Now refer to Figure 24 , which shows a block diagram of a system 2400 according to an embodiment of the present disclosure. The system 2400 may include one or more processors 2410, 2415, which are coupled to a controller hub 2420. In one embodiment, the controller hub 2420 includes a graphics memory controller hub (GMCH) 2490 and an input / output hub (IOH) 2450 (which may be on separate chips); the GMCH 2490 includes a memory and graphics controller coupled to a memory 2440 and a coprocessor 2445; the IOH 2450 couples input / output (I / O) devices 2460 to the GMCH 2490. Alternatively, one or both of the memory and graphics controllers are integrated within the processor (as described herein), the memory 2440 and the coprocessor 2445 are directly coupled to the processor 2410, and the controller hub 2420 and the IOH 2450 are in a single chip. The memory 2440 may include, for example, an acceleration module 2440A to store code that, when executed, causes one or more to perform any method of the present disclosure.

[0342] The optionality of the additional processor 2415 is indicated by a dashed line in Figure 24 . Each processor 2410, 2415 may include one or more of the processing cores described herein and may be a certain version of the processor 2300.

[0343] The memory 2440 can be, for example, a dynamic random access memory (DRAM), a phase change memory (PCM), or a combination of both. For at least one embodiment, the controller hub 2420 communicates with the (one or more) processors 2410, 2415 via a multi-drop bus (such as a frontside bus (FSB)), a point-to-point interface (such as a Quickpath Interconnect (QPI)), or a similar connection 2495.

[0344] In one embodiment, the coprocessor 2445 is a special-purpose processor, such as a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on. In one embodiment, the controller hub 2420 can include an integrated graphics accelerator.

[0345] Among the physical resources 2410, 2415, there can be various differences in terms of a range of value metrics including architectural characteristics, microarchitectural characteristics, thermal characteristics, power consumption characteristics, and so on.

[0346] In one embodiment, the processor 2410 executes instructions that control general types of data processing operations. Coprocessor instructions can be embedded within these instructions. The processor 2410 recognizes these coprocessor instructions as being of a type that should be executed by the attached coprocessor 2445. Accordingly, the processor 2410 issues these coprocessor instructions (or control signals representing coprocessor instructions) to the coprocessor 2445 over a coprocessor bus or other interconnect. The (one or more) coprocessors 2445 receive and execute the received coprocessor instructions.

[0347] Now refer to Figure 25 , in which a block diagram of a first more specific exemplary system 2500 in accordance with an embodiment of the present disclosure is shown. As Figure 25 shown, the multi-processor system 2500 is a point-to-point interconnect system and includes a first processor 2570 and a second processor 2580 coupled via a point-to-point interconnect 2550. Each of the processors 2570 and 2580 can be some version of the processor 2300. In one embodiment of the present disclosure, the processors 2570 and 2580 are the processors 2410 and 2415, respectively, and the coprocessor 2538 is the coprocessor 2445. In another embodiment, the processors 2570 and 2580 are the processor 2410 and the coprocessor 2445, respectively.

[0348] Processors 2570 and 2580 are shown as including integrated memory controller (IMC) units 2572 and 2582, respectively. Processor 2570 also includes point-to-point (P-P) interfaces 2576 and 2578 as part of its bus controller unit; similarly, second processor 2580 includes P-P interfaces 2586 and 2588. Processors 2570, 2580 can exchange information via point-to-point (P-P) interface 2550 using P-P interface circuits 2578, 2588. As Figure 25 shown, IMCs 2572 and 2582 couple the processors to respective memories, namely memory 2532 and memory 2534, which may be part of the main memories locally attached to the respective processors.

[0349] Processors 2570, 2580 can each exchange information with chipset 2590 via point-to-point interface circuits 2576, 2594, 2586, 2598 through individual P-P interfaces 2552, 2554. Chipset 2590 can optionally exchange information with coprocessor 2538 via high performance interface 2539. In one embodiment, coprocessor 2538 is a specialized processor, such as a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on.

[0350] A shared cache (not shown) may be included in either processor or outside both processors, but connected to the processors via a P-P interconnect such that local cache information of either or both processors can be stored in the shared cache when the processors are placed in a low power mode.

[0351] Chipset 2590 can be coupled to a first bus 2516 via interface 2596. In one embodiment, first bus 2516 can be a Peripheral Component Interconnect (PCI) bus, or a bus such as a rapid PCI bus or another third generation I / O interconnect bus, although the scope of the present disclosure is not limited thereto.

[0352] As Figure 25As shown, various I / O devices 2514 may be coupled to a first bus 2516, and a bus bridge 2518 that couples the first bus 2516 to a second bus 2520. In one embodiment, one or more additional processors 2515, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (e.g., a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array, or any other processor, are coupled to the first bus 2516. In one embodiment, the second bus 2520 may be a low pin count (LPC) bus. Various devices may be coupled to the second bus 2520, such as including a keyboard and / or a mouse 2522, a communication device 2527, and a storage unit 2528, such as a disk drive or other mass storage device, which may include instructions / code and data 2530 in one embodiment. Additionally, audio I / O 2524 may be coupled to the second bus 2520. Note that other architectures are possible. For example, instead of Figure 25 a point-to-point architecture, the system may implement a multi-drop bus or other such architecture.

[0353] Now refer to Figure 26 , in which a block diagram of a second more specific exemplary system 2600 according to an embodiment of the present disclosure is shown. Figure 25 And Figure 26 Similar elements in Figure 25 are labeled with similar reference numerals, and Figure 26 certain aspects of Figure 26 are omitted from

[0354] Figure 26 to avoid obscuring Figure 26 other aspects of

[0355] Now refer to Figure 27 , in which a block diagram of a SoC 2700 according to an embodiment of the present disclosure is shown. Figure 23 Similar elements in Figure 27In [the figure], one or more interconnect units 2702 are coupled to: an application processor 2710 that includes a set of one or more cores 202A-N, and one or more shared cache units 2306; a system agent unit 2310; one or more bus controller units 2316; one or more integrated memory controller units 2314; a set of one or more coprocessors 2720, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 2730; a direct memory access (DMA) unit 2732; and a display unit 2740 for coupling to one or more external displays. In one embodiment, one or more coprocessors 2720 include dedicated processors such as a network or communication processor, a compression engine, a GPGPU, a high throughput MIC processor, an embedded processor, and the like.

[0356] Embodiments disclosed herein (e.g., of the mechanisms) may be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments of the present disclosure may be implemented as a computer program or program code executed on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0357] Program code, such as Figure 25 the code 2530 shown in [the figure], may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as; a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0358] The program code may be implemented in a high-level procedural or object-oriented programming language to communicate with the processing system. If desired, the program code may also be implemented in assembly or machine language. In fact, the mechanisms described herein are not limited in scope to any particular programming language. In any case, the language may be a compiled or interpreted language.

[0359] One or more aspects of at least one embodiment can be implemented by representative instructions stored on a machine-readable medium that represent various logic within a processor. When read by a machine, these representative instructions cause the machine to fabricate the logic to perform the techniques described herein. Such representations, referred to as “IP cores,” can be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities to be loaded into the fabrication machines that actually fabricate the logic or processor.

[0360] Such machine-readable storage media can include, but are not limited to, non-transitory tangible arrangements of articles manufactured or formed by a machine or device, including storage media such as hard disks, any other type of disk, including floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and magneto-optical disks, semiconductor devices such as read-only memory (ROM), random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EEPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase change memory (PCM), magnetic or optical cards, or any other type of medium suitable for storing electronic instructions.

[0361] Accordingly, embodiments of the present disclosure also include non-transitory tangible machine-readable media that contain instructions or contain design data that define the structural, circuit, device, processor, and / or system features described herein, such as a Hardware Description Language (HDL). Such embodiments may also be referred to as program products.

[0362] Emulation (including binary translation, code morphing, etc.)

[0363] In some cases, an instruction converter may be used to convert instructions from a source instruction set to a target instruction set. For example, the instruction converter may transform (e.g., using static binary translation, dynamic binary translation including dynamic compilation), morph, emulate, or otherwise convert the instructions to one or more other instructions to be processed by the core. The instruction converter may be implemented in software, hardware, firmware, or a combination thereof. The instruction converter may be on the processor, off the processor, or part on the processor and part off the processor.

[0364] Figure 28 is a block diagram contrasting an embodiment in accordance with the present disclosure with the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set. In the illustrated embodiment, the instruction converter is a software instruction converter, although alternatively, the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof. Figure 28 shows that a program in a high-level language 2802 can be compiled using an x86 compiler 2804 to generate x86 binary code 2806, which can be natively executed by a processor having at least one x86 instruction set core 2816. A processor having at least one x86 instruction set core 2816 represents any such processor: such a processor can execute compatibly or otherwise process (1) a substantial portion of the instruction set of the x86 instruction set core or (2) the target code version of an application or other software running on a processor having at least one x86 instruction set core to perform substantially the same functions as a processor having at least one x86 instruction set core, so as to achieve substantially the same results as a processor having at least one x86 instruction set core. The x86 compiler 2804 represents a compiler operable to generate x86 binary code 2806 (e.g., target code), which can be executed on a processor 2816 having at least one x86 instruction set core with or without additional linking processing. Similarly, Figure 28It is shown that a program in a high-level language 2802 can be compiled using a replacement instruction set compiler 2808 to generate replacement instruction set binary code 2810, which can be natively executed by a processor 2814 that does not have at least one x86 instruction set core (e.g., a processor having a core that executes the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California and / or the ARM instruction set of ARM Holdings plc of Sunnyvale, California). An instruction converter 2812 is used to convert the x86 binary code 2806 into code that can be natively executed by a processor 2814 that does not have an x86 instruction set core. This converted code is not likely to be the same as the replacement instruction set binary code 2810 because instruction converters that can do this are difficult to make; however, the converted code will implement the general operations and be composed of instructions from the replacement instruction set. Thus, the instruction converter 2812 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have an x86 instruction set processor or core to execute the x86 binary code 2806 through emulation, simulation, or any other process.

Claims

1. A device, comprising: A plurality of multipliers for calculating a product by multiplying integer values corresponding to input data elements of a neural network operation; A plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in a floating-point mode to shift the product based on the maximum exponent of the product, and wherein when the input data element is an integer data element, the shifter operates in an integer mode to bypass shifting the product; One or more adders for generating a sum from the output of the shifter; and An accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums.

2. The apparatus according to claim 1, wherein, The accumulation unit is configured to operate in different modes for different data precisions.

3. The device according to claim 2, wherein, The accumulation unit is configured to shift the sum by different amounts in different modes and to accumulate the shifted sum with one or more other shifted sums generated by shifting the one or more other sums.

4. The device according to claim 1, further comprising an exponent unit for: Calculate the exponent of the product, where, Calculating the exponent of a product includes adding the exponents of two input data elements corresponding to the product; and Selecting the maximum exponent from the exponents of the product.

5. The device according to claim 1, wherein The one or more adders are a plurality of adders arranged in a tree structure.

6. The device according to claim 1, wherein, The integer value is the mantissa of the input data element in floating-point mode.

7. The device according to claim 1, wherein The neural network operation includes vector multiplication, and the input data elements are data elements in two vectors.

8. A device, comprising: A data storage unit for storing input data elements of a neural network operation; A plurality of multipliers for calculating a product by multiplying integer values corresponding to the input data elements, the integer values being represented by bits obtained from the data storage unit; A plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in a floating-point mode to shift the product based on the maximum exponent of the product, and wherein when the input data element is an integer data element, the shifter operates in an integer mode to bypass shifting the product; One or more adders for generating a sum from the output of the shifter; and An accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums.

9. The apparatus according to claim 8, wherein, The accumulation unit is configured to operate in different modes for different data precisions.

10. The apparatus according to claim 9, wherein, The accumulation unit is configured to shift the sum by different amounts in different modes and to accumulate the shifted sum with one or more other shifted sums generated by shifting the one or more other sums.

11. The device according to claim 8, further comprising an exponent unit for: Calculate the exponent of the product, where, Calculating the exponent of a product includes adding the exponents of two input data elements corresponding to the product; and Select the maximum exponent from the exponents of the product.

12. The device according to claim 8, wherein, The one or more adders are a plurality of adders arranged in a tree structure.

13. The apparatus according to claim 8, wherein, The bit is a mantissa bit of the input data element.

14. The apparatus according to claim 8, wherein, The neural network operation includes vector multiplication, and the input data element is a data element in two vectors.

15. An apparatus, comprising: A plurality of multipliers for calculating a product by multiplying integer values corresponding to input data elements of a neural network operation; An exponent unit for operating in different modes for different data precisions, wherein the exponent unit determines the maximum exponent of the product in floating-point mode and is bypassed in integer mode; A plurality of shifters for operating in different modes for different data precisions, wherein when the input data element is a floating-point data element, the shifter operates in floating-point mode to shift the product based on the maximum exponent of the product, and when the input data element is an integer data element, the shifter operates in integer mode to bypass shifting the product; One or more adders for generating a sum from the output of the shifter; and An accumulation unit for generating an output of the neural network operation from the sum generated by the one or more adders and one or more other sums.

16. The apparatus according to claim 15, wherein, The accumulation unit is for operating in different modes for different data precisions.

17. The apparatus according to claim 16, wherein The accumulation unit is for shifting the sum by different amounts in different modes and for accumulating the shifted sum with one or more other shifted sums generated by shifting the one or more other sums.

18. The device according to claim 15, wherein, The one or more adders are a plurality of adders arranged in a tree structure.

19. The device according to claim 15, wherein, The integer value is the mantissa of the input data element in floating-point mode.

20. The apparatus according to claim 15, wherein The neural network operation includes vector multiplication, and the input data element is a data element in two vectors.

Images