Floating point dynamic range extension

Abstract

The present disclosure generally relates to techniques for adjusting a digital representation (e.g., format) of a variable before and / or after performing one or more arithmetic operations on the variable. In particular, the present disclosure relates to scaling a range of variables to a suitable representation based on available hardware (e.g., hard logic) in an integrated circuit device. For example, an input in a first digital format (e.g., bfloat16) may be scaled to a second digital format (e.g., a semi-precision floating point) such that a circuit implemented to receive an input in the second digital format may perform one or more arithmetic operations on the input. Further, the output generated by the circuit may be scaled back to the first digital format. Accordingly, an arithmetic operation, such as a dot product, performed in the first format may be emulated by scaling an input to and / or an output from an arithmetic operation performed in another format.

Description

[0001] This application is a divisional application of the invention patent application with the application date of August 27, 2019, named “Floating Point Dynamic Range Extension” and application number 201910795048.9. Background Art

[0002] The present disclosure relates generally to integrated circuits, such as field programmable gate arrays (FPGAs). More specifically, the present disclosure relates to techniques for adjusting (e.g., scaling) variables before and after processing so that operations performed on the variables in a first digital format can be emulated by operations performed in another digital format using circuit elements of the integrated circuit (e.g., programmable logic of the FPGA).

[0003] This section is intended to introduce the reader to various aspects of the technology that may be related to various aspects of the present disclosure described and / or claimed below. It is believed that this discussion helps to provide the reader with background information to facilitate a better understanding of various aspects of the present disclosure. Therefore, it should be understood that these statements should be read from this perspective, rather than admitted as prior art.

[0004] Integrated circuits can represent variables according to many different formats. For example, variables can be represented in single-precision floating point format, half-precision floating point format, bfloat16 format, etc. Each format (e.g., digital representation) can provide different advantages in terms of memory usage, precision of representable values, range of representable values, etc. In some embodiments, the application of the variable in the integrated circuit (such as operation and / or processing) can specify a suitable format for the variable. For example, in machine learning applications, an increased format range (such as bfloat16) may be beneficial compared to the range of the half-precision floating point format. However, in some embodiments, the digital formats that can be used for representation in the integrated circuit may be limited by available hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Various aspects of the present disclosure may be better understood upon reading the following detailed description and referring to the drawings, in which:

[0006] Figure 1 is a block diagram of a system for implementing a scaling circuit according to an embodiment;

[0007] Figure 2 is a block diagram of an integrated circuit in which a scaling circuit according to an embodiment may be implemented;

[0008] Figure 3 is a block diagram of a digital signal processing (DSP) circuit according to an embodiment;

[0009] Figure 4 is a range diagram of a set of variables input to a DSP circuit according to an embodiment;

[0010] Figure 5 is a block diagram of an arithmetic operation simulation circuit according to an embodiment, which includes a circuit communicatively coupled to Figure 3 Scaling circuit of DSP circuit;

[0011] Figure 6 is a flow chart of a process for adjusting the representation (e.g., format) of a number before and after processing according to an embodiment;

[0012] Figure 7 is a flow chart of a process for scaling a set of inputs from a first format to a second format according to an embodiment;

[0013] Figure 8 is a block diagram of an extended arithmetic operation simulation circuit according to an embodiment; and

[0014] Fig. 9 is a block diagram of a data processing system according to an embodiment. DETAILED DESCRIPTION

[0015] One or more specific embodiments will be described below. In order to provide a concise description of these embodiments, not all features of the actual implementation are described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions can be made to achieve the developer's specific goals, such as complying with system-related and business-related constraints, which may vary from one implementation to another. In addition, it should be understood that such development efforts may be complex and time-consuming, but are still routine tasks of design, manufacturing and processing for ordinary technicians who benefit from this disclosure.

[0016] As discussed in further detail below, embodiments of the present disclosure generally relate to adjusting the digital representation (e.g., format) of a variable before and / or after performing one or more arithmetic operations on the variable. More specifically, the present disclosure relates to scaling a variable to a suitable representation based on available hardware (e.g., hard logic) in an integrated circuit. For example, an input in a first digital format (e.g., bfloat16) can be scaled to a second digital format (e.g., half-precision floating point) so that a digital signal processing (DSP) circuit implemented to receive an input in the second digital format can perform one or more arithmetic operations on the input. In addition, in some embodiments, an output generated by a DSP circuit in a second or third digital format (e.g., single-precision floating point) can be scaled back to the first digital format. Therefore, an arithmetic operation such as a dot product performed in a first format can be simulated by scaling to the input of the arithmetic operation performed in the second format and / or the output from the arithmetic operation performed in the second format.

[0017] Thus, the integrated circuit may include a pre-scaling circuit communicatively coupled to an input of the DSP circuit and / or a post-scaling circuit communicatively coupled to an output of the DSP circuit. As will be discussed in further detail below, the DSP circuit may be implemented to perform a multiplication operation such as a dot product on a set of inputs having a second digital format. In this way, a pre-scaling circuit may be implemented to scale a set of inputs from a first digital format to a second digital format. To this end, the pre-scaling circuit may determine the maximum sum of the exponents of the input pairs that can be multiplied in the DSP circuit. That is, for example, the pre-scaling circuit may determine the maximum exponent value produced by multiplying two inputs together. To avoid overflow at the DSP circuit, the pre-scaling circuit may then scale the exponent of each of the set of inputs to a suitable range according to the second digital format based at least in part on the maximum exponent value. Thus, the input may be scaled to the second format and routed to the DSP circuit. After the DSP circuit performs one or more arithmetic operations on the scaled input, the post-scaling circuit may scale the output of the DSP circuit back to the first format. More specifically, based at least in part on the maximum exponent value determined by the pre-scaling circuit and the first format, the exponent of the output may be scaled to a range corresponding to the first format. Thus, although the arithmetic operation is performed in a different format, the scaled output emulates the result of performing the arithmetic operation in the first digital format.

[0018] Considering the foregoing, Figure 1 A block diagram of a system 10 that can implement arithmetic operations is shown. A designer may wish to implement functionality such as the scaling operations of the present disclosure on an integrated circuit device 12, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In some cases, the designer may specify a high-level program to be implemented, such as an OpenCL program, which may enable the designer to more efficiently and easily provide programming instructions to configure a set of programmable logic units for the integrated circuit device 12 without requiring specific knowledge of a low-level hardware description language (e.g., Verilog or VHDL). For example, because OpenCL is very similar to other high-level programming languages, such as C++, a designer of programmable logic that is familiar with such programming languages ​​may have a reduced learning curve to implement new functionality in the integrated circuit device 12 than a designer who needs to learn an unfamiliar low-level hardware description language.

[0019] Designers can use design software 14 (such as Intel's The design software 14 may use a compiler 16 to convert the high-level program into a lower-level description. The compiler 16 may provide machine-readable instructions representing the high-level program to the host 18 and the integrated circuit device 12. The host 18 may receive a host program 22 that may be implemented by the kernel program 20. In order to implement the host program 22, the host 18 may transmit instructions from the host program 22 to the integrated circuit device 12 via a communication link 24, which may be, for example, a direct memory access (DMA) communication or a peripheral component interconnect expression (PCIe) communication. In some embodiments, the kernel program 20 and the host 18 may enable the configuration of a scaling circuit 26 (e.g., a combinational circuit) to be implemented on the integrated circuit device 12. The scaling circuit 26 may include circuits and / or other logic elements, and may be configured, for example, to scale a variable from a first digital representation to a second digital representation.

[0020] Although the techniques described herein involve the use of high-level programs, in some embodiments, a designer may generate and / or specify a low-level program, such as the low-level hardware description language described above, using design software 14. Furthermore, in some embodiments, system 10 may be implemented without a separate host program 22. Furthermore, in some embodiments, the techniques described herein may be implemented in a circuit as a non-programmable circuit design. Therefore, the embodiments described herein are intended to be illustrative and not limiting.

[0021] Turning now to a more detailed discussion of integrated circuit device 12, Figure 2 An example of an integrated circuit device 12 as a programmable logic device such as a field programmable gate array (FPGA) is shown. In addition, it should be understood that the integrated circuit device 12 can be any other suitable type of programmable logic device (e.g., an application specific integrated circuit and / or an application specific standard product). As shown, the integrated circuit device 12 can have an input / output circuit 42 for driving signals out of the device via an input / output pin 44 and receiving signals from other devices. Interconnection resources 46, such as global and local vertical and horizontal wires and buses, can be used to route signals on the integrated circuit device 12. In addition, the interconnection resources 46 may include fixed interconnects (wires) and programmable interconnects (i.e., programmable connections between corresponding fixed interconnects). Programmable logic 48 may include combinational and sequential logic circuits. For example, programmable logic 48 may include a lookup table, a register, and a multiplexer. In various embodiments, programmable logic 48 may be configured to perform custom logic functions. Programmable interconnects associated with interconnection resources may be considered as part of programmable logic 48.

[0022] A programmable logic device, such as the integrated circuit device 12, may include a programmable element 50 with the programmable logic 48. For example, as discussed above, a designer (e.g., a customer) may program (e.g., configure) the programmable logic 48 to perform one or more desired functions. For example, some programmable logic devices may be programmed by configuring their programmable elements 50 using a mask programming arrangement that is performed during semiconductor fabrication. Other programmable logic devices are configured after semiconductor fabrication operations are completed, such as by programming their programmable elements 50 using electrical programming or laser programming. In general, the programmable element 50 may be based on any suitable programmable technology, such as fuses, antifuses, electrically programmable read-only memory technology, random access memory cells, mask-programmed elements, and the like.

[0023] Many programmable logic devices are electrically programmed. Using an electrical programming arrangement, the programmable element 50 can be formed by one or more memory cells. For example, during programming, configuration data is loaded into the memory cell using pins 44 and input / output circuits 42. In one embodiment, the memory cell can be implemented as a random access memory (RAM) cell. The use of the memory cell based on RAM technology described here is intended only as an example. In addition, because these RAM cells are loaded with configuration data during programming, they are sometimes referred to as configuration RAM cells (CRAM). These memory cells can each provide a corresponding static control output signal, which controls the state of the associated logic components in the programmable logic 48. For example, in some embodiments, the output signal can be applied to the gate of the metal oxide semiconductor (MOS) transistor in the programmable logic 48.

[0024] Now go to Figure 3In some embodiments, the integrated circuit device 12 may include a digital signal processing (DSP) circuit 60, such as a multiply-accumulate (MAC) circuit, a DSP block, an arithmetic circuit, or a DSP slice (e.g., a portion of a DSP block), implemented to perform one or more arithmetic operations (e.g., dot products) on an input. In addition, in some embodiments, the DSP circuit 60 may include hardened logic (e.g., a hardened MAC circuit, a hardened DSP block, a hardened arithmetic circuit, a hardened DSP slice, etc.) to perform one or more arithmetic operations. The one or more arithmetic operations may produce a result having a specific digital representation (e.g., format and / or range). In addition, in some embodiments, the digital representation of the result may not match the digital representation of the original input. For example, in the illustrated embodiment, the DSP circuit 60 includes an input circuit 62, which is implemented to receive a plurality of half-precision floating point (e.g., FP16) inputs. Thus, each input includes sixteen bits, one of which represents a sign bit of the number, five bits represent an exponent of the number, and ten bits represent a mantissa (e.g., fraction) of the number. In addition, after determining the dot product of the inputs, DSP circuit 60 outputs a single precision floating point (e.g., SP) result that includes a single sign bit, an 8-bit exponent field, and a 23-bit mantissa field (e.g., thirty-two bits total). However, the illustrated format of the inputs and outputs is not meant to be limiting. In fact, the inputs and outputs may be in any suitable format.

[0025] To perform the one or more arithmetic operations on a set of inputs (e.g., to determine a dot product of the set of inputs), DSP circuitry 60 may include a plurality of multipliers 64. While input circuitry 62 may receive inputs having a first digital representation (e.g., a half-precision floating point format), multipliers 64 may receive inputs in a second internal digital format (e.g., a half-precision floating point format) of DSP circuitry 60. Figure 3 The DSP circuit 60 may further include a plurality of multiplication results (e.g., FP16+++) and a plurality of multiplication results (e.g., FP16+++). For example, to account for potential overflow, the multiplication results may be formatted using a 1-bit sign field, an 8-bit exponent field, and a 10-bit mantissa field. In other embodiments, the multiplication results may be maintained in a half-precision floating point format, or may be formatted according to another suitable digital format (e.g., single-precision floating point, etc.), which may depend on the format of the group of inputs to the DSP circuit 60.

[0026] The DSP circuit 60 may also include a suitable number of adders 66 (e.g., floating point adders) and / or a suitable number of stages of an adder tree 68 to sum the multiplication results. The adder 66 may be implemented to sum the multiplication results according to an internal digital format of the DSP circuit 60, which may be the same or different format than the format of the multiplication results. In addition, in some embodiments, the final sum of each multiplication result may be determined by, for example, a single-precision adder 66A in the final adder stage of the adder tree 68. Thus, the single-precision adder 66A may output a 32-bit result having a 1-bit sign field, an 8-bit exponent field, and a 23-bit mantissa field. To this end, the 10-bit fraction field of the input to the single-precision adder 66A may be extended to 23 bits before they are summed. In other embodiments, the final sum may be determined by adder 66, which is implemented to produce a result in another digital format (e.g., half-precision floating point, FP16+++, extended precision, etc.), which may depend on the format of the set of inputs to DSP circuit 60, the format used to initially sum the multiplication results, etc.

[0027] However, in some embodiments, inputs for arithmetic operations performed by DSP circuit 60 may also not be formatted according to the digital format expected at input circuit 62 (e.g., half-precision floating point). For example, in some embodiments, input circuit 62 may receive input having a 1-bit sign field, an 8-bit exponent field, and a 7-bit fraction field (e.g., bfloat16). Thus, in some embodiments, the input may be scaled from one format to another format before being received at input circuit 62. Furthermore, in some embodiments, it may be desirable to produce an output whose format does not correspond to (e.g., match) the single-precision floating point format produced by single-precision adder 66A. For example, continuing the above example, in the event that DSP circuit 60 receives an input in bfloat16 format, it may be desirable to scale the output back to bfloat16. In this way, the output of DSP circuit 60 may be scaled from one format to another format.

[0028] Although the DSP circuit 60 is shown as being implemented to determine dot products, the DSP circuit 60 may also be implemented to perform any suitable multiply-accumulate function and / or other arithmetic operations. Furthermore, the format of the inputs, outputs, and any intermediate values ​​of the DSP circuit 60 may be in any suitable digital format. Thus, bfloat16 inputs may be scaled to half-precision floating point, extended-precision inputs may be scaled to single-precision floating point, and other combinations. Thus, the embodiments described herein are intended to be illustrative and not limiting.

[0029] To better illustrate the scaling of the input and / or output of the DSP circuit 60, Figure 4A range diagram 70 is depicted for an example set of variables (e.g., P0, P1, P2, and P3) input to the DSP circuit 60. The range diagram 70 illustrates a change in the unbiased range (e.g., exponential range) of the set of variables, which may result from scaling and / or arithmetic operations. For example, a first range 72A may represent the range of each of the set of variables input to the DSP circuit 60. Thus, for a set of variables each formatted as bfloat16, the first range 72A may extend from '-126' to '127'. More specifically, as described in more detail below, because pairs of corresponding inputs may be multiplied at the multiplier 64 of the DSP circuit 60, each of the illustrated set of variables may represent the range of the corresponding product of a pair of variables. Thus, the set of variables may represent the sum of corresponding pairs of exponents.

[0030] The second range 72B can represent that the input circuit 62 is implemented to receive a range such as a half-precision floating point. Therefore, in some embodiments, the second range 72B can extend from '-14' to '15'. In addition, the third range 72C can represent the range of the product output by the multiplier 64. In this way, because the exponent of a pair of inputs to the multiplier 64 can be summed during the multiplication operation, the third range 72C can be twice the second range 72A, and an additional bit for normalization can be included. Therefore, in some embodiments, the third range 72B can extend from '-28' to '31'. In addition, because the DSP circuit 60 includes a two-stage adder 66 in the adder tree 68, the range diagram 70 includes two adder ranges (e.g., the fourth range 72D and the fifth range 72E), each of which extends the maximum value of the previous range by bit (e.g., the third range 72C and the fourth range 72D, respectively) to cope with overflow from the addition operation. In addition, range map 70 includes a first internal range 74A, which may correspond to the range of a first embodiment of the internal digital format of DSP circuit 60, such as FP16+++. Range map 70 also includes a second internal range 74B, which may represent the range of a second embodiment of the internal digital format of DSP circuit 60, such as half-precision floating point.

[0031] As discussed in more detail below, in some embodiments, the scaling of the set of variables input to the input circuit 62 may depend in part on the internal range of the DSP circuit 60 (e.g., 74A or 74B). For example, in order to maximize the amount of data (e.g., precision) retained in the set of variables by the input circuit 62 and to prevent and / or reduce internal overflow within the DSP circuit 60, the set of variables may be scaled based in part on the second range 72B and the internal range (e.g., 74A or 74B). For example, in an embodiment where the internal range corresponds to the first internal range 74A (e.g., FP16+++ range), the maximum product (e.g., P0) represented by the set of variables may be scaled (e.g., pre-scaled) before being input to the input circuit 62 so that the unbiased exponent of the corresponding scaled product (e.g., SP0) is '30'. Therefore, in some embodiments, the exponent pair corresponding to the maximum product (P0) may each be scaled to an unbiased value of '15', which may maximize the use of the second range 72B to represent each exponent in the set of variables. Additionally, the remaining exponents of the variables in the set of variables (e.g., P1, P2, and P3) may be scaled according to the same technique and / or offset to produce the remaining scaled products (e.g., SP1, SP2, and SP3, respectively). Additionally, because the inner range corresponds to the first inner range 74A, subsequent arithmetic operations (e.g., additions) performed on the set of scaled products in the DSP circuit 60 may not result in overflow (e.g., loss of data). Thus, the result (R) produced by the DSP circuit 60 may be scaled to produce a scaled result (SR) within a sixth range 72F, which may be the same as the first range 72A, without losing data included in the set of variables.

[0032] However, in embodiments where the inner range corresponds to the second inner range 74B (e.g., a half-precision floating point range), for example, the maximum product (e.g., P0) represented by the set of variables may be scaled (e.g., pre-scaled) prior to input to the input circuit 62 such that the unbiased exponent of the corresponding scaled product (e.g., 'SP0') is less than '15'. More specifically, the maximum product (e.g., P0) may be scaled, as described in more detail below, such that after two addition stages implemented by the adder tree 68, the exponent of the final sum (R') does not exceed the maximum value (e.g., 15) of the second inner range 74B. Additionally, the remaining variables in the set of variables (e.g., P1, P2, and P3) may be scaled according to the same technique and / or offset to produce the remaining scaled products (e.g., SP1', SP2', and SP3', respectively). However, as shown in area 76 of range diagram 70, the exponent of the product less than the maximum product can be scaled to a range beyond the second internal range 74B, which may result in loss of data (e.g., underflow) before any addition is calculated at adder tree 68. Therefore, the additional result (R') produced by DSP circuit 60 may not be as accurate as the result (R) produced in the embodiment with the first internal range 74A. In this way, the final result produced by scaling the additional result (R') to produce the additional scaled result (SR') may not be as accurate as the scaled result (SR). In addition, it can be understood that the scaling techniques and / or offsets applied in the embodiment with the second internal range 74B described above can be applied to the embodiment with the first internal range 74A. For example, in the embodiment where the internal range is the first internal range 74, the group of variables can be scaled to a value less than '30'. However, the scaling result produced by this embodiment may not be as accurate as the scaling techniques and / or offsets described above with reference to the embodiment with the first internal range 74A.

[0033] Furthermore, while the illustrated embodiment depicts certain ranges (e.g., 72A, 72B, 72C, 72D, 72E, and 72F) and certain internal ranges (e.g., 74A and 74B) that may correspond to certain digital formats, respectively, it will be appreciated that any suitable ranges may be applied within DSP circuit 60. Furthermore, any suitable range may be scaled to a corresponding range of input circuit 62, and the output of DSP circuit 60 may be scaled to any suitable range.

[0034] Figure 5An embodiment of an arithmetic operation simulation circuit 100 is shown, which may include a scaling circuit 26 operatively coupled to a DSP circuit 60. The scaling circuit 26 may include a pre-scaling circuit 102 implemented to adjust the format of a set of inputs (e.g., A0, A1, A2, A3, B0, B1, B2, and B3) and may include a post-scaling circuit 104 implemented to adjust the format of an output generated by the DSP circuit 60. More specifically, the pre-scaling circuit 102 may adjust to the range of the input to the DSP circuit 60, for example, by scaling the exponent of the input from a first number of bits to a second number of bits. In addition, the post-scaling circuit 104 may adjust the range of the output of the DSP circuit 60 by, for example, scaling the exponent of the output to a first number of bits (e.g., the original number of bits of the input).

[0035] As shown, in some embodiments, pre-scaling circuit 102 may include input circuit 106 that receives a set of inputs, each input having a first digital format (e.g., bfloat16). To this end, because input circuit 62 is shown implemented to receive inputs in a half-precision floating point format, pre-scaling circuit 102 may scale the exponents (e.g., eA0, eA1, eA2, eA3, eB0, eB1, eB2, eB3, and eB4) of each of the set of inputs (e.g., A0, A1, A2, A3, B0, B1, B2, and B3, respectively). More specifically, pre-scaling circuit 102 may scale the exponents of the inputs in the set of inputs from 8 bits to 5 bits to avoid overflow during arithmetic operations implemented by DSP circuit 60. For example, DSP circuit 60 includes a multiplication operation (e.g., performed by multiplier 64) that effectively sums the respective exponents of a pair of multiplied inputs (e.g., A0 and AB, A1 and B1, A2 and B2, and A3 and B3). Thus, to reduce and / or prevent overflow, each of the sums of the corresponding exponents of a pair of multiplied inputs may be scaled so as not to exceed a maximum range representable in the digital format (e.g., half-precision floating point format) expected at input circuitry 62. Thus, input circuitry 106 may route pairs of exponents corresponding to inputs multiplied at DSP circuitry 60 (e.g., A0 and B0, A1 and B1, A2 and B2, A3 and B3) to be summed at corresponding adders 103 (e.g., 103A, 103B, 103C, 103D), which may be implemented to sum integer values.

[0036] The pre-scaling circuit 102 may then determine the maximum value (e.g., M) of the pairwise sums of the exponents using, for example, the comparison circuit 107. In some embodiments, for example, the pre-scaling circuit 102 may include a set of subtractors 108 and / or comparators implemented to determine the difference between the pairs of calculated sums of the exponents. Thus, as shown, the first subtractor 108A may subtract the sum resulting from the least significant input (e.g., subtracting the sum of the exponents of A0 and B0 from the sum of the exponents of A1 and B1), and the second subtractor 108B may subtract the sum resulting from the most significant input (e.g., subtracting the sum of the exponents of A2 and B2 from the sum of the exponents of A3 and B3). In addition, the output of the first subtractor 108A may be routed to a first multiplexer 110A (mux), which may select between the sum of the exponents of the first set of inputs (e.g., A0 and B0) or the sum of the exponents of the second set of inputs (e.g., A1 and B1) based on the output. Thus, the first multiplexer 110A may use the difference provided by the first subtractor 108A to select the maximum sum between the two sums. Similarly, the second multiplexer 110B may use the difference between the two sums provided by the second subtractor 108B to select between the sum of the exponents of the third set of inputs (e.g., A2 and B2) and the sum of the exponents of the fourth set of inputs (e.g., A3 and B3). Therefore, in order to determine the maximum sum between the sums selected by the first multiplexer 110A and the second multiplexer 110B, the pre-scaling circuit 102 may include a third subtractor 108C operably coupled to the third multiplexer 110C. The third subtractor 108C may determine the difference between the corresponding sums output by the first multiplexer 110A and the second multiplexer 110B. The third multiplexer 110C may then use the difference as a selection signal to select between the corresponding sums output by the first multiplexer 110A and the second multiplexer 110B. To this end, the third multiplexer 110C may select the maximum value (M) of the sums of the corresponding index pairs.

[0037] Using the maximum value (M), the pre-scaling circuit 102 may determine the offset value (W). Thus, in some embodiments, the pre-scaling circuit 102 may include an additional subtractor 108C that may receive the maximum value (M) and an integer (e.g., '60') as inputs. In some embodiments, the integer routed to the subtractor 108C may depend on the digital format expected by the input circuit 62 of the DSP circuit 60 (e.g., half-precision floating point format) and / or the internal digital format of the DSP circuit 60 (e.g., FP16+++), as described above with reference to Figure 4 As described.

[0038] For example, in the illustrated embodiment, input circuit 62 is implemented to receive half-precision floating point format numbers, and multiplier 64 is implemented to output the product according to the FP16+++ format. In the half-precision floating point format, the maximum biased exponent value is '30' (e.g., '15' added to the biased value '15'), and thus the maximum value of the sum of two biased exponents is '60' (e.g., (15+15)+(15+15)). However, in bfloat16, the maximum biased exponent value is '254' (e.g., '127' added to the biased value '127'), and the maximum biased value of the sum of two exponents is '508' (e.g., (127+127)+(127+127)). Therefore, in order to adjust the exponent value of the input in bfloat16 and / or another format different from half-precision floating point, the maximum sum of two biased exponents (M) can be integer-scaled to '60' (e.g., the maximum biased sum of two exponents in half-precision floating point). In the illustrated embodiment, in the case where the mantissa of the product output by multiplier 64 is greater than or equal to '2', the product can have a maximum biased value of the exponent greater than the maximum sum (M) (e.g., '60'). For example, the biased exponent of the product can be '61', which overflows from the half-precision floating point range. However, because the internal digital format of the illustrated DSP circuit 60 is FP16+++, which has an increased range compared to half-precision floating point, overflow can be reduced and / or eliminated.

[0039] On the other hand, in some embodiments, such as when the internal digital format matches the digital format of input circuit 62, the integer may be determined according to an alternative technique. For example, at least where the digital format of input circuit 62 and the internal digital format are half-precision floating point, the integer may be determined based in part on the number of adder stages included in adder tree 68. As an illustrative example, the integer may be determined based on the following equation:

[0040] Integer = deviation * 3-1-adderStages,

[0041] Wherein, the term bias represents the bias corresponding to the range of the internal digital format, and the term adderStages represents the number of adder stages in the adder tree 68. Because each adder stage can increase the exponent by a single bit, subtracting the number of adder stages can reduce and / or eliminate the overflow of the range of the exponent. Therefore, for the bias value of 15 corresponding to the bias of the half-precision floating point, and the adder tree 68 shown including two adder stages, the value '42' (e.g., 15*3-1-2) can be selected to provide the value of the exponent (e.g., input to the DSP circuit 60 and / or the variable determined by the DSP circuit 60) with an opportunity to grow (e.g., up to '45'). In addition, although the integer is described above as being determined in part based on the range of the half-precision floating point and / or based on the range of both FP16+++ and the half-precision floating point, any suitable integer can be used for another digital format or a combination of digital formats to mitigate range overflow in subsequent calculations. Therefore, the embodiments are intended to be illustrative rather than restrictive.

[0042] To determine the appropriate offset value (W), the additional subtractor 108D may subtract an integer from the maximum value of the sum (M). Thus, using the offset value (W) generated by the additional subtractor 108C, the pre-scaling circuit 102 may reduce each of the sums of exponents to an appropriate range based on the digital format (e.g., half-precision floating point) input to the DSP circuit 60 and / or the internal digital format of the DSP circuit 60. More specifically, the pre-scaling circuit 102 may subtract the offset value (W) from each of the sums of exponents so that the maximum sum of exponents (M) is reduced to an appropriate range, which may reduce and / or eliminate range overflow in the DSP circuit 60, as described above. For example, using the illustrated case of scaling the sums to half-precision floating point exponents (e.g., 5-bit exponents), the sums of exponents may be scaled to a maximum value of '60'.

[0043] Starting with the scaled sum of the exponents, the pre-scaling circuit 102 may then use the exponent adjustment circuit 112 to calculate a new corresponding exponent for each input (e.g., A3, A2, A1, A0, B3, B2, B1, and B0). More specifically, in some embodiments, the pre-scaling circuit 102 may split each of the scaled sums of the exponents from a 10-bit value into two 5-bit values, as shown. In order to determine the new corresponding exponents for the inputs, the pre-scaling circuit 102 may determine whether the scaled sum for a particular pair of exponents is even or odd. If the scaled sum is even, the pre-scaling circuit may output a new exponent for each of the pair of inputs (e.g., A0 and B0) by right-shifting the scaled sum (e.g., dividing the scaled sum by two). Thus, in the case of a sum of exponents corresponding to a maximum value (M) (which is then scaled to '60' (e.g., an even number)), the pre-scaling circuit 102 may generate '30' as the new exponent for each of the corresponding pair of inputs. On the other hand, if the scaled sum is odd, one of the exponents of the pair may be determined by right shifting the scaled sum, and the other exponent may be determined by adding '1' to the right shifted scaled sum. Furthermore, if the scaled exponent is negative, which in some cases indicates that at least one of the original inputs was '0', the pre-scaling circuit 102 may force one of the new exponents in the pair of inputs to be '0'. Furthermore, since the new exponents are equalized, the half-precision input exponent range is maximized.

[0044] Thus, for a first input (e.g., A0) of a pair of inputs, the pre-scaling circuit 102 may include a multiplexer 110 that may output a value of a scaled exponential sum divided by two (e.g., bits [5:1] shifted right by one bit) or a '0', depending on a selection signal provided by an OR gate 113 (e.g., a logical OR gate). The OR gate 113 may receive one or more most significant bits (MSBs) of the scaled exponential sum (e.g., [9:6]) and may determine a logical OR of the bits. Thus, the output of the OR gate 113 may indicate whether the scaled exponential sum is negative. Thus, as described above, the multiplexer 110 may output a '0' or a scaled exponential sum divided by two based on whether the scaled exponential sum is negative.

[0045] In addition, for the second input of the pair of inputs (e.g., B0), the pre-scaling circuit 102 can route the first bit of the scaled exponent sum (e.g., [0]) to the first input of the adder 103E (e.g., an integer adder), and can route the next four bits (e.g., [5:1]) to the second input of the adder 103E. If the scaled exponent sum is an even number, the value of the first bit is '0'. Therefore, the adder 103E will output four bits ([5:1]) that have been right-shifted one bit from their original bit positions. Therefore, as described above, the adder 103E will output the value of the scaled exponent sum divided by two as the new exponent of the second input of the pair of inputs (e.g., B0). However, if the scaled exponent sum is an odd number, the value of the first bit will be '1'. Therefore, the adder 103E will sum the first bit with the fourth bit [5:1], which have been right-shifted one bit from their original bit positions. Therefore, as described above, the adder 103E outputs '1' plus the scaled exponent and the value divided by 2 as the new exponent of the second input (eg, B0) of the pair of inputs.

[0046] As described herein, the exponent of the input is scaled by pre-scaling circuit 102. Additionally, in some embodiments, pre-scaling circuit 102 may adjust the fraction of the input. In some embodiments, for example, the fraction of the input may be different in size than the fraction format expected at input circuit 62. Thus, pre-scaling circuit 102 and / or additional circuitry and / or logic may zero pad the fraction with an appropriate number of bits or truncate an appropriate number of bits from the fraction prior to receiving the scaled input at input circuit 62. Additionally, for each input, the corresponding fraction of the input may be routed to be available simultaneously with the corresponding new exponent of the input at input circuit 62, as shown by routing 114 (e.g., wiring and / or electrical connections).

[0047] As referenced above Figure 3As discussed, the DSP circuit 60 may then receive each scaled input at the input circuit 62 and may perform a plurality of arithmetic operations on the input. More specifically, the DSP circuit 60 may perform a dot product operation on each input and may output a single precision floating point format result. Thus, the post-scaling circuit 104 may include circuits and / or logic suitable for scaling the result from a single precision floating point format to another format, such as the original format of the input received at the input circuit 106 (e.g., bfloat16). More specifically, the post-scaling circuit 104 may include circuits and / or logic suitable for scaling the range of the result back to the original range of the input received at the input circuit 106 (e.g., bfloat16). Thus, the post-scaling circuit 104 may route the exponent (eSUM') of the result to the first input of the adder 103F. In addition, the post-scaling circuit 104 may route the result offset value (Wout) to the second input of the adder 66. To determine the result offset value (Wout), post-scaling circuit 104 may route the maximum sum value (M) to subtractor 108. The subtractor may subtract an integer (e.g., '284') from the maximum sum value (M) to generate a result offset value (Wout). Since the goal of the result offset value (Wout) is to scale the exponent of the result back to the original range of the input received at input circuit 106, the integer may be selected based in part on the deviation of the original input format, the integer input to subtractor 108D (e.g., the integer used to scale down to the exponent of the input of input circuit 106), the deviation of the format of the input received at input circuit 62, and / or a suitable combination thereof. For example, for the illustrated embodiment, two times the deviation of the original input format (e.g., 2*127=254) may be summed with two times the deviation of the format of the input received at the input circuit 62 minus the integer (e.g., 2*15) input to the subtractor 108D (e.g., 60) to produce an integer value of '284' (e.g., (2*127)+(60-(2*15))=284). In addition, in embodiments where a different integer value is applied at the subtractor 108D, which may depend on one or more digital formats implemented in the DSP circuit 60, the integer value applied at the subtractor 108 of the post-scaling circuit 104 may be appropriately adjusted. For example, in the example described above where the integer applied at the subtractor 108D is '42', the integer value '266' (e.g., (2*127)+(42-(2*15))=266) may be routed to the subtractor 108 of the post-scaling circuit 104.

[0048] Although not shown, the post-scaling circuit 104 may additionally include circuitry and / or logic to handle situations when the exponent (eSUM') of the result is '0' and / or when the exponent (eSUM) of the scaled sum (S) is negative or greater than or equal to the maximum exponent value allowed by the output format. If the exponent (eSUM') of the result is '0', the result offset value (Wout) may be refreshed to '0' to maintain the value '0' of the exponent (eSUM) of the scaled sum (S). Therefore, the post-scaling circuit 104 may include, for example, a logic gate and / or multiplexer 110 that is implemented to determine whether the exponent (eSUM') of the result is '0' and select between the result offset value (Wout) and '0' based on the determination. If the biased exponent (eSUM) of the scaled sum is negative, a circuit such as the multiplexer 110 may forward '0' to the exponent (eSUM) of the scaled sum. In addition, the post-scaling circuit 104 may include circuitry and / or logic to handle situations where the exponent of the scaled sum (eSUM) equals or exceeds the maximum exponent value of the output format (e.g., single-precision floating point) of the DSP circuit 60. In the illustrated embodiment, for example, if the exponent of the scaled sum (eSUM) is greater than or equal to the maximum exponent value of the single-precision floating point (e.g., '255'), circuitry (not shown) such as the multiplexer 110 may forward the exponent of the scaled sum (eSUM) to the exponent of the scaled sum (eSUM), and may refresh the value of the fraction of the scaled sum (fSUM) to zero.

[0049] In addition, while the illustrated scaling circuit 26 is suitable for adjusting the format of the input to the DSP circuit 60 and the format of the output from the DSP circuit, other embodiments may only scale the input or the output. In addition, the pre-scaling circuit may include any suitable circuit and / or logic to determine the maximum value (M). For example, in addition to or as an alternative to the illustrated combination of the subtractor 108 and the multiplexer 110, the pre-scaling circuit 102 may include different circuits and / or logic implemented to determine the maximum value (M), such as a comparator. In addition, in some embodiments, the pre-scaling circuit 102 may be implemented to convert the input from a digital format other than bfloat16 and / or the post-scaling circuit 104 may be implemented to convert the output to a digital format other than bfloat16. Therefore, the integers used to determine the offset value (W) and the resulting offset value (Wout) may be adjusted. In some embodiments, for example, each integer may be programmed based on the digital format of the input to the pre-scaling circuit and the expected digital format of the input received at the DSP circuit 60. Thus, integers may be stored in programmable mode registers and / or suitable memory locations and updated based on the implementation of DSP circuit 60 and / or the format of the input to pre-scaling circuit 102. In any case, the embodiments described herein are intended to be illustrative and not limiting.

[0050] Now go to Figure 6 , shows an example of a process 140 for adjusting the representation (e.g., format) of numbers before and after processing. Generally, process 140 includes scaling a set of raw inputs to DSP circuit 60 from a first format to a second format (process block 142), performing operations on the scaled inputs (process block 144), and scaling the results produced by the DSP circuit to the first format (process block 146).

[0051] Although the following description of process 140 is described in a particular order, which represents a particular embodiment, it should be noted that process 140 may be performed in any suitable order. In addition, embodiments of process 140 may omit process blocks and / or include suitable additional process blocks. Although process 140 is described as being implemented by scaling circuitry 26 (e.g., pre-scaling circuitry 102 and post-scaling circuitry 104) and DSP circuitry 60, portions of process 140 may be implemented by any suitable circuitry and / or logic. For example, in some embodiments, process 140 may be implemented at least in part by executing instructions stored in a tangible, non-transitory computer-readable medium (such as a memory) using processing circuitry such as one or more processors.

[0052] As shown, in some embodiments, process 140 can start by scaling a set of inputs to DSP circuit 60 from a first format to a second format (process block 142). For example, an input having a bfloat16 floating point format (e.g., a 1-bit sign field, an 8-bit exponent field, and a 7-bit fraction field) can be scaled to a half-precision floating point format. Although the example input is described as being scaled from bfloat16 to half precision, any suitable input format can be scaled to any other suitable format. For example, the input can be received as a single-precision, double-precision, or custom numeric format, among other formats, and can be scaled to half precision, bfloat16, another custom numeric format, and the like.

[0053] Now go to Figure 7 , shows an example of a process 160 for scaling the set of inputs from a first format to a second format. In general, process 160 includes summing the exponents of each pair of corresponding original inputs (process block 162), determining a maximum value (M) of the sums (process block 164), calculating an offset value (W) using the maximum value (M) of the sums (process block 166), adjusting the sums using the offset value (W) (process block 168), determining a corresponding new exponent for each original input using the corresponding adjusted sums (process block 170), and forming a corresponding scaled input in the second format for each original input using the corresponding new exponent (process block 172).

[0054] Although the following description of process 160 is described in a particular order, which represents a particular embodiment, it should be noted that process 160 may be performed in any suitable order. In addition, embodiments of process 160 may omit process blocks and / or include suitable additional process blocks. Although process 160 is described as being implemented by pre-scaling circuit 102, portions of process 160 may be implemented by any suitable circuitry and / or logic. For example, in some embodiments, process 160 may be implemented at least in part by executing instructions stored in a tangible, non-transitory computer-readable medium (such as a memory) using processing circuitry such as one or more processors.

[0055] As shown, in some embodiments, process 160 may begin by summing the exponents of each pair of original inputs (process block 162). As described above, for example, a set of adders 103 in pre-scaling circuit 102 may sum the input pairs corresponding to the dot product input pairs. For example, the set of adders 103 may sum the exponents of the first input (e.g., A0) and the second input (e.g., B0) to be multiplied by each other at DSP circuit 60.

[0056] The illustrated process 160 then proceeds to determine the maximum value (M) of the sum of the exponents of each pair of inputs (process block 164). In order to determine the maximum value (M) of the sum of the exponents, the sums can be compared to each other. Therefore, in some embodiments, the subtractor 108 can determine the difference between a pair of sums. The sign t of the difference determined by the subtractor 108 can then be used to select the sum with a higher value at, for example, the multiplexer 110. In addition, any suitable number of subtractors 108 and multiplexers 110 can be used in series (e.g., sequentially) and / or in parallel to perform a suitable number of comparisons to determine the maximum value (M) of the sum. Additionally or alternatively, the exponential sums can be compared by other suitable logic and / or circuits to identify the maximum value (M) of the sum.

[0057] The identified maximum value (M) of the sum may then be used to calculate an offset value (W) (process block 166). The offset value (W) may be used to scale the sum of the exponents based on the range of the digital format (e.g., half precision) expected by the input circuit 62 of the DSP circuit 60 and / or the internal digital format of the DSP circuit 60. Thus, in some embodiments, the offset value (W) may be calculated by subtracting an integer value from the maximum value (M) using a subtractor 108D. The integer value may represent the sum of the maximum biased exponent values ​​of the range of the digital format of the input circuit 62. Thus, in order to scale the input to a half precision floating point format, an integer value of '60' (e.g., 30+30) may be used. Additionally or alternatively, the integer value may be selected to reduce or eliminate range overflow during operations implemented in the DSP circuit 60.

[0058] After calculating the offset value (W), the sum of the exponents of the input pair may be adjusted (e.g., scaled) using the offset value (process block 168). More specifically, the offset value (W) may be subtracted from each of the sum of the exponents of the input pair. As a result, the sum of the exponents corresponding to the maximum value (M) of the sum may be reduced to a suitable range to mitigate range overflow in the DSP circuit 60. For example, in the case where the second format is half precision, the sum of the exponents corresponding to the maximum value (M) may be adjusted down to '60'.

[0059] Using the corresponding adjusted sums, a corresponding new exponent can then be determined for each of the original inputs (process block 170). For a first input and a second input, if the adjusted sum of the exponents of the first input and the second input is an even number, a new exponent for each of the first input and the second input can be determined by dividing the adjusted sum by two. Thus, the adjusted exponent of the first input can be equal to the adjusted exponent of the second input. On the other hand, if the adjusted sum is an odd number, the adjusted exponent of the first input can be determined by taking the floor of the adjusted sum divided by two, and the adjusted exponent of the second input can be determined by taking the floor of the adjusted exponent divided by two and then adding '1'. Additionally, if the adjusted sum is negative (e.g., less than '0'), the first input can be forced to '0'.

[0060] After determining the new exponent for each original input, a set of scaled inputs may be formed using the corresponding new exponents (process block 172). More specifically, the corresponding remaining bits (e.g., sign bit and / or mantissa) of each input may be combined with the corresponding new exponent to form the corresponding scaled input in the second digital format. In some embodiments, if the precision (e.g., bit width) of the mantissa does not match the precision of the second digital format, the mantissa may be truncated or zero-padded as appropriate. For example, scaling the input from bfloat16 to half precision may involve zero-padding the mantissa from seven bits to ten bits. In addition, by combining the remaining bits, such as the sign bit and mantissa, with the corresponding new exponent, the format of each scaled input may be suitable for input to the DSP circuit 60.

[0061] Now return to Figure 6 , the illustrated embodiment of process 140 then proceeds to perform an operation on the scaled inputs (process block 144). In some embodiments, the operation involves a dot product operation. For example, after combining the remaining bits of each input with the corresponding new input in the second digital format, input circuitry 62 may route the scaled inputs to DSP circuitry 60, which may use the scaled inputs to determine a dot product.

[0062] The operation performed on the input may produce a result, which may be represented in a second digital format or a third digital format, such as single precision. For example, in some embodiments, in order to reduce the overflow generated by the sum of multiple half-precision operands, one or more single-precision adders and / or single-precision combinational circuits may be used to produce a single-precision result. Thus, the result of the operation may be scaled to a first format (e.g., the original format of the input) (process block 146). In some embodiments, scaling the result to the first format may involve scaling the range of the result back to the original range of the first format. Thus, scaling the result may involve determining a result offset (Wout) using a maximum value (M) of the sum of exponents and an integer. For example, the result offset (Wout) may be determined by subtracting an integer from the maximum value (M) of the sum. The integer may be determined and / or programmed based on the format of the input (e.g., the first format), the format expected at the input circuit 62 of the DSP circuit 60 (e.g., the second format), the internal digital format of the DSP circuit 60, or a suitable combination thereof. In addition, if the exponent is non-zero, the result offset (Wout) may be summed with the exponent of the result to scale the exponent. If the exponent is zero, the exponent may be summed with zero and / or the summing operation may be bypassed so that the exponent remains zero. Thus, while the operation is performed in another format, the format of the output from DSP circuit 60 may be adjusted to the format of the input by, for example, post-scaling circuit 104. That is, for example, the digital representation of the input may be adjusted before and after processing so that an operation performed in a first format may be emulated by an operation performed in another format.

[0063] Now go to Figure 8 Although the DSP circuit 60 described herein is implemented with four multipliers 64 (e.g., implemented to receive up to eight independent inputs), the techniques described herein may be applied to larger multiplier structures (e.g., dot product structures), such as the extended arithmetic operation emulation circuit 180. Therefore, in some embodiments, the pre-scaling circuit 102 may include an additional input circuit 106. In addition, since the pre-scaling circuit 102 may determine the maximum value (M) of the sum of the exponent pairs based on each input to the pre-scaling circuit 102, the sum of the exponent pairs corresponding to the additional input circuit 106 may be used to determine the maximum value of the sum (M). For example, in Figure 8 In the illustrated embodiment of , pre-scaling circuit 102 is implemented to receive thirty-two inputs (e.g., two vectors (A and B) of sixteen inputs). Thus, pre-scaling circuit 102 may determine a maximum value (M) of sixteen sums of pairs of exponents. As described above, pre-scaling circuit 102 may then scale each of the thirty-two inputs using at least an offset value (W) determined using the maximum value (M).

[0064] As further shown, the scaled inputs may be routed in groups to a suitable number of DSP circuits 60. For example, the scaled inputs may be routed in groups of eight to DSP circuits 60 having groups of four multipliers. Additionally or alternatively, the DSP circuits 60 may be implemented with a greater or lesser number of multipliers 64, which may change the number of inputs included in each group routed to the DSP circuits 60 and / or the number of groups. Each DSP circuit 60 may then determine a portion of the final dot product. For example, the DSP circuits 60 may determine a first product for a first pair of inputs, may determine a second product for a second pair of inputs, and may output a sum of the first product and the second product.

[0065] To produce the final result of the dot product of the inputs, the output of each DSP circuit 60 may be summed. Therefore, in some embodiments, the extended arithmetic operation emulation circuit 180 may include one or more adders 66, which may be constructed in an adder tree 68, implemented to sum the outputs of the DSP circuits 60. In addition, because each DSP circuit 60 may produce a single-precision floating-point output, the adders 66 may be implemented to add the single-precision floating-point inputs and produce a single-precision floating-point sum. Therefore, the final dot product produced by the sum of each output of the DSP circuit 60 may be formatted as a single-precision floating-point number.

[0066] To this end, the extended arithmetic operation emulation circuit 180 may include a post-scaling circuit 104 to scale the final dot product. As shown, for example, the final dot product may be scaled from a single-precision floating point format to bfloat16. To this end, the extended multiplier structure may route the result offset value (Wout) to the post-scaling circuit 104. As described above, the post-scaling circuit 104 may use the result offset value (Wout) to scale the exponent of the final dot product back to the original range of the input of the pre-scaling circuit 102. In order to determine the result offset value (Wout), the maximum sum value (M) determined at the pre-scaling circuit 102 may be routed to the subtractor 108. The subtractor may subtract an integer (e.g., '284') determined based on a first format (e.g., bfloat16), a second format (e.g., half-precision floating point), an internal digital format of the DSP circuit 60 (e.g., FP16+++), or a combination thereof from the maximum sum value (M) to generate a result offset value (Wout).

[0067] Although implemented to receive bfloat16 input in the illustrated embodiment of the extended arithmetic operation emulation circuit 180, the extended arithmetic operation emulation circuit 180 may be implemented to receive any suitable digital format, such as half-precision floating point, single-precision floating point, and / or extended precision format. In addition, as described above, the DSP circuit 60 may be implemented to receive any suitable format. Therefore, the pre-scaling circuit 102 of the extended multiplier structure may be implemented to adjust the format of the received input to a format suitable for the DSP circuit 60, such as half-precision floating point, single-precision floating point, etc. In addition, the post-scaling circuit 104 may be implemented to adjust the final dot product to any format corresponding to the format of the input received by the pre-scaling circuit 102. Therefore, the embodiments described herein are intended to be illustrative rather than restrictive.

[0068] Furthermore, in some embodiments, the techniques described herein may be implemented recursively. For example, in some embodiments, the adder tree 68 of the extended arithmetic operation simulation circuit 180 may be replaced by one or more hierarchical levels of additional arithmetic operation simulation circuits 100 and / or additional extended arithmetic operation simulation circuits 180. Thus, instead of summing the outputs of the DSP circuit 60 of the extended arithmetic operation simulation circuit 180, the outputs may be scaled and routed to the additional arithmetic operation simulation circuit 100. More specifically, the post-scaling circuit 104 of the extended arithmetic operation simulation circuit 180 may adjust each output of the illustrated DSP circuit 60 to the original range and / or digital format (e.g., bfloat16) of the input of the extended arithmetic operation simulation circuit 180, and the scaled outputs may be routed to the additional arithmetic operation simulation circuit 100. The additional arithmetic operation simulation circuit 100 may, as described with reference to Figure 5 The present invention includes a pre-scaling circuit 102 adapted to scale each scaled output to a digital format (e.g., half-precision floating point) suitable for the DSP circuit 60 of the additional arithmetic operation emulation circuit 100. The digital format suitable for the DSP circuit 60 of the additional arithmetic operation emulation circuit 100 may be the same as or different from the digital format suitable for the DSP circuit 60 of the extended arithmetic operation emulation circuit 180.

[0069] The DSP circuit 60 may then perform one or more arithmetic operations on the scaled output. The one or more arithmetic operations may be the same or different from the one or more arithmetic operations performed by the DSP circuit 60 of the extended arithmetic operation simulation circuit 180. For example, the DSP circuit 60 may calculate dot products and / or additional MAC operations. In addition, the DSP circuit 60 described herein includes eight independent inputs. Therefore, half of the inputs of the DSP circuit 60 may be used to perform arithmetic operations on the inputs to the additional arithmetic operation simulation circuit 100. Alternatively, an embodiment of the DSP circuit 60 (e.g., input circuit 62) implemented with four independent inputs may be included in the additional arithmetic operation simulation circuit 100.

[0070] The additional arithmetic operation emulation circuit 100 may use the post-scaling circuit 104 to scale the output of the DSP circuit 60 back to the original format (e.g., bfloat16) received at the pre-scaling circuit 102 of the additional arithmetic operation emulation circuit 100. In some embodiments, the scaled output of the additional arithmetic operation emulation circuit 100 may represent the final result of a series of one or more recursive arithmetic operations performed at one or more arithmetic operation emulation circuits 100 and / or the extended arithmetic operation emulation circuit 180. Alternatively, the scaled output of the additional arithmetic operation emulation circuit may then be fed into another arithmetic operation circuit 100.

[0071] Furthermore, in some embodiments, instead of scaling the output of the DSP circuit 60 of the extended arithmetic operation simulation circuit 180 to the original format of the input of the extended arithmetic operation simulation circuit 180 (e.g., bfloat16) before routing the output to the additional arithmetic operation simulation circuit 100, the output may be routed directly to the additional arithmetic operation simulation circuit 100. In this case, the additional arithmetic operation simulation circuit 100 may then scale the output from, for example, a single-precision floating point format to a format suitable for the DSP circuit 60 of the additional arithmetic operation simulation circuit 100 (e.g., half-precision floating point). Furthermore, the output of the DSP circuit 60 may be scaled back to the format (e.g., single-precision floating point) output by the DSP circuit 60 of the extended arithmetic operation simulation circuit 180 by the post-scaling circuit 104 of the additional arithmetic operation simulation circuit 100. Thus, the post-scaling circuit 104 of the extended arithmetic operation simulation circuit 180 may scale the output of the additional arithmetic operation simulation circuit 100 to the format 104 of the input of the extended arithmetic operation simulation circuit 180 (e.g., from a single-precision floating point format to bfloat16).

[0072] In addition, the integrated circuit device 12 may be a data processing system, or may be a component of a data processing system. For example, the integrated circuit device 12 may be Fig. 9Components of data processing system 200 are shown. Data processing system 200 may include host processor 202, memory and / or storage circuitry 204, and network interface 206. Data processing system 200 may include more or fewer components (e.g., electronic displays, user interface structures, application specific integrated circuits (ASICs)). Host processor 202 may include any suitable processor that can manage data processing requests for data processing system 200 (e.g., for performing encryption, decryption, machine learning, video processing, speech recognition, image recognition, data compression, database search ranking, bioinformatics, network security pattern recognition, spatial navigation, etc.). The data processing system 200 may include a processor or a reduced instruction processor (e.g., a reduced instruction set computer (RISC), an advanced RISC machine (ARM) processor). The memory and / or storage circuit 204 may include a random access memory (RAM), a read-only memory (ROM), one or more hard disk drives, flash memory, etc. The memory and / or storage circuit 204 may hold data to be processed by the data processing system 200. In some cases, the memory and / or storage circuit 204 may also store a configuration program (bitstream) for programming the integrated circuit device 12. The network interface 206 may allow the data processing system 200 to communicate with other electronic devices. The data processing system 200 may include several different packages, or may be contained in a single package on a single package substrate.

[0073] In one example, the data processing system 200 may be part of a data center that processes a variety of different requests. For example, the data processing system 200 may receive data processing requests via the network interface 206 to perform encryption, decryption, machine learning, video processing, speech recognition, image recognition, data compression, database search ranking, bioinformatics, network security pattern recognition, spatial navigation, or some other specialized tasks. The host processor 202 may program the programmable logic structure of the integrated circuit device 12 with an adder suitable for implementing the requested task. For example, the host processor 202 may instruct the configuration data (bitstream) stored on the memory and / or storage circuit 204 to be programmed into the programmable logic structure of the integrated circuit device 12. The configuration data (bitstream) may represent a circuit design for the scaling circuit 26, which may be mapped to the programmable logic according to the techniques described herein to adjust the digital representation of the input and / or output of the hard logic (e.g., the DSP circuit 60). By adjusting the digital representation of the input, the arithmetic operation performed in the first format may be simulated by the scaling result of the arithmetic operation performed in the second format. In this way, integrated circuit device 12 may assist data processing system 200 in performing a requested task even when integrated circuit device 12 lacks hardware support for the digital format of one or more variables (eg, inputs) involved in processing the requested task.

[0074] Although the embodiments set forth in this disclosure may allow for various modifications and alternative forms, specific embodiments have been shown in the drawings by way of example and have been described in detail herein. For example, any suitable combination of the embodiments and / or techniques described herein may be implemented. Therefore, it should be understood that the disclosure is not intended to be limited to the specific forms disclosed. The present disclosure will cover all modifications, equivalents, and substitutions that fall within the spirit and scope of the disclosure defined by the following claims.

[0075] The technology proposed and claimed herein is cited and applied to specific examples and substantial purposes of a practical nature that clearly improve the art and are not abstract, intangible, or purely theoretical. In addition, if any claim appended to the end of this specification contains one or more elements designated as "means for [performing] the [function] ..." or "steps for [performing] the [function] ...", it is intended that these elements are to be interpreted under 35 U.S.C. § 112(f). However, for any claim containing elements designated in any other manner, it is intended that these elements should not be interpreted under 35 U.S.C. § 112(f).

Claims

1. An integrated circuit, include: A conversion circuit, wherein the conversion circuit can be configured as: receiving a set of inputs in a first digital format; as well as generating a set of converted inputs by converting the set of inputs from the first digital format to a second digital format, wherein the second digital format includes an exponent value that is less than an exponent value of the first digital format; as well as An arithmetic circuit is communicatively coupled to the conversion circuit and is configurable to perform an arithmetic operation on the set of converted inputs to produce an output.

2. The integrated circuit according to claim 1, in, The arithmetic operations include multiplication and accumulation.

3. The integrated circuit according to claim 2, in, The arithmetic circuit can be configured to generate the output in a third digital format that is larger than the first digital format.

4. The integrated circuit of claim 3, comprising a post-conversion circuit configurable to receive the output from the arithmetic circuit and convert the output from the third digital format to the first digital format.

5. The integrated circuit according to claim 4, in, The conversion circuit can be configured to scale the set of inputs based on a scaling factor.

6. The integrated circuit according to claim 5, in, The conversion circuit can be configured to determine the scaling factor based on a maximum value of the set of inputs.

7. An integrated circuit as claimed in claim 6, comprising a memory configurable to store the maximum value.

8. The integrated circuit according to claim 1, in, The second digital format includes a 4-digit exponent or a 5-digit exponent.

9. The integrated circuit according to claim 1, in, The conversion circuit can be configured to convert between the first digital format and the second digital format by adjusting the exponent value of the first digital format from a first digit to a second digit.

10. The integrated circuit according to claim 1, in, The first numeric format includes half-precision floating point numbers.

11. A method, include: receiving, via processing circuitry, a set of inputs in a first digital format; generating, via the processing circuit, a set of converted inputs by converting the set of inputs from the first digital format to a second digital format, wherein the second digital format includes an exponent value that is less than an exponent value of the first digital format; determining, via the processing circuit, a scaling factor based on a maximum value of the set of converted inputs; generating, via the processing circuit, a set of scaled converted inputs by scaling the set of converted inputs according to the scaling factor; and An output is generated, via the processing circuit, by performing an arithmetic operation on the set of scaled converted inputs, wherein the output is in a third digital format that is larger than the first digital format.

12. The method of claim 11, comprising converting the output from the third digital format back to the first digital format via the processing circuit.

13. The method according to claim 11, in, Generating the set of scaled inputs via the processing circuit includes converting the exponent value of the first digital format from a first digit to a second digit.

14. The method of claim 11 , comprising determining, via the processing circuit, a target digital format prior to generating the set of scaled inputs, in, The target digital format is the second digital format.

15. The method according to claim 11, in, The arithmetic operations include multiplication and accumulation.

16. A system, include: substrate; a processor mounted on the substrate; a memory mounted on the substrate; as well as An integrated circuit as claimed in any one of claims 1 to 10, wherein the integrated circuit is mounted on the substrate and is communicatively coupled to the processor and the memory.

17. A computer readable medium comprising instructions which, when executed by at least one processor, cause the processor to perform the method according to any one of claims 11 to 15.

18. An apparatus comprising means for performing the method according to any one of claims 11 to 15.

19. A computer program product comprising instructions which, when executed by at least one processor, cause the processor to perform the method according to any one of claims 11 to 15.

Images