Multiplier unit and method and apparatus for calculating the dot product of floating point values

Abstract

A multiplier unit and a method and apparatus for calculating a dot product of floating point values are disclosed. The apparatus includes an array of multiplier units each including integer logic to multiply integer values of corresponding elements of two vectors, exponent logic to add exponent values of corresponding elements of the two vectors to form un-biased exponent values, and a local shifter to form a first shifted value by shifting an integer product value in a predetermined direction by a number of bits based on a difference between the un-biased exponent value corresponding to the integer product value and a maximum un-biased exponent value of the array of multiplier units. An adder tree adds the shifted values output from the local shifters of the array of multiplier units to form an output, and an accumulator accumulates the output of the adder unit.

Description

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 008,544, filed April 10, 2020, and U.S. Application No. 17 / 153,871, filed January 20, 2021, the disclosures of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The subject matter disclosed herein relates to computing units. More specifically, the subject matter disclosed herein relates to computing unit architectures and methods for computing floating point dot products. BACKGROUND

[0003] Dot products of activation values and weight values are operations that are commonly computed by deep neural network (DNN) accelerators. Activation values and weight values can be represented by 16-bit half-precision floating point (FP16) values. FP16 values can be represented by a sign, an exponent, and a fraction. As an example, Figure 1A The half-precision FP16 representation is depicted including 1 sign bit, 5 exponent bits (bias = 15), and 10 fraction bits (with 1 hidden bit). Figure 1B A table is shown setting forth example representations of different types of FP numbers. Figure 1B The representation in is for exponents not equal to 0 or infinity (1...1), and for the bias 15 (127) of FP16. The acronym NAN represents not-a-number. SUMMARY

[0004] Example embodiments provide an apparatus for computing a dot product of a first vector and a second vector, where the apparatus can include a multiplier cell array, a max tree cell, an adder tree, and an accumulator. The first vector can be activation values and the second vector can be weight values. A multiplier cell of the multiplier cell array can include integer logic, exponent logic, and a local shifter. The integer logic can multiply integer values of corresponding elements of the first vector and the second vector to form a product integer value, where the first vector and the second vector can include floating point values. The exponent logic can add exponent values corresponding to the integer values of the corresponding elements of the two vectors to form an un-biased exponent value corresponding to the product integer value. The local shifter can form a first shifted value by shifting the product integer value a number of bits in a predetermined direction based on a difference between the un-biased exponent value corresponding to the product integer value and a maximum un-biased exponent value of the multiplier cell array being less than or equal to a predetermined maximum bit shift capability of the local shifter. The max tree cell can determine the maximum un-biased exponent value of the multiplier cell array. The adder tree can add the first shifted values output from the local shifters of the multiplier cell array to form a first output, and the accumulator can accumulate the first output of the adder tree. In one embodiment, the apparatus can further include a mask generator that generates a first mask that incorporates the first shifted values to the adder tree based on the difference between the un-biased exponent value corresponding to the product integer value and the maximum un-biased exponent value being less than or equal to the predetermined maximum bit shift capability of the local shifter corresponding to the first shifted value, where the adder tree can add the first shifted values output from the local shifters of the multiplier cell array and incorporated to the adder tree to the first mask to form the first output. The mask generator can generate the first mask during a first cycle. In another embodiment, the mask generator can generate a second mask that incorporates the first shifted values to the adder tree based on the difference between the un-biased exponent value corresponding to the product integer value and the maximum un-biased exponent value being greater than the predetermined maximum shift capability of the local shifter corresponding to the first shifted value, where the adder tree can add the first shifted values output from the local shifters of the multiplier cell array and incorporated to the adder tree to the second mask to form a second output. The apparatus can further include a secondary shifter incorporated to the adder tree and can form a second shifted value by shifting the second output from the adder tree by the predetermined maximum bit shift capability of the local shifter, and the accumulator can further accumulate the second output of the adder tree. The mask generator can generate the second mask during a second cycle. In one embodiment, the activation values and the weight values can include 16-bit floating point (FP16) values. In another embodiment, the activation values and the weight values can be 32-bit floating point (FP32) values.

[0005] Example embodiments provide a multiplier unit that can include integer logic, exponent logic, and a local shifter. The integer logic can multiply integer values of elements of a first vector and corresponding elements of a second vector to form product integer values. The first vector can be activation values and the second vector can be weight values. The exponent logic can add exponent values corresponding to the integer values of the corresponding elements of the two vectors to form un-biased exponent values corresponding to the product integer values. The local shifter can form first shifted values by shifting the product integer values a number of bits in a predetermined direction based on a difference between the un-biased exponent values corresponding to the product integer values and a predetermined value being less than or equal to a predetermined maximum bit shift capability of the local shifter. The multiplier unit can be part of a multiplier unit array, a max tree unit, an adder tree, and an accumulator. The max tree unit can determine the predetermined value, where the predetermined value can include a maximum un-biased exponent value of the multiplier unit array. The adder tree can add the first shifted values output from the local shifters of the multiplier unit array to form a first output. The accumulator can accumulate the first output of the adder tree. A mask generator can generate a first mask to incorporate the first shifted values to the adder tree based on the difference between the un-biased exponent values corresponding to the product integer values and the maximum un-biased exponent value being less than or equal to the predetermined maximum bit shift capability of the local shifter corresponding to the first shifted values. The adder tree can add the first shifted values output from the local shifters of the multiplier unit array and incorporated to the adder tree with the first mask to form the first output. The mask generator can generate the first mask during a first cycle. The mask generator can generate a second mask to incorporate the first shifted values to the adder tree based on the difference between the un-biased exponent values corresponding to the product integer values and the maximum un-biased exponent value being greater than the predetermined maximum bit shift capability of the local shifter corresponding to the first shifted values. The mask generator can generate the second mask during a second cycle. The adder tree can add the first shifted values output from the local shifters of the multiplier unit array and incorporated to the adder tree with the second mask to form a second output. The multiplier unit array can further include a helper shifter incorporated to the adder tree and forming second shifted values by shifting the second output from the adder tree by the predetermined maximum bit shift capability of the local shifter, and the accumulator can further accumulate the second output of the adder tree. In one embodiment, the activation values and the weight values can be 16-bit floating point (FP16) values. In another embodiment, the activation values and the weight values can be 32-bit floating point (FP32) values.

[0006] An example embodiment provides a method of computing a dot product of floating point values, where the method can include: multiplying, by integer logic of a multiplier cell array, integer values of elements of a first vector by integer values of corresponding elements of a second vector element-wise to form integer product values, where the first vector can include n elements of 16-bit floating point values, the second vector can include n elements of 16 floating point values, where n is an integer greater than 1; adding, by exponent logic of the multiplier cell array, exponent values of elements of the first vector and exponent values of corresponding elements of the second vector element-wise to form exponent sum values corresponding to the integer product values, respectively; determining a maximum exponent sum value among the exponent sum values; subtracting, by the exponent logic, the maximum exponent sum value from each of the exponent sum values to form relative exponent values corresponding to the integer product values, respectively; right shifting, by first local shifters of the multiplier cell array, the first integer product values by corresponding relative exponent values to form the integer product values aligned with the integer product value corresponding to the maximum exponent sum value, each first local shifter including a first predetermined maximum number of bit shifts, the first predetermined maximum number being less than an entire bit range of the exponent sum values of the first vector and the exponent sum values of the second vector, and the first integer product value corresponding to the relative exponent value being less than or equal to the first predetermined maximum number of bit shifts; and adding the first integer product values aligned with the integer product value corresponding to the maximum exponent sum value to form the dot product of the first vector and the second vector. In one embodiment, the method can further include: right shifting, by an auxiliary shifter, the second integer product values by a second predetermined number of bit shifts to form the second integer product values aligned with the integer product value corresponding to the maximum exponent sum value, where the second predetermined number of bit shifts can include the first predetermined maximum number of bit shifts. The entire bit range of the exponent values of the first vector and the exponent values of the second vector can include 58 bits, and the first predetermined maximum number of bit shifts plus the second predetermined number of bit shifts can be less than or equal to 58 bits. In one embodiment, the elements of the first vector and the elements of the second vector can include 16-bit floating point (FP16) values. In another embodiment, the elements of the first vector and the elements of the second vector can include 32-bit floating point (FP32) values. BRIEF DESCRIPTION OF DRAWINGS

[0007] In the following sections, aspects of the subject matter disclosed herein will be described with reference to exemplary embodiments illustrated in the drawings, wherein:

[0008] Figure 1A Depiction of a half-precision FP16 representation including a sign bit, an exponent bit, and a mantissa bit;

[0009] Figure 1B Table setting forth example representations of different types of FP numbers;

[0010] Figure 2depicting an example of a first embodiment of a dot product computation architecture 200 according to the subject matter disclosed herein that aligns exponents of a mantissa product so that the mantissas can be added while also being optimized for area and power;

[0011] Figure 3A showing a distribution histogram of example maximum exponent values for deep learning FP16 data;

[0012] Figure 3B showing a distribution histogram of example maximum negative exponent values for deep learning FP16 data;

[0013] Figure 3C showing a distribution histogram of example product exponent values for deep learning FP16 data;

[0014] Figure 4A and Figure 4B depicting an example embodiment of a dot product computation architecture that can be optimized for computing dot products of deep learning networks according to the subject matter disclosed herein;

[0015] Figure 5A and Figure 5B depicting additional details of another example embodiment of a dot product computation architecture that provides multi-cycle operations according to the subject matter disclosed herein;

[0016] Figure 6 is a flowchart of an example method for computing dot products of deep learning networks according to the subject matter disclosed herein; and

[0017] Figure 7 depicting an electronic device including a CNN accelerator that includes one or more dot product computation architectures disclosed herein. DETAILED DESCRIPTION

[0018] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. However, it will be understood by those skilled in the art that the disclosed aspects can be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the subject matter disclosed herein.

[0019] References throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" or "in an embodiment" or "according to one embodiment" (or other similar phrases) throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In this respect, as used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Furthermore, in one or more embodiments, the particular features, structures, or characteristics can be combined in any suitable manner. Moreover, according to the context of discussion herein, a singular term can include its corresponding plural form and a plural term can include its corresponding singular form. Similarly, a term with a hyphenated form can occasionally be used in conjunction with its unhyphenated form, and vice versa. Also, terms with an article (e.g., "a", "an", "the", "said", etc.) can be used with or without the article, and terms without an article can be used with or without an article. Such occasional uses of the article and other similar forms will be subject to the context in which it is used. Moreover, the use of the terms "first", "second", "third", etc. to describe a common but distinct feature, structure, or characteristic can be used interchangeably with at least one of the other terms as possible.

[0020] Furthermore, according to the context of discussion herein, a singular term can include its corresponding plural form and a plural term can include its corresponding singular form. It should also be noted that the various figures (including component diagrams) shown and discussed herein are only meant to be illustrative and not drawn to scale. Similarly, the various waveform and timing diagrams are shown for illustrative purposes only. For example, the dimensions of some elements can be exaggerated relative to other elements for clarity. Further, repeated reference numbers can be reused in different drawings in order to simplify the drawings and facilitate the discussion.

[0021] The terminology used herein is for the purpose of describing some exemplary embodiments only and is not intended to limit the claimed subject matter. Unless the context clearly indicates otherwise, the singular forms used herein are intended to include the plural forms as well. It will also be understood that the terms “comprising” and / or “including” as used in this specification indicate the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Unless thus clearly defined, the terms “first,” “second,” etc., as used herein, serve as labels for nouns following them and do not indicate any type of order (e.g., spatial, temporal, logical, etc.). Furthermore, the same reference numerals may be used between two or more figures to denote parts, components, blocks, circuits, units, or modules having the same or similar functions. However, such use is merely for simplification and ease of discussion and does not imply that the construction or architectural details of such components or units are identical across all embodiments or that such commonly referenced components / modules are the only way to implement some of the exemplary embodiments disclosed herein.

[0022] It will be understood that when an element or layer is referred to as being on, "connected to," or "bonded to" another element or layer, it may be directly on, directly connected to, or directly bonded to that other element or layer, or there may be intermediate elements or layers. In contrast, when an element or layer is referred to as being "directly on," "directly connected to," or "directly bonded to" another element or layer, there are no intermediate elements or layers. The same reference numerals always denote the same element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] Unless clearly defined as such, the terms “first,” “second,” etc., used herein serve as labels for nouns that follow them and do not indicate any type of order (e.g., spatial, temporal, logical, etc.). Furthermore, the same reference numerals may be used between two or more figures to denote parts, components, blocks, circuits, units, or modules having the same or similar functions. However, such use is merely for simplification and ease of discussion and does not imply that the construction or structural details of such components or units are identical across all embodiments, or that such commonly referenced parts / modules are the only way to implement some of the exemplary embodiments disclosed herein.

[0024] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject pertains. It will also be understood that, unless clearly defined herein, terms (such as those defined in general dictionaries) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and shall not be interpreted in an idealized or overly formalistic manner.

[0025] As used herein, the term "module" means any combination of software, firmware, and / or hardware configured to provide the functionality described herein in conjunction with the module. For example, software may be implemented as a software package, code, and / or instruction set or instructions, and the term "hardware" as used in any implementation described herein may include single or combined components, hardwired circuitry, programmable circuitry, state machine circuitry, and / or firmware storing instructions executed by the programmable circuitry. Modules may be implemented collectively or individually as part of a larger system (e.g., but not limited to, integrated circuits (ICs), system-on-a-chip (SoCs), components, etc.).

[0026] The subject matter disclosed herein provides an architecture for computing the dot product of floating-point values ​​in deep learning data, optimized for common scenarios in terms of area and power. The architecture optimization can be based on the distribution of the range of exponent values ​​to be processed. Depending on the distribution of the exponent range, the architecture can be optimized to mantissa alignment and compute the dot product within one cycle. In other embodiments, the architecture can be optimized to mantissa alignment and compute the dot product within two or three cycles. In one embodiment, the floating-point value can be an FP16 value. In another embodiment, the floating-point value can be an FP32 value or a bfloat16 value. Area optimization of the architecture can be achieved with a relatively small space compared to the space used by an architecture that covers the entire range of exponent values ​​associated with a particular floating-point format.

[0027] In some cases, because adding relatively small floating-point values ​​to large floating-point values ​​will not significantly adversely affect the dot product calculation, the alignment of the mantissa of the relatively small floating-point value with the relatively large floating-point value can be ignored or approximated by truncation without significant performance degradation. In one embodiment, the alignment capability of the architecture disclosed herein may be smaller than the range of exponent values ​​expected to be processed, and relatively small floating-point values ​​can be ignored or partially truncated, still providing sufficiently accurate results. In the case of partial truncation, correct alignment should still occur, but only a portion of the aligned product is added to the other values.

[0028] DNN accelerators typically compute the dot product of FP16 activation values ​​and their corresponding FP16 weights. Considering only the normalized number of FP16 values, X = [x0, ..., x...] n-1 ] TIt can be defined as a vector of FP16 activation values, W = [w0, ..., w n-1 ] T A dot product can be defined as a vector of weighted FP16 values. The calculation of the dot product of FP16 values ​​involves element-wise multiplication of the signed mantissas of the two vectors and addition of the exponents used to calculate the product exponent. For example, the dot product p of X and W can be determined as:

[0029]

[0030] Where i is the index. It is the weight value w i symbols, It is the weight value w i The index, It is the weight value w i The last digit, It is the activation value x i symbols, It is the activation value x i The index, It is the activation value x i The last digit, s i It is the sign of the product value, e i It is the exponent of the product, m i It is the last digit of the product value.

[0031] If the exponents of the product are (usually) aligned with the exponent with the maximum value, the products (mantissas) in the FP16 dot product calculation can be added together. Figure 2 An example of a first embodiment of a dot product computing architecture 200 according to the subject matter disclosed herein is depicted, which aligns the exponents of mantissa multiplication so that the mantissas can be added, while also being optimized for area and power. In one embodiment, architecture 200 may include multiplier units 2010 to 201 n-1 An array of (where n is an integer), addition units (i.e., adder trees) 202, and accumulation logic units (or accumulation logic) 203. A maximum tree logic unit (or maximum tree) 207 may be incorporated into the exponent logic portion 205 of each multiplier unit 201. Multiplier units 201 (e.g., multiplier units 2010 to 201) n-1 Any one of them can be configured to include a module of hardware configured to provide the functionality described herein in conjunction with multiplier unit 201.

[0032] Each multiplier unit 201 includes an integer (mantissa) logic section (or integer logic) 204, an exponentiation logic section (or exponentiation logic) 205, and a shifter 206. The integer logic section 204, exponentiation logic section 205, and shifter 206 may be formed from discrete components (such as transistors, interconnect conductors, bias components, and / or discrete logic components). Each multiplier unit 201 receives the mantissa and exponent of the corresponding elements of an input vector X and an input vector W. In one embodiment, vector X may include elements as activation values, and vector W may include elements as weight values. The integer logic section 204 receives values ​​(e.g., activation value 1.m). x and the corresponding weight value 1.m w The two mantissas are multiplied to form the product mantissa (or mantissa product). The product mantissa is output to shifter 206. Although the example activation values ​​and corresponding weight values ​​are given as normalized numbers, the subnormal value (0.m) is... x and / or 0.m w It is also supported by multiplier unit 201.

[0033] The mantissa alignment can be achieved by shifting the product mantissa by shifting the product mantissa by the difference between the exponent of the product mantissa and the maximum exponent of the product mantissa in the array of multiplier units 201. For FP16, the exponent range of the product is [-28, 30], so in extreme cases, the alignment shift of FP16 can be up to 58 bits. In this case, the alignment overhead of the dot product computing architecture covering the entire range of FP16 exponents can be significant (up to 58 bits), and the addition logic can involve, for example, an 80-bit adder tree.

[0034] The distribution of the difference between the exponent values ​​and the maximum exponent value in FP16 data associated with deep learning typically does not cover the entire 58-bit exponent value range of FP16. Instead, the distribution of the difference between the exponent values ​​and the maximum exponent value usually tends to cover a smaller range of values. For example, Figure 3A The distribution histogram of example maximum exponent values ​​is shown for deep learning FP16 data. Figure 3B A histogram showing the distribution of example maximum negative exponent values ​​in deep learning FP16 data is provided. Finally, Figure 3C The histogram shows the distribution of example product exponent values ​​for deep learning FP16 data.

[0035] The topic disclosed here is available for use. Figures 3A to 3C The relatively limited range of the bell-shaped distribution of the deep learning FP16 data shown provides a dot-product architecture that is optimized in terms of area and power for the typical conditions encountered when computing FP16 operations on deep learning data. Specifically, Figures 3A to 3CThe properties of the deep learning FP16 data shown can be used to provide a dot product architecture with a right-shift capability that is much smaller than the 58-bit shift required to accommodate the entire range of exponential values ​​of normalized FP16 data.

[0036] Return to Figure 2 The exponent logic section 205 adds the exponents of corresponding elements of vectors X and W as part of the multiplication operation provided by multiplier unit 201. Exponent logic unit 205 may include a first adder 209 and a second adder 210. The first adder 209 activates the exponent value e based on FP16. x and the corresponding FP16 weight exponent value e w The sum of these values ​​determines the unbiased exponent value e0. The second adder 210 subtracts the maximum exponent value e from the unbiased exponent value e0. m This forms the relative exponential value e′0. The maximum exponential value e m It can be determined by the maximum tree 207. The relative exponent value e′0 can be used to control the amount of right shift applied to the mantissa product value by shifter 206.

[0037] The exponentiation logic unit 205 and the maximum tree logic unit 207 of each multiplier unit 201 together form the exponentiation processing unit 208 of architecture 200. The maximum tree logic unit 207 determines the maximum exponent value e. m The maximum tree logic unit 207 is combined with each multiplier logic unit 2010 to 201. n-1 and receives each unbiased exponent value e0 to e from the exponent logic unit 205 of the array. n-1 The maximum tree logic unit 207 determines the received unbiased exponent values ​​e0 to e... n-1 Maximum exponential value e m and the maximum exponent value e m The output is fed into the input of the second adder 210 in each exponential logic unit 205.

[0038] Shifter 206 can be configured to right-shift (e.g., right-shift) the mantissa product value up to the maximum right shift R-Shift. max Among them, R-Shift max The selection can be based on the distribution of the range of exponent values ​​expected to be processed by the dot product computation architecture 200 for deep learning FP16 data. (When encountering values ​​greater than R-Shift) max In the case of an exponent value, the multi-cycle technique described below can be used to align the product values. The relative exponent value e′0 output from the exponent logic unit 205 is used to control the number of right shift bits provided by the shifter 206 for a given dot product. In an example embodiment, R-Shift maxIt can be selected to be 8 bits to consider an example range of exponential values ​​for deep learning FP16 data processed by architecture 200. By limiting shifter 206 to, for example, 8 bits, the optimized dot product architecture 200 can provide dot product computation operations that are already optimized in terms of area and power. It should be understood that R-Shift max It can be selected as any integer value.

[0039] The aligned product values ​​output from each shifter 206 are input to the adder unit 202 and added together. The output of the adder unit 202 is accumulated in the accumulator logic unit 203. This is because the exponent of the sum of the adder tree 202 is the maximum exponent value e. m Therefore, the maximum exponent value e m The sum is also input to the accumulation logic unit 203. This sum is then added to the value stored in the accumulation logic unit 203 (i.e., accumulation is performed in the accumulation logic unit 203). For example, consider that the accumulation logic unit 203 is storing a previous sum. This sum is a floating-point number with an integer value (adder tree output) and an exponent value (maximum exponent). The next sum may have a different exponent (i.e., another maximum exponent value e). m Maximum exponential value e m The input is fed into the accumulator logic unit 203 to align / add the value stored in the accumulator with the new sum value from the adder tree.

[0040] Figure 4A and Figure 4B A second example embodiment of a dot product computing architecture 400 according to the subject matter disclosed herein is depicted. The dot product computing architecture 400 uses a multi-cycle technique to align the exponents of the mantissa multiplication, allowing the mantissas to be added, while also being optimized for area and power. Architecture 400 may include multiplier units 4010 to 401. n-1 The array (where n is an integer), adder unit (i.e., adder tree) 402, auxiliary shifter 411, and accumulator logic unit 403. Maximum tree logic unit 407 can be incorporated into each multiplier unit 401 (e.g., 4010 to 401). n-1 The exponential logic section 405 of the multiplier unit 401. The multiplier unit 401 may be configured to include modules of hardware configured to provide the functions described herein in conjunction with the multiplier unit 401.

[0041] Each multiplier unit 401 may include an integer logic unit 404, an exponent logic unit 405, and a local shifter 406. The integer logic unit 404, the exponent logic unit 405, and the shifter 406 may be formed from discrete components (such as transistors, interconnects, bias components, and / or discrete logic components). Each multiplier unit 401 receives the elements of the input vector X and the mantissa and exponent of the corresponding elements of the input vector W. The integer logic unit 404 receives a value (e.g., an activation value 1.m).x and the corresponding weight value 1.m w The two mantissas are multiplied together to form the product mantissa. The product mantissa is output to the local shifter 406. Sign multiplier 413 i (For example, 4130 to 413) n-1 The sign signal is also input to the local shifter 406. Figure 2 similar, Figure 4A and Figure 4B The example activations and corresponding weight values ​​in the table are given as normalization numbers, and the subnormalized values ​​(0.m) are given as normalization numbers. x and / or 0.m w This is also supported by multiplier unit 401. The product tail value is output to local shifter 409.

[0042] The exponent logic unit 405 may include a first adder 409 and a second adder 410. The first adder 409 activates the exponent value e based on FP16. x and the corresponding FP16 weight exponent value e w The sum of these values ​​determines the unbiased exponent value e0. The second adder 410 subtracts the maximum exponent value e from the unbiased exponent value e0. m This forms the relative exponential value e′0. The maximum exponential value e m It can be determined by the maximum tree 407. The relative exponent value e′0 is used to generate the local shift. Local shift The right shift amount applied to the mantissa product is controlled by the local shifter 406.

[0043] Local shifter 406 is configured to right-shift the mantissa product value up to the maximum right shift R-Shift. max Among them, R-Shift max The choice can be based on the distribution of the range of exponent values ​​in the deep learning FP16 data expected to be processed by the dot product computation architecture 400. Architecture 400 is able to handle encounters greater than R-Shift by using multi-period techniques to align product values. max The case of the exponential value. In one example embodiment, R-Shift max It can be selected as 8 bits to consider an example distribution of the exponential value range of deep learning FP16 data to be processed by the architecture 400. It should be understood that R-Shift... max It can be selected as any integer value.

[0044] The exponentiation logic unit 405 and the maximum tree logic unit 407 of each multiplier unit 401 together form the exponentiation processing unit 408 of architecture 400. The maximum tree logic unit 407 determines the maximum exponent value e. mThe maximum tree logic unit 407 is combined with each multiplier logic unit 4010 to 401. n-1 and receives each unbiased exponent value e0 to e from the exponent logic unit 405 of the array. n-1 The maximum tree logic unit 407 determines the received unbiased exponent values ​​e0 to e... n-1 Maximum exponential value e m and the maximum exponent value e m The output is fed into the input of the second adder 410 in each exponential logic unit 405.

[0045] The aligned mantissa products output from each local shifter 406 are input to the adder unit 402 and added. The output of the adder unit 402, as described below, is accumulated in the accumulation logic unit 404 after any additional shifts that can be provided by the auxiliary shifter 411 using a multi-cycle technique. The auxiliary shifter 411 provides an increased range of exponent differences that architecture 400 may encounter, while maintaining a relatively small physical area dedicated to shifting exponent values ​​compared to, for example, a 58-bit shifter that would cover the entire range of exponents for FP16 values. For example, if the local shifters 406 and the auxiliary shifter 411 were 8-bit shifters, the total physical area of ​​the shifters dedicated to architecture 400 would be n times the area of ​​the 8-bit local shifter 411 plus the area of ​​an 8-bit auxiliary shifter 411, which is equal to (n+1) × (the area of ​​an 8-bit shifter). In contrast, the area of ​​shifters dedicated to conventional architectures is n × (the area of ​​a 58-bit shifter). It should be understood that the auxiliary shifter 411 is not limited to an 8-bit shifter, but can be a shifter of any shift bit size. For example, in one embodiment, the auxiliary shifter 411 can be a 32-bit shifter.

[0046] The output of auxiliary shifter 411 is accumulated in accumulator logic unit 403. This is because the exponent of the sum of adder tree 402 and the output from auxiliary shifter 411 is the maximum exponent value e. m Therefore, the maximum exponent value e m The sum is also input to the accumulation logic unit 403. Then, this sum is added to the value stored in the accumulation logic unit 403 (i.e., accumulation is performed in the accumulation logic unit 403). For example, consider that the accumulation logic unit 403 is storing a previous sum. The sum is a floating-point number with an integer value (adder tree output) and an exponent value (maximum exponent). The next sum may have a different exponent (i.e., another maximum exponent value e). m Maximum exponential value e m The input is fed into the accumulator logic unit 403 to align / add the value stored in the accumulator with the new sum value from the adder tree.

[0047] Figure 4B Depicting Figure 4AAdditional details of a second embodiment of the dot product computation architecture 400, not depicted in the text. More specifically, Figure 4B Example embodiments of the exponent processing unit (EHU) 408 and masking unit 414 based on the subject matter disclosed herein are described in detail. Figure 4B As shown, the EHU 408 includes a mask generator / cycle counter 412, which is integrated with R-Shift. max and output each relative exponent value e′ (e.g., e′0 to e′). n-1 The mask generator / cycle counter 412 uses R-Shift. max The output, along with each relative exponent value e′, determines the mask and local shift for each product. (For example, to The mask generator / cycle counter 412 also generates a periodic # signal. The mask unit 414 may include n AND gates 4150 to 4150. n Mask unit 414 receives the mask output from mask generator / cycle counter 412. i (For example, mask0 to mask) n-1 )Signal.

[0048] During operation, during period #k, at k×R-Shift max With (k+1)×R-Shift max Products with relative exponent values ​​e′ within the range between these values ​​will be aligned, and the mask generator / cycle counter 412 will output a mask with a value of 1 for these products. i Signal. mask i The signal is input to AND gate 415 i (For example, 4150 to 415) n-1 One of the input terminals. During period #k, not in k×R-Shift. max With (k+1)×R-Shift max Products with relative exponent values ​​e′ within the range are not masked in the dot product calculation of that period, and the mask generator / period counter 412 outputs a mask with a value of 0 for these products. i Signal. The mask generator / cycle counter 412 shifts the amount of the unmasked product. Determined as The value of period #k is used to shift the remaining k×R-Shift through auxiliary shifter 411. max Displacement.

[0049] In one embodiment, since the addition of relatively small floating-point values ​​to large floating-point values ​​will not significantly adversely affect the dot product calculation, a mask signal can be generated to mask very small floating-point values.

[0050] Figure 5A and Figure 5B An example of double-period alignment processing for an example dot product computation architecture 500 for processing a pair of four-element vectors, based on the subject matter disclosed herein, is depicted. Although architecture 500 is depicted as processing a pair of four-element vectors, it should be understood that, in combination with... Figure 5A and Figure 5B The operational details described are generally the same, regardless of the number of vector elements processed by the Architecture 500.

[0051] Architecture 500 includes four multiplier units (not shown), an adder tree 502, an accumulator logic unit 503, an exponentiation processing unit (EHU) 508, and an auxiliary shifter 511. Figure 5A and Figure 5B The masking unit is not explicitly depicted. Each multiplier unit may include a local shifter (r-shift) 506 (e.g., 5060 to 5063) and an AND gate 515 (e.g., 5150 to 5153). For this example, the local shifter 506 and the auxiliary shifter 511 may be configured to have an R-shift of equal to 5 bits. max EHU 508 may include... Figure 4B The EHU 408 shown in the diagram is similar to an adder, maximum tree, and mask generator / cycle counter.

[0052] Reference Figure 5A and Figure 5B Consider the case where four multiplier units generate example products A, B, C, and D with exponents of 10, 2, 3, and 8, respectively. Products A through D are then input to local shifters 5060 through 5063, respectively. Figure 5A During the first period (i.e., period #0) depicted in EHU 508, the maximum tree determines that the exponent value of product A, 10, is the maximum exponent e. m The relative exponents e′ of the four products (e.g., e′) A to e′ D The absolute value (i.e., the local shift) to The values ​​are 0, 8, 7, and 2 respectively. That is, for the product A, the relative exponent e′ is 0, 8, 7, and 2. A Yes: e A –e m =10 – 10 = 0. For the product B, the relative exponent e′ B Yes: e B –em =2 – 10 = -8. For the product C, the relative exponent e′ C Yes: e C –e m =3–10 = -7, for the product D, the relative exponent e′ D Yes: e D –e m =8 – 10 = -2. Because the exponent and e m Alignment is ensured, so the relative exponent is calculated with a negative sign, and the mantissa is always shifted to the right. The absolute value of the relative exponent e′ is input to the individual local shifters 506.

[0053] Still during the first cycle, mask generator 508 generates masks mask0 to mask3 based on the relative exponent value e′ of the given product. Mask signals mask0 to mask3 are applied to the inputs of the corresponding AND gates 5150 to 5153. If the relative exponent e′ has a greater than the R-Shift of the local shifter 506... max If the absolute value of the relative exponent e' is less than or equal to the R-Shift of the local shifter 506, then the mask signal value 0 is generated. max If the absolute value of the product A and product D is less than 5, then a mask signal value of 1 is generated. In this example, the local shifter 506 has a maximum shift capability of 5, and the exponents of products A and D are within 5 shift bits (0 shift bits and 2 shift bits, respectively). The mask generator 508 will generate a mask signal value of 1 for these two products. The exponents of products B and C both exceed 5 shift bits (8 shift bits and 7 shift bits, respectively), therefore, the mask generator / cycle counter 508 will generate a mask signal value of 0 for products B and C.

[0054] The mask signal values ​​for the A and D products allow the outputs of local shifters 5060 (output A >> 0) and 5063 (output D >> 2) to be output to adder tree 502. Auxiliary shifter 511 does not require additional shifting for alignment.

[0055] exist Figure 5B During the second cycle (i.e., cycle #0) depicted, EHU 508 determines that the remaining two products B and C can be processed using local shifter 506 in conjunction with auxiliary shifter 511. Products B and C are shifted right by 3 bits and 2 bits, respectively, by shifters 5061 and 5062. Auxiliary shifter 511 shifts the outputs from both shifters 5061 and 5062 an additional 5 bits to the right, aligning the mantissas. The mask generator / cycle counter 508 outputs a cycle # signal equal to 1, which can be used to control auxiliary shifter 511.

[0056] Therefore, during the second cycle, because for these two products, the relative exponent e′ of the local shifter 506 is reduced by R-Shift. max The absolute value is equal to or less than R-Shift. max The mask generator / cycle counter 508 generates mask signals mask1 and mask2 with a value of 1 for products B and C. That is, for product B, the absolute value is equal to 8-5=3, which is less than 5 bits. For product C, the absolute value is equal to 7-5=2, which is less than 5 bits. Mask signals mask1 and mask2 are applied to the inputs of AND gates 5151 and 5152, so that the B product and the C product are output to adder tree 502 during the second cycle. Since products A and D have already been output to adder tree 502, for these products, mask generator 508 generates mask signals mask0 and mask3 as 0. Mask signals mask0 and mask3 are applied to the inputs of AND gates 5150 and 5153.

[0057] Figure 5A and Figure 5B The operational example described herein completes the dot product calculation in two cycles. The sequence disclosed herein is based on the range of exponent values ​​of the product to be processed, and the R-Shift provided by the local shifter 506. max and R-Shift provided by auxiliary shifter 511 max At one end of the spectrum, the amount of shift provided by the local shifter 506 and the auxiliary shifter 511 can be selected to take into account the entire range of exponential values ​​expected to be processed for a given deep learning dataset, allowing the dot product calculation to occur within one cycle. At the other end of the spectrum, the R-Shift provided by the local shifter 506 and the auxiliary shifter 511... max It can be selected to consider the distribution of the range of exponential values ​​expected to be processed for a given deep learning dataset, such that the dot product calculation occurs within a given number of cycles or fewer. Other cycle configurations are also available. It should also be understood that the R-Shift of the local shifter 506... max Unlike the R-Shift of the auxiliary shifter 511 max .

[0058] Furthermore, different groups or clusters of the dot product computation architecture disclosed herein can be formed based on different ranges of exponent values ​​to be processed, so that any pauses between groups or clusters that may occur during multi-cycle processing can be minimized or optimally utilized in the overall design of the CNN accelerator.

[0059] Figure 6 This is a flowchart of an example method 600 for computing the dot product of a deep learning network, based on the subject matter disclosed herein. (See also...) Figures 4A to 6 This method is inFigure 6 Starting at 601, at 602, the period # value is set to 0. At 603, the integer logic unit 404 of the array of multiplier units 401 multiplies the mantissas of the elements of the first vector X element by the mantissas of the corresponding elements of the second vector W to form a mantissa product. In one embodiment, the first vector X may include n elements of 16-bit floating-point values, and the second vector W may include n elements of 16-bit floating-point values, where n is an integer greater than 1. The first and second vectors may be vectors from a deep learning dataset.

[0060] In step 604, the exponent logic unit 405 of the array of multiplier unit 401 adds the exponent values ​​of the elements of the first vector X to the corresponding exponent values ​​of the elements of the second vector W element by element to form an exponent sum value e corresponding to the mantissa product value. i .

[0061] At 605, the maximum tree 407 determines the maximum exponent sum value e. m .

[0062] In 606, the exponential logic unit 405 subtracts the maximum exponential sum e from each exponential sum. m , so as to form relative exponential values ​​e′0 corresponding to the mantissa product values ​​respectively.

[0063] At 607, the local shifters 406 of the array of multiplier units 401 right-shift the first mantissa product value by a corresponding shift amount m, to form a mantissa product value aligned with the mantissa product corresponding to the maximum exponent sum value. Each local shifter unit 406 can be configured to have a maximum number of shift bits R-Shift less than the entire bit range of the exponent sum values ​​of the first and second vectors. max Having less than or equal to R-Shift max The corresponding shift amount m i The mantissa product remains unmasked and has a value greater than R-Shift. max The corresponding shift amount m i The mantissa product value is masked. Furthermore, the first mantissa product value corresponding to the relative exponent value, which is less than or equal to the maximum number of shift bits in the local shifter unit 406, is not masked.

[0064] At 608, the adder unit 402 adds the mantissa product value aligned with the mantissa product value corresponding to the maximum exponent sum value to form a partial dot product of the first vector X and the second vector W. At this time, the mantissa product value of the mask is not added to the partial dot product. It has a displacement bit R-Shift of less than or equal to the maximum number of bits in the local shifter unit 406. maxThe mantissa products of the corresponding relative exponent value e′ are aligned and summed within the first period. As follows, there is a maximum number of displacement bits R-Shift greater than the local shifter unit 406. max The mantissa product of the corresponding relative exponent value e′ is further shifted so that it becomes aligned and is added in subsequent periods.

[0065] At 609, determine whether all mantissa product values ​​have been added and accumulated by the accumulation logic unit 404. If yes, the method ends at 610. If no, the process continues to 611, where the cycle # increments by 1. At 612, each mantissa product value masked in the previous cycle is right-shifted by an amount equal to the relative shift e′ - (Cycle# × R - Shift). max The shift amount m i Previously added mantissa products are masked. Additionally, mantissa products with a relative shift e′ greater than ((current period #+1)×R-Shift) are also masked.

[0066] At 613, addition unit 402 adds the current mantissa product values ​​aligned with the mantissa product values ​​corresponding to the maximum exponent sum to form a partial dot product of the first vector X and the second vector W. Mantissa product values ​​of any mask are not added to the partial dot product. The process continues to 609, where it is determined whether all mantissa product values ​​have been added and accumulated by accumulation logic unit 404. If yes, the method ends at 610. If no, the process continues to 611.

[0067] Figure 7 An electronic device (or electronic system) 700 including a CNN accelerator is described, the CNN accelerator including one or more dot product computing architectures disclosed herein. The electronic device 700 can also be used in, but is not limited to, computing devices, personal digital assistants (PDAs), laptop computers, mobile computers, netbooks, wireless phones, cellular phones, smartphones, digital music players, or wired or wireless electronic devices. The electronic device 700 may include a controller 710, input / output (I / O) devices 720 (such as, but not limited to, keypads, keyboards, displays, touchscreen displays, cameras, and / or image sensors), memory (or memory devices) 730, an interface 740, a graphics processing unit (GPU) 750, and an image processing unit 760, all coupled to each other via a bus 770. The controller 710 may include, for example, at least one microprocessor, at least one digital signal processor, at least one microcontroller, etc. The memory 730 may be configured to store command codes or user data to be used by the controller 710. One or both of the GPU 750 and the image processing unit 760 may include one or more dot product computing architectures disclosed herein.

[0068] Electronic device 700 and its various system components may include image processing unit 760. Interface 740 may be configured to include a wireless interface configured to transmit data to or receive data from a wireless communication network using radio frequency (RF) signals. Wireless interface 740 may include, for example, an antenna. Electronic Systems 700 can also be used for communication interface protocols of communication systems, such as, but not limited to: Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), North American Digital Communications (NADC), Extended Time Division Multiple Access (E-TDMA), Wideband CDMA (WCDMA), CDMA2000, Wi-Fi, Municipal Wi-Fi (MuniWi-Fi), Bluetooth, Digital Enhanced Wireless Communication (DECT), Wireless Universal Serial Bus (Wireless USB), Fast Low Latency Access with Seamless Switching Orthogonal Frequency Division Multiplexing (Flash-OFDM), IEEE 802.20, General Packet Radio Service (GPRS), iBurst, Wireless Broadband (WiBro), WiMAX, WiMAX Advanced, Universal Mobile Telecommunications Service - Time Division Duplex (UMTS-TDD), High-Speed ​​Packet Access (HSPA), Evolved Data Optimized (EVDO), Long Term Evolution Advanced (LTE-Advanced), Multichannel Multipoint Allocation Service (MMDS), and 5G.

[0069] Embodiments of the subject matter and operation described in this specification may be implemented as digital electronic circuits, or as computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer storage medium to execute or control the operation of a data processing device. Optionally or additionally, the program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access storage array or device, or a combination thereof. Furthermore, while the computer storage medium is not a propagation signal, it may be a source or destination of computer program instructions encoded in artificially generated propagation signals. The computer storage medium may also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). Furthermore, the operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0070] While this specification may contain numerous specific details of implementation, these details should not be construed as limiting the scope of any claimed subject matter, but rather as descriptions of specific features of particular embodiments. Specific features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination, or even initially claimed in this way, in some cases, one or more features from a claimed combination may be removed from that combination, and the claimed combination may involve sub-combinations or variations thereof.

[0071] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring the operations to be performed in the specific order shown or in sequential order, or to perform all the shown operations to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments; it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

[0072] Therefore, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the appended claims. In some cases, the actions set forth in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes described in the drawings do not necessarily require the specific order or sequence shown to obtain the desired result. In some embodiments, multitasking and parallel processing may be advantageous.

[0073] As those skilled in the art will recognize, the innovative concepts described herein can be modified and altered in a wide range of applications. Therefore, the scope of the claimed subject matter should not be limited to any specific exemplary teachings discussed above, but rather is defined by the appended claims.

Claims

1. An apparatus for calculating the dot product of a first vector and a second vector, the apparatus comprising: The multiplier unit array includes integer logic, exponential logic, and a local shifter. The integer logic multiplies the integer values ​​of the elements of a first vector and the corresponding integer values ​​of the elements of a second vector to form an integer product value. The first and second vectors include floating-point values. The exponential logic adds the exponential values ​​corresponding to the integer values ​​of the corresponding elements of the two vectors to form an unbiased exponential value corresponding to the integer product value. The local shifter forms a first shift value by shifting the integer product value by a certain number of bits in a predetermined direction, based on the fact that the absolute value of the difference between the unbiased exponential value corresponding to the integer product value and the maximum unbiased exponential value of the multiplier unit array is less than or equal to the predetermined maximum shift capability of the local shifter. The maximum tree cell determines the maximum unbiased exponent value of the multiplier cell array. The adder tree adds the first shift value output from the local shifter of the multiplier unit array to form the first output; as well as The accumulator accumulates the first output of the adder tree. The apparatus further includes a mask generator, which generates a second mask that incorporates the first shift value into the adder tree, based on the fact that the absolute value of the difference between the unbiased exponent value corresponding to the integer product value and the maximum unbiased exponent value is greater than a predetermined maximum shift capability of the local shifter corresponding to the first shift value. Specifically, the adder tree will take the local shifter output from the multiplier unit array and combine it with the first shift value of the adder tree and the second mask to form the second output. The device further includes: an auxiliary shifter, incorporated into the adder tree and forming a second shift value by shifting a second output from the adder tree by a predetermined maximum shift capability of the local shifter; and The accumulator also accumulates the second output of the adder tree.

2. The apparatus according to claim 1, wherein, The mask generator generates a first mask that combines the first shift value into the adder tree, based on the fact that the absolute value of the difference between the unbiased exponent value corresponding to the integer product value and the maximum unbiased exponent value is less than or equal to the predetermined maximum shift capability of the local shifter corresponding to the first shift value. The adder tree will take the local shifter output from the multiplier cell array and combine it with the first shift value of the adder tree and add it with the first mask to form the first output.

3. The apparatus according to claim 2, wherein, The mask generator generates the first mask during the first cycle.

4. The apparatus according to claim 1, wherein, The mask generator generates a second mask during the second cycle.

5. The apparatus according to claim 1, wherein, The first vector includes activation values, and the second vector includes weight values.

6. The apparatus according to claim 5, wherein, The activation and weight values ​​include 16-bit floating-point values.

7. The apparatus according to claim 5, wherein, Activation and weight values ​​include 32-bit floating-point values.

8. A multiplier unit, comprising: Integer logic multiplies the integer values ​​of the elements of the first vector and the corresponding integer values ​​of the elements of the second vector to form an integer product. Exponential logic adds the exponents corresponding to the integer values ​​of the corresponding elements of two vectors to form the unbiased exponent value corresponding to the integer product. as well as A local shifter, based on the premise that the absolute value of the difference between the unbiased exponent value corresponding to the integer product value and a predetermined value is less than or equal to a predetermined maximum shift capability of the local shifter, forms a first shift value by shifting the integer product value by a certain number of bits in a predetermined direction. The multiplier unit is part of the multiplier unit array, which includes: The maximum tree cell is determined by a predetermined value, which includes the maximum unbiased exponent value of the multiplier cell array. The adder tree adds the first shifted value from the local shifter output of the multiplier unit array to form the first output; and The accumulator accumulates the first output of the adder tree. The multiplier unit further includes a mask generator, which generates a second mask that combines the first shift value into the adder tree, based on the fact that the absolute value of the difference between the unbiased exponent value corresponding to the integer product value and the maximum unbiased exponent value is greater than a predetermined maximum shift capability of the local shifter corresponding to the first shift value. Specifically, the adder tree will take the local shifter output from the multiplier unit array and combine it with the first shift value of the adder tree and the second mask to form the second output. The multiplier unit array also includes: an auxiliary shifter, incorporated into the adder tree, and forming a second shift value by shifting a second output from the adder tree by a predetermined maximum shift capability of the local shifter; and The accumulator also accumulates the second output of the adder tree.

9. The multiplier unit according to claim 8, wherein, The mask generator generates a first mask that combines the first shift value into the adder tree, based on the fact that the absolute value of the difference between the unbiased exponent value corresponding to the integer product value and the maximum unbiased exponent value is less than or equal to the predetermined maximum shift capability of the local shifter corresponding to the first shift value. Specifically, the adder tree will take the local shifter output from the multiplier unit array and combine it with the first shift value of the adder tree and the first mask to form the first output, and The mask generator generates the first mask during the first cycle.

10. The multiplier unit according to claim 8, wherein, The mask generator generates a second mask during the second cycle.

11. The multiplier unit according to claim 8, wherein, The first vector includes activation values, and the second vector includes weight values.

12. The multiplier unit according to claim 11, wherein, The activation and weight values ​​include 16-bit floating-point values.

13. The multiplier unit according to claim 11, wherein, Activation and weight values ​​include 32-bit floating-point values.

14. A method for calculating the dot product of floating-point values, the method comprising: The integer logic of the multiplier unit array multiplies the integer values ​​of the elements of the first vector element by the corresponding integer values ​​of the elements of the second vector element to form an integer product value. The first vector includes n 16-bit floating-point elements, and the second vector includes n 16-bit floating-point elements, where n is an integer greater than 1. By using the exponential logic of the multiplier unit array, the exponential values ​​of the elements of the first vector and the corresponding elements of the second vector are added element by element to form the exponential and sum values ​​corresponding to the integer product values, respectively. Determine the largest exponential sum among the exponential sums; By using exponential logic, the largest exponential sum is subtracted from each exponential sum in the exponential sum to form relative exponential values ​​that correspond to integer product values ​​respectively; The first local shifter of the multiplier unit array right-shifts the absolute value of the relative exponent value corresponding to the first integer product value to form an integer product value aligned with the integer product value corresponding to the maximum exponent sum value. Each first local shifter includes a first predetermined maximum number of shift bits, which is less than the entire bit range of the exponent sum values ​​of the first and second vectors, and the first integer product value corresponding to the relative exponent value is less than or equal to the first predetermined maximum number of shift bits. Add the first integer product value aligned with the integer product value corresponding to the maximum exponent sum value to form the dot product of the first and second vectors. The method further includes: shifting the second integer product value to the right by a second predetermined number of shift bits using an auxiliary shifter to form a second integer product value aligned with the integer product value corresponding to the maximum exponent sum value, wherein the second predetermined number of shift bits includes a first predetermined maximum number of shift bits.

15. The method according to claim 14, wherein, The entire bit range of the exponent values ​​of the first and second vectors comprises 58 bits, and The first predetermined maximum number of displacement bits plus the second predetermined number of displacement bits is less than or equal to 58 bits.

16. The method of claim 14, wherein, The elements of the first vector and the elements of the second vector include 16-bit floating-point values.

17. The method of claim 14, wherein, The elements of the first vector and the elements of the second vector include 32-bit floating-point values.

Images