Computing dot products at hardware accelerator

Abstract

A computing device including a hardware accelerator configured to train a machine learning model by computing a first product matrix comprising a plurality of first inner products. Computing the first product matrix may include the steps of: receiving a first matrix comprising a plurality of first vectors and a second matrix comprising a plurality of second vectors. Each first vector may include a first shared exponent and a plurality of first vector elements. Each second vector may include a second shared exponent and a plurality of second vector elements. For each first vector, computing the first product matrix may further include the step of: computing a first inner product of the first and second vectors. The first inner product may include a first inner product exponent, a first inner product sign, and a first inner product mantissa. Training the first machine learning model may further include the step of: storing the first product matrix in memory.

Description

[Technical Field]

[0001] This application relates to calculating the inner product at a hardware accelerator. [Previous Technology]

[0002] Matrix multiplication operations are frequently performed in machine learning applications when training and inferring machine learning models. These matrix multiplication operations are often performed on large matrices (e.g., with tens or hundreds of thousands of columns and rows) and can be very computationally intensive in terms of memory and processor utilization. [Summary of the Invention]

[0003] According to one aspect of this application, a computing device including a hardware accelerator is provided, the hardware accelerator being configured to train a machine learning model at least partially by computing a first product matrix including a plurality of first inner products. The step of computing the first product matrix may include the following steps: receiving a first matrix including a plurality of first vectors and a second matrix including a plurality of second vectors. Each of the plurality of first vectors may include a first shared exponent and a plurality of first vector elements. Each of the plurality of second vectors may include a second shared exponent and a plurality of second vector elements. For each of the plurality of first vectors, the step of computing the first product matrix may further include the following step: computing a first inner product between the first vector and a second vector among the plurality of second vectors. The first inner product may include a first inner product exponent, a first inner product sign, and a first inner product mantissa. Training the first machine learning model may further include the following step: storing the first product matrix in memory.

[0004] This [Summary] provides a simplified description of the selection of concepts further described in the [Description] section below. This [Summary] is not intended to determine the key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to embodiments that address any or all the drawbacks pointed out in any part of this application.

Implementation Method

[0020] To perform matrix multiplication more efficiently when training machine learning models, the following systems and methods are provided. FIG1A schematically depicts a computing device 10 according to an exemplary embodiment. The computing device 10 may include a processor 12, memory 14, and a hardware accelerator 16. The processor 12 may be a general-purpose processor, while the hardware accelerator 16 may be dedicated to performing a subset of computational tasks. The hardware accelerator 16 may be configured to perform a subset of computational tasks more efficiently than the processor 12, and the processor 12 may be configured to offload such computational tasks to the hardware accelerator 16. The hardware accelerator 16 may be dedicated to performing matrix multiplication. The memory 14 included in the computing device 10 may include volatile memory and / or non-volatile memory. The memory 14 may be communicatively coupled to the processor 12 and the hardware accelerator 16, such that the processor 12 and the hardware accelerator 16 may store data in the memory 14 and retrieve data from the memory 14.

[0021] In some examples, the functionality of computing device 10 may be distributed among a plurality of networked physical computing devices, rather than being provided in a single physical computing device. For example, computing device 10 may be instantiated in a data center, and one or more components of computing device 10 may be provided in a plurality of physical computing devices located in the data center and connected via a network. The physical computing devices located in the data center may be configured to communicate with one or more client computing devices, which may be located outside the data center and may at least partially instantiate one or more components of computing device 10.

[0022] As shown in the example of FIG1A, the memory 14 of the computing device 10 may store a machine learning model 62. The machine learning model 62 may include one or more matrices encoding attributes of the machine learning model 62. For example, the one or more matrices may be matrices of neuron weights or biases. At the processor 12, the computing device 10 may be configured to receive instructions to train the machine learning model 62 at least in part by performing matrix multiplication operations on the one or more matrices included in the machine learning model 62. For example, the instructions may be instructions to perform gradient descent iterations on the neuron weights of a deep neural network, instructions to generate samples at the generator of a generative adversarial network, or instructions to perform some other operations by which the machine learning model 62 can be trained. The processor 12 may be further configured to offload the matrix multiplication operations to the hardware accelerator 16, as described above. Thus, the computing device 10 may be configured to train the machine learning model 62 at least in part at the hardware accelerator 16.

[0023] The hardware accelerator 16 may be configured to train the machine learning model 62 at least in part by computing a first product matrix 60. The first product matrix 60 may include a plurality of first inner products 40, which may be inner products of a plurality of first vectors 22 included in the first matrix 20 and a plurality of second vectors 32 included in the second matrix 30. In some examples, as discussed in further detail below, the plurality of first inner products may be included in the first product matrix 60 in the form of a plurality of normalized first inner products 50 (on which an exponential normalization operation has been performed). After the first product matrix 60 has been generated, the hardware accelerator 16 may be further configured to store the first product matrix 60 in memory 14. Thus, the machine learning model 62 stored in memory 14 may be updated by computing the first product matrix 60. It should be understood that other tasks utilizing matrix multiplication may also be performed outside the field of machine learning.

[0024] FIG1B schematically illustrates components of a hardware accelerator 16 included in a computing device 10 according to one example. The hardware accelerator 16 may include a controller 70, which may be configured to receive instructions from a processor 12. Furthermore, the controller 70 may be further configured to transmit control instructions to other components of the hardware accelerator 16 and memory 14. For example, the controller 70 may be configured to send a direct memory access (DMA) request to memory 14, the DMA request instructing a DMA controller included in memory 14 to read data into the hardware accelerator 16. As another example, the controller 70 may be configured to transmit instructions to an output buffer 78 of the hardware accelerator 16 in which a first product matrix 60 is stored. The instructions transmitted to the output buffer 78 may be instructions to transfer the first product matrix 60 to memory 14.

[0025] As shown in the example of FIG1B, the hardware accelerator 16 may further include a first input buffer 72A and a second input buffer 72B. As described above, the step of calculating the first product matrix 60 at the hardware accelerator 16 may include the following steps: receiving a first matrix 20 comprising a plurality of first vectors 22 and a second matrix 30 comprising a plurality of second vectors 32. The first matrix 20 may be received at the first input buffer 72A, and the second matrix 30 may be received at the second input buffer 72B.

[0026] Returning to Figure 1A, each of the plurality of first vectors 22 may include a first shared index 24 and a plurality of first vector elements 26. The first shared index 24 may be associated with all the first vector elements 26 included in the first vector 22. Alternatively, as discussed below, the first shared index 24 may be associated with a subset of the plurality of first vector elements 26. Each of the plurality of first vector elements 26 may include a respective first element sign 27 (which may be positive or negative) and a respective first element mantissa 28. The i-th value ui included in the first vector 22 may be given by the following formula: where x1 is the first shared index 24, si is the first element sign 27 of the i-th first vector element 26, and mi is the first element mantissa 28 of the i-th first vector element 26.

[0027] Similarly, each of the plurality of second vectors 32 may include a second sharing index 34 and a plurality of second vector elements 36. The second sharing index 34 may be associated with all the second vector elements 36 included in the second vector 32. Alternatively, the second sharing index 34 may be associated with a subset of the plurality of second vector elements 36. Each of the plurality of second vector elements 36 may include a respective second element symbol 37 and a respective second element mantissa 38. The j-th value vj contained in the second vector 32 may be given by the following formula: where x2 is the second sharing index 34, tj is the second element symbol 37 of the j-th second vector element 36, and nj is the second element mantissa 38 of the j-th second vector element 36.

[0028] In some examples, the first vector 22 may include a plurality of first sharing indices 24, each first sharing index 24 being associated with a plurality of first vector elements 26, and the second vector 32 may include a plurality of second sharing indices 34, each second sharing index 34 being associated with a plurality of second vector elements 36. When the data type used to express the sharing indices and its associated vector elements are shorter than the bit length of the first vector 22 and the second vector 32, the first vector 22 may include a plurality of first sharing indices 24, and the second vector 32 may include a plurality of second sharing indices 34. For example, the bit length of each of the first vector 22 and the second vector 32 may be an integer multiple of the bit length of the data type. In such an example, each of the first vector 22 and the second vector 32 may include a number of sharing indices equal to that integer.

[0029] For each of the plurality of first vectors 22, the step of calculating the first product matrix 60 at the hardware accelerator 16 may further include the following steps: calculating a first inner product 40 of the first vectors 22 and a plurality of second vectors 32. The first inner product 40 may be calculated as: where p is the first inner product 40, and ui and vi are defined as shown above. The first inner product 40 may include a first inner product exponent 42, a first inner product sign 44, and a first inner product mantissa 46.

[0030] In some examples, the step of calculating the first product matrix 60 at the hardware accelerator 16 may further include the following step: performing an exponential normalization operation on the first inner product 40 to obtain a normalized first inner product 50. The exponential normalization operation may be an operation in which one or more leading zeros are removed from the first inner product exponent 42 to obtain a normalized first inner product exponent 52. Thus, the normalized first inner product may include a normalized first inner product exponent 52, a first inner product sign 44, and a normalized first inner product mantissa 56. As shown in FIG1A, the first product matrix 60 may include a plurality of normalized first inner products 50 calculated for a plurality of first vectors 22 and a plurality of second vectors 32.

[0031] Figure 2 illustrates an example calculation of the normalized first inner product pnorm at the hardware accelerator 16. In the example of Figure 2, the first vector 20 includes four values ​​ui and the second vector includes four values ​​vj. The hardware accelerator 16 can be configured to calculate a plurality of intermediate products wi by multiplying the values ​​ui by the corresponding values ​​vj for i = j. The hardware accelerator 16 can be further configured to add the plurality of intermediate products wi to obtain the first inner product p, and for this purpose, the hardware accelerator 16 can be further configured to apply an exponential normalization operation to calculate the normalized first inner product pnorm.

[0032] Turning now to FIG. 3, the hardware accelerator 16 can be reconfigured to compute a second product matrix 160 comprising a plurality of second inner products 140. FIG. 3 illustrates an exemplary computing device 10 of FIG. 1A when the hardware accelerator 16 is reconfigured to compute the second product matrix 160. The steps of computing the second product matrix 160 may include receiving a third matrix 120 comprising a plurality of third vectors 122 and a fourth matrix 130 comprising a plurality of fourth vectors 132 at the hardware accelerator 16. The third matrix 120 and the fourth matrix 130 may be received at a first input buffer 72A and a second input buffer 72B, respectively.

[0033] Each of the plurality of third vectors 122 may include a plurality of third vector elements 126. Each of the plurality of third vector elements 126 may include its own third element exponent 124, its own third element sign 127, and its own third element mantissa 128. Similarly, each of the plurality of fourth vectors 132 may include a plurality of fourth vector elements 136. Each of the plurality of fourth vector elements 136 may include its own fourth element exponent 134, its own fourth element sign 137, and its own fourth element mantissa 138. Therefore, without including a shared exponent, each third vector 122 and each fourth vector 132 may include its own exponent in each element.

[0034] The step of calculating the second product matrix 160 may further include the following steps: for each of the plurality of third vectors 122, calculate the second inner product 140 of the third vector 122 and the fourth vector 132 of the plurality of fourth vectors 132. The second inner product 140 may include a second inner product exponent 142, a second inner product sign 144, and a second inner product mantissa 146. In some examples, the hardware accelerator 16 may be further configured to perform an exponential normalization operation on the second inner product 140 to remove one or more leading zeros. Thus, the hardware accelerator 16 may be configured to calculate a normalized second inner product 150, which includes a normalized second inner product exponent 152, a second inner product sign 144, and a normalized second inner product mantissa 156. In such an example, the normalized second inner product 150 may be included in the second product matrix 160, as shown in the example of FIG3.

[0035] Figure 4 illustrates an example computation of the normalized second inner product pnorm' at the hardware accelerator 16. In the example of Figure 4, the third vector 122 includes a complex number of third vector elements ui', and the fourth vector 132 includes a complex number of fourth vector elements vj'. For i = j, the third vector element ui' can be multiplied by the corresponding fourth vector element vj' to compute a complex number of intermediate products wi'. The hardware accelerator 16 can be further configured to perform exponential normalization on each intermediate product wi' to compute a complex number of normalized intermediate products winorm'. The hardware accelerator 16 can be further configured to add the complex number of normalized intermediate products winorm' to obtain the second inner product p', and perform exponential normalization on the second inner product p' to compute the normalized second inner product pnorm'.

[0036] In contrast to the calculation of the normalized first inner product 50 shown in Figure 2, the calculation of the normalized second inner product 150 shown in Figure 4 involves an additional exponential normalization operation on each intermediate product wi'. In the example of Figure 2, this additional exponential normalization operation can be avoided as a result of assigning a shared exponent to the first vector 22 and the second vector 32 (because each of the intermediate products wi in the example of Figure 2 has the same exponent). However, assigning separate exponents to the third vector element 126 and the fourth vector element 136, as shown in Figure 3, allows for a higher precision expression of some vector elements in the third vector element 126 and the fourth vector element 136 when the third vector 122 or the fourth vector 132 includes two or more elements with different exponents. Therefore, it may be necessary to switch between shared exponential data types and non-shared exponential data types based on the range of values ​​included in the input matrix.

[0037] Referring back to Figure 1B, the hardware accelerator 16 may include a plurality of pipeline stages 74. Each of the plurality of pipeline stages 74 may include one or more corresponding matrix multiplier modules 76. In some examples, data may pass serially through the plurality of pipeline stages 74; each pipeline stage 74 may include a plurality of multiplier modules 76 arranged in parallel. The hardware accelerator 16 may be configured to compute a plurality of corresponding product matrices at the matrix multiplier modules 76 of the plurality of pipeline stages 74. The plurality of product matrices may include a first product matrix 60 and a second product matrix 160.

[0038] In an example where the hardware accelerator 16 includes a plurality of pipeline stages 74, two or more of the pipeline stages 74 may be configured to receive respective inputs having different input types. In the example of FIG. 1B, the hardware accelerator 16 includes a first pipeline stage 74A configured to receive an input having a first input type 80A. The hardware accelerator 16 of FIG. 1B also includes a second pipeline stage 74B configured to receive an input having a second input type 80B. In some examples, one of the first input type 80A and the second input type 80B may be a shared exponential data type, while the other may be a non-shared exponential data type.

[0039] In an example where two or more pipeline stages 74 are configured to receive inputs with their respective different input types, the inputs received at the two or more pipeline stages 74 may include respective input type metadata, each indicating the respective input type of the input. In the example of FIG1B, a first pipeline stage 74A is configured to receive first input type metadata 82A, and a second pipeline stage 74B is configured to receive second input type metadata 82B. Each of the plurality of pipeline stages 74 may be configured to receive its own input type metadata. For example, the first input type metadata 82A and the second input type metadata 82B may be set as headers for a first matrix 20 and a second matrix 30. Alternatively, the first input type metadata 82A and the second input type metadata 82B may be set as headers for a plurality of first vectors 22 and a plurality of second vectors 32.

[0040] When pipeline stage 74 receives input, hardware accelerator 16 can be configured to reconfigure pipeline stage 74 based on input type metadata included in this input. For example, when input type metadata indicates that the input has a non-shared exponential data type but pipeline stage 74 is currently configured to process a vector with a shared exponential data type, hardware accelerator 16 can be further configured to reconfigure pipeline stage 74 to process data with a non-shared exponential data type. Similarly, when input type metadata indicates that the input has a shared exponential data type but pipeline stage 74 is currently configured to process a vector with a non-shared exponential data type, hardware accelerator 16 can be further configured to reconfigure pipeline stage 74 to process data with a shared exponential data type.

[0041] Figures 5A and 5B respectively illustrate two exemplary shared index data types, MSFP13 and MSFP17, which can represent a plurality of first vectors 22 and a plurality of second vectors 32. In the MSFP13 format shown in Figure 5A, the first shared index 24 may have a length of eight bits. The first vector 22 may include sixteen first vector elements 26, each element may include a first element sign 27 with a length of one bit and a first element mantissa 28 with a length of four bits. In the MSFP17 format shown in Figure 5B, the first shared index 24 may have a length of eight bits. The first vector 22 may include sixteen first vector elements 26, each element may include a first element sign 27 with a length of one bit and a first element mantissa 28 with a length of four bits. Although Figures 5A and 5B respectively show the first vectors 22 in MSFP13 and MSFP17 formats, the MSFP13 and MSFP17 formats can also be used for the second vectors 32.

[0042] Figures 5C to 5E show three types of non-shared index data: fp32, bfloat16, and fp16. In the fp32 format shown in Figure 5C, the third element symbol 127 has a length of one bit, the third element exponent 124 has a length of eight bits, and the third element mantissa 128 has a length of 23 bits (24 bits when hidden bits are included). In the bfloat16 format shown in Figure 5D, the third element symbol 127 has a length of one bit, the third element exponent 124 has a length of eight bits, and the third element mantissa 128 has a length of seven bits. In the fp16 format shown in Figure 5E, the third element symbol 127 has a length of one bit, the third element exponent 124 has a length of five bits, and the third element mantissa 128 has a length of ten bits. Although Figures 5C to 5E show the third vector 122 in fp32, bfloat16, and fp16 formats respectively, the fp32, bfloat16, and fp16 formats can also be used for the fourth vector 132.

[0043] In an example where the hardware accelerator 16 is reconfigured to process vectors with a non-shared exponential data type, each of the plurality of first vector elements 26 and each of the plurality of second vector elements 36 may include a mantissa with a first mantissa length. Furthermore, each of the plurality of third vector elements 126 and each of the plurality of fourth vector elements 136 may include a mantissa with a second mantissa length different from the first mantissa length. For example, a pipeline stage 74 of the hardware accelerator 16 may be reconfigured from a configuration receiving first vectors 22 and 32 in MSFP17 format to a configuration receiving third vectors 122 and fourth vectors 132 in fp32 format. Therefore, the first mantissa length is 8 bits, and the second mantissa length is 23 bits (24 bits including hidden bits).

[0044] When the length of the first mantissa is different from the length of the second mantissa, the length of the second mantissa may be an integer multiple of the length of the first mantissa. In some examples, the hardware accelerator 16 may be further configured to add one or more leading zeros to each of the first element mantissa 28 and each of the second element mantissa 38 or to each of the third element mantissa 128 and each of the fourth element mantissa 138, such that the length of the second mantissa is equal to an integer multiple of the length of the first mantissa. In the above example, leading zeros may be added to each of the third element mantissa 128 and each of the fourth element mantissa 138, such that the length of the second mantissa is equal to three times the length of the first mantissa.

[0045] As discussed above with reference to FIG. 1B, a plurality of first inner products 40 and a plurality of second inner products 140 can be computed at the plurality of multiplier modules 76 included in the hardware accelerator 16. In an example where the second mantissa length is an integer multiple of the first mantissa length, the hardware accelerator 16 can be further configured to reconfigure the plurality of multiplier modules 76 to receive a plurality of third vectors 122 and a plurality of fourth vectors 132, at least in part by combining the plurality of multiplier modules 76 into a multiplier supermodule 90, at which the plurality of second inner products 140 can be computed. In the example of FIG. 6A, nine 8×8 multiplier modules 76 are combined into a 24×24 multiplier supermodule 90. The nine multiplier modules 76 shown in FIG. 6A can be included in the same pipeline stage 74. Combining multiple multiplier modules 76 into a multiplier supermodule 90 may include multiplexing the outputs of the multiplier modules 76. Figure 6B illustrates the data stream through the multiplier modules 76 when they are combined into the multiplier supermodule 90 of Figure 6A. The outputs of the multiplier modules 76 can each be transmitted to an adder 94, which is configured to calculate a second inner product 140, which is the sum of the outputs of the multiplier modules 76.

[0046] In other examples, the first mantissa length may be an integer multiple of the second mantissa length. In such an example, as shown in FIG6C, the hardware accelerator 16 may be further configured to reconfigure the multiplier module 76 at least partially by dividing the multiplier module 76 into a plurality of multiplier submodules 92 to receive a plurality of third vectors 122 and a plurality of fourth vectors 132, at which a plurality of second inner products 140 may be computed. In the example of FIG6C, the multiplier module 76 that computes the plurality of first inner products 40 and the plurality of second inner products 140 is a 24×24 multiplier module 76, while each of the multiplier submodules 92 is an 8×8 multiplier submodule 92. The three multiplier submodules 92 are formed by the multiplier module 76. FIG6D shows the data stream through the multiplier module 76 when the multiplier module 76 of FIG6C is divided into a plurality of multiplier submodules 92.

[0047] In some examples, computing the first product matrix 60 at the hardware accelerator 16 may further include the step of adding the first inner product 40 to an additional inner product to obtain an inner product sum. Figure 7 shows an example computation of the inner product sum q and the normalized inner product sum qnorm. The inner product sum computed in Figure 7 may be the inner product of two vectors, where each vector includes two shared exponents. In the example computation of Figure 7, the hardware accelerator 16 is configured to compute the inner product sum q as the sum of the normalized inner products p0norm and p1norm of two pairs of four-element vectors, respectively. The hardware accelerator 16 is further configured to perform exponent normalization on the inner product sum q to obtain the normalized inner product sum qnorm. In the example of Figure 7, the paired vectors as input may be subvectors of a pair of longer vectors, which are partitioned such that the inner products of the subvectors can be computed in parallel and added together to obtain the inner product of the paired vectors. Therefore, the hardware accelerator 16 can compute the inner product of paired vectors at the cost of performing additional exponential normalization operations on the inner products p0 and p1 of the subvectors, using a larger number of parallel operations.

[0048] Turning now to FIG. 8A, a flowchart of an exemplary method 200 for use with a computing device is provided. The exemplary method 200 shown in FIG. 8A is a method for training a machine learning model at a hardware accelerator included in a computing device. The computing device may be the computing device 10 of FIG. 1A, or alternatively some other computing device.

[0049] At step 202, method 200 may include the following steps: calculating a first product matrix comprising a plurality of first inner products. At step 202, the step of calculating the first product matrix may include the following steps: at step 204, receiving a first matrix comprising a plurality of first vectors and a second matrix comprising a plurality of second vectors. The first matrix and the second matrix may be received at a first input buffer and a second input buffer included in the hardware accelerator, respectively. Each of the plurality of first vectors may include a first shared exponent and a plurality of first vector elements, and each of the plurality of second vectors may include a second shared exponent and a plurality of second vector elements. Each of the plurality of first vector elements may include a first element sign and a first element mantissa, and each of the plurality of second vector elements may include a second element sign and a second element mantissa.

[0050] At step 206, step 202 may further include the following step: for each of the plurality of first vectors, calculate a first inner product between the first vector and a second vector of the plurality of second vectors. The plurality of first inner products may be calculated at a plurality of multiplier modules included in one or more pipeline stages of the hardware accelerator. The first inner product may include a first inner product exponent, a first inner product sign, and a first inner product mantissa. In some examples, the plurality of first inner products may be elements of a first product matrix. Alternatively, when calculating the first product matrix, one or more additional operations may be performed on the plurality of first inner products.

[0051] At step 208, step 202 may further include the following step: storing the first product matrix in memory. The first product matrix may be transferred from the output buffer of the hardware accelerator to memory in a computing device included outside the hardware accelerator. This memory may be volatile memory or non-volatile memory.

[0052] Figure 8B illustrates additional steps of method 200 performed in an example in which one or more additional operations are performed when calculating the elements of the first product matrix for each of the plurality of first inner products. The steps of calculating the first product matrix at step 210 and step 202 may further include the step of performing an exponential normalization operation on the first inner products for each of the plurality of first inner products. Thus, the hardware accelerator can calculate a plurality of normalized first inner products, which in some examples may be elements of the first product matrix. In some examples, at step 212, method 200 may further include the step of adding the first inner product to the additional inner products to obtain an inner product sum. In such examples, the first inner product and the additional inner product may be subvectors of a larger vector. The calculation of the inner product of the larger vector and the additional vector can be parallelized by calculating and summing the inner products of the individual subvectors of those vectors. In the example of performing step 210, method 200 may further include the following steps: at step 214, performing an exponential normalization operation on the inner product and the product.

[0053] Figure 8C illustrates additional steps of method 200 that may be performed in some examples. At step 216, method 200 may further include the step of reconfiguring the hardware accelerator to compute a second product matrix comprising a complex number of second inner products. At step 218, method 200 may further include the step of computed the second product matrix at the reconfigured hardware accelerator.

[0054] The step of calculating the second product matrix at step 218 may include the following steps: at step 220, receiving a third matrix comprising a plurality of third vectors and a fourth matrix comprising a plurality of fourth vectors. Each of the plurality of third vectors may include a plurality of third vector elements, each third vector element including its own third element exponent, its own third element sign, and its own third element mantissa. Furthermore, each of the plurality of fourth vectors may include a plurality of fourth vector elements, each fourth vector element including its own fourth element exponent, its own fourth element sign, and its own fourth element mantissa. Therefore, the plurality of third vectors and the plurality of fourth vectors may each have a non-shared exponent data type, while the plurality of first vectors and the plurality of second vectors may each have a shared exponent data type. The plurality of third vectors and the plurality of fourth vectors may be received at the first input buffer and the second input buffer of the hardware accelerator, respectively.

[0055] At step 222, step 218 may further include the following step: for each of the complex number of third vectors, calculate a second inner product between the third vector and the fourth vector among the complex number of fourth vectors. The second inner product may include a second inner product exponent, a second inner product sign, and a second inner product mantissa. In some examples, step 218 may further include the following step: at step 224, perform an exponential normalization operation on the second inner product. The normalized second inner product may be included in the second product matrix. Alternatively, the normalized second inner product may be added to an additional inner product to obtain an inner product sum, and the inner product sum may be subjected to exponential normalization again. The normalized inner product sum may then be included in the second product matrix. At step 226, method 200 may further include the following step: storing the second product matrix in memory.

[0056] Each of the plurality of first vector elements and each of the plurality of second vector elements may include a mantissa having a first mantissa length. Furthermore, each of the plurality of third vector elements and each of the plurality of fourth vector elements may include a mantissa having a second mantissa length. The first mantissa length is different from the second mantissa length. Therefore, when the hardware accelerator is reconfigured to receive inputs with a non-shared exponential data type, the hardware accelerator may also be reconfigured to receive inputs with different mantissa lengths. In some examples, at step 216A, step 216 may include the step of reconfiguring the plurality of multiplier modules at least partially by combining the plurality of multiplier modules into a multiplier supermodule. For example, step 216A may be performed when the second mantissa length is an integer multiple of the first mantissa length. In other examples, at step 216B, step 216 may alternatively include the step of reconfiguring the multiplier modules at least partially by dividing the multiplier modules into a plurality of multiplier submodules. When the length of the first mantissa is an integer multiple of the length of the second mantissa, step 216B can be performed. In some examples, one or more bits may be added to each of the first mantissa and each of the second mantissas, or to each of the third mantissa and each of the fourth mantissas, such that the length of the second mantissa is an integer multiple of the length of the first mantissa, or such that the length of the first mantissa is an integer multiple of the length of the second mantissa.

[0057] Using the apparatus and methods discussed above, matrix multiplication can be performed at a hardware accelerator during the training of a machine learning model. By using a shared exponential data type to represent the vectors included in the matrix to which matrix multiplication is performed, the hardware accelerator can perform matrix multiplication faster due to performing fewer exponential normalization operations. Furthermore, the multiplier module included in the hardware accelerator can be dynamically reconfigured to switch between receiving shared exponential data and non-shared exponential data. By switching between shared and non-shared exponential data types, the multiplier module can efficiently compute inner products while still calculating inner products with high accuracy when the elements of the input vectors have a wide range. Therefore, when training a machine learning model using a hardware accelerator, values ​​such as neuron weights, represented in matrix form, can be updated more efficiently.

[0058] In some embodiments, the methods and processes described herein may be attached to a computing system of one or more computing devices. Specifically, such methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries and / or other computer program products.

[0059] FIG9 schematically illustrates a non-limiting embodiment of a computing system 300 capable of performing one or more of the methods and processes described above. The computing system 300 is shown in a simplified form. The computing system 300 may embody the computing device 10 described above and shown in FIG1A. Components of the computing system 300 may be materialized in one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones) and / or other computing devices, as well as wearable computing devices such as smartwatches and head-mounted augmented reality devices.

[0060] The computing system 300 includes a logic processor 302, volatile memory 304, and non-volatile storage device 306. The computing system 300 may optionally include a display subsystem 308, an input subsystem 310, a communication subsystem 312, and / or other components not shown in FIG9.

[0061] The logic processor 302 includes one or more physical means configured to execute instructions. For example, the logic processor may be configured to execute instructions that are part of one or more applications, programs, norms, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transition the state of one or more components, achieve technical effects, or otherwise achieve desired results.

[0062] The logic processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logic processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of logic processor 302 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logic processor may optionally be distributed across two or more independent devices, which may be remotely located and / or configured for collaborative processing. The various forms of the logic processor may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud computing configuration. In this case, it will be understood that these virtualized forms run on different physical logic processors on various different machines.

[0063] The non-volatile storage device 306 includes one or more physical means configured to store instructions executable by a logic processor to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile storage device 306 may be changed—for example, to retain different data.

[0064] The non-volatile storage device 306 may include removable and / or built-in physical devices. The non-volatile storage device 306 may include optical memory (e.g., CD, DVD, HD-DVD, and Blu-ray discs), semiconductor memory (e.g., ROM, EPROM, EEPROM, and flash memory), and / or magnetic memory (e.g., hard disk drives, floppy disk drives, magnetic tape drives, and MRAM), or other high-capacity storage technologies. The non-volatile storage device 306 may include non-volatile, dynamic, static, read / write, read-only, sequential access, location-addressable, file-addressable, and / or content-addressable devices. It will be understood that the non-volatile storage device 306 is configured to retain instructions even when the power supply to the non-volatile storage device 306 is cut off.

[0065] Volatile memory 304 may include a physical device containing random access memory. Volatile memory 304 is typically used by logic processor 302 to temporarily store information during the processing of software instructions. It should be understood that when volatile memory 304 is powered off, volatile memory 304 typically does not continue to store instructions.

[0066] The states of the logic processor 302, volatile memory 304, and non-volatile storage device 306 can be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), system-on-a-chip (SOCs), and complex programmable logic units (CPLDs).

[0067] The terms "module," "program," and "engine" can be used to describe a state of computing system 300, which is typically implemented in software by a processor to perform a specific function using a portion of volatile memory. This function involves specific configuration of the processor to perform variations. Therefore, a module, program, or engine can be materialized by a logic processor 302, which uses a portion of volatile memory 304 to execute instructions held by non-volatile storage device 306. It should be understood that different modules, programs, and / or engines can be materialized from the same application, service, code block, object, library, routine, API, and function. Similarly, different modules, programs, and / or engines can be materialized from different applications, services, code blocks, objects, routines, APIs, and functions. The terms "module," "program," and "engine" can include a single or a group of executable files, data files, libraries, drivers, scripts, and database records.

[0068] When a display subsystem 308 is included, the display subsystem 308 can be used to present a visual representation of the data held by the non-volatile storage device 306. The visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data held by the non-volatile storage device, thereby changing the state of the non-volatile storage device, the state of the display subsystem 308 can also be translated into a visual representation of the changes in the underlying data. The display subsystem 308 may include one or more display devices using virtually any type of technology. Such display devices may be integrated with the logic processor 302, the volatile memory 304, and / or the non-volatile storage device 306 in a shared housing, or such display devices may be peripheral display devices.

[0069] When the input subsystem 310 is included, the input subsystem 310 may include or interact with one or more user input devices, such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may include or interact with a selected Natural User Input (NUI) component. Such a component may be integrated or peripheral and may process the conversion and / or processing of input actions on-device or off-device. Exemplary NUI components may include a microphone for voice and / or sound recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; an electric field sensing component for assessing brain activity; and / or any other suitable sensor.

[0070] When a communication subsystem 312 is included, the communication subsystem 312 may be configured to communicatively couple the various computing devices described herein to each other and to other devices. The communication subsystem 312 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local area network or wide area network (such as an HDMI connection via Wi-Fi). In some embodiments, the communication subsystem may allow the computing system 300 to send and / or receive messages to and / or from other devices via a network such as the Internet.

[0071] The following paragraphs describe several embodiments of this application. According to one embodiment of this application, a computing device including a hardware accelerator is provided, which is configured to train a machine learning model at least partially by computing a first product matrix including a plurality of first inner products. The step of computing the first product matrix may include the following steps: receiving a first matrix including a plurality of first vectors and a second matrix including a plurality of second vectors. Each of the plurality of first vectors may include a first shared exponent and a plurality of first vector elements, and each of the plurality of second vectors may include a second shared exponent and a plurality of second vector elements. For each of the plurality of first vectors, the step of computing the first product matrix may further include the following step: computing a first inner product of the first vector and a second vector among the plurality of second vectors. The first inner product may include a first inner product exponent, a first inner product sign, and a first inner product mantissa. The hardware accelerator may be further configured to train a machine learning model at least partially by storing the first product matrix in memory.

[0072] According to this pattern, each of the plurality of first vector elements may include its own first element sign and its own first element mantissa. Each of the plurality of second vector elements may include its own second element sign and its own second element mantissa.

[0073] According to this configuration, the hardware accelerator can be reconfigured to compute a second product matrix comprising a plurality of second inner products, at least in part, by receiving a third matrix comprising a plurality of third vectors and a fourth matrix comprising a plurality of fourth vectors. Each of the plurality of third vectors may include a plurality of third vector elements, each third vector element including a respective third element exponent, a respective third element sign, and a respective third element mantissa. Each of the plurality of fourth vectors may include a plurality of fourth vector elements, each fourth vector element including a respective fourth element exponent, a respective fourth element sign, and a respective fourth element mantissa. The step of computing the second product matrix may further include the following step: for each of the plurality of third vectors, computing the second inner product of the third vector and the fourth vector among the plurality of fourth vectors. The second inner product may include a second inner product exponent, a second inner product sign, and a second inner product mantissa. The step of computing the second product matrix may further include the following step: storing the second product matrix in memory.

[0074] According to this configuration, each of the plurality of first vector elements and each of the plurality of second vector elements may include a mantissa having a first mantissa length. Each of the plurality of third vector elements and each of the plurality of fourth vector elements may include a mantissa having a second mantissa length. The first mantissa length may be different from the second mantissa length.

[0075] According to this configuration, a plurality of first inner products and a plurality of second inner products can be computed at a plurality of multiplier modules included in the hardware accelerator. The second mantissa length can be an integer multiple of the first mantissa length. The hardware accelerator can be further configured to reconfigure the plurality of multiplier modules to receive a plurality of third vectors and a plurality of fourth vectors at least in part by combining the plurality of multiplier modules into a multiplier supermodule, and compute the plurality of second inner products at the multiplier supermodule.

[0076] According to this configuration, a plurality of first inner products and a plurality of second inner products can be computed at a plurality of multiplier modules included in the hardware accelerator. The length of the first mantissa is an integer multiple of the length of the second mantissa. The hardware accelerator can be further configured to at least partially reconfigure the multiplier modules by dividing them into a plurality of multiplier submodules to receive a plurality of third vectors and a plurality of fourth vectors, and compute a plurality of second inner products at the plurality of multiplier submodules.

[0077] According to this configuration, the hardware accelerator may include a plurality of pipeline stages, each pipeline stage including a corresponding matrix multiplier module. The hardware accelerator may be configured to compute a plurality of corresponding product matrices at the matrix multiplier modules of the plurality of pipeline stages, including a first product matrix.

[0078] According to this configuration, two or more pipeline stages out of a plurality of pipeline stages are configured to receive their respective inputs having their own different input types.

[0079] According to this pattern, the inputs received at two or more pipeline stages may include respective input type metadata, which indicates the respective input type of the input.

[0080] Based on this state, the step of calculating the first product matrix may further include the following steps: performing exponential normalization on the first inner product.

[0081] According to this pattern, the step of calculating the first product matrix may further include the following steps: adding the first inner product to the additional inner product to obtain the inner product sum. The step of calculating the first product matrix may further include the following steps: performing an exponential normalization operation on the inner product sum.

[0082] According to another aspect of this application, a computing device including a hardware accelerator is provided, the hardware accelerator being configured to train a machine learning model at least partially by computing a first product matrix. The step of computing the first product matrix may include the following steps: configuring a multiplier module to receive input having a shared exponential data type. The step of computing the first product matrix may further include the following steps: receiving a first vector and a second vector, each having a shared exponential data type, at the multiplier module. The step of computing the first product matrix may further include the following steps: computing a first inner product of the first vector and the second vector. The hardware accelerator may be further configured to train a machine learning model at least partially by computing a second product matrix. The step of computing the second product matrix may further include the following steps: reconfiguring the multiplier module to receive input having a non-shared exponential data type. The step of computing the second product matrix may further include the following steps: receiving a third vector and a fourth vector, each having a non-shared exponential data type. The step of computing the second product matrix may further include the following steps: computing a second inner product of the third vector and the fourth vector. The hardware accelerator can be further configured to train machine learning models at least in part by storing the first and second product matrices in memory.

[0083] According to this configuration, each of the plurality of first vector elements included in the first vector and each of the plurality of second vector elements included in the second vector may include a mantissa with a first mantissa length. Each of the plurality of third vector elements included in the third vector and each of the plurality of fourth vector elements included in the fourth vector may include a mantissa with a second mantissa length. The first mantissa length may be different from the second mantissa length.

[0084] According to this configuration, a plurality of first inner products and a plurality of second inner products can be computed at a plurality of multiplier modules included in the hardware accelerator. The hardware accelerator can be configured to reconfigure the plurality of multiplier modules, at least in part, by combining the plurality of multiplier modules, including the multiplier modules, into a multiplier supermodule to receive inputs having a non-shared exponential data type and compute the second inner product at the multiplier supermodule.

[0085] According to this configuration, when the multiplier module is reconfigured to receive input with a non-shared exponential data type, the hardware accelerator can be further configured to at least partially reconfigure the multiplier module by dividing the multiplier module into a plurality of multiplier submodules, and to compute the second inner product at the plurality of multiplier submodules.

[0086] Based on this state, the steps of calculating the first product matrix and the second product matrix may further include the following steps: performing exponential normalization operation on the first inner product and the second inner product.

[0087] According to this configuration, the hardware accelerator may be further configured to perform exponential normalization on the complex intermediate product of the third vector element of the third vector and the fourth vector element of the fourth vector when calculating the second inner product.

[0088] According to another aspect of this application, a method for use with a computing device is provided. The method may include the following steps: at a hardware accelerator, training a machine learning model at least in part by calculating a first product matrix comprising a plurality of first inner products. The step of calculating the first product matrix may include the following steps: receiving a first matrix comprising a plurality of first vectors and a second matrix comprising a plurality of second vectors. Each of the plurality of first vectors may include a first shared exponent and a plurality of first vector elements, and each of the plurality of second vectors may include a second shared exponent and a plurality of second vector elements. For each of the plurality of first vectors, the step of calculating the first product matrix may further include the following step: calculating a first inner product between the first vector and a second vector among the plurality of second vectors. The first inner product may include a first inner product exponent, a first inner product sign, and a first inner product mantissa. The step of training the machine learning model may further include the following step: storing the first product matrix in memory.

[0089] According to this configuration, the method may further include the following steps: reconfiguring the hardware accelerator to at least partially compute a second product matrix comprising a plurality of second inner products by receiving a third matrix comprising a plurality of third vectors and a fourth matrix comprising a plurality of fourth vectors. Each of the plurality of third vectors may include a plurality of third vector elements, each third vector element including a respective third element exponent, a respective third element sign, and a respective third element mantissa. Each of the plurality of fourth vectors may include a plurality of fourth vector elements, each fourth vector element including a respective fourth element exponent, a respective fourth element sign, and a respective fourth element mantissa. The step of computed second product matrix may further include the following steps: for each of the plurality of third vectors, compute a second inner product of the third vector with the fourth vector among the plurality of fourth vectors. The second inner product may include a second inner product exponent, a second inner product sign, and a second inner product mantissa. The step of computed second product matrix may further include the following steps: storing the second product matrix in memory.

[0090] Based on this state, the step of calculating the first product matrix further includes the following steps: performing an exponential normalization operation on the first inner product.

[0091] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as many variations are possible. The particular conventions or methods described herein may represent one or more of any number of processing strategies. Therefore, the various actions shown and / or described may be performed in the order shown and / or described, in a different order, in parallel, or omitted. Similarly, the order of the above processes may be changed.

[0092] The subject matter of this application includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations and other features, functions, effects and / or characteristics disclosed herein, and any and all equivalents thereof. [Simplified Explanation of the Diagram]

[0005] FIG1A schematically depicts a computing device including a hardware accelerator according to an exemplary embodiment, the hardware accelerator being configured to train a machine learning model at least in part by computing a first product matrix.

[0006] Figure 1B schematically illustrates an exemplary computing device of request 1 in an example of a hardware accelerator including a plurality of pipeline stages.

[0007] Figure 2 schematically illustrates an exemplary calculation of the normalized first inner product at a hardware accelerator according to the example of Figure 1A.

[0008] Figure 3 schematically illustrates the exemplary computing device of Figure 1A when the hardware accelerator is reconfigured to compute the second product matrix.

[0009] Figure 4 schematically illustrates an exemplary calculation of the normalized second inner product at the hardware accelerator when the hardware accelerator has been reconfigured as shown in the example of Figure 3.

[0010] Figures 5A and 5B illustrate exemplary shared index data types (where vectors can be expressed) according to the example in Figure 1A.

[0011] Figures 5C to 5E illustrate exemplary non-shared index data types (where vector elements can be expressed) according to the example in Figure 3.

[0012] Figure 6A shows an exemplary plurality of multiplier modules combined into a multiplier supermodule according to the example of Figure 1B.

[0013] Figure 6B shows an exemplary data streaming path through the multiplier module of Figure 6A when the multiplier modules are combined into a multiplier supermodule.

[0014] Figure 6C shows an exemplary multiplier module divided into a plurality of multiplier submodules according to the example of Figure 1B.

[0015] Figure 6D shows an exemplary data streaming path through the multiplier module of Figure 6C when the multiplier module is divided into multiple multiplier sub-modules.

[0016] Figure 7 schematically illustrates an example calculation of the inner product of two vectors, each including two shared indices, according to the example of Figure 1A.

[0017] Figure 8A shows a flowchart of an exemplary method for training a machine learning model using a computing device, according to the example of Figure 1A.

[0018] Figures 8B to 8C show additional steps of the method of Figure 8A, which may be performed in some examples.

[0019] Figure 9 shows a schematic diagram of the computing device of Figure 1A in which an exemplary computing environment can be configured. [Biomaterial Storage]

[0094] Domestic storage information (please note in order of storage institution, date, and number): None. International storage information (please note in order of storage country, institution, date, and number): None.

Claims

1. A computing device, comprising: A hardware accelerator configured to train a machine learning model at least in part by the following steps: calculating a first product matrix comprising a plurality of first inner products, wherein the step of calculating the first product matrix comprises the following steps: receiving a first matrix comprising a plurality of first vectors and a second matrix comprising a plurality of second vectors, wherein: each of the plurality of first vectors comprises a first shared exponent and a plurality of first vector elements; and each of the plurality of second vectors comprises a second shared exponent and a plurality of second vector elements; and for each of the plurality of first vectors, calculating a first inner product of the first vector and the second vector of the plurality of second vectors, wherein the first inner product comprises a first inner product exponent, a first inner product sign, and a first inner product mantissa, wherein the first inner product exponent is equal to the sum of the first shared exponent of the first vector and the second shared exponent of the second vector; and storing the first product matrix in memory.

2. The computing device as claimed in claim 1, wherein: Each of the complex number of first vector elements includes a first element sign and a first element mantissa; and each of the complex number of second vector elements includes a second element sign and a second element mantissa.

3. The computing device as claimed in claim 1, wherein the hardware accelerator can be reconfigured to compute, at least in part, a second product matrix comprising a plurality of second inner products by means of the following steps: receiving a third matrix comprising a plurality of third vectors and a fourth matrix comprising a plurality of fourth vectors, wherein: Each of the complex number of third vectors comprises a complex number of third vector elements, each third vector element comprising a respective third element exponent, a respective third element sign, and a respective third element mantissa; and each of the complex number of fourth vectors comprises a complex number of fourth vector elements, each fourth vector element comprising a respective fourth element exponent, a respective fourth element sign, and a respective fourth element mantissa; for each of the complex number of third vectors, a second inner product of the third vector and a fourth vector of the complex number of fourth vectors is calculated, wherein the second inner product comprises a second inner product exponent, a second inner product sign, and a second inner product mantissa; and the second product matrix is ​​stored in the memory.

4. The computing device as claimed in claim 3, wherein: Each of the plurality of first vector elements and each of the plurality of second vector elements includes a mantissa having a first mantissa length; and each of the plurality of third vector elements and each of the plurality of fourth vector elements includes a mantissa having a second mantissa length, wherein the first mantissa length is different from the second mantissa length.

5. The computing device as claimed in claim 4, wherein: The plurality of first inner products and the plurality of second inner products are computed at the plurality of multiplier modules included in the hardware accelerator; the second mantissa length is an integer multiple of the first mantissa length; and the hardware accelerator is further configured to reconfigure the plurality of multiplier modules to receive the plurality of third vectors and the plurality of fourth vectors at least in part by combining the plurality of multiplier modules into a multiplier supermodule, at which the plurality of second inner products are computed.

6. The computing device as claimed in claim 4, wherein: The plurality of first inner products and the plurality of second inner products are calculated at a multiplier module included in the hardware accelerator; the first mantissa length is an integer multiple of the second mantissa length; and the hardware accelerator is further configured to reconfigure the multiplier module to receive the plurality of third vectors and the plurality of fourth vectors at least in part by dividing the multiplier module into a plurality of multiplier submodules, and to calculate the plurality of second inner products at the plurality of multiplier submodules.

7. The computing device as claimed in claim 1, wherein: The hardware accelerator includes a plurality of pipeline stages, each pipeline stage including a corresponding matrix multiplier module; and the hardware accelerator is configured to compute a corresponding plurality of product matrices at the matrix multiplier modules of the plurality of pipeline stages, the corresponding plurality of product matrices including the first product matrix.

8. The computing device as claimed in claim 7, wherein two or more pipeline stages of the plurality of pipeline stages are configured to receive respective inputs, each of which has a different input type.

9. The computing device as claimed in claim 8, wherein the inputs received at the two or more pipeline stages include respective input type metadata indicating the respective input type of the inputs.

10. The computing apparatus of claim 1, wherein the step of computing the first product matrix further comprises the step of performing an exponential normalization operation on the first inner product.

11. The computing apparatus of claim 10, wherein the step of computing the first product matrix further comprises the following steps: adding the first inner product to another inner product to obtain an inner product sum; and performing the exponential normalization operation on the inner product sum.

12. A computing device, comprising: A hardware accelerator configured to train a machine learning model at least in part by the following steps: computing a first product matrix, wherein computing the first product matrix includes the following steps: configuring a multiplier module to receive input having a shared exponent data type; receiving, at the multiplier module, a first vector and a second vector each having the shared exponent data type, wherein the first vector includes a first shared exponent and the second vector includes a second shared exponent; computing a first inner product of the first vector and the second vector, wherein a first inner product exponent of the first inner product is equal to the sum of the first shared exponent of the first vector and the second shared exponent of the second vector; computing a second product matrix, wherein computing the second product matrix includes the following steps: reconfiguring the multiplier module to receive input having a non-shared exponent data type; receiving a third vector and a fourth vector each having the non-shared exponent data type; and computing a second inner product of the third vector and the fourth vector; and storing the first product matrix and the second product matrix in memory.

13. The computing device as claimed in claim 12, wherein: Each of the plurality of first vector elements included in the first vector and each of the plurality of second vector elements included in the second vector includes a respective mantissa having a first mantissa length; and each of the plurality of third vector elements included in the third vector and each of the plurality of fourth vector elements included in the fourth vector includes a respective mantissa having a second mantissa length, wherein the first mantissa length is different from the second mantissa length.

14. The computing device as claimed in claim 13, wherein: The hardware accelerator is configured to reconfigure the plurality of multiplier modules to receive inputs having the non-shared exponential data type at least in part by combining the plurality of multiplier modules, including the multiplier modules, into a multiplier supermodule, and to compute the second inner product at the multiplier supermodule.

15. The computing device as claimed in claim 13, wherein, When the multiplier module is reconfigured to receive input with the non-shared exponent data type, the hardware accelerator is further configured to reconfigure the multiplier module, at least in part by dividing the multiplier module into a plurality of multiplier sub-modules, at which the second inner product is computed.

16. The computing apparatus of claim 12, wherein the step of computing the first product matrix and the second product matrix further comprises the step of performing an exponential normalization operation on the first inner product and the second inner product.

17. The computing device of claim 16, wherein the hardware accelerator is further configured to perform the exponential normalization operation on a complex number of intermediate products of the third vector elements of the third vector and the fourth vector elements of the fourth vector when computing the second inner product.

18. A method for use with a computing device, the method comprising the steps of: training a machine learning model at a hardware accelerator, at least in part by the steps of: calculating a first product matrix comprising a plurality of first inner products, wherein the step of calculating the first product matrix comprises the steps of: receiving a first matrix comprising a plurality of first vectors and a second matrix comprising a plurality of second vectors, wherein: Each of the plurality of first vectors includes a first shared exponent and a plurality of first vector elements; and each of the plurality of second vectors includes a second shared exponent and a plurality of second vector elements; and for each of the plurality of first vectors, a first inner product of the first vector and a second vector of the plurality of second vectors is calculated, wherein the first inner product includes a first inner product exponent, a first inner product sign, and a first inner product mantissa, wherein the first inner product exponent is equal to the sum of the first shared exponent of the first vector and the second shared exponent of the second vector; and the first product matrix is ​​stored in memory.

19. The method of claim 18, further comprising the step of: at least partially reconfiguring the hardware accelerator by the following steps to compute a second product matrix comprising a plurality of second inner products: receiving a third matrix comprising a plurality of third vectors and a fourth matrix comprising a plurality of fourth vectors, wherein: Each of the complex number of third vectors includes a complex number of third vector elements, each third vector element including a respective third element exponent, a respective third element sign, and a respective third element mantissa; and each of the complex number of fourth vectors includes a complex number of fourth vector elements, each fourth vector element including a respective fourth element exponent, a respective fourth element sign, and a respective fourth element mantissa; for each of the complex number of third vectors, a second inner product of the third vector and a fourth vector of the complex number of fourth vectors is calculated, wherein the second inner product includes a second inner product exponent, a second inner product sign, and a second inner product mantissa; and the second product matrix is ​​stored in the memory.

20. The method of claim 18, wherein the step of calculating the first product matrix further includes the step of performing an exponential normalization operation on the first inner product.

Images