Matrix approximation for matrix multiplication operations

Abstract

A processing device is provided, comprising: a memory configured to store data; and a processor configured to receive a portion of data of a first matrix including a first plurality of elements; and a portion of data of a second matrix including a second plurality of elements. The processor is also configured to determine values ​​of a third matrix by removing some products from the products of element pairs based on approximating the products of element pairs of the first and second matrices as a sum of exponents of the element pairs, and performing matrix multiplication on remaining products of element pairs of the first and second matrices.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims the interests of the pending U.S. Patent Application No. 17 / 214,784, filed on 26 March 2021, entitled "APPROXIMATION OF MATRICES FOR MATRIX MULTIPLY OPERATIONS," which is incorporated herein by reference in its entirety. This application also claims the interests of the pending Patent Application No. 17 / 214,779, filed on 26 March 2021, entitled "DATA COMPRESSOR FOR APPROXIMATION OF MATRICES FOR MATRIX MULTIPLY OPERATIONS," which is incorporated herein by reference in its entirety. [Background technology]

[0002] Matrix multiplication is a critical component across several application domains, including high-performance computing (HPC) and machine learning. It is also used in convolutional neural networks, recurrent neural networks, and other forms of artificial neural networks.

[0003] Matrix multiplication techniques employ parallelization to improve the efficiency of matrix multiplication. For example, two matrices are typically divided into smaller parts (e.g., columns, rows, and columns and rows), and the matrix multiplication operation of the two matrices is performed by performing multiple matrix multiplication calculations, each involving the multiplication of one matrix part with the other matrix part. The matrix multiplication calculations are mapped to different processor cores in a processor network and executed by different processor cores to perform the matrix multiplication operation.

[0004] A more detailed understanding can be gained from the following explanation, which is provided as an example along with the attached diagram. [Brief explanation of the drawing]

[0005] [Figure 1] This is a block diagram of an exemplary device in which one or more features of this disclosure may be implemented. [Figure 2] A block diagram illustrating an exemplary component of a processor in which one or more features of this disclosure may be implemented. [Figure 3] This flowchart illustrates an exemplary method for selecting data for matrix multiplication according to the features of this disclosure. [Figure 4A] This figure shows a first example of an array of approximate product exponential values ​​based on the features of this disclosure. [Figure 4B] This figure shows a second example of an array of approximate product exponential values ​​based on the features of this disclosure. [Figure 4C] This figure shows a third example of an array of approximate product exponential values ​​based on the features of this disclosure. [Figure 4D] This figure shows a fourth example of an array of approximate product exponential values ​​based on the features of this disclosure. [Figure 5] This figure shows an example of submatrix multiplication between two matrices A and B and an output matrix C, according to the features of this disclosure. [Modes for carrying out the invention]

[0006] As used herein, a program includes a sequence of instructions executed using one or more processors to perform a procedure or routine (e.g., an operation, calculation, function, process, or job). Processing of programmed instructions and data includes, but is not limited to, one or more processing stages, such as fetching, decoding, scheduling, executing, and decoding programmed instructions and data. Programmed instructions include, for example, application and control programs, such as operating systems. A processor includes, for example, multiple processing cores (e.g., compute units, CUs), each of which is configured to read and execute programmed instructions, such as instructions for matrix multiplication.

[0007] Matrix multiplication involves calculating the dot product of the lower parts of the data in a first matrix and a second matrix. A matrix multiplication operation involves the calculation C = A × B, where A, B, and C are matrices of sizes M × K, K × N, and M × N, respectively. Each element in matrix C is the dot product of a row in matrix A and a column in matrix B. For example, a multiplication accumulator calculates the product of pairs of values, where each value corresponds to an element in a portion of the matrix (e.g., a row, a column, a portion of a row or column, or multiple rows or columns), and adds the product to an accumulator using a hardware component known as a multiplier accumulator (MAC). For example, a 64 × 64 multiplication can be implemented as four 16 × 16 MACs or eight 8 × 8 MACs. Matrix multiplication typically involves many calculations, which are time-consuming and costly.

[0008] This application provides a device and method for efficiently performing matrix multiplication approximations. Features of this disclosure include a device and method for dynamically determining an output matrix by removing several products (i.e., products of pairs of elements of the two input matrices) along a common dimension K of the two matrices from the product used for dot product calculation of matrix multiplication of the input matrices. The products to be removed are those approximated as having the smallest exponential sum among the products used for dot product calculation. As will be described in more detail below, the target number of elements to be retained (i.e., not removed) and provided to the MAC is determined based on the sum of the bit values ​​of the approximated product value (referred to herein as the product exponential value), which begins with the sum of the most significant bits (MSB) of the product exponential value and continues through each set of corresponding next significant bits until it is determined that the target number of the product exponential value is to be retained, and then indicates which corresponding elements of matrices A and B are provided to the MAC for matrix multiplication.

[0009] A key feature of this disclosure is that it reduces the number of products without first sorting the elements of the input array according to their values, which would otherwise be costly to implement in hardware.

[0010] For example, a 64x64 product is reduced to a 48x48 product by retaining the largest approximate 48 product values ​​from the 64x64 product (i.e., removing the smallest 16 approximate values). For instance, if a 64x64 product is implemented as a 16x16 MAC, four 16x16 MACs are reduced to three, resulting in a 25% reduction in the time required to perform the task and a reduction in the energy cost of performing the task. Similarly, if a 64x64 product is implemented as an 8x8 MAC, eight 8x8 MACs are reduced to six 8x8 MACs, also resulting in a 25% time reduction. The target number of product values ​​to retain and provide to the MAC for matrix multiplication, or the target number of product values ​​to remove from the MAC (i.e., removed product values), can be any number and are determined based on various factors during runtime, such as the amount of result error acceptable by the approximation for a particular task or application. For example, when used for machine learning training, the target number of products to be removed is based on the impact of the approximation on the resulting network accuracy. Alternatively, the target number of products to be removed can be determined based on the size of the common dimension K. For instance, based on heuristics, additional products can be removed for larger values ​​of K, while additional products can be retained for smaller values ​​of K and provided to the MAC.

[0011] Features of this disclosure include performing matrix multiplication on various different data types, such as floating data types (e.g., FP32, FP16, and BF16 formats) and integer data types (e.g., int8 format).

[0012] A processing device is provided, comprising a memory configured to store data and a processor, wherein the processor is configured to receive a portion of the data of a first matrix containing a first plurality of elements and a portion of the data of a second matrix containing a second plurality of elements. The processor is also configured to determine the value of a third matrix by removing some products from the product of the pairs of elements and performing matrix multiplication on the remaining product of the pairs of elements of the first and second matrices, based on approximating the product of the pairs of elements of the first and second matrices as the sum of the exponents of the pairs of elements.

[0013] A matrix multiplication data selection method is provided, which includes receiving a portion of the data of a first matrix containing a first set of elements, and receiving a portion of the data of a second matrix containing a second set of elements. The method also includes determining the value of a third matrix by removing some products from the product of the pairs of elements, based on approximating the product of the pairs of elements of the first and second matrices as the sum of the exponents of the pairs of elements, and then performing matrix multiplication on the remaining product of the pairs of elements of the first and second matrices.

[0014] A non-temporary computer-readable storage medium is provided, which includes instructions for causing a computer to perform a matrix multiplication data selection method, the instructions comprising: receiving a portion of the data of a first matrix having a first plurality of elements; receiving a portion of the data of a second matrix having a second plurality of elements; determining the value of a third matrix by removing some products from the product of pairs of elements based on approximating the product of pairs of elements of the first and second matrices as the sum of the exponents of the pairs of elements; and performing matrix multiplication on the remaining product of pairs of elements of the first and second matrices.

[0015] Figure 1 is a block diagram of an exemplary device 100 in which one or more features of the present disclosure may be implemented. Device 100 includes, for example, a computer, a game device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100 includes a processor 102, memory 104, storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 also optionally includes an input driver 112 and an output driver 114. It should be understood that device 100 may include additional components not shown in Figure 1.

[0016] In various alternatives, the processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, each of which can be a CPU or a GPU. In various alternatives, the memory 104 is located on the same die as the processor 102 or separately from the processor 102. The memory 104 includes volatile or non-volatile memory such as random access memory (RAM), including, for example, dynamic RAM (DRAM) and static RAM (SRAM). The RAM includes, for example, cache memory, scratchpad memory, and registers.

[0017] The memory device 106 includes a fixed or removable memory device (e.g., hard disk drive, solid state drive, optical disk, flash drive). The input device 108 includes, but is not limited to, a keyboard, keypad, touch screen, touch pad, detector, microphone, accelerometer, gyroscope, biometric scanner, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE802 signals). The output device 110 includes, but is not limited to, a display, speaker, printer, tactile feedback device, one or more lights, antenna, or a network connection (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE802 signals).

[0018] The input driver 112 communicates with the processor 102 and the input device 108 to enable the processor 102 to receive inputs from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110 to enable the processor 102 to send outputs to the output device 110. Note that the input driver 112 and the output driver 114 are optional components, and the device 100 operates in the same manner in the absence of the input driver 112 and the output driver 114.

[0019] Features of the present disclosure are described herein using a CU as an example of a processor core. The CU includes one or more single instruction, multiple data (SIMD) units configured to perform operations in parallel upon request of the processor 102 according to the SIMD paradigm. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and a program counter and thus execute the same program, but can execute that program with different data. In one example, each SIMD unit includes 16 lanes, and each lane can execute the same instruction simultaneously with other lanes within the SIMD unit, but can execute that instruction with different data. The lanes can be switched off predictively if not all lanes need to execute a given instruction. The prediction can also be used to execute programs having branch control flow. More specifically, for a program having conditional branches or other instructions where the control flow is based on calculations performed by individual lanes, prediction of lanes corresponding to currently unexecuted control flow paths and serial execution of different control flow paths enables any control flow. The parallelism provided by the CU is suitable for matrix multiplication, such as matrix multiplication used in graphics-related operations such as pixel value calculation, vertex transformation, and other graphics operations.

[0020] FIG. 2 is a block diagram showing an example of the processor 102 shown in FIG. 1 in which one or more features of the present disclosure may be implemented. As shown in FIG. 2, the processor 102 includes a plurality of CUs 202. Each CU 202 includes a level N (e.g., level 1) cache controller 208 that communicates with a corresponding level N cache 204 and is configured to process data using the corresponding level N cache 204. Alternatively, a plurality of level N cache controllers 208 (e.g., the cache controller 208 of each CU 202, or a plurality of cache controllers 208 each communicating with a group of CUs 202) are used to process data.

[0021] As shown in Figure 2, the processor 102 includes a level N+1 (e.g., level 2) cache controller 210 that communicates with a level N+1 cache 206 and is configured to process data using the level N+1 cache 206. Alternatively, multiple level N+1 cache controllers, each communicating with one or more corresponding level N+1 caches, are used to process data. As shown in Figure 2, the cache controller 210 also communicates with the next cache level (e.g., level 3) and each level N cache controller 208. In addition, or alternatively, each CU 202 communicates with different types of memory 104, such as registers and scratchpad memory.

[0022] As shown in Figure 2, each CU202 includes a MAC212 and a data compressor 214 that communicates with the MAC212. The MAC212 is configured to calculate the product of element pairs of a part of a matrix (e.g., an approximate product exponential value) and to accumulate (e.g., store) the product in order to perform a matrix multiplication operation. The product of a target number that is approximated as a product with the smallest exponential sum is removed from the product that the MAC212 uses to perform the dot product calculation of the matrix multiplication.

[0023] The data compressor 214 includes hardware circuitry configured to determine which exponential values ​​(i.e., exponential sums) to retain (i.e., not delete) and provide to the MAC 212 based on their bit values, starting from their most significant bit and continuing through the corresponding next set of significant bits, until a target number of exponential values ​​are determined. The data compressor 214 is also configured to provide the retained exponential values ​​to the MAC 212 in order to perform matrix multiplication.

[0024] Figure 3 is a flowchart illustrating an exemplary method 300 for selecting data for matrix multiplication according to the features of the present disclosure.

[0025] As shown in block 302, method 300 includes receiving a portion of the data of a first matrix A and a portion of the data of a second matrix B. Figure 5 shows an example of submatrix multiplication of two matrices A and B and an output matrix C, according to the features of the present disclosure. As described above, the matrix multiplication operation includes the calculation C = A × B, where A, B, and C are matrices of size M × K, K × N, and M × N, respectively. Each element of matrix C is the dot product of a row of matrix A and a column of matrix B. For example, a portion of the matrix multiplication of matrices A and B is performed according to the features of the present disclosure by approximating the product of four elements 502(1) to 502(4) of the element array 502 of matrix B and four corresponding elements 504(1) to 504(4) of the array 504 of matrix B, thereby producing the result of sub-1(P1) of matrix C.

[0026] In Figure 5, matrices A and B are of different sizes, but matrix multiplication can be performed as long as the number of columns in one matrix is ​​equal to the number of rows in the other matrix. In the example shown in Figure 5, matrix A has 8 columns and matrix B has 8 rows. The number of columns and rows in matrices A and B are merely examples. Matrix multiplication is performed on matrices of any size that satisfy the general rules of matrix multiplication (for example, the number of columns in matrix A is equal to the number of rows in matrix B), in accordance with the features of this disclosure.

[0027] The data portions of the first and second matrices can contain any number of elements of different data types, such as integer data types (e.g., int8 format) and floating-point data types (e.g., BF16 format). The number of bits representing each element depends on the floating-point format (e.g., FP32, FP16, BF16) and integer format (e.g., int8, int16, int32) used. For example, in the floating-point format BF16, each element is represented by 16 bits, and each exponent is represented by 8 bits.

[0028] As shown in block 304, method 300 includes approximating the value of the product of element pairs of parts of the first and second matrices as the sum of the exponents of the element pairs.

[0029] When the elements of the data parts of the first and second matrices are of floating data type, the exponent value can simply be extracted from the exponent bits of each element. For example, when the element is in BF16 format, the first bit is the sign bit, bits 2 to 9 are the exponent bits, and bits 10 to 16 are the mantissa bits. Therefore, the exponent value can be extracted from the values of bits 2 to 9.

[0030] When the lower parts of the data of the first matrix A and the second matrix B contain eight elements of int data type format (int8), the exponent is determined by finding the absolute value of each element in the lower part of the data (A a [8], B a [8]), determining the number of leading zeros to be removed for each element, representing each element as 1.M * 2 e (where M is the mantissa of the element value and e is the exponent of the element value), and extracting the exponent value of each element as [number of bits - 1] - L A (where L A is the number of leading zeros of the element) by approximation.

[0031] As an example, when the absolute value of the first element of the first matrix (A a [1]) is 00001011, it is determined that four leading zeros are removed. The element is represented as 1.011 * 2 e , and the exponent value (e) for A[1] is calculated as [number of bits - 1] - L A = 7 - 4 = 3. When the absolute value of the first element of the second matrix (B a [1]) is 00101111, it is determined that two leading zeros are removed. The element is represented as 1.01111 * 2 e , and the exponent value (e) for A[1] is calculated as [number of bits - 1] - L B = 7 - 2 = 5. Therefore, the approximate product (P) of A[1] and B[1] is determined to be 14 - (L A + L B ) = 14 - 8 = 6.

[0032] The same process is carried out to determine the approximate product of the remaining seven elements of the lower part of the first matrix A (A[2-8]) and the corresponding remaining seven elements of the lower part of the second matrix B (B[2-8]). That is, the product of corresponding pairs of elements of the first matrix A and the second matrix B is approximated by 2 raised to the power of the sum of the exponents of the elements of the first matrix A and the second matrix B.

[0033] As shown in block 306, method 300 includes removing some products and determining which products should be retained for matrix multiplication of the first and second parts of matrices A and B. That is, products are essentially removed (i.e., not retained for matrix multiplication) by removing pairs of elements from the first and second matrices whose corresponding products are approximated on the sum of their exponents, resulting in the smallest product among the products of the pairs of elements. The remaining products that are not removed are retained for matrix multiplication.

[0034] Figures 4A to 4D show examples of different arrays of approximate product values ​​(referred to herein as product exponential values) approximated based on the sum of the exponents of corresponding element pairs, as described above in block 304. The process of determining which product exponential values ​​to retain and then indicating which corresponding elements of matrices A and B are provided to MAC212 in Figure 2 is described for each of the arrays of product exponential values ​​shown in Figures 4A to 4D. For simplicity, the arrays of product exponential values ​​shown in Figures 4A to 4D are 4-bit values. However, the features of this disclosure can be implemented for any number of bits used to represent the product value. In addition, for simplicity, we use eight arrays of product exponential values ​​(corresponding to eight element pairs), and use a target number of 6 from each of the eight elements as the number of product exponential values ​​to retain for the arrays of product exponential values ​​shown in Figures 4A to 4D. However, the features of this disclosure can be implemented for any number of target product exponential values.

[0035] For each of the arrays shown in Figures 4A to 4D, the decision of which product exponents to retain involves summing the values ​​of the corresponding upper bits of the product exponents, starting from the most significant bit of the product exponent, and comparing each sum to the target number of product exponents (e.g., 6) until the target number of product exponents is determined. For example, for the array of eight product exponents shown in Figure 4A, the MSBs of the eight product exponents are summed. In this case, the sum of the MSBs of the eight product exponents is 6, which is equal to the target number of six product exponents to retain. Therefore, the six product exponents with an MSB of 1 are determined to be retained (i.e., indicating which corresponding elements of matrices A and B are retained and provided to MAC212), and the two product exponents with an MSB of 0 are determined to be deleted (i.e., indicating which corresponding elements of matrices A and B are deleted and not provided to MAC212).

[0036] In the case of the array of eight product exponential values ​​shown in Figure 4B, the sum of the MSBs of the eight product exponential values ​​is 8, which is greater than the target number of the six product exponential values ​​to be held. If the sum of the MSBs is greater than 6, the next lowest most significant bits of the eight product exponential values ​​are summed. In this case, the sum of the next lowest most significant bits (i.e., the second bits) of the eight product exponential values ​​is equal to 4. Therefore, the product exponential values ​​starting with "11" are identified and held as the four largest product exponential values ​​(i.e., the four corresponding elements of matrix A and the four corresponding elements of matrix B that are held and provided to MAC212). However, since the four largest product exponential values ​​are 2 less than the target number 6, the product exponential values ​​starting with "10" are identified, the next lowest significant bit (i.e., the third bit) of the product exponential values ​​starting with "10" are summed, and the result is added to the previous sum. In this case, the sum of the third bits of the product exponential values ​​starting with "10" is equal to 2, which, when added to the previous sum of 4, equals the target number 6. Therefore, the two product exponential values ​​starting with "101" are also identified as retained product exponential values ​​(i.e., they are retained and represent two corresponding elements of matrix A and two corresponding elements of matrix B that are provided to MAC212).

[0037] In the case of the array of eight product exponential values ​​shown in Figure 4C, the sum of the MSBs of the eight product exponential values ​​is 8, which is greater than the target number of six product exponential values ​​to hold. As mentioned above, if the sum of the MSBs is greater than 6, the next lowest most significant bits of the eight product exponential values ​​are summed. In this case (the same as described above for the product exponential values ​​shown in Figure 4B), the sum of the next lowest most significant bits (i.e., the second bits) of the eight product exponential values ​​is equal to 4. Therefore, the product exponential values ​​starting with "11" are identified and held as the four largest product exponential values, but since the four largest product exponential values ​​are two less than the target number of 6, the product exponential value starting with "10" is identified, and the next lowest significant bit (i.e., the third bit) of the product exponential value starting with "10" is summed and added to the previous sum of 4. In this case, the sum of the third bits of the product exponential values ​​starting with "10" is equal to 3, which, when added to the previous sum of 4, equals 7, exceeding the target number of 6. Therefore, the least significant bit (i.e., the fourth bit) of the eight product exponents starting with "10" is summed, which is also equal to 3, and when added to the previous sum of 4, it again exceeds the target number of 6. If there are no more bits to sum and the sum of the previous sums exceeds the target number, the first two product exponents in the array that start with "10" are also retained.

[0038] For the array of eight product exponential values ​​shown in Figure 4D, the sum of the MSBs of the eight product exponential values ​​is 8, which exceeds the target number of six product exponential values ​​to be retained (this is the same as described above for the product exponential values ​​shown in Figures 4B and 4C). However, in this case, each of the remaining bits is zero. In this case, the first six product exponential values ​​in the array are retained.

[0039] The process described above with respect to Figures 4A to 4D does not involve any sorting (i.e., ordering) of the product values, which would otherwise be costly to implement in hardware.

[0040] As shown in block 308, method 300 includes selecting data (i.e., products) that were determined not to be deleted but to be retained, as described above in block 306. As shown in block 310, method 300 includes determining the dot product from the products that were determined to be retained.

[0041] In addition, the information generated by the matrix multiplication operation can be displayed on a display device (for example, output device 110 in Figure 1).

[0042] It should be understood that many modifications are possible based on the disclosures herein. Although features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0043] The methods provided can be implemented in general-purpose computers, processors, or processor cores. Suitable processors include, by example, general-purpose processors, dedicated processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines. Such processors can be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data (instructions that can be stored in a computer-readable medium), including netlists. The results of such processing can be a mask work, which is then used in a semiconductor manufacturing process to manufacture a processor that implements the features of the present disclosure.

[0044] The methods or flowcharts provided herein can be implemented in computer programs, software, or firmware embedded in non-temporary computer-readable storage media for execution by a general-purpose computer or processor. Examples of non-temporary computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. A processing device, Memory configured to store data, Equipped with a processor, The aforementioned processor, Receiving a portion of the data of a first matrix containing a first set of elements, Receiving a portion of the data of a second matrix containing a second set of elements, The value of the third matrix is The method involves calculating an exponential sum equal to the sum of the exponential values ​​of the elements of the first matrix and the exponential values ​​of the elements of the second matrix for each pair of elements, wherein the exponential sum is the exponential value of the approximate product of the product of the pair of elements, Removing the product of the target number from the product of the pairs of elements of the first and second matrices corresponding to the pair having the smallest sum of exponents, Perform matrix multiplication on the remaining product of the elements of the first matrix and the second matrix to determine an approximate value of the third matrix, The decision will be made by, It is configured to do, Processing device.

2. It also features multiple multiplier accumulators (MACs), The number of MACs used to perform the matrix multiplication is reduced by the number of products that are removed. The processing device according to claim 1.

3. The aforementioned processor, Starting from the most significant bit, sum the values ​​of the corresponding significant bits of the aforementioned exponential sum, The set of products to be retained is determined by comparing each of the results of the above sums with the target number, such that the number of products to be deleted is equal to the target number. This is configured to determine which product to use to perform the matrix multiplication. The processing device according to claim 1.

4. The first and second plurality of elements are integer data type formats. The processing device according to claim 1.

5. The aforementioned processor, Determining the absolute values ​​of the first and second plurality of elements, For each element, the number of leading zeros in the representation of the absolute value is determined, wherein the number of leading zeros is the number of consecutive "0" bits from the most significant bit to the least significant bit in the fixed-width binary representation of the absolute value before the first "1" bit appears. For each element, the exponential value is approximated as the difference between the number of bits in the element minus 1 and the number of leading zeros in the absolute value of the element. The system is configured to extract the exponential value of the aforementioned element. The processing device according to claim 4.

6. The aforementioned processor processes each element to 1.M ＊ 2 ｅ The system is configured to calculate the aforementioned exponential value for each element by expressing it as follows: in the formula, M is the mantissa and e is the aforementioned exponential value. The processing device of claim 5.

7. The first and second plurality of elements are floating data type formats. The processing device according to claim 1.

8. The number of target products to be removed is determined during the execution of the matrix multiplication based on at least one of the following: the amount of acceptable result error, the impact on the resulting network accuracy, and the size of the common dimension K between the first matrix and the second matrix. The processing device according to claim 1.

9. With additional display devices, The information generated from the matrix multiplication is displayed on the display device. The processing device according to claim 1.

10. A method for a computer to select matrix multiplication data, Receiving a portion of the data of a first matrix containing a first set of elements, Receiving a portion of the data of a second matrix containing a second set of elements, The value of the third matrix is The method involves calculating an exponential sum equal to the sum of the exponential values ​​of the elements of the first matrix and the exponential values ​​of the elements of the second matrix for each pair of elements, wherein the exponential sum is the exponential value of the approximate product of the product of the pair of elements, Removing the product of the target number from the product of the pairs of elements of the first and second matrices corresponding to the pair having the smallest sum of exponents, Perform matrix multiplication on the remaining product of the elements of the first matrix and the second matrix to determine an approximate value of the third matrix, This includes determining by, method.

11. Sum the values ​​of the corresponding significant bits of the exponential sum, starting from the most significant bit, The set of products to be retained is determined by comparing each of the results of the above sums with the target number, such that the number of products to be deleted is equal to the target number. This further includes determining which product to use to perform the matrix multiplication, The method of claim 10.

12. The first and second plurality of elements are integer data type formats. The method of claim 10.

13. Determining the absolute values ​​of the first and second plurality of elements, For each element, the number of leading zeros in the representation of the absolute value is determined, wherein the number of leading zeros is the number of consecutive "0" bits from the most significant bit to the least significant bit in the fixed-width binary representation of the absolute value before the first "1" bit appears. For each element, the exponential value is approximated as the difference between the number of bits in the element minus 1 and the number of leading zeros in the absolute value of the element. This further includes extracting the exponential value of the aforementioned element. The method according to claim 12.

14. Each element is 1. M ＊ 2 ｅ This expression includes calculating the exponential value for each element, where M is the mantissa and e is the exponential value. The method according to claim 13.

15. The first and second plurality of elements are floating data type formats. The method of claim 10.

16. The number of targets to be deleted is determined during the execution of the matrix multiplication based on at least one of the following: the amount of acceptable result error, the impact on the resulting network accuracy, and the size of the common dimension K between the first matrix and the second matrix. The method of claim 10.

17. The information generated from the aforementioned matrix multiplication is displayed. The method of claim 10.

18. A computer-readable storage medium containing instructions for causing a computer to perform a method for selecting matrix multiplication data, The aforementioned instruction is, Receiving a portion of the data of a first matrix containing a first set of elements, Receiving a portion of the data of a second matrix containing a second set of elements, The value of the third matrix is The method involves calculating an exponential sum equal to the sum of the exponential values ​​of the elements of the first matrix and the exponential values ​​of the elements of the second matrix for each pair of elements, wherein the exponential sum is the exponential value of the approximate product of the product of the pair of elements, Removing the product of the target number from the product of the pairs of elements of the first and second matrices corresponding to the pair having the smallest sum of exponents, Perform matrix multiplication on the remaining product of the elements of the first matrix and the second matrix to determine an approximate value of the third matrix, This includes determining by, Computer-readable storage medium.

Images