Method and system for improving floating point matrix operation efficiency

Abstract

The invention discloses a method and system for improving floating point matrix operation efficiency. The method comprises the following steps of: extracting sample matrixes in a floating point matrix operation to obtain a sample matrix sequence; determining an optimum calibration parameter sequence of the sample matrix sequence; and transforming calibration parameters in the optimum calibration parameter sequence and applying the transformed calibration parameter sequence into all the matrix sequence operations. By utilizing the method and system disclosed by the invention, the floating point matrix sequence operation response time can be shortened and the operation precision can be ensured.

Description

technical field [0001] The invention relates to the field of floating-point matrix operations, in particular to a method and system for improving the efficiency of floating-point matrix operations. Background technique [0002] In the fields of speech and image processing, it is often necessary to perform calculations on floating-point matrix sequences. For example, using neural network language models in speech recognition requires a large number of floating-point matrix sequences W that conform to certain distribution rules. m×n(1) , W m×n(2) ,...,W m×n(N) and x n×p(1) 、X n×p(2) ,...,X n×p(N) (m, n, p, and N are all positive integers) do multiplication operation, that is, W m×n(1) × R n×p(1) , W m×n(2) × R n×p(2) ,...,W m×n(N) × R n×p(N) , these areas require faster computing response time. If you directly operate on the floating-point matrix sequence, take the float (single-precision floating-point number) matrix as an example, since the float occupies 4 bytes,...

AI Technical Summary

Problems solved by technology

[0003] The existing floating-point matrix fixed-point solution is to multiply the entire matrix by the same 2 n Scaling parameters, since the entire matrix uses the same scalar value n, when the distribution of floating-point values ​​in some rows is quite different, the response time and accuracy of the operation cannot be guaranteed at the same time, that is, after the fixed-point number is restored to a floating-point number, it will produce precision loss

For example, the value of one row of the matrix is ​​relatively small, and the value of the other row is relatively large. The two rows are (1.1, 1.2, 1.3) and (20, 30, 40) respectively. It needs to be multiplied by 3 at the same time, then all the numbers in the first line will be fixed-point and the result will be 3, the precision loss will be great, and the precision loss will be further enlarged after the floating-point matrix multiplication, addition, and subtraction operations; and if you want to keep one decimal places, the two rows need to be multiplied by at least 16 respectively, then the floating-point matrix must be fixed-point converted to short type, and the response time of short type operation is twice that of char type

In addition, the existing method enlarges all floating-point numbers by 2 at the same time n Multiply scaling, the size of the number increases exponentially, and the risk of exploding the scale is very high during matrix multiplication, addition, and subtraction operations

And the existing 2 n The calibration method needs to calibrate all the matrices one by one in the matrix sequence operation, which consumes a lot of operation response time

Images