A fast multiplication of matrix and its transpose under low memory

Abstract

The application discloses a low-memory matrix and a fast multiplication method of the matrix and its transpose, which comprises the following steps: 1) according to obtained matrix data, the matrix data is stored in a text file according to a specified format, and the matrix data is read into a memory by using a fast reading module; 2) the original matrix in the step 1) is transmitted into a display memory, and the answer matrix is simplified, vectorized and grouped and then transmitted into the display memory; 3) parallel calculation is carried out in groups, and serial calculation is carried out between groups; and 4) a plurality of arrays obtained in the step 3) are combined, and are converted with the answer matrix, and finally the solution is output. The application can utilize the parallel calculation framework of a GPU to fully exert the performance of a display card under the condition that the size of the display memory of the hardware is low, and large-scale matrix multiplication and its transpose multiplication can be quickly completed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a fast multiplication of matrices and their transposes under low video memory conditions. Background Technology

[0002] In today's rapidly developing field of artificial intelligence, an increasing number of deep learning models are demonstrating their power. However, the practical application of deep learning models in industry remains a significant challenge. One major difficulty is the sheer number of parameters in deep learning models, requiring high-performance hardware, particularly a graphics card with ample video memory. This means that the cost of a hardware device running a deep learning model is substantial. Model distillation, a method that compresses deep learning models, allows them to run on lower-configuration hardware platforms with minimal performance changes. As is well known, deep learning models are essentially composed of a series of matrix multiplications and fundamental functions. Among matrix multiplications, there is a very common and special type: matrix multiplication with its transpose. This type of multiplication is widely used in least squares estimation in parameter theory, observability balance in control systems, and KL transform in signal transmission.

[0003] Let matrix A have dimensions of 31441 × 32. There are two approaches to performing multiplication. The first is on a CPU platform, using a loop to traverse the rows and columns of the matrix, then directly multiplying and summing the results. The second is on a GPU platform, using the platform's matrix multiplication interface for parallel computation to obtain the final result. Comparing the two methods, taking matrix A as an example, performing one transpose multiplication requires... Order-of-magnitude operations.

[0004] The advantage of the first method is its low memory cost, allowing for computation at a lower cost even for large-scale matrices. The disadvantage is that even with the NumPy library for accelerated computation, the CPU's serial processing mechanism still requires a significant amount of time. On our platform, processing matrix A using the NumPy library took 26 hours.

[0005] The advantage of the second method is that, with sufficient video memory, it can be completed in seconds thanks to the parallel architecture of the GPU, making it fast. The disadvantage is that it requires at least 5GB of video memory to perform this operation, which is very expensive. When we performed grouped multiplication on matrix A on a graphics card with 2GB of video memory, it took about 5 minutes.

[0006] In conclusion, current methods cannot strike a balance between performance and cost. Summary of the Invention

[0007] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide a fast multiplication of matrices and their transposes under low video memory conditions.

[0008] To strike a balance between performance and cost, this invention employs fast matrix loading, matrix vectorization, and grouped computation (serial computation between groups and parallel computation within groups) to enable large-scale matrix and transpose multiplication to be performed quickly with lower video memory.

[0009] The technical solution of the present invention is as follows:

[0010] A fast matrix and transpose multiplication method with low video memory requirements includes the following steps:

[0011] 1) Based on the obtained matrix data, store it in a text file according to the specified format, and use a fast read module to read the matrix data into memory;

[0012] 2) Transfer the original matrix from step 1) to the video memory, and simplify, vectorize, and group the answer matrix before transferring it to the video memory;

[0013] 3) Parallel computation is performed within groups, while serial computation is performed between groups;

[0014] 4) Merge the multiple arrays obtained in step 3), transform them with the answer matrix, and output the final solution.

[0015] Step 1) specifically includes:

[0016] 1.1) The size of this matrix is ,in Let be the row number of the matrix. Let be the number of columns in the matrix, and let be the maximum width of the characters in each element of the matrix. Store it in a text file, the first... The row storage matrix of the first row Line elements, with each line of elements separated by commas;

[0017] 1.2) The fast read module first calls the C language's fopen and fread functions to read the entire text file as characters, and then, based on the results obtained in step 1.1),... , , The matrix is ​​read into memory using a loop and the C language's atof_ function, denoted as . .

[0018] Step 2) specifically includes:

[0019] 2.1) Will The matrix is ​​transferred to the video memory;

[0020] 2.2) The solution obtained by multiplying a matrix by its transpose (denoted as matrix A) It is still a matrix with size . Furthermore, since the matrix is ​​symmetric, the number of valid values ​​in the matrix is... To simplify it, we only need to calculate the matrix. In One value is sufficient;

[0021] 2.3) In order to store We store these values ​​using a one-dimensional array, the length of which is [value]. ;

[0022] 2.4) In practice, The value is usually much larger than Therefore, the one-dimensional array described in 2.3) cannot be directly passed into the video memory. Thus, the one-dimensional array described in 2.3) is divided into... A one-dimensional array, from 1 to... The length of the one-dimensional array of the group is , No. The length of the group is , record the The length of the group array is , No. The array of groups is ,in Indicates to Rounding;

[0023] 2.5) Traversal Group, in calculating the first Before the group Passed into video memory, the first After the group calculation is completed, Passed into memory.

[0024] Step 3) specifically includes:

[0025] 3.1) Section Group 1 The values ​​of the elements are:

[0026] (1)

[0027] in Representation matrix The Line number One element, Representation matrix The Line number There are elements, among which , The value is:

[0028] (2)

[0029] (3)

[0030] in Indicates to Round down. The value is:

[0031] (4)

[0032] It should be noted that, due to the first The values ​​of the group are calculated in the GPU, therefore the first... The values ​​within each group are computed in parallel. The calculations are performed sequentially between the groups;

[0033] Step 4) specifically includes:

[0034] 4.1) After the above steps, we obtained... A collection of one-dimensional arrays ,Will Let ... ,in The length is ;

[0035] 4.2) Create a one-dimensional array With matrix The mapping relationship is as follows:

[0036] (5)

[0037] in Representation matrix The one located at the line, number Column elements, Represents array The One element, , , The definition is as follows:

[0038] (6)

[0039] (7)

[0040] By traversal , to obtain the corresponding and The final result That is, a matrix The product of its transpose and the result.

[0041] The beneficial effects of this invention are: it can utilize the parallel computing framework of the GPU to fully leverage the performance of the graphics card, and quickly complete large-scale matrix and transpose multiplication, even with a relatively small amount of video memory. This improves the performance by approximately 2340 times compared to the first scheme of the background technology and by 7.5 times compared to the second scheme of the background technology, and also has stronger adaptability. Attached Figure Description

[0042] Fig. 1 This is a flowchart of the present invention;

[0043] Fig. 2 This is a schematic diagram illustrating matrix simplification, vectorization, grouping, and grouping calculations in the demonstration example. Fig. 3 This is a schematic diagram illustrating how array merging and array mapping generate the final result in the demonstration example. Detailed Implementation

[0044] The algorithm implementation scheme of the present invention will be described below with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] Reference Figs. 1-3 :

[0047] Suppose a text file stores a matrix of size 4×3, i.e., N=4, D=3, and the content of the text file is as follows:

[0048] 0.9, 1.2, 0.7

[0049] 1.1, 0.4, 0.3

[0050] 0.8, 0.3, 0.1

[0051] 2.1, 0.7, 0.2

[0052] First, use the fast read module to load the matrix into memory to obtain the matrix. As shown in Table 1 below;

[0053]

[0054] Table 1 Matrix value

[0055] Then determine the matrix. The matrix after multiplication of its transpose The dimensions are shown in the following formula. The size is :

[0056]

[0057] Because it is Since it's a symmetric matrix, the number of valid elements is 10. Here, we choose G=4, meaning it's divided into four groups. The lengths of the one-dimensional arrays for the four groups are 3, 3, 3, and 1 respectively. Taking the calculation of the second group as an example, we first divide the one-dimensional array of the first group... Read from video memory, then create a one-dimensional array. And Move it into video memory; then... We calculate the three elements simultaneously. Taking the first element of the second group as an example, according to the formula in step 3.1), we get... Then retrieve the matrix. The elements in the first and fourth rows are multiplied at corresponding positions and then summed to obtain the value of the first element of the second group. The rest are similar. Finally, four one-dimensional arrays of lengths 3, 3, 3, and 1 are obtained. The entire process is as follows: Fig. 2 As shown. Next, the four groups of one-dimensional arrays of different lengths are concatenated according to their group numbers to obtain a one-dimensional array of length 10. Then, according to the calculation formula in step 4.1), the array is... Mapping to a matrix The final matrix The values ​​are shown in Table 2 below. This process is as follows: Fig. 3 As shown.

[0058]

[0059] Table 2 Matrix value

[0060] Example 2

[0061] This embodiment describes a product recommendation method applying the present invention's fast matrix and transpose multiplication method under low video memory conditions, comprising the following steps:

[0062] Suppose that 5 users watched 5 different movies, as shown in Table 3 below:

[0063]

[0064] Table 3 Movie IDs Viewed by Users

[0065] There are currently ten movies, with movie IDs as follows: ~ They are all represented by a three-dimensional vector, as shown in Table 4 below.

[0066]

[0067] Table 4. Vectors of Movies

[0068] 1) Based on the obtained matrix data, store it in a text file according to the specified format, and use a fast read module to read the matrix data into memory;

[0069] Save the data in Table 4 in a text file according to the format described in step 1), and then use a fast read module to read the matrix data into memory to obtain the matrix. As shown in Table 5 below;

[0070]

[0071] Table 5 Matrix

[0072] 2) Transfer the original matrix from step 1) to the video memory, and simplify, vectorize, and group the answer matrix before transferring it to the video memory;

[0073] matrix The data is passed into the video memory, due to the matrix... The size is Therefore, the matrix The matrix after multiplying it by its transpose The size is Due to the matrix Since it is a symmetric matrix, it has 55 valid elements. Here, we choose G=4, which means it is divided into four groups. The lengths of the one-dimensional arrays of the four groups are 14, 14, 14, and 13, respectively.

[0074] 3) Parallel computation is performed within groups, while serial computation is performed between groups;

[0075] Taking the calculation of the second group as an example, first, the array of the first group... Read from video memory, then create a one-dimensional array. And Move it into video memory; then... The 14 elements in the group are calculated simultaneously. Taking the first element of the second group as an example, according to the formula in step 3.1), we get... Extract the matrix The second and sixth row elements (that is, the movie) And movies The vectors are multiplied by their corresponding positions and then summed to get the value of the first element of the second group. The remaining 13 elements in the second group are calculated simultaneously during this process.

[0076] 4) Merge the multiple arrays obtained in step 3), transform them with the answer matrix, and output the final solution.

[0077] Step 3) will result in four one-dimensional arrays with lengths of 14, 14, 14, and 13 respectively. Then, these four arrays of different lengths are concatenated in order of their group numbers to obtain a one-dimensional array of length 55. ; Calculate the array according to the formula in step 4.1). Mapping to a matrix The final matrix The values ​​are shown in Table 6 below. Indicates user With movies The similarity.

[0078]

[0079] Table 6. Similarity between items and users

[0080] 5) Based on users For example, recommendations are made. Table 3 shows that users... I like movies According to Table 6, we can find that the movie With movies More similar, so the film Recommended to users It should be noted that in low-memory environments, due to insufficient video memory, users... ~ The calculations need to be performed sequentially before recommendations are made. This invention can provide users with... ~ Recommendations will be made simultaneously.

[0081] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A fast matrix and transpose multiplication method under low video memory conditions, characterized in that, Includes the following steps: 1) Based on the obtained matrix data, store it in a text file according to the specified format, and use a fast read module to read the matrix data into memory; 2) Transfer the original matrix from step 1) to the video memory, and simplify, vectorize, and group the answer matrix before transferring it to the video memory; 3) Parallel computation is performed within groups, while serial computation is performed between groups; 4) Merge the multiple arrays obtained in step 3), transform them with the answer matrix, and output the final solution; Step 1) specifically refers to: 1.1) The size of this matrix is ,in Let be the row number of the matrix. Let be the number of columns in the matrix, and let be the maximum width of the characters in each element of the matrix. Store it in a text file, the first... The row storage matrix of the first row Line elements, with each line of elements separated by commas; 1.2) The fast read module first calls the C language's fopen and fread functions to read the entire text file as characters, and then, based on the results obtained in step 1.1),... , , The matrix is ​​read into memory using a loop and the C language's atof_ function, denoted as . ; Step 2) specifically refers to: 2.1) Will The matrix is ​​transferred to the video memory; 2.2) The solution obtained by multiplying a matrix by its transpose (denoted as matrix A) It is still a matrix with size . Furthermore, since the matrix is ​​symmetric, the number of valid values ​​in the matrix is... To simplify it, we only need to calculate the matrix. In One value is sufficient; 2.3) In order to store The values ​​are stored in a one-dimensional array, the length of which is [value]. ; 2.4) In practice, The value is usually much larger than Therefore, the one-dimensional array described in 2.3) cannot be directly passed into the video memory. Thus, the one-dimensional array described in 2.3) is divided into... A one-dimensional array, from 1 to... The length of the one-dimensional array of the group is The length of group G is , record the The length of the group array is , No. The array of groups is ,in Indicates to Rounding; 2.5) Traversal Group, in calculating the first Before the group Passed into video memory, the first After the group calculation is completed, Passed into memory; Step 3) specifically refers to: 3.1) Section Group 1 The values ​​of the elements are: （1） in Representation matrix The Line number One element, Representation matrix The Line number There are elements, among which , The value is: （2） （3） in Indicates to Round down. The value is: （4） It should be noted that, due to the first The values ​​of the group are calculated in the GPU, therefore the first... The values ​​within each group are computed in parallel. The calculations are performed sequentially between the groups; Step 4) specifically involves: 4.1) After the above steps, we obtained... A collection of one-dimensional arrays ,Will Let ... ,in The length is ; 4.2) Create a one-dimensional array With matrix The mapping relationship is as follows: （5） in Representation matrix The one located at the line, number Column elements, Represents array The One element, , , The definition is as follows: （6） （7） By traversal , to obtain the corresponding and The final result That is, a matrix The product of its transpose and the result.

Images