GPU adaptive memory optimization method and device, equipment and medium

Abstract

The invention provides a GPU self-adaptive memory optimization method and device, equipment and a medium, and relates to the technical field of computer system structures, and the method comprises the steps: obtaining a target sparse matrix; optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrixes; for any optimized sub-block matrix, based on a plurality of minimum memory units contained in the GPU, allocating the same target minimum memory unit to each array corresponding to the optimized sub-block matrix; for any array of any optimized sub-block matrix, calculating a memory initial address and a memory termination address of the array in a corresponding target minimum memory unit to obtain a target storage address of the array; and storing each array to a corresponding target storage address to obtain an optimized GPU memory. According to the method, the use efficiency of the GPU memory is improved, the GPU memory is saved, the data locality during calculation is improved, and the speed of calculating the function based on the block matrix is increased.

Description

Technical Field

[0001] The present application relates to the technical field of computer architecture, and in particular to a GPU adaptive memory optimization method, device, equipment and medium. Background Art

[0002] Sparse matrix-vector multiplication (SpMV) and sparse matrix multiplication (SpMM) are important mathematical operations widely used in computational science and engineering. In these two operations, the matrix is ​​represented as a sparse matrix, that is, most of the elements are zero. This characteristic of sparse matrices enables efficient saving of memory and computing resources when processing large-scale data. However, due to the irregularity of SpMV and SpMM operations and the limited storage bandwidth of related memories, it is very challenging to implement faster and parallel SpMV and SpMM operations on GPUs. The existing technology currently mainly accelerates the calculation of SpMV and SpMM by processing sparse matrices in blocks.

[0003] Although block processing of sparse matrices has the advantages of improving the data limitation of dense vectors, easier load balancing, and greater simplicity and flexibility, it also has the following problems:

[0004] First, dividing the sparse matrix into multiple smaller blocks can allocate tasks more finely and easily achieve load balancing, but it will increase communication overhead.

[0005] Secondly, the sparse format data structure inside the sub-blocks of functions such as SpMV and SpMM usually includes multiple arrays. The existing technology is to Malloc the multiple arrays separately. However, since the sub-blocks are usually very small and the sub-blocks are usually sparse matrices, the memory size required for each array is actually very small, which is much smaller than the applied memory unit; this leads to huge memory fragments inside the generated multiple GPU memory units, which greatly wastes the precious memory resources of the GPU, and also greatly reduces the data locality of the data in the sub-block.

[0006] Finally, the space waste caused on GPUs of different brands is also different. For example, on Nvidia GPU, each time a memory allocation is requested, it is allocated with 512B as the basic memory unit. On domestic DCU, each time a memory allocation is requested, it is allocated with 4KB as the basic memory unit. Under the blocking mechanism, the array size corresponding to the data structure in each block is much smaller than the basic memory unit mentioned above. This leads to a large performance difference on GPUs of different brands when the blocking method is applied to matrix data on the GPU. Summary of the invention

[0007] The purpose of the embodiments of the present application is to provide a GPU adaptive memory optimization method, device, equipment and medium to solve the above-mentioned problems existing in the prior art, realize GPU load balancing, maximize the parallelism of calculations while avoiding waste of memory space, and improve the related calculation performance based on block matrices on the GPU.

[0008] In a first aspect, a GPU adaptive memory optimization method is provided, which may include:

[0009] Obtain a target sparse matrix; wherein each element in the target sparse matrix includes the value of the element and the horizontal coordinate and the vertical coordinate of the element in the target sparse matrix;

[0010] Optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix includes at least two arrays;

[0011] For any optimized sub-block matrix, based on multiple minimum memory units included in the GPU, allocating the same target minimum memory unit to each array corresponding to the optimized sub-block matrix; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0012] For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array;

[0013] Store each array to the corresponding target storage address to obtain the optimized GPU memory.

[0014] In an optional implementation, the target sparse matrix is ​​optimized to obtain a plurality of optimized sub-block matrices, including:

[0015] Based on the number of threads included in the GPU thread bundle and the capacity of the minimum memory unit, the target sparse matrix is ​​divided into blocks to obtain a plurality of sub-block matrices; wherein the number of rows and the number of columns of any sub-block matrix are equal to the number of threads;

[0016] Extracting matrix features of each sub-block matrix;

[0017] For any sub-block matrix, the matrix features of the sub-block matrix are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein the data storage format selection model is obtained by training based on multiple different historical sub-block matrices and the optimal data storage formats of the corresponding historical sub-block matrices;

[0018] According to the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain an optimized sub-block matrix.

[0019] In an optional implementation, optimizing the storage format of the data in the sub-block matrix according to the optimal data storage format to obtain an optimized sub-block matrix includes:

[0020] Convert the storage format of each horizontal coordinate and each vertical coordinate in the sub-block matrix into the optimal data storage format to obtain a position quantity row of the sub-block matrix;

[0021] Extracting the value of each element in the sub-block matrix as an element value row of the sub-block matrix;

[0022] If the memory capacity required by the data type of any position quantity in the position quantity row is greater than the memory capacity required by the data type of any element value in the element value row, the element value row is used as the first row of the optimized sub-block matrix, and the position quantity row is used as the second row of the optimized sub-block matrix;

[0023] If the memory capacity required for the data type of any position quantity in the position quantity row is smaller than the memory capacity required for the data type of any element value in the element value row, the element value row is used as the second row of the optimized sub-block matrix, and the position quantity row is used as the first row of the optimized sub-block matrix.

[0024] In an optional implementation, the method further includes:

[0025] Obtaining GPU configuration information; wherein the GPU configuration information includes: the memory first address of the target minimum memory unit and the GPU alignment rule;

[0026] Calculating the memory start address and the memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array includes:

[0027] If the array is the array corresponding to the first row of the optimized sub-block matrix, the memory first address of the corresponding target minimum memory unit is used as the memory start address of the array;

[0028] The sum of the memory first address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the memory end address of the array; wherein the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rule;

[0029] If the array is not the array corresponding to the first row of the optimized sub-block matrix, the memory end address corresponding to the previous row of the corresponding row of the array is used as the memory start address of the array;

[0030] The sum of the memory start address, the memory capacity required for the corresponding row of the array and the memory padding amount of the array is used as the memory end address of the array.

[0031] In an optional implementation, the target minimum memory unit is any one of a plurality of minimum memory units, or is a plurality of consecutive minimum memory units.

[0032] In an optional implementation, the configuration information of the GPU further includes: a memory capacity of the target minimum memory unit;

[0033] After obtaining a plurality of sub-block matrices, the method further includes:

[0034] Calculate the memory capacity occupied by each sub-block matrix;

[0035] From the obtained multiple sub-block matrices, select the sub-block matrix that occupies the largest memory capacity as the target sub-block matrix;

[0036] Multiplying the memory capacity of the target minimum memory unit by a preset memory capacity threshold to obtain a memory capacity block threshold;

[0037] If the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, the step of extracting matrix features of each sub-block matrix is ​​performed.

[0038] In an optional implementation, the method further includes:

[0039] If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity block threshold, dividing the number of threads by the preset thread threshold to obtain a block parameter;

[0040] The target sparse matrix is ​​divided into blocks using the block parameters as the number of rows and columns of each sub-block matrix to obtain multiple sub-block matrices, and the execution step is returned to: the memory capacity occupied by each sub-block matrix is ​​calculated until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity block threshold.

[0041] In a second aspect, a GPU adaptive memory optimization device is provided, which may include:

[0042] An acquisition unit, configured to acquire a target sparse matrix; wherein each element in the target sparse matrix includes a value of the element and a horizontal coordinate and a vertical coordinate of the element in the target sparse matrix;

[0043] An optimization unit, used for optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays;

[0044] an allocation unit, for allocating the same target minimum memory unit to each array corresponding to any optimized sub-block matrix based on multiple minimum memory units included in the GPU; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0045] A calculation unit, used for calculating, for any array of any optimized sub-block matrix, a memory start address and a memory end address of the array in a corresponding target minimum memory unit, to obtain a target storage address of the array;

[0046] The storage unit is used to store each array to a corresponding target storage address to obtain an optimized GPU memory.

[0047] In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0048] Memory, used to store computer programs;

[0049] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0050] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0051] The present application provides an adaptive GPU perception function that masks the hardware differences between different types of GPUs, so that when using related functions based on the matrix blocking mechanism, such as SpMV or SpMM functions, it can automatically adapt to the programming language and hardware characteristics of different types of GPUs, and then obtain relevant thread configuration information and the actual size of the Malloc memory unit block under different types of GPUs, providing information for subsequent optimization steps.

[0052] This application determines the size of the sub-block matrix after the sparse matrix is ​​divided based on the number of threads, which can make full use of the SIMD computing resources of GPUs of different types, find the best point to achieve load balancing effect, maximize the parallelism of calculation, and achieve the purpose of accelerated calculation; this application uses the data storage format selection model to select the optimal data storage format for each sub-block matrix, maximizing the computing performance of each sub-block matrix. This application can dynamically adjust the sub-block size and data storage format to achieve better parallel processing efficiency and memory usage efficiency.

[0053] This application allocates a minimum memory unit to each sub-block matrix, and each array of each sub-block matrix is ​​stored in the same minimum memory unit, thereby realizing unified space management of the memory corresponding to different arrays of the same sub-block matrix and optimizing the GPU memory unit; at the same time, a sub-block array that originally required multiple memory applications is applied for once, which greatly reduces the generation of memory fragmentation and makes more effective use of memory resources, thereby saving GPU memory and improving data locality during calculation, thereby speeding up the calculation of functions based on block matrices.

[0054] The present application improves the locality of data in the GPU by compactly storing the data of all sub-block matrices, thereby reducing access delays and accelerating data processing speed; the present application greatly improves the memory utilization efficiency and computing speed of the GPU; the present application can eliminate the hardware differences between different types of GPUs, so that the latest calculation functions based on the matrix partitioning mechanism, such as SpMV or SpMM, can be optimized to different degrees on different types of GPUs, so that each GPU can exert its maximum performance.

[0055] This application can identify and adapt to the GPU characteristics of different manufacturers, such as the different sizes of basic memory allocation units, and optimize cross-platform computing and memory management strategies; this application enables the GPU to perform sparse matrix calculations more efficiently, reduce invalid calculations and waste of memory resources, and significantly improve the computing performance of functions such as SpMV and SpMM based on the blocking mechanism. It also provides a flexible and effective solution for optimization in different GPU environments, and can achieve better performance and memory management on different GPUs, thereby expanding its application scope in industry and scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A GPU adaptive memory optimization control system architecture diagram provided in an embodiment of the present application;

[0058] Figure 2 An architecture diagram of an adaptive matrix partitioning module provided in an embodiment of the present application;

[0059] Figure 3 An architecture diagram of an adaptive memory optimization module provided in an embodiment of the present application;

[0060] Figure 4 A flowchart of a GPU adaptive memory optimization method provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of a data storage format selection model training method provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the structure of a GPU adaptive memory optimization device provided in an embodiment of the present application;

[0063] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] The GPU adaptive memory optimization method provided in the embodiment of the present application can be applied to Figure 1 In the GPU adaptive memory optimization control system architecture shown, the system may include: a hardware layer, a preprocessing layer (i.e., an adaptive memory perception module), a sparse matrix structure layer (i.e., an adaptive matrix partitioning module), a sub-block internal structure layer (i.e., an adaptive memory optimization module), a sub-block internal algorithm layer (i.e., an algorithm module), an external algorithm, and an external function interface layer;

[0066] Among them, the pre-processing layer is used to obtain the configuration information of the graphics processing unit (i.e., GPU);

[0067] Sparse matrix structure layer, such as Figure 2 As shown, it is used to divide the target sparse matrix into multiple sub-block matrices based on the number of GPU threads;

[0068] The internal structure layer of the sub-block, such as Figure 3As shown, it is used to determine the optimal data storage format of each sub-block matrix, and optimize each sub-block matrix based on the optimal data storage format of each sub-block matrix to obtain an optimized sub-block matrix; allocate a minimum memory allocation unit to each sub-block matrix; and simultaneously calculate the target storage address of any array of any sub-block matrix on the minimum memory allocation unit, and store each array to the corresponding target storage address;

[0069] The sub-block internal algorithm layer is used to call the reconstructed internal sub-block related kernel functions to calculate the data stored in each minimum memory unit of the GPU and transmit it back to the host (i.e., the general processor to which the GPU is connected);

[0070] The outer algorithm and the external function interface layer are used to receive data transmitted by the internal algorithm layer of the sub-block, and return the correct results through the interface to complete a function call and calculation.

[0071] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0072] Figure 4 A flowchart of a GPU adaptive memory optimization method provided in an embodiment of the present application. Figure 4 As shown, the method may include:

[0073] Step S410, obtaining a target sparse matrix; optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices.

[0074] In the embodiment of the present application, the target sparse matrix, that is, each element in the target sparse matrix, includes the value of the element and the horizontal and vertical coordinates of the element in the target sparse matrix; the number of rows and columns of the target sparse matrix is ​​the same. For example, the element A in the first row and first column of the target sparse matrix A 11 =[value 11 , x 11 ,y 11 ], where value 11 Indicates A 11 The value of x 11 Indicates A 11 The horizontal axis of y 11 Indicates A 11 The vertical coordinate of .

[0075] In an embodiment of the present application, a GPU includes a controller, multiple computing units (the minimum parallel computing core of a GPU), a function interface unit, a thread bundle, and multiple minimum memory units, etc.; in an embodiment of the present application, before optimizing a target sparse matrix to obtain multiple optimized sub-block matrices, the GPU adaptive memory optimization method also needs to obtain the configuration information of the GPU; wherein the configuration information of the GPU includes: the type of GPU, the alignment rules of the GPU, the compiler type of the hardware device where the GPU is located, the memory capacity of each minimum memory unit, the memory start address and the memory end address, and the number of threads contained in the thread bundle, etc. Different types of GPUs may have different configuration information.

[0076] In an embodiment of the present application, the target sparse matrix is ​​optimized to obtain a plurality of optimized sub-block matrices, including:

[0077] Based on the number of threads contained in the GPU thread bundle and the capacity of the minimum memory unit, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices; the matrix features of each sub-block matrix are extracted; for any sub-block matrix, the matrix features of the sub-block matrix are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein the data storage format selection model is trained based on multiple different historical sub-block matrices and the optimal data storage formats of the corresponding historical sub-block matrices; according to the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain the optimized sub-block matrix.

[0078] In the embodiment of the present application, the number of rows and columns of any sub-block matrix obtained by dividing the target sparse matrix are equal; the target sparse matrix is ​​preliminarily divided into blocks to obtain the preliminarily divided sub-block matrix. The number of rows and columns of each sub-block matrix obtained by dividing the target sparse matrix into blocks are equal.

[0079] In the embodiment of the present application, after obtaining a plurality of sub-block matrices, the method further includes: calculating the memory capacity occupied by each sub-block matrix; selecting the sub-block matrix with the largest memory capacity from the obtained plurality of sub-block matrices as the target sub-block matrix; multiplying the memory capacity of the target minimum memory unit by a preset memory capacity threshold to obtain a memory capacity block threshold; if the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, executing the steps of: extracting matrix features of each sub-block matrix;

[0080] If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity blocking threshold, the number of threads is divided by the preset thread threshold to obtain the blocking parameters; the blocking parameters are used as the number of rows and columns of each sub-block matrix, the target sparse matrix is ​​blocked to obtain multiple sub-block matrices, and the execution step is returned: the memory capacity occupied by each sub-block matrix is ​​calculated until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity blocking threshold.

[0081] In the embodiment of the present application, the element type contained in each sub-block matrix is ​​multiplied by the number of elements of the corresponding type and then added to obtain the memory capacity occupied by each sub-block matrix.

[0082] In an embodiment of the present application, the memory capacity threshold is pre-set, and the value range is 100%-180%; when the memory capacity threshold is set to 180%, it means that when the memory capacity occupied by the target sub-block matrix does not exceed 180% of the memory capacity of the target minimum memory unit, then the target sub-block matrix can be accommodated by two consecutive minimum memory units, that is, the memory capacity of the target minimum memory unit is the memory capacity of the two minimum memory units. The block result at this time is the final block result, and the next step can be executed: extracting the matrix features of each sub-block matrix.

[0083] In the embodiment of the present application, the thread threshold is pre-set, and its value may be 2.

[0084] In an embodiment of the present application, the memory capacity of the target minimum memory unit is determined by the number of minimum memory units constituting the target minimum memory unit and the memory capacity of the minimum memory unit; when the target minimum memory unit is any one of multiple minimum memory units, the memory capacity of the target minimum memory unit is the memory capacity of the minimum memory unit; when the target minimum memory unit is multiple consecutive minimum memory units, the memory capacity of the target minimum memory unit is determined based on the number and memory capacity of the minimum memory units.

[0085] For example, if the number of threads is 32 and the thread threshold is 2, the target sparse matrix is ​​divided into multiple 32×32 sub-block matrices; the sub-block matrix with the largest memory capacity among all sub-block matrices is selected as the target sub-block matrix, and it is calculated whether the memory capacity occupied by the target sub-block matrix exceeds the memory capacity block threshold; if it exceeds, 32 / 2=16 is used as the number of rows and columns of the block matrix, and the target sparse matrix is ​​re-blocked to obtain multiple 16×16 sub-block matrices, and it is verified again whether the memory capacity occupied by the new target sub-block matrix exceeds the memory capacity block threshold:

[0086] If it does not exceed, the block result of this time is used as the final block result of the target sparse matrix to obtain multiple 16×16 sub-block matrices;

[0087] If it exceeds, the number of threads is divided by the preset thread threshold to obtain an intermediate value; the intermediate value is divided by the preset thread threshold again to obtain a block parameter; that is, if the target sub-block matrix with the largest memory capacity occupied by the 16×16 sub-block matrix still cannot be accommodated by the target minimum memory unit, the number of threads is divided by the preset thread threshold twice to obtain a block parameter; that is, 32 / 2 / 2=8 is used as the number of rows and columns of the block matrix, and the target sparse matrix is ​​re-blocked to obtain multiple 8×8 sub-block matrices, which are verified again until the target block matrix can be accommodated by a minimum memory unit.

[0088] In practical applications, the purpose of blocking the target sparse matrix is ​​to have one computing unit responsible for processing one block matrix, and multiple threads in the computing unit execute the calculation tasks of each block matrix in parallel. In order to maximize the efficiency of parallel processing and achieve better load balancing, the key is to match the size of the sub-block (that is, the number of rows and columns) with the number of threads in the computing unit in the GPU.

[0089] Therefore, when the target sparse matrix is ​​initially divided into blocks, the embodiment of the present application sets the number of rows and columns of the sub-block matrix after the initial division to be equal to the number of threads; however, since the memory capacity occupied by the sub-block matrix may not be accommodated by a minimum memory unit and the number of threads in the computing units of different GPUs is different, the size of the fixed sub-block may affect the parallel efficiency of certain computing units. Therefore, after obtaining the sub-block matrix after the initial division, the present application compares the memory capacity of the sub-block matrix. If the memory capacity occupied by the sub-block matrix with the largest memory capacity cannot be accommodated by the target minimum memory unit, the target sparse matrix is ​​re-divided into blocks.

[0090] In another embodiment of the present application, the target sparse matrix is ​​divided into blocks based on the number of threads included in the GPU thread bundle and the capacity of the minimum memory unit, and the following steps can be adopted: analyzing the target sparse matrix; determining the block target based on the analysis result of the target sparse matrix and the capacity of the minimum memory unit; performing preliminary block division on the target sparse matrix based on the block target to obtain a sub-block matrix after preliminary block division, and adjusting the sub-block matrix after preliminary block division to obtain multiple sub-block matrices.

[0091] The embodiment of the present application determines the block parameters by the number of GPU threads and the capacity of the minimum memory unit, divides a target sparse matrix as a whole into multiple sub-block matrices, and adjusts the block parameters according to the memory capacity occupied by the sub-block matrices after the division. This not only makes the size of the sub-block matrices after the target sparse matrix is ​​divided into blocks adapt to the thread configuration in the GPU, but also helps to divide the target sparse matrix more accurately, thereby improving the computing efficiency of each computing unit and making the load between each computing unit more balanced.

[0092] In actual applications, the memory capacity of each minimum memory unit and the number of threads included in the GPU thread bundle cannot be directly obtained. The embodiment of the present application pre-writes a macro file and compiles the written macro file using the device where the GPU is located, thereby obtaining the memory capacity of each minimum memory unit and the number of threads included in the GPU thread bundle.

[0093] In an embodiment of the present application, memory query code and thread number query code are written in different hardware description languages; through macro definition and conditional selection statements, the written memory query code and thread number query code are embedded in the corresponding macro conditional program code segment to obtain a macro definition code fragment; during compilation, the device where the GPU is located will select the corresponding macro definition code fragment to execute based on its own compiler type and GPU type; the execution feedback result is obtained to obtain the memory capacity of the corresponding minimum memory unit and the number of threads contained in the GPU thread bundle.

[0094] In actual applications, Nvidia GPUs usually use the nvcc compiler to compile CUDA source code, while AMD GPUs use the hipcc compiler to compile HIP source code; therefore, when writing memory query code and thread number query code, this application will use both of the above two and other types of GPU-compilable hardware languages ​​to write; in addition, when compiling source code, macro definitions can be used in the code to identify the specific compiler that is compiling the file, thereby effectively distinguishing and identifying the compilation technologies used by different hardware.

[0095] In an embodiment of the present application, a method for obtaining the memory capacity of each minimum memory unit includes:

[0096] Based on the preset memory application capacity, a memory application request is generated; the memory application request is sent to the GPU twice in succession, and the memory addresses returned by the GPU are obtained twice; the memory addresses returned twice are subtracted to obtain the memory capacity of the minimum memory unit.

[0097] In an embodiment of the present application, the preset memory request capacity is a sufficiently small value, for example, 4kb; by sending memory request instructions for requesting a sufficiently small memory space to the GPU controller twice in succession, obtaining the memory addresses (or memory pointers) in the two instructions, and subtracting the two memory addresses, the memory capacity of the minimum memory unit of the GPU can be obtained.

[0098] In the embodiment of the present application, the matrix characteristics of each sub-block include the number of non-zero elements, the proportion of non-zero elements in each row, and the arrangement structure of non-zero elements.

[0099] In the embodiments of the present application, Figure 5As shown, a data storage format selection model is pre-constructed, and the data storage format selection model is trained using a data set constructed by multiple different historical sub-block matrices and the optimal data storage formats of the corresponding historical sub-block matrices, so as to obtain a trained data storage format selection model and obtain the optimal data storage format corresponding to the sub-block matrix, thereby achieving the optimal calculation effect for each sub-block.

[0100] In an embodiment of the present application, a method for determining the optimal data storage format of each corresponding historical sub-block matrix includes: storing each historical sub-block matrix in a different data storage format, and then performing relevant calculations to obtain the optimal data storage format of each corresponding historical sub-block matrix.

[0101] In an embodiment of the present application, different network models are used to construct a storage format selection model, and different storage format selection models are trained and cross-validated using the same data set, and the model with the highest accuracy is selected as the final data storage format selection model; the data storage format selection model is a storage format selection model constructed based on a decision tree classifier (DTC).

[0102] In the embodiment of the present application, the optimal data storage format corresponding to the sub-block matrix is ​​the optimal matrix storage format of the horizontal coordinate and / or the vertical coordinate in the sub-block matrix. Specifically, the optimal data storage format includes: column compression storage (CSC), row compression storage (CSR), triple table, link storage, COO and diagonal storage format (DIA), etc.

[0103] In the embodiment of the present application, the storage format of the data in the sub-block matrix is ​​optimized according to the optimal data storage format to obtain an optimized sub-block matrix, including:

[0104] The storage format of each horizontal coordinate and each vertical coordinate in the sub-block matrix is ​​converted into an optimal data storage format to obtain a position quantity row of the sub-block matrix;

[0105] Extract the value of each element in the sub-block matrix as the element value row of the sub-block matrix;

[0106] If the memory capacity required by the data type of any position quantity in the position quantity row is greater than the memory capacity required by the data type of any element value in the element value row, the element value row is used as the first row of the optimized sub-block matrix, and the position quantity row is used as the second row of the optimized sub-block matrix;

[0107] If the memory capacity required for the data type of any position quantity in the position quantity row is smaller than the memory capacity required for the data type of any element value in the element value row, the element value row is used as the second row of the optimized sub-block matrix, and the position quantity row is used as the first row of the optimized sub-block matrix.

[0108] In the embodiment of the present application, the memory capacity required for the element value row is obtained based on the data type of the element value multiplied by the number of all element values ​​in the element value row.

[0109] For example, the sub-block matrix contains a total of 6 elements, namely A 11 , A 12 , A 13 , A 21 , A 22 and A 23 ; A mn =[value mn , x mn ,y mn ], where value mn Indicates A mn The value of x mn Indicates A mn The horizontal axis of y mn Indicates A mn The vertical coordinate of ; m and n are both positive integers.

[0110] If the data type of the horizontal axis is the same as the optimal data type of the horizontal axis, and the data type of the vertical axis is the same as the optimal data type of the vertical axis, and it is a triple table, then based on value 11 ,value 12 ,value 13 ,value 21 ,value 22 and value 23 , get the element value row of the sub-block matrix [value 11 ,value 13 ,value 21 ,value 12 ,value 23 ,value 22 ]; and obtain the horizontal and vertical coordinate rows of the sub-block matrix respectively; based on the data type of the element value, the memory capacity required for one element data type of the element value row is 64 bits, similarly, the memory capacity required for one element data type of the horizontal coordinate row is 32 bits, and the memory capacity required for one element data type of the vertical coordinate row is 32 bits. Then the horizontal coordinate row is used as the first row of the optimized sub-block matrix, the vertical coordinate row is used as the second row of the optimized sub-block matrix, and the element value row is used as the third row of the optimized sub-block matrix.

[0111] In the embodiment of the present application, the optimized sub-block matrix includes an element value row and a position quantity row, and each row of the optimized sub-block matrix corresponds to an array; due to different optimal storage formats, the position quantity row may be the abscissa row and the ordinate row of the elements in the sub-block matrix, or may be the abscissa row, or may be the abscissa deviation row.

[0112] In the above embodiment, three arrays are obtained respectively, namely the array corresponding to the abscissa row (i.e., the first row), the array corresponding to the ordinate row (i.e., the second row), and the array corresponding to the element value row (i.e., the third row).

[0113] In the embodiment of the present application, the data types of the elements in each row of the optimized sub-block matrix are the same. Assuming that there are three arrays in the optimized sub-block matrix: array1, array2, and array3, and the types of the elements stored in each array are type1, type2, and type3 respectively, the conditions to be followed are: the sizes of each element type increase gradually, that is: sizeof(type1) < sizeof(type2) < sizeof(type3), which is a rule set for subsequent memory alignment (i.e., memory padding).

[0114] Step S420: For any optimized sub-block matrix, based on multiple minimum memory units included in the GPU, allocate the same target minimum memory unit for each array corresponding to the optimized sub-block matrix.

[0115] In the embodiment of the present application, the minimum memory unit is the minimum virtual memory interval when the GPU compilation software allocates memory; the target minimum memory unit is any one of the multiple minimum memory units or is composed of multiple consecutive minimum memory units.

[0116] In the prior art, applying for GPU memory usually involves separately executing the Malloc operation for each array in each sub-block of the matrix. This method allocates memory according to the size of the minimum memory unit. Since the arrays in the block matrix are usually small after passing through the adaptive block module, through calculation, it is found that the sizes of all the array data after blocking are usually smaller than the basic memory unit of the GPU. Therefore, the frequent Malloc operations of this method not only increase the execution time but also cause waste of memory space. In addition, the above method may also affect the locality of data. Although these arrays need to be accessed simultaneously, they may be located in different memory blocks, resulting in the need to access multiple blocks during the calculation process, thereby increasing the calculation latency.

[0117] During the calculation process, the embodiment of the present application discovered that for the data stored in each sub-block matrix, only one GPU minimum memory unit or multiple continuous minimum memory units need to be applied for. Therefore, in order to solve the problems existing in the above-mentioned prior art, the present application applies to the GPU for each sub-block matrix to form a target minimum content unit to accommodate all arrays of each sub-block.

[0118] Step S430: for any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array; store each array to the corresponding target storage address to obtain the optimized GPU memory.

[0119] In an embodiment of the present application, calculating the memory start address and the memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array includes:

[0120] If the array is the array corresponding to the first row of the optimized sub-block matrix, the memory first address of the corresponding target minimum memory unit is used as the memory start address of the array;

[0121] The sum of the memory first address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the memory end address of the array; wherein the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rule;

[0122] If the array is not the array corresponding to the first row of the optimized sub-block matrix, the memory end address corresponding to the previous row of the corresponding row of the array is used as the memory start address of the array;

[0123] The sum of the memory start address, the memory capacity required for the corresponding row of the array, and the memory padding amount of the data is used as the memory end address of the array.

[0124] In an embodiment of the present application, the amount of memory padding for any array is determined based on the data type of the data in the array and the GPU alignment rules; specifically, it may include: determining the number of alignment bytes A according to the data type of the data in the array, and if the sum of the memory start address and the memory capacity required for the corresponding row of the array is not a multiple of A, sufficient bytes need to be supplemented so that the memory end address of the array is a multiple of A.

[0125] In an embodiment of the present application, the memory starting address of the array corresponding to the first row is the memory first address of the minimum memory unit; the memory starting address of the array corresponding to the second row and / or the third row is the memory ending address of the first row and / or the second row.

[0126] In the embodiment of the present application, assuming that the length of each array is len1, len2, and len3 respectively, the padding (i.e., filling) of the current array is calculated based on the latter array to ensure that the data is aligned in the memory according to the size of the largest element type.

[0127] The embodiment of the present application divides a minimum memory unit of the GPU into multiple sub-memory units through the above method, and each sub-memory unit stores a corresponding array, which not only ensures that all data in the sub-block are stored continuously in the memory, thereby improving the locality of the data; at the same time, it reduces the need for multiple memory applications (Malloc), thereby reducing memory fragmentation and improving memory utilization efficiency; it not only reduces the problem of accessing multiple memory blocks in calculations, but also reduces the generation of memory fragmentation, greatly improving computing efficiency and overall memory utilization efficiency.

[0128] In an embodiment of the present application, each array is stored to a corresponding target storage address, including: an adaptive memory copy function uses Memcpy (memory copy function) to copy data from the host memory to the optimized GPU memory with the help of the memory start address, memory end address and sub-block data stored in the host memory of the memory space corresponding to each array; the whole process ensures the correct layout and alignment of the data structure on the GPU to optimize access speed and processing efficiency.

[0129] In an embodiment of the present application, the optimized GPU memory is obtained, that is, the matrix data after memory optimization based on the GPU and the target matrix characteristics is obtained; after obtaining the optimized GPU memory, the method also includes: designing a corresponding matrix calculation kernel for the memory-optimized matrix data to implement the memory-optimized matrix calculation function.

[0130] In an embodiment of the present application, a memory-optimized matrix calculation function is implemented based on a matrix calculation kernel, including: using a function call interface to call related functions, calculating the data stored in each minimum memory unit of the optimized GPU memory, and returning the calculation results to the GPU host. The interface module of the host receives the data and returns the correct results through the interface, completing a function call and calculation.

[0131] The embodiment of the present application can effectively improve the computational efficiency of the function by optimizing the memory of the GPU.

[0132] Corresponding to the above method, the embodiment of the present application also provides a GPU adaptive memory optimization device, such as Figure 6 As shown, the GPU adaptive memory optimization device includes:

[0133] An acquisition unit 610 is used to acquire a target sparse matrix; wherein each element in the target sparse matrix includes the value of the element and the horizontal coordinate and the vertical coordinate of the element in the target sparse matrix;

[0134] The optimization unit 620 is used to optimize the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix includes at least two arrays;

[0135] The allocation unit 630 is used to allocate the same target minimum memory unit to each array corresponding to the optimized sub-block matrix based on multiple minimum memory units included in the GPU for any optimized sub-block matrix; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0136] A calculation unit 640 is used to calculate the memory start address and the memory end address of any array of any optimized sub-block matrix in the corresponding target minimum memory unit to obtain the target storage address of the array;

[0137] The storage unit 650 is used to store each array to a corresponding target storage address to obtain an optimized GPU memory.

[0138] The functions of each functional unit of the GPU adaptive memory optimization device provided in the above embodiments of the present application can be achieved through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the GPU adaptive memory optimization device provided in the embodiments of the present application will not be repeated here.

[0139] The present application also provides an electronic device, such as Figure 7 As shown, it includes a processor 710 , a communication interface 720 , a memory 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 , and the memory 730 communicate with each other via the communication bus 740 .

[0140] Memory 730, for storing computer programs;

[0141] The processor 710 is used to execute the program stored in the memory 730 to implement the following steps:

[0142] Obtain a target sparse matrix; wherein each element in the target sparse matrix includes the value of the element and the horizontal coordinate and vertical coordinate of the element in the target sparse matrix;

[0143] Optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays;

[0144] For any optimized sub-block matrix, based on multiple minimum memory units contained in the GPU, the same target minimum memory unit is allocated to each array corresponding to the optimized sub-block matrix; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0145] For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array;

[0146] Store each array to the corresponding target storage address to obtain the optimized GPU memory.

[0147] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0148] The communication interface is used for communication between the above electronic device and other devices.

[0149] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0150] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0151] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 4The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0152] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes any one of the GPU adaptive memory optimization methods in the above embodiments.

[0153] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the GPU adaptive memory optimization methods in the above embodiments.

[0154] Those skilled in the art will appreciate that the embodiments in the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt a complete hardware embodiment, a complete software embodiment, or a form of an embodiment combining software and hardware. Moreover, the present application may adopt a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0159] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application fall within the scope of the claims and their equivalents in the embodiments of the present application, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A GPU adaptive memory optimization method, characterized in that: The method comprises: Obtain a target sparse matrix; wherein each element in the target sparse matrix includes the value of the element and the horizontal coordinate and the vertical coordinate of the element in the target sparse matrix; Optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix includes at least two arrays; For any optimized sub-block matrix, based on multiple minimum memory units included in the GPU, allocating the same target minimum memory unit to each array corresponding to the optimized sub-block matrix; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory; For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array; Store each array to the corresponding target storage address to obtain the optimized GPU memory.

2. The method according to claim 1, characterized in that The target sparse matrix is ​​optimized to obtain a plurality of optimized sub-block matrices, including: Based on the number of threads included in the GPU thread bundle and the capacity of the minimum memory unit, the target sparse matrix is ​​divided into blocks to obtain a plurality of sub-block matrices; wherein the number of rows and the number of columns of any sub-block matrix are equal to the number of threads; Extracting matrix features of each sub-block matrix; For any sub-block matrix, the matrix features of the sub-block matrix are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein the data storage format selection model is obtained by training based on multiple different historical sub-block matrices and the optimal data storage formats of the corresponding historical sub-block matrices; According to the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain an optimized sub-block matrix.

3. The method according to claim 2, characterized in that According to the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain an optimized sub-block matrix, including: Convert the storage format of each horizontal coordinate and each vertical coordinate in the sub-block matrix into the optimal data storage format to obtain a position quantity row of the sub-block matrix; Extracting the value of each element in the sub-block matrix as an element value row of the sub-block matrix; If the memory capacity required by the data type of any position quantity in the position quantity row is greater than the memory capacity required by the data type of any element value in the element value row, the element value row is used as the first row of the optimized sub-block matrix, and the position quantity row is used as the second row of the optimized sub-block matrix; If the memory capacity required for the data type of any position quantity in the position quantity row is smaller than the memory capacity required for the data type of any element value in the element value row, the element value row is used as the second row of the optimized sub-block matrix, and the position quantity row is used as the first row of the optimized sub-block matrix.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining GPU configuration information; wherein the GPU configuration information includes: the memory first address of the target minimum memory unit and the GPU alignment rule; Calculating the memory start address and the memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array includes: If the array is the array corresponding to the first row of the optimized sub-block matrix, the memory first address of the corresponding target minimum memory unit is used as the memory start address of the array; The sum of the memory first address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the memory end address of the array; wherein the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rule; If the array is not the array corresponding to the first row of the optimized sub-block matrix, the memory end address corresponding to the previous row of the corresponding row of the array is used as the memory start address of the array; The sum of the memory start address, the memory capacity required for the corresponding row of the array and the memory padding amount of the array is used as the memory end address of the array.

5. The method according to claim 1, characterized in that The target minimum memory unit is any one of a plurality of minimum memory units, or a plurality of continuous minimum memory units.

6. The method according to claim 2, characterized in that in, The configuration information of the GPU also includes: the memory capacity of the minimum memory unit; After obtaining a plurality of sub-block matrices, the method further includes: Calculate the memory capacity occupied by each sub-block matrix; From the obtained multiple sub-block matrices, select the sub-block matrix that occupies the largest memory capacity as the target sub-block matrix; Multiplying the memory capacity of the target minimum memory unit by a preset memory capacity threshold to obtain a memory capacity block threshold; If the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, the step of extracting matrix features of each sub-block matrix is ​​performed.

7. The method according to claim 6, characterized in that The method further comprises: If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity block threshold, dividing the number of threads by the preset thread threshold to obtain a block parameter; The target sparse matrix is ​​divided into blocks using the block parameters as the number of rows and columns of each sub-block matrix to obtain multiple sub-block matrices, and the execution step is returned to: the memory capacity occupied by each sub-block matrix is ​​calculated until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity block threshold.

8. A GPU adaptive memory optimization device, characterized in that: The device comprises: An acquisition unit, configured to acquire a target sparse matrix; wherein each element in the target sparse matrix includes a value of the element and a horizontal coordinate and a vertical coordinate of the element in the target sparse matrix; An optimization unit, used for optimizing the target sparse matrix to obtain a plurality of optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays; an allocation unit, for allocating the same target minimum memory unit to each array corresponding to any optimized sub-block matrix based on multiple minimum memory units included in the GPU; wherein the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory; A calculation unit, used for calculating, for any array of any optimized sub-block matrix, a memory start address and a memory end address of the array in a corresponding target minimum memory unit, to obtain a target storage address of the array; The storage unit is used to store each array to a corresponding target storage address to obtain an optimized GPU memory.

9. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images