A GPU adaptive memory optimization method, apparatus, device, and medium

Abstract

This application provides a GPU adaptive memory optimization method, apparatus, device, and medium, relating to the field of computer architecture technology. The method includes: obtaining a target sparse matrix; optimizing the target sparse matrix to obtain multiple optimized sub-block matrices; for any optimized sub-block matrix, allocating the same target minimum memory unit to each array corresponding to the optimized sub-block matrix based on the multiple minimum memory units contained in the GPU; for any array of any optimized sub-block matrix, calculating the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array; and storing each array to the corresponding target storage address to obtain the optimized GPU memory. This application improves the efficiency of GPU memory utilization, saves GPU memory, improves data locality during computation, and accelerates the speed of functions based on block matrices.

Description

Technical Field

[0001] This application relates to the field of computer architecture technology, and more specifically, to a GPU adaptive memory optimization method, apparatus, device, and medium. Background Technology

[0002] Sparse matrix-vector multiplication (SpMV) and sparse matrix multiplication (SpMM) are important mathematical operations widely used in computer science and engineering. In these two operations, the matrix is ​​represented as a sparse matrix, meaning that most of its elements are zero. This characteristic of sparse matrices allows for efficient saving of memory and computational resources when processing large-scale data. However, due to the irregularity of SpMV and SpMM operations and the limited bandwidth of associated memory, implementing faster parallel SpMV and SpMM operations on GPUs is very challenging. Current techniques mainly accelerate SpMV and SpMM computations by processing sparse matrices in blocks.

[0003] While block-based processing of sparse matrices offers advantages such as mitigating the data limitations of dense vectors, facilitating load balancing, and providing greater simplicity and flexibility, it also presents the following challenges:

[0004] First, dividing the sparse matrix into multiple smaller blocks allows for more granular task allocation and facilitates load balancing, but it increases communication overhead.

[0005] Secondly, the sparse data structures within sub-blocks of functions like SpMV and SpMM typically consist of multiple arrays. Existing techniques allocate each array separately using Malloc. However, since sub-blocks are usually small and sparse matrices, the memory required for each array is actually very small, much smaller than the allocated memory unit. This results in huge memory fragmentation within the generated GPU memory units, greatly wasting the GPU's valuable memory resources and significantly reducing the data locality within the sub-block.

[0006] Finally, the space wastage caused by different brands of GPUs also varies. For example, on Nvidia GPUs, memory is allocated in 512-byte units each time a memory request is made, while on domestic DCUs, memory is allocated in 4-byte units each time a memory request is made. Under the block mechanism, the array size corresponding to the data structure in each block is much smaller than the aforementioned basic memory unit. This leads to significant performance differences on different brands of GPUs when applying the block method to matrix data. Summary of the Invention

[0007] The purpose of this application is to provide a GPU adaptive memory optimization method, apparatus, device and medium to solve the above-mentioned problems existing in the prior art, realize GPU load balancing, maximize the parallelism of computation while avoiding memory space waste, and improve the relevant computation performance based on block matrix on GPU.

[0008] Firstly, a GPU adaptive memory optimization method is provided, which may include:

[0009] Obtain the target sparse matrix; wherein each element in the target sparse matrix contains the value of the element and the x-coordinate and y-coordinate of the element in the target sparse matrix;

[0010] The target sparse matrix is ​​optimized to obtain multiple optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays.

[0011] For any optimized sub-block matrix, based on the multiple minimum memory units contained in the GPU, the same target minimum memory unit is allocated to each array corresponding to the optimized sub-block matrix; wherein, the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0012] For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array;

[0013] Each array is stored in its corresponding target memory address to obtain the optimized GPU memory.

[0014] In an optional implementation, the target sparse matrix is ​​optimized to obtain multiple optimized sub-block matrices, including:

[0015] Based on the number of threads contained in the GPU thread bundle and the capacity of the smallest memory unit, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices; wherein the number of rows and columns of any sub-block matrix is ​​equal to the number of threads;

[0016] Extract the matrix features of each sub-block matrix;

[0017] For any sub-block matrix, the matrix features of the sub-block matrix are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein, the data storage format selection model is trained based on multiple different historical sub-block matrices and the optimal data storage format of the corresponding historical sub-block matrices;

[0018] Based on the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain the optimized sub-block matrix.

[0019] In an optional implementation, the storage format of the data in the sub-block matrix is ​​optimized according to the optimal data storage format to obtain an optimized sub-block matrix, including:

[0020] The storage format of each x-coordinate and each y-coordinate in the sub-block matrix is ​​converted into the optimal data storage format to obtain the position data row of the sub-block matrix;

[0021] Extract the values ​​of each element in the sub-block matrix and use them as the element value rows of the sub-block matrix;

[0022] If the memory required for the data type of any position quantity in the position quantity row is greater than the memory required for the data type of any element value in the element value row, then the element value row is taken as the first row of the optimized sub-block matrix, and the position quantity row is taken as the second row of the optimized sub-block matrix.

[0023] If the memory required for the data type of any position quantity in the position quantity row is less than the memory required for the data type of any element value in the element value row, then the element value row is used as the second row of the optimized sub-block matrix, and the position quantity row is used as the first row of the optimized sub-block matrix.

[0024] In an optional implementation, the method further includes:

[0025] Obtain GPU configuration information; wherein, the GPU configuration information includes: the memory starting address of the target minimum memory unit and the GPU alignment rules;

[0026] Calculate the starting and ending addresses of the array in the corresponding target minimum memory unit to obtain the target storage address of the array, including:

[0027] If the array is the array corresponding to the first row of the optimized sub-block matrix, then the memory starting address of the corresponding target minimum memory unit is used as the memory starting address of the array;

[0028] The sum of the starting memory address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the memory termination address of the array; wherein, the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rules;

[0029] If the array is not the array corresponding to the first row of the optimized sub-block matrix, then the memory termination address corresponding to the row above the corresponding row of the array is used as the memory start address of the array.

[0030] The sum of the starting memory address, the memory capacity required for the corresponding row of the array, and the memory padding amount of the array is used as the ending memory address of the array.

[0031] In an optional implementation, the target minimum memory unit is any one of a plurality of minimum memory units, or a plurality of consecutive minimum memory units.

[0032] In an optional implementation, the configuration information of the GPU further includes: the memory capacity of the target minimum memory unit;

[0033] After obtaining multiple sub-block matrices, the method further includes:

[0034] Calculate the memory capacity occupied by each sub-block matrix;

[0035] From the multiple sub-block matrices obtained, select the sub-block matrix with the largest memory usage as the target sub-block matrix;

[0036] Multiply the memory capacity of the target minimum memory unit by a preset memory capacity threshold to obtain the memory capacity block threshold;

[0037] If the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, then the following step is executed: extract the matrix features of each sub-block matrix.

[0038] In an optional implementation, the method further includes:

[0039] If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity block threshold, then the number of threads is divided by the preset thread threshold to obtain the block parameters;

[0040] Using the block parameters as the number of rows and columns of each sub-block matrix, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices. The execution steps are then returned: calculate the memory capacity occupied by each sub-block matrix until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity block threshold.

[0041] Secondly, a GPU adaptive memory optimization device is provided, which may include:

[0042] An acquisition unit is used to acquire a target sparse matrix; wherein each element in the target sparse matrix contains the value of the element and the x-coordinate and y-coordinate of the element in the target sparse matrix;

[0043] An optimization unit is used to optimize the target sparse matrix to obtain multiple optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays;

[0044] An allocation unit is used to allocate the same target minimum memory unit to each array corresponding to the optimized sub-block matrix based on multiple minimum memory units contained in the GPU, for any optimized sub-block matrix; wherein, the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0045] The calculation unit is used to calculate the memory start address and memory end address of any array in the corresponding target minimum memory unit for any optimized sub-block matrix, so as to obtain the target storage address of the array;

[0046] The storage unit is used to store each array to the corresponding target storage address to obtain the optimized GPU memory.

[0047] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0048] Memory, used to store computer programs;

[0049] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0050] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.

[0051] This application provides an adaptive GPU-aware function that masks the hardware differences of various types of GPUs. This enables the use of matrix-based block-based functions, such as SpMV or SpMM, to automatically adapt to the programming languages ​​and hardware characteristics of different types of GPUs. In turn, it obtains relevant thread configuration information and the actual size of Malloc memory blocks under different types of GPUs, providing information for subsequent optimization steps.

[0052] This application determines the size of sub-block matrices after partitioning a sparse matrix based on the number of threads. This fully utilizes the SIMD computing resources of different types of GPUs, finds the optimal point for load balancing, maximizes computational parallelism, and accelerates computation. Furthermore, this application employs a data storage format selection model to choose the optimal data storage format for each sub-block matrix, maximizing the computational performance of each sub-block matrix. This application can dynamically adjust the sub-block size and data storage format, achieving better parallel processing efficiency and memory utilization efficiency.

[0053] This application allocates a minimum memory unit for each sub-block matrix, and all arrays of each sub-block matrix are stored in the same minimum memory unit, realizing unified space management of memory corresponding to different arrays of the same sub-block matrix and optimizing GPU memory units. At the same time, sub-block arrays that originally required multiple memory allocations are allocated in a single allocation, which greatly reduces the generation of memory fragmentation, makes fuller use of memory resources, saves GPU memory, improves data locality during computation, and speeds up the computation of functions based on block matrices.

[0054] This application improves data locality in the GPU by tightly storing all sub-block matrix data, thereby reducing access latency and accelerating data processing speed. This application greatly improves the memory utilization efficiency and computing speed of the GPU. This application can eliminate the hardware differences between different types of GPUs, so that the latest matrix block-based computing functions, such as SpMV or SpMM, can be optimized to different degrees on different types of GPUs, allowing each GPU to exert its maximum performance.

[0055] This application can identify and adapt to the characteristics of GPUs from different manufacturers, such as the different sizes of basic memory allocation units, and optimize cross-platform computing and memory management strategies. This application enables GPUs to perform sparse matrix calculations more efficiently, reduce ineffective computation and memory resource waste, and significantly improve the computational performance of functions such as SpMV and SpMM based on block mechanisms. It also provides a flexible and effective solution for optimization in different GPU environments, and can achieve better performance and memory management on different GPUs, thereby expanding its application scope in industrial and scientific research fields. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A GPU adaptive memory optimization control system architecture diagram is provided for embodiments of this application;

[0058] Figure 2 An architecture diagram of an adaptive matrix partitioning module provided in an embodiment of this application;

[0059] Figure 3 An architecture diagram of an adaptive memory optimization module provided in an embodiment of this application;

[0060] Figure 4 A flowchart illustrating a GPU adaptive memory optimization method provided in this application embodiment;

[0061] Figure 5 This is a schematic diagram of a data storage format selection model training method provided in an embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the structure of a GPU adaptive memory optimization device provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0065] The GPU adaptive memory optimization method provided in this application embodiment can be applied to, for example, Figure 1 The GPU adaptive memory optimization control system architecture shown can include: a hardware layer, a preprocessing layer (i.e., an adaptive memory awareness module), a sparse matrix structure layer (i.e., an adaptive matrix partitioning module), a sub-block internal structure layer (i.e., an adaptive memory optimization module), a sub-block internal algorithm layer (i.e., an algorithm module), an outer algorithm layer, and an external function interface layer.

[0066] The preprocessing layer is used to obtain the configuration information of the graphics processing unit (GPU).

[0067] Sparse matrix structure layers, such as Figure 2 As shown, this is used to divide the target sparse matrix into multiple sub-block matrices based on the number of GPU threads;

[0068] Sub-block internal structure layers, such as Figure 3As shown, the optimal data storage format for each sub-block matrix is ​​determined, and each sub-block matrix is ​​optimized based on the optimal data storage format to obtain the optimized sub-block matrix; a minimum memory allocation unit is allocated for each sub-block matrix; at the same time, the target storage address of any array of any sub-block matrix on the minimum memory allocation unit is calculated, and each array is stored to the corresponding target storage address;

[0069] The sub-block internal algorithm layer is used to call the reconstructed internal sub-block related kernel functions to perform calculations on the data stored in each smallest memory unit of the GPU and send it back to the host (i.e., the general-purpose processor to which the GPU is linked).

[0070] The outer algorithm layer and the external function interface layer are used to receive data passed from the internal algorithm layer of the sub-block and return the correct result through the interface, thus completing a function call and calculation.

[0071] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0072] Figure 4 This is a flowchart illustrating a GPU adaptive memory optimization method provided in an embodiment of this application. Figure 4 As shown, the method may include:

[0073] Step S410: Obtain the target sparse matrix; optimize the target sparse matrix to obtain multiple optimized sub-block matrices.

[0074] In this embodiment, the target sparse matrix, i.e., each element of the target sparse matrix, contains the element's value and its x and y coordinates within the target sparse matrix; the number of rows and columns of the target sparse matrix are the same. For example, element A in the first row and first column of the target sparse matrix A... 11 =[value 11 x 11 y 11 ], where value 11 A represents 11 The value of x 11 A represents 11 x-coordinate; y 11 A represents 11 The ordinate.

[0075] In this embodiment, the GPU includes a controller, multiple computing units (the GPU's smallest parallel computing cores), function interface units, thread bundles, and multiple smallest memory units. Before optimizing the target sparse matrix to obtain multiple optimized sub-block matrices, the GPU adaptive memory optimization method also needs to obtain the GPU's configuration information. This configuration information includes: the GPU type, the GPU alignment rules, the compiler type of the hardware device hosting the GPU, the memory capacity of each smallest memory unit, the memory start address and memory end address, and the number of threads contained in the thread bundle. Different types of GPUs may have different configuration information.

[0076] In this embodiment of the application, the target sparse matrix is ​​optimized to obtain multiple optimized sub-block matrices, including:

[0077] Based on the number of threads in a GPU thread bundle and the capacity of the smallest memory unit, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices. The matrix features of each sub-block matrix are extracted. For any sub-block matrix, the matrix features are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix. The data storage format selection model is trained based on multiple different historical sub-block matrices and the optimal data storage format of the corresponding historical sub-block matrices. Based on the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain the optimized sub-block matrix.

[0078] In this embodiment, the number of rows and columns of any sub-block matrix obtained by dividing the target sparse matrix are equal; the target sparse matrix is ​​initially divided into blocks to obtain sub-block matrices after initial block division. The number of rows and columns of each sub-block matrix obtained by dividing the target sparse matrix are equal.

[0079] In this embodiment of the application, after obtaining multiple sub-block matrices, the method further includes: calculating the memory capacity occupied by each sub-block matrix; selecting the sub-block matrix with the largest memory capacity from the obtained multiple sub-block matrices as the target sub-block matrix; multiplying the memory capacity of the target smallest memory unit by a preset memory capacity threshold to obtain a memory capacity block threshold; if the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, then the step of: extracting the matrix features of each sub-block matrix is ​​performed.

[0080] If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity block threshold, the number of threads is divided by the preset thread threshold to obtain the block parameters; the block parameters are used as the number of rows and columns of each sub-block matrix to divide the target sparse matrix into multiple sub-block matrices, and the execution steps are returned: calculate the memory capacity occupied by each sub-block matrix until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity block threshold.

[0081] In this embodiment, the memory capacity occupied by each sub-block matrix is ​​obtained by multiplying the element type contained in each sub-block matrix by the number of elements of the corresponding type and then summing the results.

[0082] In this embodiment, the memory capacity threshold is preset and ranges from 100% to 180%. When the memory capacity threshold is set to 180%, it means that when the memory capacity occupied by the target sub-block matrix does not exceed 180% of the memory capacity of the target minimum memory unit, the target sub-block matrix can be accommodated by two consecutive minimum memory units. That is, the memory capacity of the target minimum memory unit is the memory capacity of two minimum memory units. The block division result at this time is the final block division result, and the next step can be performed: extracting the matrix features of each sub-block matrix.

[0083] In this embodiment of the application, the thread threshold is preset, and its value can be 2.

[0084] In the embodiments of this application, the memory capacity of the target minimum memory unit is determined by the number of minimum memory units constituting the target minimum memory unit and the memory capacity of the minimum memory unit; when the target minimum memory unit is any one of a plurality of minimum memory units, the memory capacity of the target minimum memory unit is the memory capacity of the minimum memory unit; when the target minimum memory unit is a plurality of consecutive minimum memory units, the memory capacity of the target minimum memory unit is determined based on the number of minimum memory units and the memory capacity.

[0085] For example, if the number of threads is 32 and the thread threshold is 2, the target sparse matrix is ​​divided into multiple 32×32 sub-block matrices. The sub-block matrix with the largest memory usage among all sub-block matrices is selected as the target sub-block matrix. It is then calculated whether the memory usage of this target sub-block matrix exceeds the memory capacity block threshold. If it does, 32 / 2 = 16 is used as the number of rows and columns of the sub-block matrix, and the target sparse matrix is ​​re-divided into multiple 16×16 sub-block matrices. This process is then repeated to verify whether the memory usage of the newly obtained target sub-block matrix exceeds the memory capacity block threshold.

[0086] If the value does not exceed the limit, the current block division result will be used as the final block division result of the target sparse matrix, resulting in multiple 16×16 sub-block matrices.

[0087] If the number of threads exceeds the preset thread threshold, the number of threads is divided by the preset thread threshold to obtain an intermediate value. This intermediate value is then divided by the preset thread threshold again to obtain the block parameters. That is, if the target sub-block matrix with the largest memory capacity in the 16×16 sub-block matrix still cannot be accommodated by the target smallest memory unit, the number of threads is divided by the preset thread threshold twice to obtain the block parameters. That is, 32 / 2 / 2 = 8 is used as the number of rows and columns of the block matrix. The target sparse matrix is ​​re-blocked to obtain multiple 8×8 sub-block matrices. This process is repeated until the target block matrix can be accommodated by a single smallest memory unit.

[0088] In practical applications, the purpose of target sparse matrix partitioning is to have one computing unit be responsible for processing one partitioned matrix, and multiple threads within the computing unit execute the computation tasks of each partitioned matrix in parallel. In order to maximize the efficiency of parallel processing and achieve better load balancing, the key is to match the size of the sub-blocks (i.e., the number of rows and columns) with the number of threads within the computing unit in the GPU.

[0089] Therefore, in the initial segmentation of the target sparse matrix, the number of rows and columns of the sub-block matrix after the initial segmentation are made equal to the number of threads. However, since the memory capacity occupied by the sub-block matrix may not be accommodated by a single smallest memory unit and the number of threads in the computing units of different GPUs is different, fixing the size of the sub-block may affect the parallel efficiency of some computing units. Therefore, after obtaining the sub-block matrix after the initial segmentation, the memory capacity of the sub-block matrix is ​​compared. If the memory capacity occupied by the sub-block matrix with the largest memory capacity cannot be accommodated by the target smallest memory unit, the target sparse matrix is ​​re-segmented.

[0090] In another embodiment of this application, the target sparse matrix is ​​divided into blocks based on the number of threads contained in the GPU thread bundle and the capacity of the smallest memory unit. The steps are as follows: analyze the target sparse matrix; determine the block target based on the analysis results of the target sparse matrix and the capacity of the smallest memory unit; perform preliminary block division on the target sparse matrix based on the block target to obtain the preliminary block sub-block matrix; adjust the preliminary block sub-block matrix to obtain multiple sub-block matrices.

[0091] This application embodiment determines the partitioning parameters by using the number of GPU threads and the capacity of the smallest memory unit. It divides a target sparse matrix into multiple sub-block matrices and adjusts the partitioning parameters according to the memory capacity occupied by the partitioned sub-block matrices. This not only makes the size of the sub-block matrices after partitioning the target sparse matrix adapt to the thread configuration in the GPU, but also helps to more accurately partition the target sparse matrix, thereby improving the computational efficiency of each computing unit and making the load between the computing units more balanced.

[0092] In practical applications, the memory capacity of each minimum memory unit and the number of threads contained in the GPU thread bundle cannot be directly obtained. In this embodiment, a macro file is written in advance and the macro file is compiled by the device where the GPU is located, thereby obtaining the memory capacity of each minimum memory unit and the number of threads contained in the GPU thread bundle.

[0093] In this embodiment, memory query code and thread count query code are written using different hardware description languages. These codes are then embedded into corresponding macro conditional program code segments using macro definitions and conditional selection statements to obtain macro definition code fragments. During compilation, the device containing the GPU selects the corresponding macro definition code fragment for execution based on its compiler type and GPU type. By obtaining the execution feedback results, the memory capacity of the corresponding minimum memory unit and the number of threads contained in the GPU thread bundle can be obtained.

[0094] In practical applications, Nvidia GPUs typically use the nvcc compiler to compile CUDA source code, while AMD GPUs use the hipcc compiler to compile HIP source code. Therefore, this application will use hardware languages ​​that include both of these and other types of GPU-compilable languages ​​when writing memory query code and thread count query code. In addition, when compiling source code, macro definitions can be used in the code to identify the specific compiler that is compiling the file, thereby effectively distinguishing and identifying the compilation technology used by different hardware.

[0095] In this embodiment of the application, the method for obtaining the memory capacity of each smallest memory unit includes:

[0096] Based on the preset memory request capacity, a memory request is generated; the memory request is sent to the GPU twice in succession, and the memory addresses returned by the GPU are obtained twice; the memory addresses returned twice are subtracted to obtain the memory capacity of the smallest memory unit.

[0097] In this embodiment, the preset memory request capacity is a sufficiently small value, such as 4kb. By sending two consecutive memory request instructions to the GPU controller to request a sufficiently small memory space, the memory address (or memory pointer) in the two instructions is obtained, and the memory capacity of the smallest memory unit of the GPU is obtained by subtracting the two memory addresses.

[0098] In the embodiments of this application, the matrix features of each sub-block include the number of non-zero elements, the proportion of non-zero elements in each row, and the arrangement structure of non-zero elements.

[0099] In the embodiments of this application, such as Figure 5As shown, a data storage format selection model is pre-built, and the data storage format selection model is trained using a dataset constructed from multiple different historical sub-block matrices and the optimal data storage format of the corresponding historical sub-block matrices. The trained data storage format selection model is obtained, and the optimal data storage format corresponding to the sub-block matrix is ​​obtained, so that the computation of each sub-block achieves the optimal effect.

[0100] In this embodiment of the application, the method for determining the optimal data storage format of each corresponding historical sub-block matrix includes: storing each historical sub-block matrix using different data storage formats, and then performing relevant calculations to obtain the optimal data storage format of each corresponding historical sub-block matrix.

[0101] In this embodiment of the application, different network models are used to construct storage format selection models. The same dataset is used to train and cross-validate the different storage format selection models, and the model with the highest accuracy is selected as the final data storage format selection model. The data storage format selection model is a storage format selection model based on a decision tree classifier (DTC).

[0102] In this embodiment, the optimal data storage format corresponding to the sub-block matrix is ​​the optimal matrix storage format in which the x-coordinate and / or y-coordinate of the sub-block matrix are located. Specifically, the optimal data storage formats include: Column Compressed Storage (CSC), Row Compressed Storage (CSR), triplet table, linked storage, COO, and Diagonal Storage Format (DIA), etc.

[0103] In this embodiment of the application, the storage format of the data in the sub-block matrix is ​​optimized according to the optimal data storage format to obtain the optimized sub-block matrix, including:

[0104] Convert the storage format of each x-coordinate and y-coordinate in the sub-block matrix into the optimal data storage format to obtain the position data row of the sub-block matrix;

[0105] Extract the values ​​of each element in the sub-block matrix and use them as the element value rows of the sub-block matrix;

[0106] If the memory required for the data type of any position quantity in the position quantity row is greater than the memory required for the data type of any element value in the element value row, then the element value row will be used as the first row of the optimized sub-block matrix, and the position quantity row will be used as the second row of the optimized sub-block matrix.

[0107] If the memory required for the data type of any positional quantity in the positional quantity row is less than the memory required for the data type of any element value in the element value row, then the element value row will be used as the second row of the optimized sub-block matrix, and the positional quantity row will be used as the first row of the optimized sub-block matrix.

[0108] In this embodiment, the memory capacity required for the element value row is obtained by multiplying the data type of the element value by the number of all element values ​​in the element value row.

[0109] For example, the sub-block matrix contains a total of 6 elements, namely A 11 A 12 A 13 A 21 A 22 and A 23 A mn =[value mn x mn y mn ], where value mn A represents mn The value of x mn A represents mn x-coordinate; y mn A represents mn The ordinate of the coordinate; m and n are both positive integers.

[0110] If the data type of the x-axis is the same as the optimal data type for the x-axis, and the data type of the y-axis is the same as the optimal data type for the y-axis, and the table is a triplet table, then based on the value... 11 value 12 value 13 value 21 value 22 and value 23 This yields the element value row [value] of the sub-block matrix. 11 value 13 value 21 value 12 value 23 value 22 The x-coordinate rows and y-coordinate rows of the sub-block matrix are obtained respectively. Based on the data type of the element values, the memory required to obtain one element data type in the element value row is 64 bits. Similarly, the memory required to obtain one element data type in the x-coordinate row is 32 bits, and the memory required to obtain one element data type in the y-coordinate row is 32 bits. Therefore, the x-coordinate rows are used as the first row of the optimized sub-block matrix, the y-coordinate rows as the second row, and the element value rows as the third row.

[0111] In the embodiment of the present application, the optimized sub-block matrix includes an element value row and a position quantity row, and each row of the optimized sub-block matrix corresponds to an array; due to different optimal storage formats, the position quantity row may be the abscissa row and the ordinate row of the element in the sub-block matrix, or may be the abscissa row, or may be the abscissa deviation quantity row.

[0112] In the above embodiment, three arrays are obtained respectively, namely the array corresponding to the abscissa row (i.e., the first row), the array corresponding to the ordinate row (i.e., the second row), and the array corresponding to the element value row (i.e., the third row).

[0113] In the embodiment of the present application, the data types of the elements in each row of the optimized sub-block matrix are the same. Assume that there are three arrays in the optimized sub-block matrix: array1, array2, and array3, and the types of the elements stored in each array are type1, type2, and type3 respectively. The conditions to be followed are: the sizes of each element type increase gradually, that is: sizeof(type1) < sizeof(type2) < sizeof(type3). This is a rule set for subsequent memory alignment (i.e., memory padding).

[0114] Step S420: For any optimized sub-block matrix, based on multiple minimum memory units included in the GPU, allocate the same target minimum memory unit to each array corresponding to the optimized sub-block matrix.

[0115] In the embodiment of the present application, the minimum memory unit is the minimum virtual memory interval when the GPU compilation software allocates memory; the target minimum memory unit is any one of the multiple minimum memory units or is composed of multiple consecutive minimum memory units.

[0116] In the prior art, applying for GPU memory usually involves performing a Malloc operation separately for each array in each sub-block of the matrix. This method allocates memory according to the size of the minimum memory unit. Since after passing through the adaptive block module, the arrays in the block matrix are usually small, through calculation, it is found that the sizes of all the array data after block division are usually smaller than the basic memory unit of the GPU. Therefore, the frequent Malloc operations of this method not only increase the execution time but also cause waste of memory space. In addition, the above method may also affect the locality of data. Although these arrays need to be accessed simultaneously, they may be located in different memory blocks, resulting in the necessity to access multiple blocks during the calculation process, thereby increasing the calculation latency.

[0117] In the calculation process, this application found that for the data stored in each sub-block matrix, only one GPU minimum memory unit or multiple consecutive minimum memory units need to be requested. Therefore, in order to solve the problems existing in the prior art, this application requests a consecutive minimum memory unit from the GPU for each sub-block matrix to form the target minimum content unit, so as to accommodate all the arrays of each sub-block.

[0118] Step S430: For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array; store each array to the corresponding target storage address to obtain the optimized GPU memory.

[0119] In this embodiment of the application, the starting address and ending address of the array in the corresponding target minimum memory unit are calculated to obtain the target storage address of the array, including:

[0120] If the array is the array corresponding to the first row of the optimized sub-block matrix, then the memory starting address of the corresponding target minimum memory unit is used as the memory starting address of the array;

[0121] The sum of the starting memory address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the ending memory address of the array; where the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rules;

[0122] If the array is not the array corresponding to the first row of the optimized sub-block matrix, then the memory termination address of the row above the corresponding row of the array is used as the memory start address of the array.

[0123] The sum of the starting memory address, the memory capacity required for the corresponding row of the array, and the amount of memory padding for the data is used as the ending memory address of the array.

[0124] In the embodiments of this application, the amount of memory padding for any array is determined based on the data type of the data in the array and the GPU alignment rules; specifically, it may include: determining the number of alignment bytes A according to the data type of the data in the array; if the sum of the memory start address and the memory capacity required for the corresponding row of the array is not a multiple of A, then sufficient bytes need to be added so that the memory end address of the array is a multiple of A.

[0125] In this embodiment, the memory starting address of the array corresponding to the first row is the memory starting address of the smallest memory unit; the memory starting address of the array corresponding to the second row and / or the third row is the memory ending address of the first row and / or the second row.

[0126] In the embodiments of this application, assuming that the lengths of each array are len1, len2, and len3 respectively, the padding of the current array is calculated based on the next array to ensure that the data is aligned in memory according to the size of the largest element type.

[0127] This application embodiment divides a minimum memory unit of the GPU into multiple sub-memory units using the above method. Each sub-memory unit stores a corresponding array, which not only ensures that all data within the sub-block is stored contiguously in memory, thereby improving data locality, but also reduces the need for multiple memory allocations (Malloc), thereby reducing memory fragmentation and improving memory utilization efficiency. This not only reduces the problem of accessing multiple memory blocks during computation, but also reduces the generation of memory fragmentation, greatly improving computational efficiency and overall memory utilization efficiency.

[0128] In this embodiment of the application, storing each array to the corresponding target storage address includes: an adaptive memory copy function uses the memory start address, memory end address and sub-block data stored in the host memory of each array to copy the data from the host memory to the optimized GPU memory using Memcpy (memory copy function); the whole process ensures the correct layout and alignment of the data structure on the GPU to optimize access speed and processing efficiency.

[0129] In this embodiment of the application, the optimized GPU memory is obtained as the matrix data optimized by memory based on the GPU and the target matrix features. After obtaining the optimized GPU memory, the method further includes: designing a corresponding matrix calculation kernel for the memory-optimized matrix data and implementing the memory-optimized matrix calculation function.

[0130] In this embodiment of the application, a memory-optimized matrix calculation function is implemented based on the matrix calculation kernel, including: calling relevant functions using a function call interface, calculating the data stored in each smallest memory unit of the optimized GPU memory, and returning the calculation result to the host of the GPU. The host's interface module receives the data and returns the correct result through the interface, thus completing one function call and calculation.

[0131] This application embodiment optimizes GPU memory, which can effectively improve the computational efficiency of functions.

[0132] Corresponding to the above method, this application also provides a GPU adaptive memory optimization device, such as... Figure 6 As shown, the GPU adaptive memory optimization device includes:

[0133] The acquisition unit 610 is used to acquire the target sparse matrix; wherein each element in the target sparse matrix contains the value of the element and the x-coordinate and y-coordinate of the element in the target sparse matrix;

[0134] The optimization unit 620 is used to optimize the target sparse matrix to obtain multiple optimized sub-block matrices; wherein any optimized sub-block matrix contains at least two arrays;

[0135] Allocation unit 630 is used to allocate the same target minimum memory unit to each array corresponding to the optimized sub-block matrix based on the multiple minimum memory units contained in the GPU; wherein, the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory;

[0136] The calculation unit 640 is used to calculate the memory start address and memory end address of any array in the corresponding target minimum memory unit for any array of any optimized sub-block matrix, so as to obtain the target storage address of the array.

[0137] Storage unit 650 is used to store each array to the corresponding target storage address to obtain optimized GPU memory.

[0138] The functions of each functional unit of the GPU adaptive memory optimization device provided in the above embodiments of this application can be implemented through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the GPU adaptive memory optimization device provided in the embodiments of this application will not be repeated here.

[0139] This application also provides an electronic device, such as... Figure 7 As shown, it includes a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740.

[0140] Memory 730 is used to store computer programs;

[0141] When the processor 710 executes the program stored in the memory 730, it performs the following steps:

[0142] Obtain the target sparse matrix; where each element in the target sparse matrix contains the value of the element and the x and y coordinates of the element in the target sparse matrix;

[0143] The target sparse matrix is ​​optimized to obtain multiple optimized sub-block matrices; each optimized sub-block matrix contains at least two arrays.

[0144] For any optimized sub-block matrix, based on the multiple minimum memory units contained in the GPU, the same target minimum memory unit is allocated to each array corresponding to the optimized sub-block matrix; where the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory.

[0145] For any array of any optimized sub-block matrix, calculate the starting and ending addresses of the array in the corresponding target minimum memory unit to obtain the target storage address of the array;

[0146] Each array is stored in its corresponding target memory address to obtain the optimized GPU memory.

[0147] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0148] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0149] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0150] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0151] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 4The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0152] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when run on a computer, cause the computer to execute any of the GPU adaptive memory optimization methods in the above embodiments.

[0153] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the GPU adaptive memory optimization methods in the above embodiments.

[0154] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0159] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims in this application and their equivalents, then this application also intends to include these modifications and variations.

Claims

1. A GPU adaptive memory optimization method, characterized in that, The method includes: Obtain the target sparse matrix; wherein each element in the target sparse matrix contains the value of the element and the x-coordinate and y-coordinate of the element in the target sparse matrix; Based on the number of threads contained in the GPU thread bundle and the capacity of the smallest memory unit, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices; wherein the number of rows and columns of any sub-block matrix is ​​equal to the number of threads. Extract the matrix features of each sub-block matrix; For any sub-block matrix, the matrix features of the sub-block matrix are input into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein, the data storage format selection model is trained based on multiple different historical sub-block matrices and the optimal data storage format of the corresponding historical sub-block matrices; Based on the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain an optimized sub-block matrix; wherein any optimized sub-block matrix contains at least two arrays; For any optimized sub-block matrix, based on the multiple minimum memory units contained in the GPU, the same target minimum memory unit is allocated to each array corresponding to the optimized sub-block matrix; wherein, the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory; For any array of any optimized sub-block matrix, calculate the memory start address and memory end address of the array in the corresponding target minimum memory unit to obtain the target storage address of the array; Each array is stored in its corresponding target memory address to obtain the optimized GPU memory.

2. The method as described in claim 1, characterized in that, Based on the optimal data storage format, the storage format of the data in the sub-block matrix is ​​optimized to obtain an optimized sub-block matrix, including: The storage format of each x-coordinate and each y-coordinate in the sub-block matrix is ​​converted into the optimal data storage format to obtain the position data row of the sub-block matrix; Extract the values ​​of each element in the sub-block matrix and use them as the element value rows of the sub-block matrix; If the memory required for the data type of any position quantity in the position quantity row is greater than the memory required for the data type of any element value in the element value row, then the element value row is taken as the first row of the optimized sub-block matrix, and the position quantity row is taken as the second row of the optimized sub-block matrix. If the memory required for the data type of any position quantity in the position quantity row is less than the memory required for the data type of any element value in the element value row, then the element value row is used as the second row of the optimized sub-block matrix, and the position quantity row is used as the first row of the optimized sub-block matrix.

3. The method as described in claim 2, characterized in that, The method further includes: Obtain GPU configuration information; wherein, the GPU configuration information includes: the memory starting address of the target minimum memory unit and the GPU alignment rules; Calculate the starting and ending addresses of the array in the corresponding target minimum memory unit to obtain the target storage address of the array, including: If the array is the array corresponding to the first row of the optimized sub-block matrix, then the memory starting address of the corresponding target minimum memory unit is used as the memory starting address of the array; The sum of the starting memory address, the memory capacity required for the first row of the optimized sub-block matrix, and the memory padding amount of the array is used as the memory termination address of the array; wherein, the memory padding amount of the array is determined based on the data type of the data in the array and the GPU alignment rules; If the array is not the array corresponding to the first row of the optimized sub-block matrix, then the memory termination address corresponding to the row above the corresponding row of the array is used as the memory start address of the array. The sum of the starting memory address, the memory capacity required for the corresponding row of the array, and the memory padding amount of the array is used as the ending memory address of the array.

4. The method as described in claim 1, characterized in that, The target minimum memory unit can be any one of a plurality of minimum memory units, or a plurality of consecutive minimum memory units.

5. The method as described in claim 1, characterized in that, in, The GPU configuration information also includes: the memory capacity of the smallest memory unit; After obtaining multiple sub-block matrices, the method further includes: Calculate the memory capacity occupied by each sub-block matrix; From the multiple sub-block matrices obtained, select the sub-block matrix with the largest memory usage as the target sub-block matrix; Multiply the memory capacity of the target minimum memory unit by a preset memory capacity threshold to obtain the memory capacity block threshold; If the memory capacity occupied by the target sub-block matrix is ​​not greater than the memory capacity block threshold, then the following step is executed: extract the matrix features of each sub-block matrix.

6. The method as described in claim 5, characterized in that, The method further includes: If the memory capacity occupied by the target sub-block matrix is ​​greater than the memory capacity block threshold, then the number of threads is divided by the preset thread threshold to obtain the block parameters; Using the block parameters as the number of rows and columns of each sub-block matrix, the target sparse matrix is ​​divided into blocks to obtain multiple sub-block matrices. The execution steps are then returned: calculate the memory capacity occupied by each sub-block matrix until the memory capacity occupied by the target sub-block matrix is ​​not less than the memory capacity block threshold.

7. A GPU adaptive memory optimization device, characterized in that, The device includes: An acquisition unit is used to acquire a target sparse matrix; wherein each element in the target sparse matrix contains the value of the element and the x-coordinate and y-coordinate of the element in the target sparse matrix; An optimization unit is used to divide the target sparse matrix into multiple sub-block matrices based on the number of threads in the GPU thread bundle and the capacity of the smallest memory unit; wherein the number of rows and columns of any sub-block matrix is ​​equal to the number of threads; extract the matrix features of each sub-block matrix; for any sub-block matrix, input the matrix features of the sub-block matrix into a pre-trained data storage format selection model to obtain the optimal data storage format corresponding to the sub-block matrix; wherein the data storage format selection model is trained based on multiple different historical sub-block matrices and the optimal data storage format of the corresponding historical sub-block matrices; optimize the storage format of the data in the sub-block matrix according to the optimal data storage format to obtain an optimized sub-block matrix; wherein any optimized sub-block matrix contains at least two arrays; An allocation unit is used to allocate the same target minimum memory unit to each array corresponding to the optimized sub-block matrix based on multiple minimum memory units contained in the GPU, for any optimized sub-block matrix; wherein, the minimum memory unit is the minimum virtual memory interval when the GPU allocates memory; The calculation unit is used to calculate the memory start address and memory end address of any array in the corresponding target minimum memory unit for any optimized sub-block matrix, so as to obtain the target storage address of the array; The storage unit is used to store each array to the corresponding target storage address to obtain the optimized GPU memory.

8. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images