A GPU resource-aware matrix multiplication parallel performance analysis model construction method

Abstract

The application discloses a GPU resource-aware matrix multiplication parallel performance analysis model construction method, and relates to the technical field of high-performance computing, which is based on the Roofline principle, combines a bandwidth model, a resource load model and an instruction dependency model to construct a performance analysis model, and is successfully applied to a matrix multiplication scene to realize parallel performance quantification under resource-aware matrix multiplication application.The application can realize resource awareness without platform limitation, can distinguish parameter settings for maximizing resource use and optimizing performance in the matrix multiplication scene through the performance analysis model, and lays a foundation for subsequent parallel program optimization work; the hardware parameters used by the application are completely based on publicly available GPU hardware parameters, so that non-GPU professionals can also use the model to evaluate the performance of programs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-performance computing, in particular to a GPU resource-aware matrix multiplication parallel performance analysis model construction method. BACKGROUND

[0002] With the rapid development of information technology, the global computing power demand is growing exponentially, and the field of high-performance computing is undergoing unprecedented development and change. Modern technology has put forward higher requirements for computing power, which poses a more severe challenge to traditional CPU architecture. Therefore, the current research and industry have begun to focus on how to effectively improve computing performance and resource utilization. Modern image processing units integrate a large number of parallel hardware to form thousands of lightweight context switches, which have higher performance in high-throughput and high-latency applications, and have given birth to more emerging fields, of which the most significant change is the extension of the artificial intelligence (AI) field to a larger scale of learning, and the essence of GPU-accelerated AI field computing can be largely attributed to the GPU's calculation of matrix operations. In the field of high performance, general matrix multiplication is the core task of realizing efficient matrix operations, and its optimization not only can fully tap the potential of GPU parallel computing, but also has important significance for improving model training speed and inference efficiency.

[0003] The emergence of parallel programming languages has greatly reduced the development difficulty of GPU programming, but GPU programming is still considered a professional field because, in fact, more consideration is given to the performance optimization of the parallel program after porting, and changing the kernel launch parameters of the parallel program is a direct and easy performance optimization method. Because it can determine the operation mapping relationship between software and hardware, determine the use degree of resources such as registers and shared memory, limit the number of active thread blocks in the multi-processor in the GPU, and affect the actual task parallel granularity. Therefore, although these parameters are logically independent of the program itself, they are the foundation work for all subsequent optimization work, and incorrect or unreasonable launch parameters often lead to low resource utilization, especially in the matrix multiplication scenario, irregular cases will further increase the complexity of parameter tuning. Therefore, the selection of launch parameters not only needs to consider the hardware characteristics of the GPU architecture, but also needs to adjust to the specific shape and other characteristics of the matrix. How to achieve the optimal configuration of the launch parameters under multiple constraints has become a major problem in GPU programming and performance optimization.

[0004] In order to determine the best performance of the parallel program, it is often necessary to rely on experience and a large number of repeated verification experiments and tests to adjust the parameters, which is very time-consuming. Especially, the upgrade of GPU architecture makes parameter optimization a moving target problem, and the slight difference between platforms may make the original best parameters and optimized performance meaningless, affecting the portability and performance repeatability of the parallel program. In order to cope with many challenges, the automatic tuning method based on performance prediction has gradually become an important tool to solve the generalization problem of GPU parameter optimization. SUMMARY

[0005] In view of the above technical problems in the related art, the present application provides a GPU resource-aware matrix multiplication parallel performance analysis model construction method, which can solve the above problems.

[0006] To achieve the above technical purpose, the technical scheme of the present application is as follows:

[0007] A GPU resource-aware matrix multiplication parallel performance analysis model construction method, comprising the following steps:

[0008] S100, a resource-aware model is constructed, including obtaining the total number of theoretical thread blocks that can be executed in parallel by the GPU under the current setting through the classic occupancy model, and obtaining the total thread block number of the current parallel task division according to the start-up parameters, and then obtaining the influence rate of the tail effect, constructing the load occupancy model, and realizing the resource awareness of the load imbalance situation;

[0009] S200, the memory access mode conforming to the application characteristics is established according to the memory access address mapping mode of the matrix multiplication application, and the bandwidth model is constructed, including obtaining the average memory data redundancy by combining the memory address mapping mode of the matrix multiplication in the global memory and the shared memory, and then constructing the thread bundle reuse distance concept, and the time access method generates the average thread bundle cache line reuse distance with space-time locality SM, and obtains the memory data locality with the current application characteristics;

[0010] S300, a performance analysis model combining the memory model and the computing intensity is constructed based on the Roofline principle, including obtaining the computing intensity of the corresponding computing-memory instruction by counting the computing and each memory level operation instruction under the current matrix multiplication application, and realizing the performance analysis model of the unit floating point computing frequency combined with the bandwidth model in S200;

[0011] S400, establish the instruction dependency relationship under the calculation intensity, combine the calculation intensity calculation method of the calculation-visit instruction, build a simple instruction dependency relationship chain with the calculation instruction as the medium, simulate the GPU instruction emission process to estimate the overall running time, obtain the calculation instruction change rate under the influence of instruction dependency, realize the quantization of instruction dependency, and then optimize the performance analysis model;

[0012] S500, build a scale-aware parallel comprehensive performance analysis model, including dividing the influence of task scale on the optimized performance analysis model in S400, building a stress factor, realizing a more perfect comprehensive performance analysis model, and improving the scale-aware capability.

[0013] Further, the tail effect in S100 refers to the phenomenon that GPU cannot fully start all SMs to complete parallel computing, resulting in resource waste and reduced computing efficiency due to idle waiting of part of SMs during the computing process.

[0014] Further, in the resource-aware model construction of S100, the theoretical awareness model is obtained through the programming interface cudaOccupancyMaxPotentialBlockSize() to calculate the theoretical load occupancy rate; the number of thread blocks in parallel task division is obtained through the start parameter setting; and the estimation of tail effect is realized by load quantization through the load ratio after the actual task is evenly distributed to each SM.

[0015] Further, the thread bundle reuse distance in S200 is the number of non-repeated cache lines generated within the time interval from the next access to the cache line accessed by a thread bundle in the same active thread bundle in the same SM.

[0016] Further, the specific steps of S100 are as follows:

[0017] S101, use cudaOccupancyMaxPotentialBlockSize() to obtain the number of thread blocks that can be executed in parallel by each SM, and then obtain the total number of thread blocks that can be processed in parallel in GPU according to the number of SMs in GPU;

[0018] S102, obtain the task division size using the parameters about BlockDim part in the start parameter, and obtain the total task division thread block number according to the task scale;

[0019] S103, obtain the parallel execution proportion of full-load operation by dividing the number of task thread blocks divided by the start parameter by the number of thread blocks that can be processed in parallel by GPU, and then divide by the upper limit value of the proportion to obtain the load proportion of tail effect;

[0020] S104 The load ratio obtained by S103 is multiplied by the load theoretical occupancy to obtain the load occupancy under the influence of the tail effect, which is further expressed as a load-aware resource model.

[0021] Further, the specific steps of S200 are as follows:

[0022] S201, by mapping the program global memory access address to the memory block in units of 128 bytes, the number of memory blocks corresponding to each thread bundle, also known as the number of cache lines, is estimated.

[0023] S202, according to the number of thread bundles executed in parallel by each SM obtained from the resource-aware model, and the loop part in the memory access process is unfolded, the corresponding cache line mapping is found by the method of S201;

[0024] S203, after obtaining the cache line mapping of each thread bundle, a stack with cache mode is constructed, the cache line update stack top of each thread bundle is stacked in order, and the number of cache lines passed in the stacking process is counted to obtain the corresponding reuse distance. The sum of the reuse distances of all thread bundles is averaged to obtain the corresponding cache line number of each thread bundle;

[0025] S204, the average cache line number obtained in S203 is normalized by the number of cache lines corresponding to the data block accessed by the thread bundle, and the reciprocal is obtained to obtain the final reuse distance parameter;

[0026] S205, the reuse distance parameter is combined with the load occupancy obtained in S104 to obtain the bandwidth influence parameter, and the corresponding peak bandwidth of the GPU platform is combined to obtain the bandwidth estimate.

[0027] Further, the specific steps of S300 are as follows:

[0028] S301, according to the matrix multiplication algorithm, the corresponding theoretical multiplication and addition times are obtained, and combined with the block parameter set by the start parameter, the global memory access times after introducing shared memory are obtained;

[0029] S302, using the calculation intensity calculation method, the theoretical multiplication and addition times are divided by the access data volume of the corresponding global memory access times to obtain the calculation intensity of the corresponding global memory;

[0030] S303, combined with the calculation intensity obtained in S302, the bandwidth estimate obtained in S205 is multiplied to obtain the corresponding unit floating point calculation times.

[0031] Further, the specific steps of S400 are as follows:

[0032] S401, obtain the number of instructions under different instructions, and construct the simple dependency relationship between instructions by using the calculation instruction as a medium;

[0033] S402, establishing a transmission flow model between instructions according to an instruction transmission method in the GPU, and estimating an overall running time;

[0034] S403, dividing the total execution time estimated in S402 by the number of executed calculation instructions to obtain an average unit calculation instruction execution time, and comparing the average unit calculation instruction execution time with a theoretical unit calculation instruction execution time to obtain an instruction dependency impact factor;

[0035] S404, dividing the instruction dependency factor by the performance analysis model obtained in S300 to realize optimization of the performance analysis model.

[0036] Further, the specific steps of S500 are as follows:

[0037] S501, based on the optimized performance analysis model in S400, obtaining a corresponding calculation intensity division threshold value according to the Roofline principle, and calculating a theoretical calculation intensity of a current calculation application, and dividing the two to obtain a task size division ratio;

[0038] S502, using a nonlinear function to change the division ratio to obtain an emphasis factor;

[0039] S503, using the emphasis factor to emphasize the calculation-global memory calculation intensity obtained in S302 and the bandwidth impact parameter in S205, respectively, to obtain a final comprehensive performance analysis model.

[0040] The present application has the following advantages: the present application can realize resource perception without platform limitation, and can distinguish parameter settings for maximizing resource use and optimizing performance through a performance analysis model in a matrix multiplication scenario, thereby laying a foundation for subsequent parallel program optimization work and being an important step for performance portability of parallel programs; the hardware parameters used in the present application are completely based on publicly available GPU hardware parameters, so that non-GPU professionals can also use the model to evaluate the performance of programs; the present application fully considers hardware resource impact factors and captures task characteristics of matrix multiplication applications, and can realize resource perception and task characteristic perception, has strong applicability, and can effectively alleviate performance bottlenecks in the field of high-performance computing. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0042] The present application will be further described in detail below according to the drawings.

[0043] Figure 1 is a schematic diagram of the data redundancy calculation method used in the global memory access pattern description of the GPU resource-aware matrix multiplication parallel performance analysis model construction according to the embodiments of the present application;

[0044] Figure 2 is a schematic diagram of the thread bundle reuse distance calculation method used in the GPU resource-aware matrix multiplication parallel performance analysis model construction according to the embodiments of the present application;

[0045] Figure 3 is a schematic diagram of the GPU running total time calculation method under the matrix multiplication instruction dependency used in the GPU resource-aware matrix multiplication parallel performance analysis model construction according to the embodiments of the present application;

[0046] Figure 4 is a schematic diagram of the Roofline model used in the GPU resource-aware matrix multiplication parallel performance analysis model construction according to the embodiments of the present application;

[0047] Figure 5 is a diagram showing the use basis of the emphasis factor in the GPU resource-aware matrix multiplication parallel performance analysis model construction according to the embodiments of the present application;

[0048] Figure 6 is a fitting relationship quantization table of the proposed model, the traditional occupancy model and time based on two different GPU platforms of V100 and A100 in the GPU resource-aware matrix multiplication parallel performance analysis model according to the embodiments of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application belong to the scope of protection of the present application.

[0050] A GPU resource-aware matrix multiplication parallel performance analysis model construction method is disclosed according to the application, which is based on the Roofline principle, combines a bandwidth model, a resource load model and an instruction dependency model to construct a performance analysis model, and is successfully applied to a matrix multiplication scene to realize parallel performance quantification under resource-aware matrix multiplication application, and the main principle is as follows: (1) tracking global and shared memory access mode: taking the access logic of global and shared memory as input, the average access transaction number of each thread bundle in the global memory access process is counted, the average memory bank conflict number of each thread bundle in the shared memory access process is counted, and the data redundancy rate in the current application scene is calculated. (2) constructing a thread bundle level reuse distance theory: the average number of thread bundles of each SM is estimated in sequence and the access cycle is unfolded, the cache line replacement strategy is realized with the hardware cache size as the limit, the average cache line number of thread bundles in each SM is counted, and the data locality is quantitatively described. (3) capturing resource changes under uneven load: the number of thread blocks corresponding to the parallel task size and the blocking strategy is calculated, the number of thread blocks that can be executed in parallel is obtained according to the occupancy rate model, a new occupancy rate model under uneven load is obtained, and resource awareness under uneven load is realized. (4) establishing the instruction dependency relationship under the calculation intensity: the number of calculation and memory access instructions of the parallel computing task is counted to obtain the corresponding calculation intensity relationship, the dependency relationship between the access instructions is constructed with the calculation instruction as the medium, the simple dependency relationship chain between the calculation and each storage unit is realized, and then the instruction dependency relationship is quantified by combining the GPU instruction emission method. (5) realizing the comprehensive performance analysis model combining the access and calculation characteristics: the bandwidth and resource quantization indexes obtained from (1) to (4) are combined to construct the access and calculation model, then the comprehensive performance analysis model is constructed based on the Roofline theory, and the strength factor is constructed, and finally the task size-aware comprehensive performance analysis model is realized.

[0051] In one specific embodiment of the application, as shown in Figure 1 The data redundancy calculation method in the global memory access mode description is shown. The method distinguishes whether the thread access position is continuous by counting the mapping of the access address and the memory address in the thread bundle, counts the number of data addresses falling into the aligned and continuous 128-byte memory blocks, obtains the number of 128-byte memory blocks corresponding to the number of threads, compares with the actual effective data to obtain the change of data efficiency, and also reflects the data redundancy characteristics under the application access mode. At the same time, this method can also be represented as the number of cache lines of the number of threads, which prepares for the subsequent data reuse distance.

[0052] The reuse distance of a warp is the number of non-repeated cache lines accessed by a certain warp within the time interval that the cache line is next accessed by the same warp that is active in the same SM. This warp-level reuse distance theory not only reflects the data locality between warps, but also reflects the data locality within a warp by counting the cache lines accessed within the warp. The specific reuse distance RD is calculated according to the following formula:

[0053] (1)

[0054] wherein RD represents the reuse distance, which is an abbreviation of Reuse Distance, represents the average reuse distance of each warp in the entire memory access process, represents the reuse distance corresponding to the thread warp unfolded in time sequence, represents the sequence length of the time sequence, represents the number of warps active in the SM (Streaming Multiprocessor) under the current resource limit, represents the number of non-repeated cache lines corresponding to the data accessed by the current SM.

[0055] In the calculation of cache reuse distance, a cache stack structure with time sequence awareness is designed to accurately depict the warp-level memory access pattern. The core mechanism is to build a virtual cache stack that conforms to the hardware execution characteristics. The idea can be divided into three key points: building a cache stack, building time locality, and building spatial locality.

[0056] In terms of cache stack construction, the system constructs a dynamic memory access sequence according to the warp scheduling order. The cache line request initiated by the first thread of each warp will trigger a push operation, and the corresponding cache line will be pushed into the virtual cache stack and arranged in the order of push. At the same time, in terms of push strategy, the application uses a time sequence maintenance strategy - the first accessed cache line is placed at the bottom of the stack instead of the top. This reverse push order design effectively maintains the time sequence of warp execution, avoiding the reuse distance of stack top calculation being too small and ignoring the unordered characteristics of warp access. It is worth noting that the calculation of reuse distance needs to maintain the capacity constraint characteristics of the physical cache stack to ensure that the storage hardware limitations are considered.

[0057] In terms of time locality, for the loop access behavior of the same warp, the application converts it into a virtual warp, and the loop time sequence is converted into multiple logical warp entities participating in calculation. This virtual warp processing method not only preserves the time dimension characteristics of the original memory access pattern, but also realizes the cross-period memory association analysis.

[0058] In the aspect of spatial locality, the stack top update strategy is adopted. When a new thread bundle proposes an access request for a non-repeated cache line, the corresponding data block will be sequentially stacked according to the stack bottom strategy and moved to the top of the stack after distance calculation; if the access is for an existing cache line, it will be directly moved to the top of the stack.

[0059] Finally, the reuse distance calculation needs to introduce a normalization factor, using the cumulative reuse distance divided by the number of data cache lines actually required by the SM to access, to avoid the dimensional error influence brought by the task scale. At the same time, the normalized reuse distance calculation needs to be divided by the number of active thread bundles actually residing in the SM, to better estimate the actual locality characteristics of each access request, and using the actual thread bundle method can eliminate the radix bias brought by virtual thread bundle expansion, ensuring that the calculation result strictly corresponds to the hardware execution characteristics.

[0060] As shown in Figure 2 is an example of a reuse distance. In the example, the program design accesses the global memory according to the thread bundle arrangement template, and under the resource allocation restriction, one active thread block can be supported in each SM, where the thread is set to (8, 8). That is, 2 active thread bundles can be executed in parallel in the SM, and each thread bundle is accessed in the manner of (8, 4). There are two loops Loop in the program design, so there will be 4 groups of thread bundle accesses in the SM under virtual expansion, at this time Nwarp= 2, Nloop= 2.

[0061] First, in the first loop Loop-0, the first threads T0, T8, T 16 , T 24 , T 32 , T 40 , T 48 , T 56The data cache lines to be accessed will be put into the cache. Take the first thread bundle Warp0 as an example, the cache lines C3, C2, C1, C0 are put into the cache. The first row first thread T0 accesses the top element C0, and the reuse distance is 0, and C0 is kept at the top of the stack; T8 accesses C1 and updates the top element, and the reuse distance is 1, and other threads are similar. Therefore, the reuse distance of the thread bundle Warp0 in this example is RDw0=0+1+2+3=6. Since Warp0 and Warp1 are two parallel thread bundles, they will simultaneously initiate a write request to the cache, but the service of the write request will be implemented in a certain order, in order to preserve the order characteristics, the new cache line of Warp1 will be placed at the bottom of the stack, at this time the cache line order in the cache stack is C7, C6, C5, C4, C0, C1, C2, C3. When the cache line is read, the corresponding cache line will be updated to the top of the stack, specifically T32 accesses C4, the reuse distance is 4, and C4 is updated to the top of the stack; T40 accesses C5, the reuse distance is 5, and C5 is updated to the top of the stack, and so on. Finally, the reuse distance of Warp1 is RDw1=4+5+6+7=22.

[0062] As for the calculation of the time reuse distance, when the first loop ends, the second round of Loop-1 virtual thread bundle will start to access in parallel, and the access method is the same as Loop-0. Since the cache line at the top of the current cache stack coincides with the access of Warp1 of Loop-1, Warp1 will be accessed in priority from the perspective of locality, at this time the cache lines C7, C6, C5, C4 are sequentially accessed, and a new reuse distance RDw2=0+1+2+3=6 is generated, and the stack top is updated to generate a cache distribution C0, C1, C2, C3, C7, C6, C5, C4. Then, Warp0 will continue to access C3, C2, C1, C0 in order, generate a reuse distance of RDw3=4+5+6+7=22, and update and generate a new cache distribution C7, C6, C5, C4, C3, C2, C1, C0.

[0063] The total reuse distance corresponding to the access mode is RDw0+RDw1+RDw2+RDw3=6+22+6+22=56, and the final reuse distance is RD=56 / 8 / 2=3.5.

[0064] In one specific embodiment of the present application, as Figure 3 The estimation model of the overall time combined with the calculation intensity is shown. The time consumption of the entire calculation execution process is the sum of the execution time of a single thread bundle under parallel execution and the redundant time consumption in the scheduling process, as shown in formulas 2-4:

[0065] (2)

[0066] (3)

[0067] (4)

[0068] wherein, represents the GPU hardware unit operation time, represents the GPU hardware shared memory unit access time, represents the current hardware thread bundle scheduler number, represents the current operation corresponding hardware unit number, represents the current hardware condition loading and storage unit number, represents the current access mode under the number of storage bank conflicts occurred in the shared memory, represents the current active thread bundle number, represents the number of threads in each thread bundle, represents the number of calculation operations in each thread bundle, and AI represents the calculation intensity, and respectively represent the calculation intensity between global memory access and calculation operation, and shared memory access and calculation operation.

[0069] By constructing the instruction-level SIMT model, a simple estimation of the overall running time T can be obtained, and combined with the known calculation times, the operation unit time under the influence of instruction dependency and shared memory access conflict can be obtained. Then, by comparing the changed operation unit time and the unit operation time not affected, the quantitative influence factor can be obtained, as shown in the following formulas 5 and 6:

[0070] (5)

[0071] (6)

[0072] wherein T represents the overall running time estimated by the proposed SIMT model method; represents the calculation intensity between global memory access and calculation instruction under the current application, represents the unit calculation time estimated by the proposed SIMT model method, represents the GPU hardware unit operation time, and the calculation method is shown in formula (3), represents the quantitative influence factor of instruction dependency on calculation operation.

[0073] In one specific embodiment of the present application, as Figure 4The Roofline principle model is shown. The model is a performance analysis index commonly used in the field of high performance. The application is based on the principle of Roofline, and the performance prior analysis model is realized by constructing bandwidth model and computing intensity model. The specific calculation method is shown in formulas 7-9:

[0074] (7)

[0075] (8)

[0076] (9)

[0077] Wherein, The computing intensity between global memory access and computing instruction under the current application is represented, BW represents the actual bandwidth estimation considering load balancing and reuse distance, The theoretical peak bandwidth of GPU global memory is, The occupancy rate considering load balancing is represented, The theoretical occupancy rate can be obtained by interface cudaOccupancyMaxPotentialBlockSize(), The number of times of full load running of SM of GPU under the current task is represented, and FLOPS represents the unit floating point calculation times, The instruction dependence quantization influence factor of computing operation is represented, and the specific calculation method is shown in formula (5).

[0078] In one specific embodiment of the application, as Figure 5 The use of emphasis factor is shown by experimental graph. As shown in Figure 5 The different factors in the proposed performance index appear two-level differentiation in the calculation-intensive application, that is, the computing intensity, the reuse distance and the occupancy rate show different degrees of sensitive characteristics. Therefore, according to the sensitivity change phenomenon and the analysis function of Roofline model, the standard computing intensity is used to divide the parallel computing task, and the intensity factor is set to highlight the effect between the indexes under different tasks, which is shown in formulas 10-12:

[0079] (10)

[0080] (11)

[0081] (12)

[0082] where p is an emphasis factor used to control the fluctuation of the corresponding parameter to achieve the effect of emphasizing different parameters in different task scenarios, β is used to control the change speed of the emphasis factor p, and a is used to control the change range of the emphasis index, represents the basic computing intensity of the current application, i.e., the computing amount of the application divided by the corresponding data amount, represents the computing intensity threshold under the Roofline index, i.e., the peak computing intensity of the GPU under the corresponding computing type and the peak bandwidth of the global memory. .

[0083] By combining the threshold value, the new computing intensity in the performance model can distinguish the changes before and after the threshold value, and can eliminate the deviation caused by the dimensional difference; at the same time, due to the different influencing factors in the memory-limited and computing-limited scenarios, the use of the logical function in combination with the threshold value can accurately capture the dynamic changes in the memory-limited and computing-limited scenarios; the reason for using the logical function is that it can represent the fast and slow law under the performance bottleneck scenario change, which is not achieved by the linear function.

[0084] As shown in Figure 6 , it can be seen that the index proposed in the present application shows strong correlation under different scales and GPU models, and the absolute value of the correlation coefficient exceeds 0.8, indicating that it has strong performance capture ability in the matrix multiplication scenario. At the same time, for the occupancy index, it shows a certain inverse proportional relationship in the performance capture ability in the small-scale matrix scenario, and the correlation further decreases in the large-scale matrix scenario. The performance analysis model proposed in the present application effectively introduces the description of computing intensity on the basis of the occupancy model, and can realize more accurate capture of the change of performance influencing factors. Different scale matrix multiplication experiments further verify the effectiveness of the proposed performance analysis index in the GPU performance evaluation. The experimental results show that the proposed index shows high correlation under different scales of matrix and GPU models, which is better than the traditional occupancy model. At the same time, through the correlation analysis, it is also revealed that the performance index has advantages in capturing computing intensity, memory reuse, etc.

[0085] To sum up, by means of the technical scheme of the present application, the hierarchical performance analysis model is innovatively constructed and multi-dimensional optimization breakthrough is realized by researching around heterogeneous hardware resources and task characteristic perception. The research cuts in from the dual perspectives of GPU hardware and task characteristics. At the memory level, based on the global memory and shared memory storage hierarchy, the memory transaction and storage bank conflict principle are analyzed, and the thread bundle level reuse distance theory is proposed. At the calculation level, the instruction dependency recognition method based on the calculation intensity is proposed by expanding the traditional pipeline logic according to the instruction dependency relationship, and the instruction level SIMT pipeline model is constructed. At the same time, the unified parallel performance analysis model is constructed based on the Roofline theory, the application boundary of the traditional model is broken through, and it is suitable for memory and calculation intensive tasks. The prediction accuracy is enhanced after the experiment is fine-tuned. The present application fully considers various influencing factors, and constructs the performance analysis model based on the authoritative principle, can realize the high fitting with time, capture the performance change, realize the resource perception, has strong applicability and strong usability, and is suitable for application in various heterogeneous computing systems.

[0086] The above merely describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a GPU resource-aware matrix multiplication parallel performance analysis model, characterized in that, The method comprises the following steps: S100, constructing a resource-aware model, including obtaining the total number of theoretical thread blocks that can be executed in parallel by the GPU under the current setting through a classic occupancy model, obtaining the total number of thread blocks of the current parallel task division according to the start parameter, and then obtaining the influence rate of the tail effect, constructing a load occupancy model, and realizing resource awareness of the load uneven situation; S200, establishing a memory access mode in accordance with the memory address mapping mode of the matrix multiplication application and constructing a bandwidth model, including obtaining the average memory data redundancy by combining the memory address mapping mode of the matrix multiplication in the global memory and the shared memory, and then constructing the thread bundle reuse distance concept, expanding the time access method to generate the average thread bundle cache line reuse distance with spatial and temporal locality SM, and obtaining the memory data locality with the current application characteristics; S300, constructing a performance analysis model combining the memory model and the computing intensity based on the Roofline principle, including obtaining the computing intensity of the corresponding computing-memory instruction by counting the computing and various memory level operation instructions under the current matrix multiplication application, and realizing the performance analysis model of the unit floating point computing frequency in combination with the bandwidth model in S200; S400, establishing the instruction dependency relationship under the computing intensity, constructing a simple instruction dependency relationship chain with the computing instruction as the medium, simulating the GPU instruction emission process to estimate the overall running time, obtaining the computing instruction change rate under the influence of instruction dependency, realizing the quantification of instruction dependency, and then optimizing the performance analysis model; S401, obtaining the number of instructions under different instructions, and constructing a simple dependency relationship between instructions with the computing instruction as the medium; S402, establishing a transmission pipeline model between instructions according to the instruction emission method in the GPU to estimate the overall running time; S403, dividing the total execution time estimated in S402 by the number of executed computing instructions to obtain the average unit computing instruction execution time, and comparing it with the theoretical unit computing instruction execution time to obtain the instruction dependency influence factor; S404, dividing the instruction dependency factor by the performance analysis model obtained in S300 to realize the optimization of the performance analysis model; S500, constructing a scale-aware parallel comprehensive performance analysis model, including dividing the performance analysis model optimized in S400 by the task scale influence to construct an emphasis factor, realizing a more perfect comprehensive performance analysis model, and improving the scale awareness ability; S501, based on the performance analysis model optimized in S400, obtaining the corresponding computing intensity division threshold value according to the Roofline principle, and calculating the theoretical computing intensity of the current computing application, and dividing the two to obtain the task scale division ratio; S502, using a nonlinear function to change the division ratio to obtain an emphasis factor, and the specific formula is as follows: ， Wherein p is an emphasis factor, used to control the fluctuation of the corresponding parameter, to achieve the effect of emphasizing different parameters in different task scenarios, β is used to control the change speed of the emphasis factor p, and α is used to control the change range of the emphasis index, Indicates the theoretical calculation strength of the current application, Indicates the calculation strength division threshold under the Roofline index; S503, using the emphasis factor to emphasize the global memory computing intensity and the bandwidth influence parameter respectively to obtain the final comprehensive performance analysis model.

2. The GPU resource-aware matrix multiplication parallel performance analysis model construction method according to claim 1, characterized in that, The tail effect in S100 refers to the phenomenon that the GPU cannot fully start all SMs to complete parallel computing, resulting in resource waste and reduced computing efficiency due to the idle waiting of some SMs during the computing process.

3. The GPU resource-aware matrix multiplication parallel performance analysis model construction method of claim 1, wherein, In the construction of the resource perception model of S100, the theoretical perception model is obtained through the programming interface cudaOccupancyMaxPotentialBlockSize() to calculate the theoretical load occupancy rate; the number of thread blocks in parallel task division is obtained through the start parameter setting; and the estimation of the tail effect is achieved by quantifying the load through the load ratio after the actual task is evenly distributed to each SM.

4. The GPU resource-aware matrix multiplication parallel performance analysis model construction method of claim 1, wherein, The thread bundle reuse distance in S200 is the number of non-repeated cache lines generated within the time interval in which the cache line accessed by a certain thread bundle is accessed next time for the active thread bundle in the same SM.

5. The GPU resource-aware matrix multiplication parallel performance analysis model construction method of claim 3, wherein, The specific steps of S100 are as follows: S101, use cudaOccupancyMaxPotentialBlockSize() to obtain the number of thread blocks that can be executed in parallel by each SM, and then obtain the total number of thread blocks that can be processed in parallel in the GPU according to the number of SMs in the GPU; S102, use the parameters about BlockDim part in the start parameter to obtain the task division size, and obtain the total task division thread block number according to the task size; S103, divide the number of task thread blocks divided by the start parameter by the number of thread blocks that can be processed in parallel by the GPU to obtain the parallel execution ratio of full-load operation, and then divide by the upper bound value to obtain the load ratio of the tail effect; S104, multiply the load ratio obtained by S103 by the theoretical load occupancy rate to obtain the load occupancy rate under the influence of the tail effect, and then express it as a load-aware resource model.

6. The GPU resource-aware matrix multiplication parallel performance analysis model construction method according to claim 5, characterized in that, The specific steps of S200 are as follows: S201, estimate the number of memory blocks corresponding to each thread bundle by mapping the global memory access address of the program to memory blocks with a unit of 128 bytes, also known as cache line number; S202, according to the number of thread bundles executed in parallel by each SM obtained by the resource perception model, and unfolding the loop part in the memory access process, find the corresponding cache line mapping by the method of S201; S203, after obtaining the cache line mapping of each thread bundle, construct a stack with cache mode, stack the cache line update of each thread bundle in order, and count the number of cache lines passed in the stacking process to obtain the corresponding reuse distance, and take the average of the sum of the reuse distances of all thread bundles to obtain the average cache line number corresponding to each thread bundle; S204, normalize the average cache line number obtained in S203 by the cache line number corresponding to the data block accessed by the thread bundle, take the reciprocal, and obtain the final reuse distance parameter; S205, combine the reuse distance parameter with the load occupancy rate obtained in S104 to obtain the bandwidth influence parameter, and combine the peak bandwidth corresponding to the GPU platform to obtain the bandwidth estimation.

7. The GPU resource-aware matrix multiplication parallel performance analysis model construction method according to claim 6, characterized in that, The specific steps of S300 are as follows: S301、According to the matrix multiplication algorithm, the corresponding theoretical multiplication and addition times are obtained, and the global memory access times after introducing the shared memory are obtained in combination with the block parameters set by the start parameters; S302、Using the calculation intensity calculation method, the theoretical multiplication and addition times are divided by the access data volume of the corresponding global memory access times to obtain the calculation intensity of the corresponding global memory; S303、In combination with the calculation intensity obtained in S302, the bandwidth estimation obtained in S205 is multiplied to obtain the corresponding unit floating point calculation times.

Images