Queuing theory-based Stencil calculation memory access concurrency performance prediction method

Abstract

The invention relates to a Sencil calculation memory access concurrency performance prediction method based on a queuing theory. The method comprises the following steps: calculating memory access missing numbers under different cache capacities by utilizing a memory access missing number prediction model calculated by Stencil; obtaining CPU storage subsystem information, and extracting a processing process of the memory access request in the hardware device supporting memory access concurrency according to the CPU storage subsystem information; according to the CPU storage subsystem information, the memory access missing number, the processing process and the assumed condition, a memory access concurrency performance prediction model is constructed by using the queuing theory and the Littele law; calculating the memory access concurrency bottleneck of each memory access stream according to the memory access concurrency performance prediction model, and calculating the actual memory access bandwidth according to the memory access concurrency of the memory access concurrency bottleneck and the dynamic memory access delay by utilizing the Littele law; according to the actual execution bandwidth and the data access amount, the actual execution time of Sencil calculation is obtained through calculation. By adopting the method, the prediction precision of the Sencil calculation performance can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a Stencil computing memory access concurrency performance prediction method based on queuing theory. BACKGROUND

[0002] Stencil computation on structured grid is the core of many scientific computing programs, such as Jacobi iteration in PDE solvers. In the stencil computation process, updating the value of a single cell of a multi-dimensional grid needs to access the neighboring cells in a fixed pattern, and in one scanning iteration, all array elements except the boundary need to be updated in this way. In practical applications, the stencil computation needs to repeat the iteration scanning process multiple times. The characteristics of the stencil computation result in two inherent problems: non-continuous memory access pattern and low operation density. The performance of the stencil computation is severely limited by memory access, and the computing unit spends most of the time waiting for memory data to arrive, so it can only achieve a low proportion of the theoretical peak performance. In order to reduce the execution time of many scientific computing applications, most of the optimization work is used for the performance optimization of the stencil computation.

[0003] However, these manual optimization methods for stencil computation are tedious and time-consuming, and most of the performance-based optimizations are based on rough performance estimates, or just optimizations without analyzing performance bottlenecks and boundaries. To alleviate the tedious optimization process and optimization cost of user supervision, automatic tuning frameworks achieve the optimization of algorithms by automating the search for optimal parameters, and the core part of these frameworks uses performance models to guide decisions on the best algorithm parameters, thread execution configurations, and code modifications for stencil computation. Performance models can be seen as mathematical formulas that quantify the analyzed program and hardware platform, which can predict performance by inputting some characteristic parameters of the corresponding stencil computation without executing any stencil computation program. The prediction accuracy of performance models directly determines the tuning effect of these automatic tuning frameworks. Existing performance models are divided into statistical models and analytical models. Statistical models are based on existing performance test data, and use statistical methods or machine learning to establish models. They can relatively accurately predict performance, but cannot be used to analyze performance bottlenecks. Analytical models are based on reasonable assumptions about real hardware architectures and application programs, and use simplified machine models to describe the interaction between code and hardware. They quantify the execution process and performance of application programs on related hardware architectures and find performance bottlenecks, but the limitations based on assumptions often result in lower performance prediction accuracy. Roofline model and ECM model are two widely used general-purpose analytical performance models. Roofline model is based on memory bandwidth and peak performance of computation, and it can give the performance upper limit of a computing kernel on a specific hardware architecture according to its operation density. Roofline model does not distinguish between memory hierarchies and does not consider the impact of multi-level cache structures on memory access. ECM model is an extension and refinement of Roofline model, which fully considers the memory hierarchy. However, ECM model approximates that the sum of data transfer times of each layer of the storage hierarchy in the memory process is the total memory access overhead, and existing research does not specify that intermediate caches cannot communicate with upper and lower caches simultaneously, and it uses peak bandwidth instead of the real data transfer bandwidth of the loop kernel, which affects the prediction accuracy of ECM model. Cruz model is an analytical performance model specifically designed for stencil computation, which improves on previous work and fully considers the characteristics of modern multi-layer storage structures. Modern CPU processors use non-blocking cache technology to hide dynamic memory access latency using concurrent memory access to improve memory performance, and MLP (Memory Level Parallelism) is introduced to measure this feature.The C-AMAT (Concurrent-AMAT) model improves the problem that the AMAT model cannot accurately measure the memory access performance of the non-blocking cache, introduces the memory access concurrency as a model parameter, simultaneously represents the locality and concurrency of the memory access, and unifies the mutual influence of the information represented by the AMAT and the MLP in one formula. However, the C-AMAT model needs to increase a dedicated memory access concurrency statistical hardware device on the existing CPU processor, and therefore cannot be used to quantitatively analyze the influence of the concurrency on the memory access performance. The deficiencies of the ECM model and the Cruz model introduced above are that the MLP characteristics of the modern CPU are not considered, and the ECM model and the Cruz model have an unreasonable assumption in analyzing the memory bandwidth of the stencil calculation. The bandwidth of the STREAM benchmark is directly used to replace the actual memory bandwidth of the stencil calculation, which is a direct reason for the low prediction accuracy of the model.

[0004] Therefore, how to improve the prediction accuracy of the existing performance model for the stencil calculation so as to improve the prediction result of the performance model on the execution performance of the stencil calculation on the existing computer architecture, and more accurately find the performance bottleneck, becomes a technical problem to be solved. SUMMARY

[0005] Therefore, how to improve the prediction accuracy of the existing performance model for the stencil calculation so as to improve the prediction result of the performance model on the execution performance of the stencil calculation on the existing computer architecture, and more accurately find the performance bottleneck, becomes a technical problem to be solved.

[0006] A method for predicting the memory access concurrency performance of a stencil calculation based on queuing theory, the method comprising: calculating the memory access miss number under different cache capacities by using a pre-constructed memory access miss number prediction model of the stencil calculation; obtaining CPU storage subsystem information, and extracting the processing process of a memory access request in a hardware device supporting memory access concurrency according to the CPU storage subsystem information; the CPU storage subsystem information comprises cache parameters, buffer parameters, DRAM delay and dynamic memory access delay; constructing a memory access concurrency performance prediction model by using queuing theory and Little's law according to the CPU storage subsystem information, the memory access miss number, the processing process and a pre-set assumption condition; Little's law is used to establish the relationship between the memory bandwidth of the memory access flow and the memory access concurrency and the average delay of the memory access request; the average delay of the memory access request is obtained by weighting the dynamic memory access delay and the memory access delay of each level of cache according to the memory hit ratio; and the memory access concurrency performance prediction model is used to predict the maximum memory access concurrency provided by the hardware device for different memory access flows of the stencil calculation; According to the memory access concurrency performance prediction model, the memory access concurrency bottleneck of each memory access flow is calculated, and according to Little's law, the actual memory bandwidth is calculated according to the memory access concurrency of the memory access concurrency bottleneck and the dynamic memory access delay. According to the actual execution bandwidth and the data access amount, the actual execution time of the Stencil calculation is calculated; the data access amount is derived from the memory access miss number.

[0007] In one of the embodiments, the memory access miss number prediction model calculates the cache miss number according to the hardware characteristic parameters and the software characteristic parameters of the Stencil calculation; the hardware characteristic parameters include the cache structure and the cache capacity of different cache levels; the software characteristic parameters include the Stencil calculation mode and the storage structure of the grid data.

[0008] In one of the embodiments, the assumption conditions include a first assumption condition, a second assumption condition and a third assumption condition; the first assumption condition includes that the memory bandwidths of different memory access flows in a memory access period are equal; the second assumption condition includes that when the ratio of the actual use bandwidth of the cache to the limit bandwidth is lower than a preset threshold, the increase amount of the memory access delay of the cache compared with the basic delay does not exceed a preset delay increment threshold; and the third assumption condition includes that only one hardware device becomes the concurrency bottleneck.

[0009] In one of the embodiments, the cache parameters include the cache capacity of each cache level and the cache access delay; and the buffer parameters include the size of the loading buffer, the storage buffer, the reservation station, the reordering buffer, the line padding buffer and the miss state holding register.

[0010] In one of the embodiments, the step of calculating the dynamic memory access delay includes: drawing a curve of the dynamic memory access delay changing with the bandwidth usage ratio by using a memory access delay test tool; fitting a dynamic memory access delay formula according to the curve of the dynamic memory access delay changing with the bandwidth usage ratio, and calculating the dynamic memory access delay according to the dynamic memory access delay formula.

[0011] In one of the embodiments, the processing process of the memory access request in the hardware device includes: after the memory access request arrives at the hardware device, the memory access request enters a service queue to wait for processing, and after the processing is completed, the hardware resource is released; the maximum length of the service queue is equal to the maximum number of entries of the corresponding hardware device, and the processing order follows the first-in-first-out rule.

[0012] In one of the embodiments, the method for constructing the memory access concurrency performance prediction model according to the CPU memory subsystem information, the memory access miss number, the processing procedure and the pre-set assumption condition comprises: abstracting the processing procedure of the memory access request in the hardware device supporting the memory access concurrency as a queuing system through the queuing theory to construct the memory access concurrency model corresponding to each hardware device, wherein the task is the memory access request received by the corresponding hardware device, the task arrival law is the number of memory access requests in each clock cycle, the service station is the hardware device supporting the memory access concurrency, the service time is the average delay of the memory access request, and the service queue includes the memory access requests waiting for processing and the memory access requests being processed; mapping the Little's Law to the queuing system according to the CPU memory subsystem information, the memory access miss number and the pre-set assumption condition to establish the relationship between the memory access bandwidth of a single memory access flow and the memory access concurrency degree and the memory access delay; and establishing the prediction model for predicting the maximum memory access concurrency degree provided by each hardware device for different memory access flows of the Stencil calculation.

[0013] In one of the embodiments, the method for calculating the memory access concurrency degree bottleneck of each memory access flow according to the memory access concurrency performance prediction model comprises: calculating the maximum memory access concurrency degree provided by the hardware device supporting the memory access concurrency for each memory access flow through the memory access concurrency performance prediction model; and comparing the maximum memory access concurrency degree provided by each hardware device for each memory access flow, and selecting the minimum value as the memory access concurrency degree bottleneck of the corresponding memory access flow.

[0014] In one of the embodiments, the hardware device supporting the memory access concurrency comprises a load buffer, a store buffer, a miss status holding register and a line fill buffer.

[0015] In one of the embodiments, the method further comprises: assisting the design of the hardware processor supporting the memory access concurrency in the CPU architecture according to the actual memory access bandwidth and the actual execution time of the Stencil calculation.

[0016] The Stencil computing memory access concurrency performance prediction method based on queuing theory can quickly and accurately calculate the memory access missing number under different cache capacities through the pre-constructed memory access missing number prediction model, provide a quantitative basis for subsequent performance analysis, obtain the cache parameters, buffer parameters and other information of the CPU storage subsystem, determine the processing logic of the memory access request in the hardware device, fully adapt to the characteristics of the modern CPU storage architecture, combine the memory access missing number and the hardware processing process, use queuing theory and Little's law to construct a memory access concurrency performance prediction model, can quantitatively predict the maximum memory access concurrency degree provided by the hardware for different memory access streams, break through the limitation of the traditional model ignoring the MLP characteristics, and then derive the actual memory bandwidth through the memory access concurrency degree bottleneck, avoid the unreasonable assumption of replacing the real bandwidth with the peak bandwidth, and finally calculate the execution time combined with the actual bandwidth and the data access amount. The Stencil computing performance prediction precision can be significantly improved, the performance bottleneck can be accurately located, and effective guidance can be provided for the hardware design supporting memory access concurrency. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of the Stencil computing memory access concurrency performance prediction method based on queuing theory in an embodiment is shown. Figure 2 A 3d7p Stencil Cache Line missing distribution graph in an embodiment is shown. Figure 3 A DRAM memory access concurrency degree and delay relationship curve graph in an embodiment is shown. Figure 4 A queuing theory model schematic diagram of the memory access concurrency process on the LFB in an embodiment is shown. Figure 5 An experimental platform Intel Ice Lake memory subsystem graph in an embodiment is shown. Figure 6 A memory access concurrency process schematic diagram supported by hardware in an embodiment is shown. Figure 7 A memory access concurrency process schematic diagram on a single hardware component described by queuing theory in an embodiment is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0019] The technical problem to be solved by the present application is that the single-core prediction accuracy of the existing performance model is too low to accurately quantify and analyze the execution performance of Stencil calculation. The existing analytical performance model is based on the assumption of maximizing memory bandwidth and cannot accurately predict the actual execution bandwidth of Loop Kernel. The present application first determines a memory concurrency prediction model based on memory concurrency analysis. This method combines hardware information providing memory concurrency to quickly predict the actual execution bandwidth of Loop Kernel using queuing theory and Little's Law theorem. Based on the actual execution bandwidth and data access volume, the actual execution time of Stencil and other Loop Kernel can be obtained, which can also assist in guiding the design of hardware processors supporting memory concurrency in existing CPU architectures.

[0020] In one embodiment, as shown in Figure 1 a queuing theory-based Stencil calculation memory concurrency performance prediction method is provided, comprising the following steps: Step 102, using a pre-constructed memory miss number prediction model of Stencil calculation to calculate the memory miss number under different cache capacities.

[0021] The memory miss number prediction model refers to a model that quantitatively predicts the number of memory requests that miss the cache under different cache capacities by analyzing hardware features (such as cache structure, capacity) and software features (such as Stencil calculation mode, grid data storage structure). The memory miss number is the total number of memory requests that miss the cache. Step 102 calculates the memory miss number under different cache capacities through the pre-constructed model, providing basic data for subsequent analysis of memory pressure and derivation of data access volume, solving the problem of inaccurate evaluation of storage subsystem load due to unclear memory miss number, and laying a quantitative foundation for performance prediction.

[0022] Step 104, obtaining CPU storage subsystem information, and extracting the processing process of memory requests in hardware devices supporting memory concurrency based on the CPU storage subsystem information.

[0023] The CPU storage subsystem information includes cache parameters, buffer parameters, DRAM delay, and dynamic memory delay. The hardware devices supporting memory concurrency are used to process memory requests in parallel. The processing process of memory requests refers to the receiving, queuing, processing, and resource release process of memory requests in the above hardware devices. Step 104 extracts the processing process of the hardware devices to clarify the underlying mechanism of memory concurrency, solves the problem of unclear hardware processing logic, and provides hardware-level logical support for subsequent construction of a concurrency performance prediction model.

[0024] In step 106, a memory access concurrency performance prediction model is constructed according to the CPU storage subsystem information, the number of missing memory accesses, the processing procedure and the pre-set assumption conditions by using queuing theory and Little's Law.

[0025] Queuing theory is a mathematical theory for studying queuing phenomena in a system, which is used to abstract the processing procedure of memory access requests into a task-service station-queue model. Little's Law is a core law in queuing theory, which is used to establish the relationship among the average number of tasks, the task arrival rate and the average service time in a system. In the method of the present application, Little's Law is used to establish the relationship among the memory access bandwidth, the memory access concurrency degree and the average delay of memory access requests of a memory access stream. The average delay of memory access requests is obtained by weighting the dynamic memory access delay and the memory access delay of each level of cache according to the memory access hit ratio. The memory access concurrency performance prediction model is used to predict the maximum memory access concurrency degree provided by a hardware device for different memory access streams of a stencil calculation. The memory access stream refers to a sequence of memory access requests classified according to the storage hierarchy or the operation type (such as a main memory read memory access stream and an L1 cache read memory access stream). Each point in a stencil calculation template corresponds to a data access forming a memory access stream. The memory access concurrency degree refers to the number of memory access requests processed simultaneously by the hardware device. In step 106, the prediction model is constructed by using queuing theory and Little's Law, which converts the complex memory access concurrency process into a quantifiable mathematical relationship, solves the problem of being unable to quantitatively evaluate the support capability of hardware for memory access concurrency, and realizes accurate prediction of the maximum concurrency degree of different memory access streams.

[0026] In step 108, the memory access concurrency degree bottleneck of each memory access stream is calculated according to the memory access concurrency performance prediction model, and the actual memory access bandwidth is calculated according to the memory access concurrency degree of the memory access concurrency degree bottleneck and the dynamic memory access delay by using Little's Law.

[0027] The memory access concurrency degree bottleneck refers to the minimum value in the maximum concurrency degrees provided by each hardware device for a memory access stream, that is, the key factor limiting the concurrency performance of the memory access stream. The actual memory access bandwidth refers to the actual data transmission rate of the memory access stream under the constraint of the concurrency degree bottleneck. In step 108, the concurrency degree bottleneck is determined and the actual memory access bandwidth is calculated, which solves the problem of overestimation of the bandwidth caused by not considering the limitation of hardware resources, and accurately reflects the performance upper limit of the storage subsystem under actual constraints.

[0028] In step 110, the actual execution time of the stencil calculation is calculated according to the actual execution bandwidth and the data access amount.

[0029] Actual execution time refers to the actual time taken for a Stencil computation from start to finish. Data access volume is derived from the number of memory access misses, which refers to the total amount of data that needs to be transferred during memory access, specifically derived from the number of memory access misses combined with the data block size at each level of storage. Step 110 calculates the execution time using actual memory access bandwidth and data access volume, solving the problem of traditional methods' difficulty in accurately predicting the actual time taken for Stencil computation, and providing a reliable performance reference for program optimization and hardware design.

[0030] The aforementioned Stencil computation memory access concurrency performance prediction method based on queuing theory utilizes a pre-built memory access miss prediction model to quickly and accurately calculate the number of memory access misses under different cache capacities, providing a quantitative basis for subsequent performance analysis. By acquiring information such as cache parameters and buffer parameters of the CPU storage subsystem, the processing logic of memory access requests in the hardware device is clarified, fully adapting to the characteristics of modern CPU storage architecture. Combining the number of memory access misses with the hardware processing, a memory access concurrency performance prediction model is constructed using queuing theory and Little's law. This model can quantitatively predict the maximum memory access concurrency provided by the hardware for different memory access streams, overcoming the limitation of traditional models ignoring MLP characteristics. Furthermore, the actual memory access bandwidth is derived through the memory access concurrency bottleneck, avoiding unreasonable assumptions that peak bandwidth substitutes for true bandwidth. Finally, the execution time is calculated by combining the actual bandwidth and data access volume. This invention significantly improves the accuracy of Stencil computation performance prediction, accurately locates performance bottlenecks, and provides effective guidance for hardware design that supports memory access concurrency.

[0031] In one embodiment, the memory miss prediction model calculates the cache miss number based on hardware and software characteristic parameters calculated by Stencil: the hardware characteristic parameters include cache structure and cache capacity at different cache levels; the software characteristic parameters include Stencil calculation mode and storage structure of grid data. In this embodiment, the cache structure includes cache hierarchy information, the Stencil computation mode refers to information such as the dimension, template size, and iteration step size of the computation template, and the grid data storage structure includes the array layout in memory, data block size and dimension partitioning, and the storage distinction between boundary data and internal data. A missing memory access prediction model is used to plot a missing memory access distribution map. This map is then used to statistically analyze the trigger frequency of various types of memory access requests in different storage levels, thereby approximately determining the number of memory access flows in each storage level. Alternatively, the Cruz missing memory access model can also be used for the missing memory access prediction model.

[0032] Specifically, taking 3D-stencil as an example: consider a Given a grid size of l=1, calculate a Plane needs of and Z-X planes, is the size of each plane containing ghost points, is the size of each plane not containing ghost points, and their sizes are in words. The total data to be saved into the cache when the cache write policy uses write-back method is The total data to be saved into the cache when the cache write policy uses write-through method is The requested data from the CPU is obtained by static analysis tools, and the Cache Line miss from each cache level is estimated by the following formula:

[0033] where with double-precision floating-point as array data elements, W is the number of words per cache line, is the number of planes read from the next cache level due to possible compulsory, conflict, or capacity misses per k iterations, is obtained by the following Cache Line miss patterns and rules:

[0034] where R1: , R2: , R3: , R4: , is the word length (single or double precision), is the factor of the proportion of data required by the k central planes relative to the entire data set , is the size of each cache level, L1 is 48 KB, L2 is 1.25 MB, and L3 is 48 MB. Finally, based on the above four rules, the average Cache Line miss distribution of each point of 3d7p Stencil when the grid data is updated each time is obtained, as shown in Figure 2 Figure 3d7p Stencil Cache Line miss distribution.

[0035] ​In one embodiment, the assumption conditions include a first assumption condition, a second assumption condition, and a third assumption condition; the first assumption condition includes that the memory bandwidths of different memory flows in a memory cycle are equal; the second assumption condition includes that when the ratio of the actual usage bandwidth of the cache to the limit bandwidth is lower than a preset threshold, the increase of the memory delay of the cache compared to the base delay does not exceed a preset delay increment threshold; and the third assumption condition includes that only one hardware device becomes the concurrency bottleneck.

[0036] In one embodiment, the cache parameters include the cache capacities of the cache levels and the cache access delays; and the buffer parameters include the sizes of the load buffer, the store buffer, the reservation station, the reorder buffer, the line fill buffer, and the miss status holding register.

[0037] In one embodiment, the step of calculating the dynamic memory delay includes: drawing a curve of the dynamic memory delay changing with the bandwidth usage ratio by using a memory delay test tool; fitting a dynamic memory delay formula according to the curve of the dynamic memory delay changing with the bandwidth usage ratio; and calculating the dynamic memory delay according to the dynamic memory delay formula.

[0038] In this embodiment, the curve of the memory delay changing with the bandwidth usage ratio is obtained by using a memory delay test tool and a formula is fitted, as shown in Figure 3 The figure shows the relationship curve of the DRAM memory concurrency and the delay, the relationship curve formula of the DRAM memory delay and the memory bandwidth usage ratio is fitted, which can quantitatively describe the nonlinear growth rule of the DRAM memory delay (especially the queuing delay) when the bandwidth usage ratio increases, and breaks through the limitations of the traditional fixed delay assumption. The average delay of the memory request is the overall delay average calculated by considering the DRAM memory delay and the cache memory delay of each level and weighting according to the hit ratio of the corresponding memory request, which includes both the low delay when the cache is hit and the high delay of the DRAM access. Through this formula, the memory concurrency performance prediction model can dynamically calculate the delay according to the actual bandwidth load, so that the deduction based on the queuing theory and the Little's law is closer to the actual behavior of the hardware, and the accuracy of the prediction of the memory performance in the Stencil computing and other scenarios is improved. At the same time, the formula provides a mathematical tool for analyzing the correlation mechanism of the bandwidth usage ratio and the delay, which helps to accurately locate the performance bottleneck caused by the bandwidth competition and provides quantitative guidance for optimizing the hardware resource configuration (such as the buffer size) and the program memory mode.

[0039] In one embodiment, the processing process of the memory request in the hardware device includes: the memory request enters a service queue to wait for processing after arriving at the hardware device, and the hardware resource is released after the processing is completed; the maximum length of the service queue is equal to the maximum number of entries of the corresponding hardware device, and the processing order follows the first-in-first-out rule.

[0040] In one embodiment, according to the CPU storage subsystem information, the number of memory access misses, the processing procedure and the pre-set assumption condition, the memory access concurrency performance prediction model is constructed by using the queuing theory and the Little's law, including: abstracting the processing procedure of the memory access request in the hardware device supporting the memory access concurrency as a queuing system by using the queuing theory to construct the memory access concurrency model corresponding to each hardware device, wherein the task is the memory access request received by the corresponding hardware device, the task arrival law is the number of memory access requests in each clock cycle, the service station is the hardware device supporting the memory access concurrency, the service time is the average delay of the memory access request, and the service queue includes the memory access requests waiting for processing and the memory access requests being processed; according to the CPU storage subsystem information, the number of memory access misses and the pre-set assumption condition, the Little's law is mapped to the queuing system to establish the relationship between the memory access bandwidth of a single memory access flow and the memory access concurrency degree and the memory access delay; for different memory access flows of the Stencil calculation, the prediction model for predicting the maximum memory access concurrency degree provided by each hardware device for different memory access flows is established.

[0041] In the embodiment, the memory access concurrency model of a single hardware device is described by taking the MSHR and the LFB as examples. 1) Queuing theory model of concurrent memory access procedure on the MSHR: Task: read memory access request from the CPU.

[0042] Task arrival speed: the memory access request arrives at the speed of 2 load / cycle.

[0043] Service queue: unfinished memory access request recorded by the MSHR.

[0044] Maximum length of the service queue: when the memory access request hits the L1 cache completely, if the memory access delay of the L1 cache is 4 cycles and the maximum throughput rate of the data transmission between the L1 and the register is 2 load / cycle.

[0045] According to Little's Law, the maximum concurrency of memory access requests that hit L1 cache is 8, which requires 8 MSHR entries to be occupied simultaneously. When all memory access requests hit L3 cache, if the memory access delay of L3 cache is 34 cycles, and the maximum throughput of data transmission between L3 and L2 is 1 Cache Line / cycle. For AVX256 instructions, the maximum concurrency of memory access requests that hit L3 cache is 68 loads, and MSHR cannot provide sufficient memory concurrency to fully utilize the memory bandwidth of L3 cache. When there are memory access requests that do not hit L1, the data returned by the lower-level cache competes with the data returned by L1 cache for the memory bandwidth between L1 and the register, causing waiting when the data is returned and increasing the delay of the memory access request. However, when the memory bandwidth utilization is not high, the increase in the average delay of the memory access request is small, which can be ignored compared to the delay of accessing the main memory.

[0046] 2) as shown in Figure 4 is a queuing theory model diagram of the memory access concurrency process on LFB, Figure 4 CPU: is the initiator of the task (memory access request), which constantly generates memory access requests (such as reading / writing memory data). LFB: as a service station or queue carrier, temporarily stores and processes memory access requests from CPU. nextlevelcache (next-level cache): is the service target of the memory access request (for example, after LFB processing, the request will be sent to the next-level cache to finally obtain data); FIFO (First In First Out): indicates that the request processing order in LFB is first-come-first-served. The queuing theory model of the memory access concurrency process on LFB is similar to MSHR: Task: L1 memory read miss, replacement request for modified cache line, and non-temporal write request.

[0047] Task arrival rate: memory access misses arrive at LFB at a peak bandwidth of 2load+1store / cycle.

[0048] Service queue: LFB allocates resources in cache line unit, multiple load / store requests accessing the same cache line share the same LFB entry. If LFB allocates at most 10 different cache line memory requests, the maximum length of the service queue: when all memory requests hit the L3 cache, the memory access delay of the L3 cache is 34 cycles, and the maximum throughput of data transmission between L3 and L2 is 1 CacheLine / cycle. For AVX instructions, the maximum concurrency of memory access requests that hit the L3 cache is 34 Cache Lines.

[0049] While LFB supports at most 10 Cache Lines of concurrent memory access (20 AVX loads) at the same time, LFB becomes the bottleneck of memory access concurrency compared with the maximum concurrency that MSHR can provide.

[0050] In one embodiment, calculating the memory access concurrency bottleneck of each memory access stream according to the memory access concurrency performance prediction model comprises: calculating the maximum memory access concurrency provided by the hardware device supporting memory access concurrency for each memory access stream through the memory access concurrency performance prediction model; comparing the maximum memory access concurrency provided by each hardware device corresponding to each memory access stream, and selecting the minimum value as the memory access concurrency bottleneck of the corresponding memory access stream.

[0051] In one embodiment, the hardware device supporting memory access concurrency includes a load buffer, a store buffer, a miss status holding register, and a line fill buffer.

[0052] In one embodiment, the method further comprises: assisting the design of the hardware processor supporting memory access concurrency in the CPU architecture according to the actual memory bandwidth and the actual execution time calculated by the Stencil.

[0053] In a specific embodiment, the process of establishing the memory access concurrency prediction model comprises: Firstly, the hardware and software factors affecting the stencil cache miss number are analyzed. The hardware factors affecting the stencil cache miss rate include the structure and capacity of the cache in the memory system. The software factors include the stencil computation mode and the storage structure of the grid data. The influence of each factor on the memory miss number is qualitatively analyzed. For the stencil computation of the multi-level cache storage structure model and the array data structure, the influence of the stencil computation mode and the storage structure of the grid data on the memory miss number is quantitatively analyzed. The cache capacity conditions for different types of data reuse in the structured grid stencil computation are given. The memory miss number prediction model is established and the miss number calculation formula is given.

[0054] Secondly, the Intel X86 multi-core processor is selected as the experimental platform processor. The naïve Jacobi-Stencil is written as the experimental target program and runs on the experimental platform. At the same time, the PAPI, Likwid and other performance analysis tools are configured to measure the cache miss number and execution time of the stencil.

[0055] Thirdly, the stencil computation memory access concurrency prediction model without prefetch on the CPU single core is constructed. 3.1 According to the computation and memory access delay characteristics of the hardware architecture, the memory access time overhead and computation time overhead of the stencil computation on the typical CPU architecture are theoretically analyzed. It is concluded that the computation time overhead is much smaller than the memory access time overhead.

[0056] 3.2 The memory access delay increases exponentially with the increase of the bandwidth usage rate. The main reason is the increase of the queuing delay in the total delay. The memory access delay of DRAM with the increase of the memory bandwidth usage rate is obtained by the memory access delay test tool Memory Latency Checker provided by Intel. According to the experimental result change graph, the relationship curve formula of the DRAM memory access delay and the memory bandwidth usage rate is fitted. 3.3 The information of the storage subsystem of the experimental platform is analyzed in detail, including the cache capacity, the size of the Load Buffer, the size of the Store Buffer, the size of the RS, the size of the ROB, the size of the LFB, the size of the MSHR, the memory access delay of accessing the cache at each level, and the DRAM memory access delay obtained according to the relationship curve in 3.2. The memory subsystem structure diagram is constructed according to the obtained memory subsystem information, and the hardware information supporting the memory access concurrency and the memory access concurrency process are obtained according to the memory subsystem information. Figure 5The experimental platform Intel Ice Lake memory subsystem diagram is shown as follows: Figure 6 The hardware-supported memory access concurrency process is shown in the schematic diagram, which shows the allocation and flow of different types of memory access requests (L1 load uop, memory load uop, NT-store uop) in hardware resources (load buffer, MSHR, LFB, store buffer): red blocks represent L1 load requests, blue blocks represent memory load requests, and orange blocks represent NT-store requests; These requests in turn occupy resources such as load buffer, MSHR, LFB, or store buffer, reflecting the process of memory access requests from initiation to processing, gradually occupying hardware resources.

[0057] 3.4 Detailed analysis of the memory access concurrency degree of stencil computation: the data access corresponding to each point of the stencil in the stencil computation iteration process forms a memory stream, so each point in the stencil corresponds to a memory stream. The update calculation of different data units in the stencil computation does not have a data dependency relationship, so each stream can be executed in parallel. At the same time, the data access of each unit update calculation in the stencil computation to obtain data from different neighbor units also does not have a data dependency relationship, so there is a certain parallelism between different memory streams. These two reasons make the memory concurrency degree of stencil computation very high. Compared with the hardware devices that provide concurrent memory support in the hardware memory subsystem introduced in the previous section, the parallelism of stencil computation itself is much higher than the memory concurrency degree that the hardware devices can provide, so the bottleneck of memory performance is the memory concurrency degree that the hardware devices can provide.

[0058] 3.5 Build a memory concurrency model based on queuing theory: Figure 7 The queuing theory is used to describe the memory access concurrency process on a single hardware component. Figure 7 In the middle, the left is a three-dimensional grid (dimension I , J , K ), which shows the multi-layer data in the stencil computation (such as K 1、 K , KFigure 3.1: Stencil's memory access pattern and its corresponding queueing theory model. Left: the structure of a Stencil kernel (a 3x3 stencil kernel with 1 level of cache); Right: the distribution of a certain grid point (the center red dot) and its surrounding neighbors (other color dots), representing the data of surrounding neighbors that need to be accessed when computing the center point. Define the queueing theory properties as: Task: a memory access request of the CPU. Task arrival pattern: N memory access requests / cycle. Service station: the part of the L1 cache (MSHR) that receives and processes memory access requests, or the lower level cache (LFB). Service time: the average latency of a memory access request. Service queue: the service queue is composed of memory access requests that are being executed, a memory access request leaves the queue after its data is returned, the maximum length of the service queue is the maximum concurrency supported by the hardware resources that provide memory access concurrency.

[0059] 3.6 Build the memory access concurrency performance prediction model of Stencil based on queueing theory and Little's Law, based on the following assumptions: 1) Assumption 1: the memory bandwidth of different streams in a memory access cycle is equal: the modern CPU instruction is sequentially emitted, out-of-order executed and in-order submitted mechanism, which ensures that the number of memory access requests of different streams completed in each memory access cycle is equal, that is, the memory bandwidth of different streams in a memory access cycle is equal. 2) Assumption 2: when the actual usage bandwidth of the cache is much lower than the limit bandwidth, the memory latency of the cache increases very little: the memory latency of the modern cache is very small, and the actual usage bandwidth of the Stencil calculation is much lower than the peak bandwidth of the cache. 3) Assumption 3: there is only one hardware device that becomes the concurrency bottleneck: the number of memory access requests of different streams executed in a memory access cycle is approximately equal, so the usage of different hardware device resources in each memory access cycle is basically the same, which ensures that different memory access cycles will not have different concurrency bottlenecks. At the same time, when one hardware device is saturated, the pipeline stalls, and other hardware devices will not be allocated resources. The memory access concurrency performance prediction model quantitatively predicts the maximum memory access concurrency provided by each hardware device that provides memory access concurrency support for different memory access streams. The number of memory access streams of different memory levels (main memory read access The number of main memory write access is The number of L1 cache read access is The number of L2 cache read access is The number of L3 cache read access is The number of L3 cache read access ) can be approximated from the missing distribution graph. Each hardware device handles memory requests from multiple memory streams concurrently, and the total concurrency of the hardware device is equal to the sum of the concurrency provided for different memory streams. Take LFB as an example for analysis: LFB provides memory access for main memory write access , main memory read access , L2 cache read access , and L3 cache read access . The LFB provides memory access concurrency support, and gives the relationship between the concurrency provided by the LFB for different memory streams and the maximum concurrency, where is the maximum memory access concurrency of the LFB, is the memory access concurrency provided by the LFB for a single main memory write access , is the memory access concurrency provided by the LFB for a single main memory read access , is the memory access concurrency provided by the LFB for a single L2 cache read access , is the memory access concurrency provided by the LFB for a single L3 cache read access . According to assumption 1, the memory bandwidth used by different streams is equal, and the formula is obtained: . According to Little's Law, the relationship between the memory bandwidth of a single stream, the memory access concurrency, and the memory access delay is established, and the memory bandwidth of the main memory read access is given by the formula: , is the delay of accessing the main memory. The maximum concurrency provided by the LFB for each type of stream is obtained by combining the above formulas, and the maximum concurrency provided by the LFB for the main memory read access is given by the same calculation method as the maximum concurrency provided by the MSHR for the main memory read access: . The calculation methods of the concurrency provided by the LB and the SB for different types of streams are different from those of the MSHR and the LFB, because the resources allocated by the LB and the SB for memory requests cannot be released until the instructions are submitted. The concurrency provided by the LB and the SB for different types of streams can be directly calculated according to the proportion of the number of different types of streams. and the maximum concurrency provided by the LB for the main memory read access The maximum degree of concurrency provided. Assumption 2 of the memory access concurrency model states that there is only one hardware device that will become the concurrency bottleneck, so the minimum of the concurrency provided by different hardware devices for memory read request MR and memory write request MW is found respectively to determine the concurrency bottleneck. MR actual concurrency: , MW actual concurrency: . According to the above formula, the memory bandwidth actually used is calculated by using Little's Law based on the concurrency bottleneck: , .

[0060] The above embodiment gives the steps of using the present application to establish a Stencil computation memory access concurrency performance model. The first step gives a Stencil computation memory access miss quantization formula, which can quickly predict the miss number and miss rate of Stencil computation under different cache capacities; the third step analyzes in detail the memory access concurrency support hardware in the processor and the memory access concurrency of Stencil itself according to the miss number of Stencil computation under different cache capacities, quantitatively analyzes the memory access concurrency process of Stencil and other Loop Kernels on a single-core CPU based on queuing theory and Little's Law, and constructs a memory access concurrency prediction model to predict the actual execution bandwidth of Loop Kernels. As can be seen, the present application can quickly predict the miss number and actual execution bandwidth of Stencil computation under different cache capacities, improve the prediction accuracy of existing performance models, and guide the design of hardware that provides memory access concurrency. Compared with the traditional method, the present application does not use a simulator, and the undisclosed hardware parameters are obtained through corresponding bandwidth delay analysis. The present application predicts the actual execution bandwidth of Loop Kernels, solving the problem that existing methods cannot obtain the single-core execution bandwidth. According to the predicted actual execution bandwidth and memory access miss number, the present application can obtain the actual execution time of Loop Kernels, and the prediction accuracy is improved compared with existing performance models.

[0061] It should be understood that, although the steps in the flowchart of Figure 1 are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in may include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0062] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present disclosure.

[0063] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A queuing theory based Stencil computation memory access concurrency performance prediction method, characterized in that, The method comprises: calculating the number of memory access misses of different cache capacities by using a pre-constructed Stencil calculation memory access miss number prediction model; obtaining CPU storage subsystem information, and extracting a processing process of a memory access request in a hardware device supporting memory access concurrency according to the CPU storage subsystem information; the CPU storage subsystem information comprises cache parameters, buffer parameters, DRAM delay and dynamic memory access delay; constructing a memory access concurrency performance prediction model by using queuing theory and Little's Law according to the CPU storage subsystem information, the number of memory access misses, the processing process and pre-set assumption conditions; Little's Law is used to establish the relationship between the memory bandwidth of a memory access flow and the memory access concurrency degree and the average delay of a memory access request; the average delay of the memory access request is obtained by weighting the dynamic memory access delay and the cache access delay at each level according to the memory hit ratio; the memory access concurrency performance prediction model is used to predict the maximum memory access concurrency degree provided by the hardware device for different memory access flows of Stencil calculation; calculating the memory access concurrency degree bottleneck of each memory access flow according to the memory access concurrency performance prediction model, and calculating the actual memory bandwidth according to the memory access concurrency degree of the memory access concurrency degree bottleneck and the dynamic memory access delay by using Little's Law; calculating the actual execution time of Stencil calculation according to the actual execution bandwidth and the data access amount; the data access amount is derived from the number of memory access misses.

2. The method of claim 1, wherein, The memory access miss number prediction model calculates the cache miss number according to hardware characteristic parameters and software characteristic parameters of Stencil calculation; the hardware characteristic parameters comprise cache structures and cache capacities at different cache levels; the software characteristic parameters comprise Stencil calculation modes and storage structures of grid data.

3. The method of claim 1, wherein, The assumption conditions comprise a first assumption condition, a second assumption condition and a third assumption condition; the first assumption condition comprises that the memory bandwidths of different memory access flows in a memory access period are equal; the second assumption condition comprises that when the ratio of the actual use bandwidth of a cache to the limit bandwidth is lower than a pre-set threshold, the increase of the memory access delay of the cache compared with the basic delay does not exceed a pre-set delay increment threshold; the third assumption condition comprises that only one hardware device becomes the concurrency bottleneck.

4. The method of claim 1, wherein, The cache parameters comprise cache capacities at different cache levels and cache access delays; the buffer parameters comprise the sizes of a load buffer, a store buffer, a reservation station, a reorder buffer, a line fill buffer and a miss status holding register.

5. The method of claim 1, wherein, The step of calculating the dynamic memory access delay comprises: drawing a curve of the dynamic memory access delay changing with the bandwidth usage rate by using a memory access delay test tool; fitting a dynamic memory access delay formula according to the curve of the dynamic memory access delay changing with the bandwidth usage rate, and calculating the dynamic memory access delay according to the dynamic memory access delay formula.

6. The method of claim 1, wherein, The processing process of the memory access request in the hardware device comprises: after the memory access request arrives at the hardware device, the memory access request enters a service queue to wait for processing, and the hardware resource is released after the processing is completed; the maximum length of the service queue is equal to the maximum number of entries of the corresponding hardware device, and the processing order follows the first-in-first-out rule.

7. The method of claim 1, wherein, According to the CPU memory subsystem information, the number of memory access misses, the processing procedure and the pre-set assumption condition, a memory access concurrency performance prediction model is constructed by using queuing theory and Little's law, and the model comprises the following steps: a queuing system is constructed by using queuing theory to abstract the processing procedure of the memory access request in the hardware device supporting memory access concurrency, wherein the task is the memory access request received by the corresponding hardware device, the task arrival law is the number of memory access requests in each clock cycle, the service station is the hardware device supporting memory access concurrency, the service time is the average delay of the memory access request, and the service queue comprises the memory access requests waiting for processing and the memory access requests being processed; Little's law is mapped to the queuing system according to the CPU memory subsystem information, the number of memory access misses and the pre-set assumption condition, and the relationship between the memory access bandwidth of a single memory access flow and the memory access concurrency degree and the memory access delay is established; for different memory access flows of the Stencil calculation, a prediction model for predicting the maximum memory access concurrency degree provided by each hardware device for different memory access flows is established.

8. The method of claim 1, wherein, calculating the memory access concurrency degree bottleneck of each memory access flow according to the memory access concurrency performance prediction model comprises the following steps: the maximum memory access concurrency degree provided by the hardware device supporting memory access concurrency for each memory access flow is calculated by using the memory access concurrency performance prediction model; the maximum memory access concurrency degree provided by each hardware device for each memory access flow is compared, and the minimum value is selected as the memory access concurrency degree bottleneck of the corresponding memory access flow.

9. The method of claim 1, wherein, The hardware device supporting memory access concurrency comprises a load buffer, a store buffer, a miss status holding register and a line fill buffer.

10. The method of claim 1, wherein, The method further comprises: According to the actual memory access bandwidth and the actual execution time of the Stencil calculation, the hardware processor supporting memory access concurrency in the CPU architecture is assisted in design.

Images