Methods, systems, software products, and electronic devices for simulating and analyzing the cache performance of general-purpose graphics processors.
By constructing a virtual cache topology and grid scheduling information, and combining it with memory base addresses to simulate the cache performance of GPGPU, the problem of the disconnect between cache analysis methods and hardware execution processes in existing technologies is solved, and accurate evaluation and optimization of GPGPU cache performance is achieved.
Patent Information
- Application Number
- CN202680002489.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-23
- Publication Date
- 2026-11-17
AI Technical Summary
Existing GPGPU cache analysis methods cannot accurately reflect the real behavior of complex multi-level cache architectures and the handling of diverse data layouts, resulting in inaccurate evaluation results and failing to effectively guide chip design.
By acquiring model input information and configuration input information, a flexible and accurate virtual cache topology that reflects the GPGPU hardware architecture is constructed. Combined with grid scheduling information and memory base address, the data loading and storage process is simulated to generate workload logs and cache behavior logs, enabling precise analysis of cache performance.
It improves the accuracy of cache performance analysis, enabling precise guidance for GPGPU chip cache design, optimization of cache configuration, and enhancement of chip performance and resource utilization.
Smart Images

Figure CN123125615A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cache design technology, and in particular to a method and system for simulating and analyzing the performance of a general-purpose graphics processor cache, as well as a program product and electronic device. Background Technology
[0002] General-Purpose Graphics Processing Units (GPGPUs) have become indispensable core hardware in modern high-performance computing, particularly in fields such as artificial intelligence, scientific computing, and graphics rendering, due to their massively parallel computing capabilities and high throughput. With the rise of complex AI models such as large language models and text-to-graph models, unprecedented demands have been placed on the computing power and storage performance of GPGPU chips. Against this backdrop, computing power and memory bandwidth have become core indicators for measuring the performance and actual operating efficiency of GPGPU chips.
[0003] According to the classic Roof-Line model, the theoretical performance of a chip is limited by its peak computing power and peak memory bandwidth. Computing power determines the upper limit of achievable performance (roof height), while memory bandwidth determines the slope of performance change with computational intensity in data-intensive computing (eaves). In GPGPU chip hardware design, adding General Purpose Matrix Computing (GEMM) units can improve computing power, while expanding memory bandwidth or optimizing the storage hierarchy is key to improving bandwidth effectiveness. A reasonable cache configuration is a decisive factor in balancing GPGPU chip computing power and bandwidth resources, maximizing the hardware utilization of the GPGPU chip.
[0004] However, current cache performance analysis methods for GPGPUs have significant limitations when applied to emerging and complex workloads such as large language models and text-based graph models. Existing cache performance analysis methods are often disconnected from the actual fine-grained parallel computing scheduling process of GPGPUs, making it difficult to reflect the cache access characteristics during dynamic task execution. Furthermore, existing cache analysis methods are often based on simplified or single-level cache models and idealized memory access patterns, failing to accurately simulate the real-world behavior of complex multi-level cache architectures and the handling of diverse data layouts. These shortcomings make it difficult for existing cache analysis methods to accurately evaluate and optimize GPGPU cache designs for modern AI workloads, potentially leading to underutilization of computing power or bandwidth resources in chip design. Summary of the Invention Technical issues
[0005] The technical problem this application aims to solve is that existing GPGPU cache analysis methods are disconnected from the actual hardware execution process, resulting in inaccurate evaluation results and an inability to effectively guide chip design. This application provides a simulation analysis method and system for general-purpose graphics processor cache performance, as well as a storage medium, program product, and electronic device. This method simulates and analyzes the cache performance of a target general-purpose graphics processor when running a specific workload to be evaluated, improving the accuracy of cache performance analysis and thus effectively guiding the chip cache design of the target general-purpose graphics processor.
[0006] Technical solution
[0007] To address the aforementioned technical problems, according to a first aspect of this application, a method for simulating and analyzing the cache performance of a general-purpose graphics processor (GPU) is provided, comprising: acquiring model input information and configuration input information related to a target GPU, wherein the model input information characterizes the characteristics of the workload to be evaluated on the target GPU, and the configuration input information characterizes the characteristics of the hardware architecture of the target GPU; extracting model structure information of the model to be simulated from the model input information; constructing a cache topology structure of the cache to be simulated based on the configuration input information; generating grid scheduling information for scheduling the computational tasks of the model to be simulated to the target GPU based on the model structure information and the configuration input information; allocating memory base addresses for the input and output sources of each operator in the model to be simulated based on the model structure information; simulating the data loading and storage process of the execution unit of the target GPU when running the model to be simulated based on the grid scheduling information, the cache topology structure, and the memory base addresses; and generating a workload log of the execution unit and a cache behavior log of the cache to be simulated during the simulation data loading and storage process, for use in analyzing the cache performance of the target GPU.
[0008] The model input information obtained by the technical solution of this invention characterizes the features of the workload to be evaluated on the target general-purpose graphics processor (GPU). Therefore, the model structure information of the model to be simulated extracted from the model input information can accurately reflect the internal structure and computational tasks of the workload to be evaluated on the target GPU. Furthermore, since the configuration input information obtained by this invention characterizes the hardware architecture features of the target GPU, a virtual cache topology with multi-level cache characteristics is constructed based on the configuration input information. This topology is flexibly configurable and accurately reflects the hardware architecture of the target GPU.
[0009] Based on this, grid scheduling information is generated using model structure information and configuration input information to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor. This combines the computational task characteristics from the model side with the hardware scheduling parameters from the configuration side, proactively constructing a specific task deployment scheme that conforms to the hierarchical parallel execution model of the GPGPU grid-workgroup architecture. This grid scheduling information serves as a crucial hub connecting computation and hardware, ensuring that all subsequent data loading and storage simulation processes are conducted within a context strictly aligned with the actual execution scheduling mechanism of the GPGPU. This guarantees that every simulated cache access accurately reflects the behavior that task execution should produce under the target GPGPU hardware scheduling strategy, thus fundamentally overcoming the technical deficiency of existing cache analysis methods being disconnected from the actual execution process of GPGPU hardware.
[0010] Furthermore, the technical solution of this invention allocates memory base addresses to the input and output sources of each operator in the model to be simulated based on model structure information, establishing a precise logical address mapping for all data accessed during subsequent data loading and storage simulation. Then, based on grid scheduling information, cache topology, and memory base addresses, it simulates the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated. This achieves dynamic co-simulation of computational task scheduling, hardware storage model, and data addressing mechanism, ensuring that each simulated cache access accurately reflects the behavior generated when processing a specific data layout defined by memory base addresses and model input information in the complex multi-level cache system represented by the cache topology, under the execution sequence determined by the grid scheduling information. Moreover, during the simulation of data loading and storage, the generated workload logs of the execution unit and the cache behavior logs of the cache to be simulated can completely record the interaction details between the computation side and the storage side, providing a quantifiable data foundation for analyzing the cache performance of the target general-purpose graphics processor. Therefore, based on these two logs, the performance advantages and disadvantages of different hardware configuration schemes can be quickly and effectively evaluated, accurately guiding the chip cache design of the target general-purpose graphics processor.
[0011] In one possible implementation of the first aspect, the model structure information may include a hierarchical structure diagram of the model, extracting the model structure information of the model to be simulated from the model input information, including: extracting the model layer information of the model to be simulated from the model input information; and determining the hierarchical structure diagram of the model based on the model layer information.
[0012] The technical solution of this invention extracts a hierarchical structure diagram of the model from the model input information. This hierarchical structure diagram is a structured and traversable data representation form, which provides a clear operation path for subsequent operations such as base address allocation, so that the subsequent simulation process can process every computational unit in the model without omission.
[0013] In one possible implementation of the first aspect, the information for each layer of the model may include operator type information, operator input source information, weight source information, and output source size information for each layer of the model. This information ensures that the metadata extracted from the model can meet the input requirements of subsequent key steps such as base address allocation, and guarantees at least one of the following information from the data source: the accuracy of the entire simulation process.
[0014] In one possible implementation of the first aspect, the model input information may include at least one of the following: model layer number information, model layer structure information, data type information, and memory layout information. This information further improves the accuracy of the simulation analysis of the target general-purpose graphics processor cache performance and its ability to reflect real hardware operation.
[0015] In one possible implementation of the first aspect, the configuration input information may include hardware computing power bandwidth information, cache parameter information, and data routing information of the target general-purpose graphics processor; wherein, the hardware computing power bandwidth information includes target computing power information of the computing unit and / or target memory bandwidth information; the cache parameter information includes at least one of cache level number information, cache size information of each level, and cache replacement strategy information; and the data routing information includes data loading source path information and / or data storage destination path information.
[0016] The technical solution of this invention provides accurate and complete hardware architecture input specifications for subsequent steps by structurally classifying the configuration input information. Specifically, hardware computing power and bandwidth information provides a benchmark for evaluating the matching degree of computing power and bandwidth of the workload on the target general-purpose graphics processor; cache parameter information defines a set of configurable parameters for the static attributes of each cache level when constructing the cache topology to be simulated; and data routing information provides a basis for defining the dynamic flow rules of data between cache levels and between the cache and global memory within the same topology. The input of these structured parameters enables the final constructed cache topology to become a high-fidelity, configurable virtual hardware storage model, thereby supporting the subsequent simulation process in an environment that more closely resembles real hardware constraints. Finally, the workload logs and cache behavior logs generated based on this provide quantifiable and comparable evaluation criteria for the performance impact of different cache configuration schemes on specific workloads, effectively improving the efficiency of chip cache subsystem design space exploration and the accuracy of design decisions.
[0017] In one possible implementation of the first aspect, the cache to be simulated may include a multi-level cache. The cache topology structure of the cache to be simulated is constructed based on configuration input information, including: determining the capacity, organization method, and data replacement strategy of each level of the cache to be simulated based on cache parameter information in the configuration input information, to define the hierarchical static structure of the cache; and determining the loading and storage directions of data between each level of the hierarchical static structure and between each level of the cache and global memory based on data routing information in the configuration input information, to define the data path structure of the cache; wherein the cache topology structure consists of the hierarchical static structure and the data path structure.
[0018] This provides a high-fidelity, well-defined virtual storage system model for subsequent simulation data loading and storage processes, ensuring that each cache access during the simulation is processed in a multi-level structure that conforms to the target hardware data path rules, thereby helping to generate an accurate cache behavior log.
[0019] In one possible implementation of the first aspect, the configuration input information may further include grid configuration parameters and execution unit configuration parameters. Grid scheduling information for scheduling computational tasks of the model to be simulated to the target general-purpose graphics processor is generated based on the model structure information and the configuration input information. This includes: determining the dimension of the computational grid based on the output size of each operator in the model structure information; dividing the computational grid into multiple working groups based on the grid configuration parameters in the configuration input information, determining the traversal order of the multiple working groups in the computational grid, and generating grid index information based on the traversal order; generating computational scheduling entries for each grid in the grid index information based on the grid index information and the execution unit configuration parameters in the configuration input information, and generating computational scheduling information based on the computational scheduling entries; and using the computational scheduling information as grid scheduling information. The grid configuration parameters include at least one of the following parameters: the number of working groups in the X direction of the grid, the number of working groups in the Y direction of the grid, and the working group traversal order layout.
[0020] In this way, by combining the computational characteristics of the model with the scheduling and resource configuration characteristics of the hardware, specific and executable grid scheduling information is generated, providing a precise and time-sequential task execution context for the subsequent simulation process. This ensures that the data loading and storage behavior of the simulation can closely follow the parallel scheduling logic of the target GPGPU and avoid decoupling from the hardware execution process.
[0021] In one possible implementation of the first aspect, the workgroup traversal order layout can be one of a linear X-order layout, a linear Y-order layout, or a Z-Order layout. This allows the technical solution of the present invention to simulate and compare the impact of different task scheduling strategies on data access locality. Different traversal order layouts result in different execution orders of workgroups, directly affecting memory access patterns. This enables a quantitative assessment of the impact of different scheduling strategies on the final cache hit rate and memory bandwidth utilization, providing important simulation and analysis capabilities for task scheduler optimization or strategy selection in chip design.
[0022] In one possible implementation of the first aspect, based on model structure information, memory base addresses are allocated to the input and output sources of each operator in the model to be simulated. This may include: traversing each layer and each operator in the hierarchical structure graph of the model to be simulated; and calculating and allocating the corresponding memory base addresses according to the size and data type of each input and output source of the current operator.
[0023] In this way, a precise logical starting address mapping can be established in the virtual memory space for all data that will be accessed in the simulation. The allocated memory base address is a necessary input for calculating the access address of data in memory during subsequent simulations, providing the addressing origin for each memory access operation in the simulation, and ensuring the accuracy and reproducibility of the simulated memory access behavior.
[0024] In one possible implementation of the first aspect, simulating the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated, based on grid scheduling information, cache topology, and memory base address, may include: selecting the activation matrix calculation core unit or arithmetic logic calculation unit as the current execution unit according to the type of the currently executed operator; determining the grid and workgroup corresponding to the data to be loaded or stored according to grid scheduling information; calculating the access address of the data in memory according to the memory base address and the memory layout information in the model input information; and simulating the data load hit, load miss, store hit, or store miss behavior in the cache topology according to the access address.
[0025] This ensures that each simulated data access becomes a complete behavioral chain, where a specific computing unit initiates a query for a specific memory address within a specific cache system model while executing a specific scheduled task. Through this refined process decomposition and simulation, the technical solution of this invention can accurately reflect the real and dynamic cache interaction behavior exhibited by a target general-purpose graphics processor running a model with a specific memory layout under a specific grid scheduling strategy, thereby improving the accuracy and fidelity of cache performance simulation.
[0026] In one possible implementation of the first aspect, the operator type may include a general matrix multiplication type or a general matrix-vector multiplication type. Since the operator types of this invention support these two types of computationally intensive operators, which are the most core and time-consuming in the fields of artificial intelligence and high-performance computing, the technical solution of this invention can accurately simulate and analyze the pressure characteristics of the most important computational modes in modern AI workloads (such as deep learning training and inference) on the caching system. This enables the technical solution of this invention to accurately evaluate and optimize GPGPU cache designs for modern AI workloads.
[0027] In one possible implementation of the first aspect, the workload log of the execution unit may include at least one of the following: the load loaded by the execution unit, the load stored, and the number of computational operations completed.
[0028] By recording the load loaded by the execution unit and the load stored, the total access requirements of the computing task to the memory subsystem can be directly quantified, providing a direct basis for assessing memory bandwidth pressure. Recording the number of computational operations completed by the execution unit can accurately measure the actual workload of the computing unit. Combining these three factors allows for the calculation of the computational intensity of the workload on the actual hardware model, providing a quantitative analytical basis for optimizing the chip's computing power and bandwidth resource allocation.
[0029] In one possible implementation of the first aspect, the cache behavior log of the cache to be simulated includes at least one of the following: load load, storage load, cache hit count, and cache miss count for each level of cache. This hierarchical, fine-grained cache behavior log information enables performance analysis to accurately pinpoint the bottleneck level of the caching system and quantitatively evaluate the performance gains brought about by different cache configuration parameters, providing a basis for targeted cache optimization decisions.
[0030] In one possible implementation of the first aspect, the method for simulating and analyzing the cache performance of a general-purpose graphics processor further includes: generating a cache performance analysis chart of the target general-purpose graphics processor based on the workload logs of the execution unit and the cache behavior logs of the cache to be simulated. Because the cache performance analysis chart is more intuitive, designers can quickly understand the cache performance of the target general-purpose graphics processor through it, thus helping to shorten the design process.
[0031] In one possible implementation of the first aspect, a cache performance analysis chart is used to illustrate at least one of the following relationships: the relationship between memory traffic and cache size, the relationship between memory bandwidth and cache size, and the relationship between cache miss rate and cache size. These three relationships serve as a visual basis for evaluating the performance impact of different cache configuration schemes. The cache performance analysis chart can intuitively reveal the changing patterns of key cache performance indicators—memory traffic, memory bandwidth utilization, and cache miss rate—of the target general-purpose graphics processor when running the workload to be evaluated, as well as configuration parameters such as cache size, thereby providing a design basis for selecting the optimal cache configuration parameters in actual chip design.
[0032] According to a second aspect of this application, a simulation analysis system for general-purpose graphics processor (GPU) cache performance is provided, comprising: an information acquisition unit for acquiring model input information and configuration input information related to a target GPU, wherein the model input information characterizes the characteristics of the workload to be evaluated on the target GPU, and the configuration input information characterizes the characteristics of the hardware architecture of the target GPU; a model extractor for extracting model structure information of the model to be simulated from the model input information; a cache topology builder for constructing a cache topology structure of the cache to be simulated based on the configuration input information; and a grid scheduling generator for generating a grid schedule based on the model structure information and the configuration input information. The system comprises: a grid scheduling information unit for scheduling computational tasks of the model to be simulated to the target general-purpose graphics processor; a base address generator for allocating memory base addresses for the input and output sources of each operator in the model to be simulated based on the model structure information; a performance simulation unit for simulating the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated, based on the grid scheduling information, cache topology, and memory base addresses; and a log generation unit for generating workload logs of the execution unit and cache behavior logs of the cache to be simulated during the simulation data loading and storage process, for analysis of the cache performance of the target general-purpose graphics processor.
[0033] In the simulation and analysis system for general-purpose graphics processor (GPU) cache performance provided in this application, the model input information acquired by the information acquisition unit characterizes the features of the workload to be evaluated on the target GPU. Therefore, the model structure information of the model to be simulated extracted by the model extractor from the model input information can accurately reflect the internal structure and computational tasks of the workload to be evaluated on the target GPU. Furthermore, since the configuration input information acquired by the information acquisition unit characterizes the hardware architecture features of the target GPU, the cache topology builder constructs a flexibly configurable virtual cache topology that accurately reflects the multi-level cache characteristics of the target GPU's hardware architecture based on the configuration input information.
[0034] Building upon this foundation, the grid scheduler generates grid scheduling information based on model structure and configuration input information. This information is used to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor (GPGPU). This combines the computational task characteristics from the model side with the hardware scheduling parameters from the configuration side, proactively constructing a specific task deployment scheme that conforms to the hierarchical parallel execution model of the GPGPU grid-workgroup architecture. This grid scheduling information serves as a crucial link between computation and hardware, ensuring that all subsequent data loading and storage simulation processes executed by the performance simulation unit are performed within a context strictly aligned with the actual execution scheduling mechanism of the GPGPU. This guarantees that every simulated cache access accurately reflects the behavior that task execution should produce under the target GPGPU hardware scheduling strategy, thus overcoming the technical deficiency of existing cache analysis methods being disconnected from the actual execution process of GPGPU hardware at the system level.
[0035] Furthermore, the technical solution of this invention, through a base address generator, allocates memory base addresses to the input and output sources of each operator in the model to be simulated based on model structure information, establishing a precise logical address mapping for all data accessed during subsequent data loading and storage simulation. In addition, based on grid scheduling information, cache topology, and memory base addresses, the performance simulation unit of the target general-purpose graphics processor simulates the data loading and storage process when running the model to be simulated. This achieves dynamic collaborative simulation of computational task scheduling, hardware storage model, and data addressing mechanism, ensuring that each simulated cache access accurately reflects the behavior generated when processing a specific data layout defined by memory base addresses and model input information in a complex multi-level cache system represented by the cache topology, under the execution sequence determined by the grid scheduling information. Moreover, during the simulation of data loading and storage, the log generation unit generates workload logs for the execution unit and cache behavior logs for the cache to be simulated, which can completely record the interaction details between the computation side and the storage side. This provides a quantifiable data foundation for analyzing the cache performance of the target general-purpose graphics processor, allowing for rapid and effective evaluation of the performance of different hardware configurations based on these two logs, and accurately guiding the chip cache design of the target general-purpose graphics processor.
[0036] In one possible implementation of the second aspect, the grid scheduling generator may further include a grid generator and a computation scheduling table generator; the grid generator is used to determine the dimension of the computation grid according to the output size of each operator in the model structure information, and to divide the computation grid into multiple working groups according to the grid configuration parameters in the configuration input information, and to determine the traversal order of the multiple working groups in the computation grid, and to generate grid index information based on the traversal order; and the computation scheduling table generator is used to generate a computation scheduling entry for each grid in the grid index information according to the grid index information and the execution unit configuration parameters in the configuration input information, and to generate computation scheduling information based on the computation scheduling entries, and to use the computation scheduling information as grid scheduling information.
[0037] In this way, by combining the computational characteristics of the model with the scheduling and resource configuration characteristics of the hardware, specific and executable grid scheduling information is generated, providing a precise and time-sequential task execution context for the subsequent simulation process. This ensures that the data loading and storage behavior of the simulation can closely follow the parallel scheduling logic of the target GPGPU and avoid decoupling from the hardware execution process.
[0038] In one possible implementation of the second aspect, the simulation and analysis system for general-purpose graphics processor (GPU) cache performance may further include a visualization unit for generating cache performance analysis charts of the target GPU based on the workload logs of the execution unit and the cache behavior logs of the cache to be simulated. Because the cache performance analysis charts are more intuitive, designers can quickly understand the cache performance of the target GPU through these charts, thus helping to shorten the design process.
[0039] According to a third aspect of this application, a computer-readable storage medium is also provided, on which instructions are stored, which, when executed on an electronic device, cause the electronic device to perform a simulation analysis method for the performance of a general-purpose graphics processor cache as described in the first aspect and any possible implementation thereof.
[0040] According to a fourth aspect of this application, a computer program product is also provided, the computer program product including instructions, which, when executed by one or more processors, are used to implement a simulation analysis method for the performance of a general-purpose graphics processor cache as described in the first aspect and any possible implementation of the first aspect.
[0041] According to a fifth aspect of this application, an electronic device is also provided, comprising: a memory for storing instructions, and one or more processors, wherein when the instructions are executed by the one or more processors, the processors execute a simulation analysis method for general graphics processor cache performance as described in the first aspect and any possible implementation thereof. Beneficial effects
[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0043] The technical solution of this invention acquires model input information and configuration input information. The acquired model input information characterizes the characteristics of the workload to be evaluated on the target general-purpose graphics processor (GPU), thereby extracting the model structure information of the model to be simulated from the model input information, which can accurately reflect the internal structure and computational tasks of the workload to be evaluated on the target GPU. Furthermore, since the configuration input information acquired by this invention characterizes the hardware architecture characteristics of the target GPU, a virtual cache topology with multi-level cache characteristics in the hardware architecture of the target GPU is constructed based on the configuration input information. This topology is flexibly configurable and accurately reflects the hardware architecture of the target GPU.
[0044] Based on this, grid scheduling information is generated using model structure information and configuration input information to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor. This combines the computational task characteristics from the model side with the hardware scheduling parameters from the configuration side, proactively constructing a specific task deployment scheme that conforms to the hierarchical parallel execution model of the GPGPU grid-workgroup architecture. This grid scheduling information serves as a crucial hub connecting computation and hardware, ensuring that all subsequent data loading and storage simulation processes are conducted within a context strictly aligned with the actual execution scheduling mechanism of the GPGPU. This guarantees that every simulated cache access accurately reflects the behavior that task execution should produce under the target GPGPU hardware scheduling strategy, thus fundamentally overcoming the technical deficiency of existing cache analysis methods being disconnected from the actual execution process of GPGPU hardware.
[0045] Furthermore, the technical solution of this invention allocates memory base addresses to the input and output sources of each operator in the model to be simulated based on model structure information, establishing a precise logical address mapping for all data accessed during subsequent data loading and storage simulation. Then, based on grid scheduling information, cache topology, and memory base addresses, it simulates the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated. This achieves dynamic co-simulation of computational task scheduling, hardware storage model, and data addressing mechanism, ensuring that each simulated cache access accurately reflects the behavior generated when processing a specific data layout defined by memory base addresses and model input information in the complex multi-level cache system represented by the cache topology, under the execution sequence determined by the grid scheduling information. Moreover, during the simulation of data loading and storage, the generated workload logs of the execution unit and the cache behavior logs of the cache to be simulated can completely record the interaction details between the computation side and the storage side, providing a quantifiable data foundation for analyzing the cache performance of the target general-purpose graphics processor. Therefore, based on these two logs, the performance advantages and disadvantages of different hardware configuration schemes can be quickly and effectively evaluated, accurately guiding the chip cache design of the target general-purpose graphics processor. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 According to some embodiments of the present invention, a flowchart of a simulation analysis method for the cache performance of a general-purpose graphics processor is shown;
[0048] Figure 2 According to some embodiments of the present invention, a simplified hierarchical structure diagram is shown;
[0049] Figure 3 According to some embodiments of the present invention, a cache topology is shown;
[0050] Figure 4 According to some embodiments of the present invention, a flowchart for generating grid scheduling information is shown;
[0051] Figure 5 According to some embodiments of the present invention, a schematic diagram of a linear X-order layout of a workgroup is shown;
[0052] Figure 6According to some embodiments of the present invention, a schematic diagram of a linear Y-order layout of a workgroup is shown;
[0053] Figure 7 According to some embodiments of the present invention, a schematic diagram of a Z-Order layout for a workgroup is shown;
[0054] Figure 8 According to some embodiments of the present invention, a flowchart of memory base address allocation is shown;
[0055] Figure 9 According to some embodiments of the present invention, a simplified flowchart is shown for selecting the activation matrix calculation core unit or the arithmetic logic calculation unit as the current execution unit;
[0056] Figure 10 According to some embodiments of the present invention, a computation process for a GEMM operator is shown;
[0057] Figure 11 According to some embodiments of the present invention, a relationship between memory flow and cache size, a relationship between memory bandwidth and cache size, a relationship between cache miss rate of Level 1 cache and cache size, and a relationship between cache miss rate of Level N cache and cache size are shown.
[0058] Figure 12 According to some embodiments of the present invention, a system architecture for simulating and analyzing the performance of a general-purpose graphics processor cache is shown;
[0059] Figure 13 According to some embodiments of the present invention, a structural block diagram of an electronic device is shown. Embodiments of the present invention
[0060] The illustrative embodiments of the present invention include, but are not limited to, a method and system for simulating and analyzing the performance of a general-purpose graphics processor cache, a storage medium, a program product, and an electronic device.
[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0062] As mentioned earlier, existing GPGPU cache analysis methods suffer from technical problems such as being disconnected from the actual hardware execution process, resulting in inaccurate evaluation results and an inability to effectively guide chip design.
[0063] In view of this, the present invention provides a method and system for simulating and analyzing the cache performance of a general-purpose graphics processor, as well as a storage medium, a program product, and an electronic device. This method can simulate and analyze the cache performance of a target general-purpose graphics processor when running a specific workload to be evaluated, thereby improving the accuracy of cache performance analysis and effectively guiding the chip cache design of the target general-purpose graphics processor.
[0064] Figure 1 According to some embodiments of the present invention, a flowchart of a simulation analysis method for the cache performance of a general-purpose graphics processor is shown, with reference to... Figure 1 The present invention provides a simulation analysis method for the cache performance of a general-purpose graphics processor, comprising the following steps S11 to S17:
[0065] Step S11: Obtain model input information and configuration input information related to the target general-purpose graphics processor, wherein the model input information characterizes the characteristics of the workload to be evaluated of the target general-purpose graphics processor, and the configuration input information characterizes the characteristics of the hardware architecture of the target general-purpose graphics processor.
[0066] The target general-purpose graphics processor (GPGPU) is the GPGPU for which designers need to perform cache performance analysis. For example, the target GPGPU could be a next-generation AI acceleration chip under design, codenamed "GPGPU-AI-2026," whose cache subsystem is a core component that needs to be evaluated and optimized. The simulation analysis method provided in this invention ultimately aims to provide performance profiling data and optimization guidance for the cache design of target GPGPUs like "GPGPU-AI-2026."
[0067] In some embodiments, the workload to be evaluated on the target general-purpose graphics processor (GPU) refers to a specific artificial intelligence (AI) model that needs to run efficiently on the target GPU, and the features of this model are described as the model's input information. This AI model includes, but is not limited to, Large Language Models (LLMs), text-based graph models, and convolutional models.
[0068] Step S12: Extract the model structure information of the model to be simulated from the model input information.
[0069] The model to be simulated refers to the structured data extracted from the model input information that characterizes the features of the target general-purpose graphics processor's workload to be evaluated.
[0070] In some embodiments, the model structure information includes a hierarchical structure diagram of the model. Extracting the model structure information of the model to be simulated from the model input information includes: extracting the model layer information of the model to be simulated from the model input information; and determining the hierarchical structure diagram of the model based on the model layer information.
[0071] In some embodiments, the hierarchical structure graph of the model presents the computational dependencies of the model to be simulated in a tree-like topology. Figure 2 A simplified hierarchical structure diagram is shown, see reference. Figure 2 The top-level node of this hierarchical structure diagram is the layer ID, representing the hierarchical identifier of the model. The next-level node after the layer ID is operator 1; the next-level nodes after operator 1 are operator 2 and operator 3; the next-level node after operator 2 is operator 6; and the next-level nodes after operator 3 are operator 4 and operator 5. Operators are connected by arrows to form parent-child dependencies. For example, operator 1 is the parent node, which contains two child nodes, operator 2 and operator 3. Operator 2 contains operator 6, and operator 3 contains operator 4 and operator 5. Each operator node contains information such as operator type, input, output, and data type.
[0072] The technical solution of this invention uses a hierarchical structure diagram of the model extracted from the model input information. This is a structured and traversable data representation, which provides a clear operation path for subsequent operations such as base address allocation. This allows the subsequent simulation process to process every computational unit in the model to be simulated without omission.
[0073] In some embodiments, the model layer information includes at least one of the following: operator type information, operator input source information, weight source information, and output source size information for each layer of the model. This information ensures that the metadata extracted from the model can meet the input requirements of subsequent key steps such as base address allocation, thus guaranteeing the accuracy of the entire simulation process from the data source.
[0074] In some embodiments, the model input information includes at least one of the following: model layer number information, model layer structure information, data type information, and memory layout information. This information further improves the accuracy of the simulation analysis of the target general-purpose graphics processor cache performance and its ability to reflect real hardware operation.
[0075] Step S13: Construct the cache topology structure of the cache to be simulated based on the configuration input information.
[0076] In some embodiments, the cache to be simulated does not refer to the actual physical cache hardware in the target general-purpose graphics processor, but rather to a virtual multi-level storage system model simulated by software based on configuration input information characterizing the hardware architecture features of the target general-purpose graphics processor.
[0077] In some embodiments, the configuration input information includes hardware computing power bandwidth information, cache parameter information, and data routing information of the target general-purpose graphics processor. The hardware computing power bandwidth information includes target computing power information of the computing unit and / or target memory bandwidth information. The cache parameter information includes at least one of the following: cache level number information, cache size information of each level, and cache replacement policy information. The data routing information includes data loading source path information and / or data storage destination path information. In some embodiments, the data routing information includes data loading source path information. In other embodiments, the data routing information includes data storage destination path information. In still other embodiments, the data routing information includes both data loading source path information and data storage destination path information.
[0078] The technical solution of this invention provides accurate and complete hardware architecture input specifications for subsequent steps by structurally classifying the configuration input information. Specifically, hardware computing power and bandwidth information provides a benchmark for evaluating the matching degree of computing power and bandwidth of the workload on the target general-purpose graphics processor; cache parameter information defines a set of configurable parameters for the static attributes of each cache level when constructing the cache topology to be simulated; and data routing information provides a basis for defining the dynamic flow rules of data between cache levels and between the cache and global memory within the same topology. The input of these structured parameters enables the final constructed cache topology to become a high-fidelity, configurable virtual hardware storage model, thereby supporting the subsequent simulation process in an environment that more closely resembles real hardware constraints. Finally, the workload logs and cache behavior logs generated based on this provide quantifiable and comparable evaluation criteria for the performance impact of different cache configuration schemes on specific workloads, effectively improving the efficiency of chip cache subsystem design space exploration and the accuracy of design decisions.
[0079] In some embodiments, the cache to be simulated includes a multi-level cache. Constructing the cache topology based on configuration input information includes: determining the capacity, organization, and data replacement strategy of each level of the cache to be simulated based on cache parameter information in the configuration input information, to define the hierarchical static structure of the cache; and determining the loading and storage flow of data between each level of the hierarchical static structure and between each level of the cache and global memory based on data routing information in the configuration input information, to define the data path structure of the cache. The cache topology consists of the hierarchical static structure and the data path structure. This provides a high-fidelity, well-defined virtual storage system model for subsequent simulated data loading and storage processes, ensuring that each cache access during the simulation is processed within a multi-level structure that conforms to the target hardware data path rules, thereby contributing to the generation of accurate cache behavior logs.
[0080] For example, Table 1 illustrates the cache parameter information and data routing information in a set of configuration inputs. Table 1 defines the capacity, organization method, replacement strategy, loading source, and storage purpose of each cache level. Specifically, referring to Table 1, it includes the following fields:
[0081] Cache Level: Used to identify the cache level, such as L1_A, L1_B, L2, L3, where L1_A and L1_B belong to the L1 level cache.
[0082] Here, Set: indicates the number of sets in each cache level.
[0083] Way: Represents the number of paths (set associativity) at each cache level.
[0084] Line size (B): Indicates the size of the cached line in bytes.
[0085] Replacement algorithm (Policy): Represents the cache replacement strategy, such as LRU, Random, PLRU.
[0086] LoadSrc: Indicates the level or storage unit from which data is loaded when a cache miss occurs at this level.
[0087] StoreDestination: Indicates the level or storage unit to which data is written when a storage operation occurs at this level of cache.
[0088] Table 1
[0089]
[0090] In some embodiments, based on the cache parameter information and data routing information shown in Table 1, the following can be obtained: Figure 3 The cache topology is shown. For example, based on the cache level, group, path, row size, and replacement algorithm fields in each row of Table 1, the capacity, organization, and data replacement strategy of each cache level in the cache to be simulated are determined; based on the load source and storage destination fields in each row of Table 1, the loading and storage flow of data between different cache levels, and between the cache and global memory (GM), is determined. Thus, the following is obtained: Figure 3 The cache topology shown.
[0091] refer to Figure 3The hierarchical static structure of this cache topology includes L1_A, L1_B, L2, and L3. L1_A and L1_B belong to the first-level cache, L2 is the second-level cache, and L3 is the third-level cache. Within the execution unit, "Input A," "Input B," and "Output C" represent different computational data flows when the execution unit runs the model to be simulated. For example, Input A and Input B are input matrix data, and Output C is the calculated result matrix data.
[0092] Figure 3 The data path structure of the cache topology shown is as follows:
[0093] The "Input A" data stream within the execution unit accesses the L1_A cache via a "load" operation. If L1_A is hit, the data is "returned" to the execution unit. If L1_A is missing, a "missing" signal is triggered, and data is requested from the L2 cache. If L2 is hit, the missing data is "returned" to L1_A. If L2 is missing, data is requested from the L3 cache. If L3 is hit, the missing data is "returned" to L2. If L3 is missing, data is finally requested from global memory, and then returned from global memory to L3, L2, and L1_A before being returned to the execution unit.
[0094] The "input B" data stream within the execution unit accesses the L1_B cache through the "load" operation. If a hit occurs, it "returns"; if a missing data is detected, a "missing" signal is triggered and data is requested from the L2 cache. The subsequent missing data handling process is the same as that of L1_A and will not be described in detail here.
[0095] The "Output C" data stream within the execution unit is written to the L1_B cache via the "Store to" operation, or to other caches depending on the storage destination configuration. If the L1_B cache hits, the storage is completed; if a miss occurs, a "Missing" signal is triggered, and storage space is requested from the L2 cache, or the data is sent back and then stored. The missing handling process for L2, L3, and global memory is consistent with the reverse path logic of the load operation: that is, storage support or data back is requested layer by layer from L1_B→L2→L3→GM. Simultaneously, L2 receives missing requests from L1_A and L1_B and forwards them to L3; L3 receives missing requests from L2 and forwards them to GM; after GM responds to the request from L3, the data is sent back along the path L3→L2→L1_A (or L1_B); after L2 responds to the missing requests from L1_A and L1_B, the data is sent back along the path L2→L1_A (or L1_B), thus forming a complete bidirectional path structure for multi-level cache data loading and storage.
[0096] Step S14: Generate grid scheduling information based on model structure information and configuration input information to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor.
[0097] like Figure 4As shown, the configuration input information also includes grid configuration parameters and execution unit configuration parameters. Based on the model structure information and configuration input information, grid scheduling information is generated to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor, including the following steps S141 to S144:
[0098] Step S141: Determine the dimensions of the computational grid based on the output dimensions of each operator in the model structure information.
[0099] Here, the output of the operator can be a multidimensional tensor, and the output size of the operator refers to the number of elements in each dimension of this multidimensional tensor. For example, suppose the output of the operator is a two-dimensional tensor, that is, a matrix, then the output size of this matrix is usually expressed as [M,N], where M represents the number of rows of the matrix and N represents the number of columns of the matrix.
[0100] The dimension of the computational grid refers to the number of logical units contained in the computational grid in the X-axis (horizontal) direction and the Y-axis (vertical) direction, respectively. For example, for an operator with an output size of [1024, 512], its computational grid dimension can be defined as (32, 16). This means that logically, this computational task is divided into 32 parts in the X-direction and 16 parts in the Y-direction.
[0101] Step S142: Based on the grid configuration parameters in the configuration input information, divide the computing grid into multiple working groups, determine the traversal order of the multiple working groups in the computing grid, and generate grid index information based on the traversal order.
[0102] Step S143: Based on the grid index information and the execution unit configuration parameters in the configuration input information, generate a computation scheduling entry for each grid in the grid index information, and generate computation scheduling information based on the computation scheduling entry.
[0103] Step S144: And use the computation scheduling information as grid scheduling information. The grid configuration parameters include at least one of the following: the number of workgroups in the X direction, the number of workgroups in the Y direction, and the workgroup traversal order layout.
[0104] In this way, by combining the computational characteristics of the model with the scheduling and resource configuration characteristics of the hardware, specific and executable grid scheduling information is generated, providing a precise and time-sequential task execution context for the subsequent simulation process. This ensures that the data loading and storage behavior of the simulation can closely follow the parallel scheduling logic of the target GPGPU and avoid decoupling from the hardware execution process.
[0105] In some embodiments, the workgroup traversal order layout is one of a linear X-order layout, a linear Y-order layout, or a Z-Order layout. Figures 5 to 7 The following examples illustrate three traversal order layout methods for the workgroup: linear X-order layout, linear Y-order layout, and Z-Order layout.
[0106] refer to Figure 5 There are 4 working groups in the X direction and 2 working groups in the Y direction, for a total of 8 working groups. First, traverse the 4 working groups WG0 to WG3 in the first row, and then traverse the 4 working groups WG4 to WG7 in the second row.
[0107] refer to Figure 6 There are 4 working groups in the X direction and 2 working groups in the Y direction, for a total of 8 working groups. First, iterate through the 2 working groups WG0 and WG4 in the first column, then iterate through the 2 working groups WG1 and WG5 in the second column, then iterate through the 2 working groups WG2 and WG6 in the third column, and finally iterate through the 2 working groups WG3 and WG7 in the fourth column.
[0108] refer to Figure 7 There are 4 working groups in the X direction and 2 working groups in the Y direction, for a total of 8 working groups. First, iterate through the first two working groups WG0 and WG1 in the first row, then iterate through the first two working groups WG4 and WG5 in the second row. Then iterate through the last two working groups WG2 and WG3 in the first row, and the last two working groups WG6 and WG7 in the second row.
[0109] Therefore, the technical solution of this invention can simulate and compare the impact of different task scheduling strategies on data access locality. Different traversal order layouts will lead to different execution orders of workgroups, thereby directly affecting memory access patterns. This allows for a quantitative evaluation of the impact of different scheduling strategies on the final cache hit rate and memory bandwidth utilization, providing important simulation and analysis capabilities for the optimization of task schedulers or strategy selection in chip design.
[0110] In some embodiments, the workgroup traversal order layout can also be dynamically generated based on the output size of each operator, further improving the accuracy of cache performance simulation and enhancing the practical design guidance value of the simulated cache performance results.
[0111] Step S15: Based on the model structure information, allocate memory base addresses for the input and output sources of each operator in the model to be simulated.
[0112] Here, the input source of an operator refers to the input data that an operator in the model to be simulated depends on when performing its computational function.
[0113] The output source of an operator refers to the output tensor generated by the operator's computation. For example, a matrix multiplication operator outputs a result matrix C; a convolution operator outputs a feature map.
[0114] In some embodiments, based on model structure information, memory base addresses are allocated to the input and output sources of each operator in the model to be simulated, including: traversing each layer and each operator in the hierarchical structure graph of the model to be simulated; and calculating and allocating the corresponding memory base addresses according to the size and data type of each input and output source of the current operator.
[0115] Figure 8 The diagram illustrates the detailed memory base address allocation process for an operator with two input sources, namely input source A and input source B, and one output source C. Figure 8 The flowchart shown includes the following steps S150 to S159:
[0116] Step S150: Initialize the base address.
[0117] In some embodiments, initializing the base address means setting the starting logical address of the virtual memory space to zero (base_addr = 0). This step defines the addressing origin for subsequent allocation of memory addresses for all input and output sources, ensuring that the starting point for base address calculation is unique and unambiguous.
[0118] Step S151: Traverse the model layers.
[0119] In some embodiments, traversing model layers means visiting each layer in a loop according to the hierarchical order defined in the hierarchical structure graph of the model to be simulated. This step is implemented through an outer loop, using the layer index (Layer_Id) as the loop variable, to ensure that each layer of the model and all its internal computational units can be processed by subsequent steps, thereby ensuring the coverage and completeness of memory base address allocation for the entire model.
[0120] Step S152: Obtain the current layer structure.
[0121] In some embodiments, obtaining the current layer structure means retrieving the corresponding current layer structure (Cur_Layer = LAYERS[Layer_Id]) from the model's layer set based on the current layer index (Layer_Id). The current layer structure contains model structure information for all operators within that layer and serves as the basis for subsequent traversal and processing of specific computational tasks within that layer.
[0122] Step S153: Traverse the operators of the current layer.
[0123] In some embodiments, traversing the operators of the current layer means: within the acquired current layer structure, visiting each operator in a loop according to the order defined in its hierarchical structure diagram. This step is implemented through an inner loop, using the operator index (Op_Id) as the loop variable, to ensure that the input and output sources of each operator in the current layer can be processed by subsequent steps, thereby completing the allocation of base addresses one by one.
[0124] Step S154: Process the base address allocation of input source A.
[0125] In some embodiments, processing the base address allocation of input source A refers to allocating a logical memory address for input source A of the current operator (identified by Op_Id). The calculation of its base address is based on the size and data type of input source A. Specifically, if the current layer is the first layer of the model, its base address is equal to the currently accumulated base_addr; otherwise, its base address is base_addr plus the byte space occupied by input source A after rounding up. Input source A can be, for example, the first input matrix in matrix multiplication. The size of input source A includes, for example, height and width information. The data type can be the number of bytes corresponding to input source A.
[0126] Step S155: Process the base address allocation of input source B.
[0127] In some embodiments, processing the base address allocation of input source B means allocating a logical memory address for the input source B of the current operator. The calculation principle of its base address is the same as in step S154; based on the size and data type of input source B, its base address is calculated and allocated after the address space of input source A. Input source B can, for example, be the second input matrix in matrix multiplication.
[0128] Step S156: Process the base address allocation of output source C.
[0129] In some embodiments, processing the base address allocation of the output source C means allocating a logical memory address for the output source C of the current operator. Its base address is calculated after the address space of the input source B, and address alignment is ensured by rounding up based on the size and data type of the output source C. The output source can be, for example, the result matrix of matrix multiplication.
[0130] Step S157: Update the base address.
[0131] In some embodiments, updating the base address means that after processing the output source C of the current operator, the cumulative address base_addr is updated to the base address of the output source C. This step sets the base address of the output source C of the current operator as the starting point for address calculation of the next operator or data source to be processed. This ensures that the address allocation of different operators and their data sources in the virtual memory space is strictly continuous and ordered, fundamentally avoiding address overlap and facilitating accurate addressing in subsequent simulation processes.
[0132] Step S158: Inner loop iteration.
[0133] In some embodiments, the inner loop iteration refers to: after all base address allocations for the current operator are completed, the operator index (Op_Id) is incremented by 1 (i.e., Op_Id += 1) to point to the next operator in the current layer, and the processing of the next operator continues. This step drives the inner loop that traverses the operators in the current layer until all operators in the current layer have been processed.
[0134] Step S159: Iterate through the outer loop.
[0135] In some embodiments, the outer loop iteration refers to: after completing the base address allocation for all operators in the current layer, incrementing the layer index (Layer_Id) by 1 (i.e., Layer_Id += 1) to point to the next layer in the model, and continuing to process the next layer. This step drives the outer loop that traverses the model layers until all layers of the model have been processed, thereby completing the memory base address allocation for all data sources of the entire model to be simulated.
[0136] In this way, a precise logical starting address mapping can be established in the virtual memory space for all data that will be accessed in the simulation. The allocated memory base address is a necessary input for calculating the access address of data in memory during subsequent simulations, providing the addressing origin for each memory access operation in the simulation, and ensuring the accuracy and reproducibility of the simulated memory access behavior.
[0137] Please continue to refer to this. Figure 1 The following will continue the introduction. Figure 1 The flowchart illustrates the simulation analysis method for the cache performance of a general graphics processor.
[0138] Step S16: Based on the grid scheduling information, cache topology, and memory base address, simulate the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated.
[0139] In some embodiments, based on grid scheduling information, cache topology, and memory base address, the data loading and storage process of the execution unit of the simulated target general-purpose graphics processor when running the model to be simulated is as follows.
[0140] Based on the type of operator being executed, either the activation matrix computation core unit or the arithmetic logic computation unit is selected as the current execution unit. Based on the grid scheduling information, the grid and workgroup corresponding to the data to be loaded or stored are determined. Based on the memory base address and the memory layout information in the model input information, the access address of the data in memory is calculated. Based on the access address, the data load hit, load miss, store hit, or store miss behaviors are simulated in the cache topology.
[0141] This ensures that each simulated data access becomes a complete behavioral chain, where a specific computing unit initiates a query for a specific memory address within a specific cache system model while executing a specific scheduled task. Through this refined process decomposition and simulation, the technical solution of this invention can accurately reflect the real and dynamic cache interaction behavior exhibited by a target general-purpose graphics processor running a model with a specific memory layout under a specific grid scheduling strategy, thereby improving the accuracy and fidelity of cache performance simulation.
[0142] In some embodiments, the operator type includes GEMM or GEMV types. For example, such as Figure 9 As shown, if the operator type is General Matrix Multiplication (GEMM), the matrix computation core unit is activated; if the operator type is General Matrix-Vector Multiplication (GEMV) or arithmetic logic operation, the arithmetic logic unit (ALU) is activated. The computation process of the GEMM operator is as follows: Figure 10 As shown.
[0143] Because the operator types of this invention support these two types of computationally intensive operators, which are the most core and time-consuming in the fields of artificial intelligence and high-performance computing, the technical solution of this invention can accurately simulate and analyze the pressure characteristics of the most important computing modes in modern AI workloads on the caching system. This enables the technical solution of this invention to accurately evaluate and optimize GPGPU cache designs for modern AI workloads.
[0144] Step S17: During the simulation data loading and storage process, generate the execution unit's workload log and the cache behavior log of the cache to be simulated, so as to analyze the cache performance of the target general-purpose graphics processor.
[0145] In some embodiments, the workload log of the execution unit includes at least one of the following: the load loaded by the execution unit, the storage load, and the number of computational operations completed. Recording the load loaded by the execution unit and the storage load directly quantifies the total access requirements of the computational task to the memory subsystem, providing a direct basis for assessing memory bandwidth pressure. Recording the number of computational operations completed by the execution unit accurately measures the actual workload of the computing unit. Combining these three elements allows for the calculation of the computational intensity of the workload on the actual hardware model, providing a quantitative analytical basis for optimizing the chip's computing power and bandwidth resource allocation.
[0146] In some embodiments, the cache behavior logs to be simulated include at least one of the following: load load, storage load, cache hit count, and cache miss count for each level of cache. This hierarchical, fine-grained cache behavior log information allows performance analysis to accurately pinpoint the bottleneck level of the caching system and quantitatively evaluate the performance gains from different cache configuration parameters, providing a basis for targeted cache optimization decisions.
[0147] In some embodiments, the simulation analysis method for general-purpose graphics processor (GPU) cache performance provided by the present invention further includes: generating a cache performance analysis chart of the target GPU based on the workload logs of the execution unit and the cache behavior logs of the cache to be simulated. Because the cache performance analysis chart is more intuitive, designers can quickly understand the cache performance of the target GPU through the chart, thus helping to shorten the design process.
[0148] In some embodiments, cache performance analysis charts are used to illustrate at least one of the following relationships: the relationship between memory traffic and cache size, the relationship between memory bandwidth and cache size, and the relationship between cache miss rate and cache size.
[0149] It is understood that the execution order of steps S11 to S17 above is only an illustration. In other embodiments, other execution orders may be used, and some steps may be split or combined. This is not limited here.
[0150] Figure 11 The document exemplifies the relationships between memory flow and cache size, memory bandwidth and cache size, L1 cache miss rate and cache size, and LN cache miss rate and cache size. These three relationships serve as a visual basis for evaluating the performance impact of different cache configuration schemes. The cache performance analysis charts intuitively reveal the changing patterns of key cache performance indicators—memory flow, memory bandwidth, and cache miss rate—with configuration parameters such as cache size when the target general-purpose graphics processor is running the workload under evaluation. This provides a design basis for selecting the optimal cache configuration parameters in actual chip design.
[0151] In summary, the model input information obtained by the technical solution of this invention characterizes the features of the workload to be evaluated on the target general-purpose graphics processor (GPU). Therefore, the model structure information of the model to be simulated extracted from the model input information can accurately reflect the internal structure and computational tasks of the workload to be evaluated on the target GPU. Furthermore, since the configuration input information obtained by this invention characterizes the hardware architecture features of the target GPU, a virtual cache topology with multi-level cache characteristics is constructed based on the configuration input information. This topology is flexibly configurable and accurately reflects the hardware architecture of the target GPU.
[0152] Based on this, grid scheduling information is generated using model structure information and configuration input information to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor. This combines the computational task characteristics from the model side with the hardware scheduling parameters from the configuration side, proactively constructing a specific task deployment scheme that conforms to the hierarchical parallel execution model of the GPGPU grid-workgroup architecture. This grid scheduling information serves as a crucial hub connecting computation and hardware, ensuring that all subsequent data loading and storage simulation processes are conducted within a context strictly aligned with the actual execution scheduling mechanism of the GPGPU. This guarantees that every simulated cache access accurately reflects the behavior that task execution should produce under the target GPGPU hardware scheduling strategy, thus fundamentally overcoming the technical deficiency of existing cache analysis methods being disconnected from the actual execution process of GPGPU hardware.
[0153] Furthermore, the technical solution of this invention allocates memory base addresses to the input and output sources of each operator in the model to be simulated based on model structure information, establishing a precise logical address mapping for all data accessed during subsequent data loading and storage simulation. Then, based on grid scheduling information, cache topology, and memory base addresses, it simulates the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated. This achieves dynamic collaborative simulation of computational task scheduling, hardware storage model, and data addressing mechanism, ensuring that each simulated cache access accurately reflects the behavior generated when processing a specific data layout defined by memory base addresses and model input information in the complex multi-level cache system represented by the cache topology, under the execution sequence determined by the grid scheduling information. Moreover, during the simulation of data loading and storage, the generated workload logs of the execution unit and the cache behavior logs of the cache to be simulated can completely record the interaction details between the computation side and the storage side, providing a quantifiable data foundation for analyzing the cache performance of the target general-purpose graphics processor. Therefore, based on these two logs, the performance advantages and disadvantages of different hardware configuration schemes can be quickly and effectively evaluated, accurately guiding the chip cache design of the target general-purpose graphics processor.
[0154] Furthermore, embodiments of the present invention also provide a simulation and analysis system for the cache performance of a general-purpose graphics processor, referencing... Figure 12 The present invention provides a simulation analysis system 10 for general graphics processor cache performance, which includes the features described below.
[0155] The information acquisition unit 118 is configured to acquire model input information and configuration input information related to the target general-purpose graphics processor, wherein the model input information characterizes the characteristics of the workload to be evaluated of the target general-purpose graphics processor, and the configuration input information characterizes the characteristics of the hardware architecture of the target general-purpose graphics processor.
[0156] Model extractor 114 is configured to extract model structure information of the model to be simulated from the model input information.
[0157] The cache topology builder 113 is configured to construct the cache topology structure of the cache to be simulated based on the configuration input information.
[0158] The grid scheduler 110 is configured to generate grid scheduling information for scheduling computational tasks of the model to be simulated to the target general-purpose graphics processor based on model structure information and configuration input information.
[0159] The base address generator 115 is configured to allocate memory base addresses for the input and output sources of each operator in the model to be simulated based on the model structure information.
[0160] The performance simulation unit 116 is configured to simulate the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated, based on grid scheduling information, cache topology and memory base address.
[0161] Log generation unit 117 is configured to generate workload logs of the execution unit and cache behavior logs of the cache to be simulated during the simulation data loading and storage process, so as to analyze the cache performance of the target general-purpose graphics processor.
[0162] In the simulation analysis system 10 for general-purpose graphics processor cache performance provided in this application, since the model input information acquired by the information acquisition unit 118 characterizes the characteristics of the workload to be evaluated of the target general-purpose graphics processor, the model structure information of the model to be simulated extracted by the model extractor 114 from the model input information can accurately reflect the internal structure and computational tasks of the workload to be evaluated of the target general-purpose graphics processor. Furthermore, since the configuration input information acquired by the information acquisition unit 118 characterizes the characteristics of the hardware architecture of the target general-purpose graphics processor, the cache topology builder 113 constructs a flexibly configurable virtual cache topology structure based on the configuration input information, which accurately reflects the multi-level cache characteristics in the hardware architecture of the target general-purpose graphics processor.
[0163] Based on this, the grid scheduler 110 generates grid scheduling information for scheduling the computational tasks of the model to be simulated to the target general-purpose graphics processor, according to the model structure information and configuration input information. This combines the computational task characteristics from the model side with the hardware scheduling parameters from the configuration side, proactively constructing a specific task deployment scheme that conforms to the hierarchical parallel execution model of the GPGPU grid-workgroup architecture. This grid scheduling information serves as a crucial link between computing and hardware, ensuring that all subsequent data loading and storage simulation processes executed by the performance simulation unit are performed within a context strictly aligned with the actual execution scheduling mechanism of the GPGPU. This ensures that every simulated cache access accurately reflects the behavior that task execution should produce under the target GPGPU hardware scheduling strategy, thus overcoming the technical deficiency of existing cache analysis methods being disconnected from the actual execution process of GPGPU hardware at the system level.
[0164] Furthermore, the technical solution of this application, through the base address generator 115, allocates memory base addresses to the input and output sources of each operator in the model to be simulated based on the model structure information, establishing a precise logical address mapping for all data that will be accessed during the subsequent data loading and storage simulation process. Subsequently, the performance simulation unit 116 simulates the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated, based on grid scheduling information, cache topology, and memory base addresses. This achieves dynamic collaborative simulation of computational task scheduling, hardware storage model, and data addressing mechanism, ensuring that each simulated cache access accurately reflects the behavior generated when processing a specific data layout defined by memory base addresses and model input information in the complex multi-level cache system represented by the cache topology, under the execution sequence determined by the grid scheduling information. Furthermore, during the simulation data loading and storage process, the log generation unit 117 generates the workload log of the execution unit and the cache behavior log of the cache to be simulated, which can completely record the interaction details between the computing side and the storage side. This provides a quantifiable data basis for analyzing the cache performance of the target general-purpose graphics processor. Based on these two logs, the performance of different hardware configuration schemes can be quickly and effectively evaluated, and the chip cache design of the target general-purpose graphics processor can be accurately guided.
[0165] exist Figure 12 In the illustrated embodiment, the grid schedule generator 110 further includes a grid generator 111 and a computation schedule table generator 112.
[0166] Mesh generator 111 is configured to determine the dimensions of the computational grid based on the output dimensions of each operator in the model structure information, divide the computational grid into multiple working groups based on the grid configuration parameters in the configuration input information, determine the traversal order of the multiple working groups in the computational grid, and generate grid index information based on the traversal order. Computational scheduler 112 is configured to generate computational scheduling entries for each grid in the grid index information based on the grid index information and the execution unit configuration parameters in the configuration input information, generate computational scheduling information based on the computational scheduling entries, and use the computational scheduling information as grid scheduling information. In this way, by combining the computational characteristics of the model with the scheduling and resource configuration characteristics of the hardware, specific and executable grid scheduling information is generated, providing a precise and time-sequential task execution context for the subsequent simulation process. This ensures that the data loading and storage behavior of the simulation closely follows the parallel scheduling logic of the target GPGPU, avoiding decoupling from the hardware execution process.
[0167] exist Figure 12 In the illustrated embodiment, the general-purpose graphics processor (GPU) cache performance simulation and analysis system 10 further includes a visualization unit 12, used to generate a cache performance analysis chart of the target GPU based on the workload logs of the execution unit and the cache behavior logs of the cache to be simulated. Because the cache performance analysis chart is more intuitive, designers can quickly understand the cache performance of the target GPU through the chart, thus helping to shorten the design process.
[0168] Regarding the simulation and analysis system 10 for general graphics processor cache performance in the above embodiments, the specific execution methods of each component have been described in detail in the embodiments related to the method, and will not be elaborated here.
[0169] Embodiments of this application also provide a computer-readable storage medium having instructions stored thereon that, when executed on an electronic device, cause the electronic device to perform a simulation analysis method for the performance of a general graphics processor cache as described in any of the above embodiments.
[0170] Embodiments of this application also provide a computer program product, which includes instructions that, when executed by one or more processors, are used to implement a simulation analysis method for the general graphics processor cache performance as described in any of the above embodiments.
[0171] Embodiments of this application also provide an electronic device, such as... Figure 13 As shown, the electronic device 20 includes: a memory 201 for storing instructions, and one or more processors 202, which, when the instructions are executed by the one or more processors 202, perform a simulation analysis method for the performance of a general graphics processor cache as described in any of the above embodiments.
[0172] Figure 13 The electronic device 20 shown also includes a communication interface 203. The processor 202, memory 201, and communication interface 203 are connected via a communication bus and communicate with each other.
[0173] Processor 202 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.
[0174] Communication interface 203 is configured to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.
[0175] The memory 201 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.
[0176] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features. Industrial applicability
[0177] The simulation analysis method for general-purpose graphics processor cache performance provided in this application can improve the accuracy of cache performance analysis, thereby effectively guiding the chip cache design of the target general-purpose graphics processor.
[0178] It should be noted that, in this application, each unit / module mentioned in the various device embodiments is a logical unit / module. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problem proposed by this invention. Furthermore, to highlight the innovative aspects of this invention, the above-described device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problem proposed by this invention. This does not mean that the above-described device embodiments do not contain other units / modules.
[0179] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.
[0180] Although the invention has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims
1. A method for simulating and analyzing the performance of a general-purpose graphics processor cache, comprising: Obtain model input information and configuration input information related to the target general-purpose graphics processor. Here, the model input information characterizes the characteristics of the workload to be evaluated of the target general-purpose graphics processor, and the configuration input information characterizes the characteristics of the hardware architecture of the target general-purpose graphics processor. Extract the model structure information of the model to be simulated from the model input information; Based on the configuration input information, construct the cache topology structure of the cache to be simulated; Based on the model structure information and the configuration input information, grid scheduling information is generated to schedule the computational tasks of the model to be simulated to the target general-purpose graphics processor; Based on the model structure information, memory base addresses are allocated to the input and output sources of each operator in the model to be simulated; Based on the grid scheduling information, the cache topology, and the memory base address, the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated is simulated; as well as During the simulation data loading and storage process, workload logs of the execution unit and cache behavior logs of the cache to be simulated are generated for analysis of the cache performance of the target general-purpose graphics processor.
2. The simulation analysis method according to claim 1 further includes the step of: Extract the model layer information of the model to be simulated from the model input information; and The hierarchical structure diagram of the model is determined based on the information of each layer of the model.
3. The simulation analysis method according to claim 2, wherein, The information for each layer of the model includes at least one of the following: operator type information, operator input source information, weight source information, and output source size information for each layer of the model.
4. The simulation analysis method according to claim 1 or 3, wherein, The model input information includes at least one of the following: model layer number information, model layer structure information, data type information, and memory layout information.
5. The simulation analysis method according to claim 4, wherein... The configuration input information includes the hardware computing power bandwidth information, cache parameter information, and data routing information of the target general-purpose graphics processor; the hardware computing power bandwidth information includes the target computing power information of the computing unit and / or the target memory bandwidth information; The cache parameter information includes at least one of the following: cache level number information, cache size information at each level, and cache replacement policy information; and The data routing information includes data loading source path information and / or data storage destination path information.
6. The simulation analysis method according to claim 5 further includes: Based on the cache parameter information in the configuration input information, the capacity, organization method and data replacement strategy of each level of cache in the cache to be simulated are determined to define the hierarchical static structure of the cache. as well as Based on the data routing information in the configuration input information, the loading and storage directions of data between each level of cache in the hierarchical static structure, and between each level of cache and global memory, are determined to define the data path structure of the cache. The cache topology consists of a hierarchical static structure and a data path structure.
7. The simulation analysis method according to claim 4 further includes: The dimensions of the computational grid are determined based on the output dimensions of each operator in the model structure information. Based on the grid configuration parameters in the configuration input information, the computing grid is divided into multiple working groups, the traversal order of the multiple working groups in the computing grid is determined, and grid index information is generated based on the traversal order. Based on the grid index information and the execution unit configuration parameters in the configuration input information, a computation scheduling entry is generated for each grid in the grid index information, and computation scheduling information is generated based on the computation scheduling entry; as well as The computational scheduling information is used as grid scheduling information; The grid configuration parameters include at least one of the following parameters: the number of workgroups in the X direction, the number of workgroups in the Y direction, and the workgroup traversal order layout.
8. The simulation analysis method according to claim 7, wherein, The workgroup traversal order layout is one of the following: linear X-order layout, linear Y-order layout, or Z-Order layout.
9. The simulation analysis method according to claim 4, wherein, The allocation of memory base addresses for the input and output sources of each operator in the model to be simulated, based on the model structure information, includes: Iterate through each layer and each operator in the hierarchical structure graph of the model to be simulated; and calculate and allocate the corresponding memory base address according to the size and data type of each input source and output source of the current operator.
10. The simulation analysis method according to claim 7, further comprising the process of loading and storing data during the simulation of the execution unit of the target general-purpose graphics processor running the model to be simulated based on the grid scheduling information, the cache topology, and the memory base address, wherein: Based on the type of operator currently being executed, select either the core unit for matrix computation or the unit for arithmetic logic computation as the current execution unit. Based on the grid scheduling information, determine the grid and workgroup corresponding to the data that needs to be loaded or stored; Calculate the memory access address based on the memory base address and the memory layout information in the model input information; as well as Based on the access address, simulate data load hit, load miss, storage hit, or storage miss behavior in the cache topology.
11. The simulation analysis method according to claim 10, wherein, The operator types include general matrix multiplication type or general matrix-vector multiplication type.
12. The simulation analysis method according to claim 1, wherein, The workload log of the execution unit includes at least one of the following: the load loaded by the execution unit, the load stored, and the number of computational operations completed.
13. The simulation analysis method according to claim 1, wherein, The cache behavior log to be simulated includes at least one of the following: load load, storage load, cache hit count, and cache miss count for each level of cache.
14. The simulation analysis method according to claim 1, further comprising: Based on the workload log of the execution unit and the cache behavior log of the cache to be simulated, a cache performance analysis chart of the target general-purpose graphics processor is generated.
15. The simulation analysis method according to claim 14, wherein, The cache performance analysis chart is used to show at least one of the following: the relationship between memory traffic and cache size, the relationship between memory bandwidth and cache size, and the relationship between cache miss rate and cache size.
16. A simulation and analysis system for the cache performance of a general-purpose graphics processor, comprising: The information acquisition unit is configured to acquire model input information and configuration input information related to the target general-purpose graphics processor, wherein the model input information characterizes the characteristics of the workload to be evaluated of the target general-purpose graphics processor, and the configuration input information characterizes the characteristics of the hardware architecture of the target general-purpose graphics processor. A model extractor is configured to extract model structure information of the model to be simulated from the model input information; A cache topology builder is configured to construct a cache topology structure to be simulated based on the configuration input information. A grid scheduler generator is configured to generate grid scheduling information for scheduling computational tasks of the model to be simulated to a target general-purpose graphics processor based on the model structure information and the configuration input information. A base address generator is configured to allocate memory base addresses for the input and output sources of each operator in the model to be simulated based on the model structure information. The performance simulation unit is configured to simulate the data loading and storage process of the execution unit of the target general-purpose graphics processor when running the model to be simulated, based on the grid scheduling information, the cache topology, and the memory base address; and The log generation unit is configured to generate workload logs of the execution unit and cache behavior logs of the cache to be simulated during the simulation data loading and storage process, so as to analyze the cache performance of the target general-purpose graphics processor.
17. The simulation analysis system according to claim 16, wherein, The grid scheduling generator includes a grid generator and a computation scheduling table generator; The mesh generator is used to determine the dimension of the computational mesh based on the output size of each operator in the model structure information, and to divide the computational mesh into multiple working groups based on the mesh configuration parameters in the configuration input information, and to determine the traversal order of the multiple working groups in the computational mesh, and to generate mesh index information based on the traversal order. and The computation scheduling table generator is used to generate computation scheduling entries for each grid in the grid index information based on the grid index information and the execution unit configuration parameters in the configuration input information, and to generate computation scheduling information based on the computation scheduling entries, and to use the computation scheduling information as grid scheduling information.
18. The simulation analysis system according to claim 16 further includes a visualization unit configured to generate a cache performance analysis chart of the target general-purpose graphics processor based on the workload log of the execution unit and the cache behavior log of the cache to be simulated.
19. A computer program product configured to implement a simulation analysis method for the cache performance of a general-purpose graphics processor as described in any one of claims 1-15.
20. An electronic device configured to perform a simulation analysis method for the performance of a general-purpose graphics processor cache as claimed in any one of claims 1-15.
