A design space exploration method for cache hierarchy in a multi-core system

Abstract

The application discloses a design space exploration method for cache hierarchy in a multi-core system. The method aims to optimize the cache subsystem in the multi-core system by reasonably configuring the cache hierarchy and the inter-chiplet network topology, thereby improving the system performance. The specific steps include: 1) cache miss rate and network delay modeling: modeling the cache miss rate and the network delay as functions of the cache hierarchy and the inter-chiplet network parameters; 2) optimization problem definition: defining the optimization target under the cost and power consumption constraints, and minimizing the concurrent average memory access time (C-AMAT); and 3) solving by a double-layer optimization algorithm: performing the cache subsystem optimization and the inter-chiplet network optimization by the double-layer optimization algorithm. The method provides an effective solution for the cache optimization of the multi-core system, and has a wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a design space exploration method for cache hierarchy in a multi-chip system. Background Technology

[0002] With the increasing complexity of computing systems and the growing prevalence of multi-core systems, memory performance optimization has become a core issue in computing architecture research. Existing research has proposed various memory performance modeling and optimization methods, but many shortcomings remain. The Concurrent Average Memory Access Time (C-AMAT) model expands the functionality of the traditional Average Memory Access Time (AMAT) model by incorporating memory concurrency into performance calculations, but its optimization mainly focuses on local memory characteristics, lacking global performance optimization capabilities. The Hierarchical Performance Matching (LPM) method decomposes the memory system performance optimization problem into local sub-problems, improving overall performance, but its applicability to multi-core sharing and consistency issues is limited. The Data Sharing Awareness Analysis (DSA) model effectively estimates the shared cache miss rate and solves some data consistency problems, but its scalability is insufficient under heterogeneous environments and complex multi-threaded workloads. The Shared Memory Analysis Model (SASMM) and optimal cache partitioning techniques excel in shared cache hit rate and dynamic cache partitioning, but still lack flexibility when facing dynamic workloads and heterogeneous memory requirements. Design tools and optimization models for multi-core systems (such as IntLP, NN-Baton, and Monad) have made significant progress in functional partitioning, resource optimization, and design space exploration. However, there is still room for improvement in memory hierarchy optimization and inter-core communication efficiency. Furthermore, while multi-core integration methods based on 64-core RISC-V processors and cost-sensitivity analysis of on-chip multi-core systems have addressed cost and scalability issues in the early stages of system design, they lack focus on runtime dynamic performance optimization. In heterogeneous systems, optimization methods for latency constraints and dynamic scheduling are often limited to single-architecture scenarios and are ill-suited to the complex requirements of multi-core systems. Overall, existing technologies have achieved considerable success in memory performance measurement, cache management, and design tools, but in multi-core systems, issues remain such as insufficient global performance optimization, limited dynamic adaptability, inadequate cost-performance trade-offs, and a large design space. Further research and improvement are needed to meet the demands of future high-performance computing systems. Summary of the Invention

[0003] The purpose of this invention is to address the problems of high complexity, low performance optimization efficiency, and insufficient adaptability in the design of cache hierarchy structures in existing multi-core systems. It provides a design space exploration method for cache hierarchy structures in multi-core systems, which can achieve optimal configuration of cache access performance while meeting cost and power consumption constraints.

[0004] To achieve the above objectives, the technical solution provided in this application is as follows:

[0005] This application provides a method for exploring the design space of cache hierarchy in a multi-core system, including the following steps:

[0006] (1) Obtain the architecture topology diagram of the multi-core integrated system; the architecture topology diagram includes a storage hierarchy diagram and a network topology diagram; wherein, the storage hierarchy diagram is a hierarchical connection diagram of the cache subsystem of the multi-core integrated system, used to characterize the connection relationship between each level of cache; the network topology diagram is a core connection diagram of the multi-core integrated system, used to describe the communication connection relationship between cores;

[0007] (2) Based on the architecture topology diagram, construct a system performance model; the system performance model includes a cache miss rate model, a latency model, and a memory parallelization model;

[0008] (3) Optimize the system performance model and obtain the target cache parameters; generate the target cache architecture diagram based on the target cache parameters and the storage hierarchy diagram.

[0009] Further, step (1) includes the following sub-steps:

[0010] (1.1) Cache hierarchy and access latency modeling: Based on the storage hierarchy of the multi-core system, a cache hierarchy diagram is defined to represent the hierarchical relationship of each cache level and its access latency characteristics;

[0011] (1.2) Bandwidth and delay modeling of the inter-core interconnection network: Construct an inter-core interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation and communication delay characteristics between cores.

[0012] Furthermore, step (2) includes the following sub-steps:

[0013] (2.1) Cache miss rate modeling: Based on the storage hierarchy diagram and network topology diagram, combined with the hit probability and data flow characteristics of each cache level, a miss rate calculation formula is defined, and a cache miss rate model is constructed to estimate the cache miss rate of different cache levels.

[0014] (2.2) Cache access latency modeling: Based on the connection path between nodes, establish a cache access latency model, calculate the point-to-point communication latency between two nodes based on the latency model, including zero-load latency and queuing latency; quantify the average access latency of each cache level;

[0015] (2.3) Memory parallelization modeling: Combine cache miss rate and inter-chip network latency to calculate memory parallelization parameters related to concurrent memory access time to evaluate the overall performance of multi-chip system.

[0016] Furthermore, step (3) includes the following sub-steps:

[0017] (3.1) Definition of the optimization objective function: Based on the performance requirements of the multi-core system, cost and power consumption constraints are set. Combined with the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application. Through recursion, referring to the formula for average parallel memory access time, the cache miss rate model, latency model and parallel memory access time model are integrated into the performance model.

[0018] (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the optimization subproblem of the cache subsystem and the optimization subproblem of the interconnection network topology between the core particles. The two-layer optimization algorithm is used to solve the problem alternately until the performance converges or the preset iteration limit is reached. The target cache parameters that meet the constraints are obtained, and the optimized cache architecture diagram is generated.

[0019] Furthermore, define the storage hierarchy graph G. M (V,E), where each node v ijk ∈V represents a core, cache unit, or storage unit, and node v ijk The weights are:

[0020] When j = 0, v ijk With the core as the weight w(v) ijk )= <cpi(v ijk ),ls(v ijk )>, where cpi(v ijk ) represents the ideal instruction cycle per instruction (CPI), ls(v ijk The core delay is )

[0021] When 0 <j<h k At that time, v ijk For each cache unit, the weight is w(v) ijk )= <cs(v ijk ),cl(v ijk ),as(v ijk ),

[0022] rs(v ijk ),ac(v ijk )>, where cs(v ijk ) represents the cache size, cl(v) ijk ) represents the cache line size, as(v ijk ) represents the relevance of the cache, rs(v) ijk ) represents the cache replacement strategy, ac(v ijk ) represents the number of cache accesses; the cache replacement strategy is rs(v). ijk The replacement algorithms used include FIFO, LRU, LFU, and random strategies;

[0023] When j = h k At that time, v ijk For DRAM cells, the weight is w(v) ijk )= <dt(v ijk ),bw(v ijk )>, where dt(v ijk bw(v) represents the DRAM access latency. ijk ) represents the DRAM bandwidth;

[0024] Storage hierarchy diagram G M Edge e in (V,E) ijk,mno ∈E represents node v ijk and v mno The connection relationship between them, when node v ijk With v mno When connected, e ijk,mno =1, otherwise 0;

[0025] Define the chip-to-chip interconnect network topology diagram G I (N,C), where each node n p,q ∈N represents a node on a core q, which is a core, L1 / L2 cache, network interface, or router, or simply a D2D interface; node n p,q The weights ω(c(n) p,q ,n r,s )) represents the transmission delay between nodes; network connection c(n) p,q ,n r,s Connecting adjacent nodes in C; Core-level interconnection network topology G I The number of cores in (N,C) is n. α And each core Where |R i | represents the number of cores in the i-th core;

[0026] This storage hierarchy diagram and network topology diagram are used to construct cache miss rate and network latency models for multi-core systems, in order to optimize storage access performance under power and cost constraints.

[0027] Furthermore, the cache miss rate model M(v) ijk The calculation includes calculations via node v. ijk Reachable path Π(v ijk ) is a polynomial fit used to determine cache size, cache line size, cache associativity, and replacement strategy, where Π(v ijk ) represents the distance from any core to node v ijk The set of all nodes on the reachable path; cache miss rate model M(v ijkBased on different cache replacement strategies, including FIFO, LRU, LFU and random strategies.

[0028] Further, step (2.2) includes:

[0029] Zero-load latency is defined as being related to the distance between the source and target nodes, and its calculation method includes using the chip-to-chip interconnect network topology graph G. I To perform modeling, define L(n) p,q ,n r,s ) is a sub-node n p,q to node n r,s The transmission delay consists of two parts: header packet delay L h (n p,q ,n r,s ) and serialization delay L s (n p,q ,n r,s );

[0030] Based on transmission rate μ and data packet size s i The serialization latency L is calculated using the flit size f and the number of router pipeline stages η. s (n p,q ,n r,s ) and header delay L h (n p,q ,n r,s );

[0031] Queuing delay is defined and modeled based on the G / G / 1 model, and the computation node... Queuing delay This delay is a node Multiple input channels With single output channel Queue delays between;

[0032] Calculate the total queuing delay W(n) p,q ,n r,s This delay is achieved through path Γ(n) p,q ,n r,s Each router The sum of queuing delays;

[0033] Finally, the transmission delay D(n) is obtained. p,q ,n r,s ) represents the zero-load delay L(n) p,q ,n r,s ) and queuing delay W(n) p,q ,n r,s ) and .

[0034] Further, step (3) includes:

[0035] Problem decomposition: The optimization problem is decomposed into two subproblems using a two-level optimization algorithm, namely the storage hierarchy graph G. M The optimization subproblem P1 and the network topology graph G I The optimization subproblem P2;

[0036] Iterative optimization: The optimization process adopts an iterative approach. In each iteration, only one subproblem is optimized, while the decision variables of the other subproblem are fixed as constraints.

[0037] Termination condition: The optimization process terminates when the target performance index σ satisfies the convergence condition or reaches the maximum number of iterations.

[0038] Further, step (3.2) includes:

[0039] A two-layer optimization framework is adopted, where subproblem P1 is a storage hierarchy optimization problem, and its objective is to optimize G. M To minimize the average concurrent memory access time σ(G) M );

[0040] Subproblem P2 is an interconnection network topology optimization problem, the objective of which is to optimize G. I To minimize the average concurrent memory access time σ(G) I ).

[0041] This application also provides a design space exploration device for cache hierarchy in a multi-core system, including:

[0042] The architecture diagram acquisition module is used to acquire the storage hierarchy diagram and network topology diagram of the multi-core integrated system. The storage hierarchy diagram is the connection diagram of the core, cache, and DRAM nodes of the multi-core integrated system, and the network topology diagram is the connection diagram of the nodes within / between the cores of the multi-core integrated system.

[0043] The model building module is used to build a system performance model based on the storage hierarchy graph and network topology graph; the system performance model includes a cache miss rate model, a latency model, and a memory parallelization model; and calculates the optimal performance parameters and optimal cache parameters of the storage hierarchy graph respectively;

[0044] The model optimization module is used to solve the system performance model and generate target cache parameters with optimal performance.

[0045] The model generation module is used to generate a target cache architecture diagram based on the target cache parameters, and use the optimal cache parameters corresponding to the target architecture diagram as the cache parameters of the multi-core integrated system.

[0046] The present invention has the following beneficial effects:

[0047] By modeling cache performance and optimizing design space, cache access time is significantly reduced and the overall performance of multi-core systems is improved.

[0048] Hierarchical optimization algorithms effectively reduce the complexity of optimization problems and improve design efficiency.

[0049] It supports flexible optimization for various application scenarios and can meet specific performance requirements;

[0050] While optimizing performance, cost and power consumption constraints are taken into account, thereby improving the economy and feasibility of system design.

[0051] This invention provides a systematic solution for optimizing cache performance in multi-core systems, enabling efficient cache hierarchy design in high-performance computing environments.

[0052] Compared to existing technologies, this invention can comprehensively optimize the configuration of the cache subsystem by integrating multiple models, reduce cross-core communication latency and performance loss caused by cache misses, meet the stringent requirements of high-performance computing systems for cache architecture, and thus improve the overall computing efficiency of multi-core systems. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic flowchart of the method of the present invention;

[0055] Figure 2 This is an architectural diagram of the present invention; wherein, Figure 2 (a) in the diagram is a multi-core system architecture diagram. Figure 2 (b) in the diagram is the cache hierarchy. Figure 2 (c) in the diagram represents the interconnection network topology between the core particles. Detailed Implementation

[0056] To more clearly illustrate the technical solution of the present invention and its beneficial effects, the technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be particularly noted that the embodiments described below are only for the purpose of facilitating understanding of the technical content of this application and should not be considered as limiting the scope of protection of this application. Any other implementation methods made based on the disclosure of this application without the need for creative effort by those skilled in the art should be considered to fall within the scope of protection of this application.

[0057] In this application, the term "embodiment" is used to refer to a possible implementation or technical solution. The repeated use of the term "embodiment" in the specification does not imply that all mentioned content belongs to the same embodiment, nor does it mean that different embodiments are mutually exclusive. In fact, according to actual needs and technical background, the features of different embodiments can be appropriately combined and adjusted to achieve better technical effects.

[0058] In the context of this application, "comprising" or "including" is non-exclusive, meaning that in addition to the listed contents, other related contents may be included. For example, a method may include steps not explicitly listed, and an apparatus may include other modules or units not mentioned.

[0059] In this application, the terms "connection" and "linkage" are not limited to physical connections, but may also include electrical connections, whether direct or indirect.

[0060] In implementing this application, those skilled in the art can reasonably adjust the order or structure of the various modules and steps in the technical solution according to specific circumstances. This flexibility applies not only to the design of methods and apparatus but also to the manufacturing and deployment of products or equipment. Furthermore, for those skilled in the art, further designing and optimizing the technical solution of this invention based on the disclosure of this application is a routine technical activity and should be considered an extended application of this invention, without exceeding the scope of the technical disclosure.

[0061] Through the above description, those skilled in the art can fully understand the technical content and implementation of the present invention. The technical solution of the present invention can be adjusted and optimized according to specific needs. In particular, for complex multi-core systems, the design of their storage architecture and interconnection network can achieve significant performance improvements and design flexibility through the optimization methods provided in this application.

[0062] The core technology of this invention is to model and optimize the cache hierarchy and interconnection network of a multi-core system, and propose a two-layer optimization method that combines performance modeling and design space exploration to achieve the optimal configuration of cache access performance in a multi-core system while meeting the system's cost and power consumption constraints.

[0063] This invention proposes a design space exploration method for cache hierarchy in a multi-core system, comprising the following steps:

[0064] (1) Construction of caching and interconnection models: Based on the cache hierarchy and inter-core communication characteristics of multi-core systems, a cache hierarchy diagram and an inter-core interconnection network topology diagram are constructed; specifically, the following sub-steps are included:

[0065] (1.1) Cache hierarchy and access latency modeling: Based on the storage hierarchy of the multi-core system, a cache hierarchy diagram is defined to represent the hierarchical relationship of each cache level and its access latency characteristics.

[0066] (1.2) Bandwidth and delay modeling of the inter-core interconnection network: Construct an inter-core interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation and communication delay characteristics between cores.

[0067] (2) Cache performance modeling: Establish cache miss rate model, cache access latency model, and memory parallelization model based on multi-core system to quantify performance indicators; specifically including the following sub-steps:

[0068] (2.1) Cache Miss Rate Modeling: Combining the hit probability of each level of cache and the characteristics of data flow, a formula for calculating the missing rate is defined.

[0069] (2.2) Cache access latency modeling: Based on the connection path between nodes, a cache access latency model is established to quantify the average access latency of each level of cache.

[0070] (2.3) Memory parallelization modeling: Combine cache miss rate and inter-slice network latency to calculate memory parallelization parameters related to concurrent memory access time to evaluate overall performance.

[0071] (3) Definition and solution of performance optimization problem: Under the condition of satisfying system cost and power consumption constraints, define the optimization objective function based on the performance model to minimize the average concurrent memory access time of the application, which includes the following sub-steps:

[0072] (3.1) Definition of the optimization objective function: Combining the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application.

[0073] (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the optimization subproblem of the cache subsystem and the optimization subproblem of the interconnection network topology between the core particles. The two-layer optimization algorithm is used to solve them alternately until the performance converges or the preset iteration limit is reached.

[0074] This invention introduces a two-layer optimization method to decompose the complex cache performance optimization problem into subproblems that are solved step by step, effectively reducing the problem's complexity. Simultaneously, by combining the hierarchical performance matching method (LPM), it achieves collaborative optimization of the cache subsystem and the interconnect network, significantly improving system performance while considering cost and power consumption constraints. This invention is applicable to various application scenarios and provides strong technical support for the design of high-performance multi-core systems.

[0075] See Figure 1By constructing storage hierarchy diagrams and network topology diagrams, a multi-core system is modeled, and optimization solutions are obtained by combining performance models, ultimately achieving the optimal design of the cache subsystem. The following section combines... Figure 2 The embodiments of the present invention are described in detail.

[0076] like Figure 2 As shown in (a) in this embodiment, the multi-core integrated system is modeled using two graphs, which respectively characterize its cache subsystem and interconnect subsystem.

[0077] Storage hierarchy diagram G M (V,E) is an undirected graph used to describe the memory hierarchy of a multi-core integrated system. Each node v in the graph... ijk ∈V represents a core, cache unit, or memory unit. Here, i is the node number, j is the cache level, and k is the core number. When j = 0, node v ijk Indicates core; when 0 <j<h k At that time, node v ijk Represents a cache unit; when j = h k At that time, node v ijk This represents a Dynamic Random Access Memory (DRAM) cell. (Side e) ijk,mno ∈E represents node v ijk and v mno Connectivity between them, such as the connection between two cache units or the reachability between the core and the cache. If v ijk and v mno If there is a direct connection between them, then e ijk,mno =1, otherwise 0.

[0078] Specifically, such as Figure 2 As shown in (b) of the diagram, the storage hierarchy includes multiple cores and their corresponding L1 and L2 caches, with two cores sharing the L3 cache. For example, node v 131 and v 121 There is an edge between them, representing v 131 Includes v 121 The data. On the other hand, v 131 and v 321 The absence of an edge between them indicates that there is no containment relationship between them.

[0079] The weights w(v) of the storage hierarchy graph ijk The definition is as follows:

[0080]

[0081] For the core node where j=0, the weight is... <cpi(v ijk)> represents the ideal instruction cycle count for the core. For 0 <j<h k The cache nodes, with weights equal to the set of cached parameters. <cs(v ijk ),cl(v ijk ),as(v ijk ),rs(v ijk ),ac(v ijk The numbers )> represent the cache size, cache line size, correlation, replacement strategy, and cache access count, respectively. For j = h k DRAM nodes with weights of <dt(v ijk ),bw(v ijk )> represents the DRAM access latency and bandwidth. Replacement strategy rs(v ijk The replacement algorithms used include FIFO (First In First Out), LRU (Least Recently Used), LFU (Least Frequently Used), and random strategies.

[0082] Interconnection network topology diagram G I (N,C) is another undirected graph that describes the node connection structure between and within cores. Each node n p,q ∈N represents a node in a core q, which can be a core, L1 / L2 cache, network interface (NI), router, or D2D interface between cores. Edge c(n) p,q ,n r,s )∈C represents adjacent node n p,q and n r,s The connectivity between them. If q = s, then node n p,q and n r,s They are in the same core; otherwise, they are in different cores. The edge weight ω(c(n) p,q ,n r,s )) represents the transmission delay between two nodes. Core-level interconnect network topology diagram G I The number of cores in (N,C) is n. α And each core Where |R i | represents the number of cores in the i-th core.

[0083] This storage hierarchy diagram and network topology diagram are used to construct cache miss rate and network latency models for multi-core systems, in order to optimize storage access performance under power and cost constraints.

[0084] like Figure 2 As shown in (c), the interconnection network topology diagram illustrates a two-core system, including the connections within and between cores. Cores communicate with each other via D2D interfaces, while their internal connections are made through routers.

[0085] In step (2), the system performance is modeled by combining the cache hierarchy diagram and the interconnection network topology diagram between cores, based on the cache miss rate model, latency model, and memory parallelism model. The specific implementation method is as follows:

[0086] To quantify the performance of a multi-core integrated system, this embodiment uses a cache hierarchy graph G. M and the chip interconnect topology diagram G I Establish a performance model. The modeling process is described in detail below.

[0087] ① Mapping relationship between cache nodes and network nodes

[0088] Define function Return to G M China and G I Network node n p,q The corresponding cache node v ijk ,Right now

[0089] ② Average latency modeling between cache levels

[0090] To estimate the average latency between adjacent cache levels, this embodiment calculates the point-to-point latency based on a latency model and flow theory. The system's average latency between adjacent cache levels... Defined as:

[0091]

[0092] in, Indicates from node n p,q to node n r,s The flow rate, c = 1, ..., g.

[0093] ③ Modeling of average parallel memory access time

[0094] Define the average parallel memory access time of cache level j-1 as follows: This represents the average time taken for each memory access within a CPU cycle. It is related to the next cache level. The average parallel memory access time has a recursive relationship:

[0095]

[0096] Wherein, H(v) i(j-1)k ) represents the hit time at cache level j-1, C H (v i(j-1)k M(v) represents the average hit concurrency at cache level j-1. i(j-1)k ) represents the missing rate at cache level j-1, κ(v) i(j-1)k ) represents the percentage of pure miss cycles at cache level j-1. This represents the average latency between cache levels j-1 and j.

[0097] ④ Performance Modeling

[0098] Access time σ(G) of the caching subsystem M G I The expression represents the total time spent on cache access operations in a multi-core system, i.e., the performance model, which is expressed as:

[0099]

[0100] Wherein, H(v) ijk C represents the hit time of cache level j in core k. H (v ijk M(v) represents the average hit concurrency of cache level j in core k. itk ) represents the cache miss rate of cache level j in core k, κ(v itk () represents the percentage of pure miss cycles in cache level t within core k. n is the average latency between cache levels j-1 and j. α h represents the total number of core particles. k Let |R be the total number of cache levels for kernel k. k | represents the number of core particles k, and t is the recursive series integrated up to j.

[0101] Using the performance modeling method described above, this embodiment can accurately evaluate the cache access performance in a multi-core integrated system, providing a basis for subsequent optimization.

[0102] In step (2.1), the cache miss rate is modeled to optimize the cache subsystem of the multi-core integrated system. The specific implementation method of cache miss rate modeling is described in detail below with reference to the storage hierarchy diagram.

[0103] In this embodiment of the application, the cache miss rate is determined by the storage hierarchy graph G. M Model (V,E), where G M This diagram illustrates the memory hierarchy of a multi-chip integrated system. Each node v in the diagram... ijk ∈V represents a node in the cache hierarchy, which contains cores, cache units, or memory units.

[0104] In cache miss rate modeling, for each cache node v ijk Define the set of all upper-level nodes on its reachable path as Π(v ijk )={π1(v ijk ),…,π n (v ijk )}, where π x (v ijk() represents the x-th node on the path, where n is the total number of nodes on the path. This set contains v ijk All parent nodes in the cache hierarchy that have a relationship with it (e.g., containment).

[0105] Based on this, node v ijk Cache miss rate M(v) ijk The model is based on the following formula:

[0106]

[0107]

[0108] Among them, Λ(v ijk ) is a function related to caching, determined by the cache size (Cache Size, cs(π)). x (v ijk ))), Cache Line Size (cl(π) x (v ijk ))) and Associativity, as (π x (v ijk Composed of; a x b x c x and d x α is a coefficient related to the cache miss rate. x ,β x γ x and δ x rs(π) represents the order of the corresponding polynomial. x (v ijk )) refers to cache replacement strategies, including FIFO (First In First Out), LRU (Least Recently Used), LFU (Least Frequently Used), and Random (Random Replacement), etc. This is a function that maps replacement strategies to numerical values.

[0109] For example, for node v ijk Its path set Π(v ijk It includes all nodes in the upper layer. Λv is calculated by combining the weights of cache size, cache line size, and relevance. ijk ), and calculate M(v) based on the mapping value of the replacement strategy. ijk This allows us to obtain the final cache miss rate of the node.

[0110] The modeling methods described above can accurately assess the miss rate of each cache level in the caching subsystem, providing data support for cache optimization. Combined with other models, the cache architecture of multi-core integrated systems can be further optimized to achieve more efficient system performance.

[0111] In step (2.2), the transmission delay between nodes is analyzed and modeled to optimize and predict system performance. The following section uses the network topology diagram G as an example. I (N,C) provides a detailed description of the specific implementation of delayed modeling.

[0112] In this embodiment, the latency model is used to estimate the transmission latency between two core nodes. The transmission latency consists of two parts: zero-load latency and queue latency.

[0113] ① Zero load delay

[0114] Zero-load latency represents the time required for a data packet to travel from the source node to the destination node without contention. Zero-load latency L(n p,q ,n r,s ) Delayed by header packet L h (n p,q ,n r,s ) and serialization delay L s (n p,q ,n r,s It consists of two parts, and its expression is:

[0115] L(n p,q ,n r,s ) = L h (n p,q ,n r,s )+L s (n p,q ,n r,s )

[0116] Head packet delay L h (n p,q ,n r,s ) by node n p,q and n r,s Shortest path l(n) between p,q ,n r,s The delay along the path determines the serialization delay. s (n p,q ,m r,s This is related to the data packet length and transmission rate. The formula is as follows:

[0117] l h (n p,q ,n r,s )=ψ(η·l(n p,q ,n r,s ))

[0118]

[0119] Where μ is the transmission rate, s if is the data packet length, f is the data fragment size; η represents the number of routing pipeline stages, ψ(η·l(n) p,q ,n r,s )) represents the total delay on the path.

[0120] The total length of the shortest path is obtained recursively using the Bellman equation, starting from node n. p,q and n r,s The formula is:

[0121]

[0122] in, This represents the distance between adjacent nodes in the path.

[0123] ② Queue delay

[0124] Queue latency represents the data transmission delay caused by data contention. Queue latency W(n) p,q ,n r,s Γ(n) represents the sum of the queue delays of all routing nodes along the path, where the path is Γ(n). p,q ,n r,s In the G / G / 1 model, each routing node on the path... queue delay The expression is:

[0125]

[0126] in, Indicates the percentage of queue occupancy. For data packet transmission rate, For service speed, and These are the correlation coefficients for data arrival rate and service rate, respectively.

[0127] The formula for calculating total queue delay is:

[0128]

[0129] ③ Total transmission delay

[0130] Ultimately, the total transmission delay D(n) p,q ,n r,s The result is obtained by adding the zero-load latency and the queue latency:

[0131] D(n p,q ,n r,s )=L(n p,q ,n r,s )+W(n p,q ,n r,s )

[0132] By modeling the transmission delay between nodes using the above method, the performance impact of network communication in multi-core integrated systems can be quantified, providing a theoretical basis for optimization design.

[0133] In step (2.3), the memory access characteristics of the caching subsystem are modeled by defining the average hit concurrency and the percentage of pure misses. The specific implementation method is as follows:

[0134] Memory parallelism modeling aims to quantify the hit concurrency and miss overhead of memory accesses in multi-core systems, thereby providing guidance for system performance optimization. The specific modeling process is detailed below.

[0135] Average hit concurrency C H (v ijk This is used to represent the concurrency of cache hit events during memory access, and its calculation formula is:

[0136]

[0137] Where α(v) ijk The total number of memory accesses by the kernel is represented by:

[0138] α(v ijk ) = IC(v ijk )·f mem (v ijk )

[0139] IC(v ijk The number of instructions, f, is the core. mem (v ijk H(v) represents the average number of memory accesses per instruction. ijk ) represents the core hit time (calculated in CPU cycles), h(v) ijk The core of the pure hit cycle number is x(v). ijk The core is the number of mixed hit / miss cycles. Let be the core hit concurrency in the i-th CPU cycle, and n be the total number of CPU cycles.

[0140] Pure miss rate κ(v) ijk This describes the proportion of core misses in the total misses, and its calculation formula is:

[0141]

[0142] Where, m(v) ijk The number of pure misses is the core of the cycle count.

[0143] For C H (v ijk ) and κ(vijk The model is modeled using a regression model to capture its relationship with cache parameters.

[0144] Average hit concurrency C H (v ijk The regression model for ) is:

[0145]

[0146] Among them, f x , l x and g x θ is a coefficient x , ι x and ξ x Let π be the order of the polynomial. x (v ijk From the core to v ijk The x-th node on the path, mpki(π) x (v ijk )) represents the number of memory accesses per thousand instructions for a path node, ac(π) x (v ijk )) represents the number of cached accesses for the path node. This refers to the DRAM bandwidth.

[0147] Pure miss rate k(v) ijk The regression model for ) is:

[0148]

[0149] Where, m x and o x For regression coefficients, and Let be the order of the polynomial. This represents the miss rate for the LLC (last-level cache).

[0150] By modeling the average hit concurrency and the proportion of pure miss cycles as described above, the memory access parallelism characteristics of the cache hierarchy can be effectively quantified, providing a theoretical basis for optimizing cache configuration.

[0151] In step (3), an optimization problem is defined based on the performance model, aiming to achieve the optimal performance of the multi-core system under cost and power consumption constraints. The specific implementation is as follows:

[0152] ① Cost modeling

[0153] The architectural cost of a multi-chip system consists of the cumulative cost of each component (including the chip and DRAM) and the chip assembly / integration cost. int constitute.

[0154] For the cost C of a single chipdie (v ijk This can be represented as:

[0155]

[0156] in, This represents the chip area, c1 is the cost per unit silicon area, and Y... die It is related to chip area Defect density The limiting yield related to the clustering factor α is calculated using the following formula:

[0157]

[0158] Regarding the cost C of DRAM dram (v ijk This can be represented as:

[0159]

[0160] Among them, bw(v ijk ) represents the DRAM bandwidth, c2 represents the cost per unit bandwidth, and U bw The capacity provided per unit of DRAM bandwidth.

[0161] The cost modeling formula for the entire multi-core system is as follows:

[0162]

[0163] ② Power consumption modeling

[0164] The power consumption of a multi-core system is the cumulative power consumption of all components. The modeling formula for system power consumption is as follows:

[0165]

[0166] Among them, P core (v ijk ) represents the core power consumption, P dram (v IJK ) represents the DRAM power consumption, P ROutER (v Ijk ) represents the router's power consumption, r k The number of routers in each chip.

[0167] In step (3.1), the problem is defined, and the specific implementation is as follows:

[0168] Based on the aforementioned performance, cost, and power consumption models, and under the condition of satisfying cost and power consumption constraints, the performance of the multi-core system is optimized to minimize the average memory access time σ(G). M G IAmong these, the cost and power consumption of each core must not exceed a set threshold C. T and P T The optimization problem is defined as follows:

[0169] min{σ(G M G I )}

[0170] dtG(G M G I )≤C T

[0171] P(G M G I )≤P T

[0172] The optimization objective is to minimize the application's cache access time within a power and cost budget. The search space for this problem is extremely large. For example, consider a system containing n... γ One core, n α A system with one core, where core i contains |R i | 1 core, total cache levels are h i The number of interconnected topologies across cores is t, and the optional size of the cache at layer j is . The selectable size of DRAM is

[0173] In step (3.2), the aim is to efficiently explore cache configuration and interconnect network in a large-scale design space through a two-layer optimization algorithm, combining Layered Performance Matching (LPM) and Branch-and-Bound (BNB) algorithms, thereby achieving optimal performance of the multi-core system.

[0174] In multi-core systems, due to the complexity of cache configuration and interconnect network design, the design space needs to be decomposed into two sub-problems: (a) memory hierarchy graph G M Optimize subproblem P1, (b) Inter-core and intra-core interconnection network topology G I Optimize subproblem P2. The optimization objectives are as follows:

[0175] min{σ(G M )}

[0176] min{σ(G I )}

[0177] Bilevel optimization iterates by alternately optimizing subproblems P1 and P2. In each iteration, only one subproblem is solved, while the variables of the other subproblem are fixed as constraints. For example, in the first iteration, P1 is solved while the decision variables of P2 are fixed; in the second iteration, P2 is solved. The optimization continues until the performance index σ converges or reaches the predefined iteration limit n. max The algorithm terminates when the time is right.

[0178] The convergence condition for the iteration is:

[0179] σ i -σ i-1 ≤δ or i≤n MAx

[0180] Where, σ i Let be the cache access time for the i-th iteration, δ be the convergence condition, and n be the cache access time for the i-th iteration. max This represents the upper limit of the number of iterations.

[0181] ① Storage hierarchy diagram G M optimization

[0182] The goal of the search space definition memory hierarchy graph optimization subproblem P1 is to optimize the cache hierarchy h. i Number of cores in each core | R i | Cache parameter w(v ijk The optimization objective can be expressed as:

[0183] min{σ(G M )=σ9h i ,|R i |,w(v IJk ))}

[0184] Among them, h i = j > indicates the maximum number of cache layers per core; |R i |=<|R1|,|R2|,…,|R j |> indicates the number of cores in each core particle;

[0185] This includes cache size, cache line size, correlation, and replacement strategy.

[0186] This subproblem is solved using the branch and bound algorithm. Decision variables are represented as tree nodes. Defined as the a-th node in the b-th layer. Its form is:

[0187]

[0188] Each tree node Associate a target value​ and cost and power consumption

[0189] The branching rule is that the search tree expands step by step according to its level.

[0190] Tiered Performance Matching (LPM) Pruning. Tiered Performance Matching (LPM) is a method for evaluating the performance of tiered memory systems based on data flow analysis. Using the performance model described above, the LPM model is employed to verify the utilization rate of each cache tier.

[0191] LPM matching rate LPMR(v ijk The cache level j is defined as the ratio of the request rate to the provisioning rate, as shown in the following formula:

[0192]

[0193] Wherein, λ(v) ijk ) represents the request rate; v(v ijk ) represents the supply rate; μ(v) ijk This indicates the miss rate for a memory cycle.

[0194] According to the above definition, a smaller LPMR(v) iJk ) indicates higher memory performance for cache level j.

[0195] The pruning rules defined in this embodiment are implemented as follows: Combined with LPM pruning, when a node… LPMR(v ijk When the performance exceeds the set performance threshold, pruning will be performed.

[0196] Through the above methods, this embodiment achieves efficient exploration of the design space of multi-core integrated systems, optimizes cache configuration and interconnection network topology, and improves the overall system performance.

[0197] ② Inter-chip interconnection network topology diagram G I Optimization

[0198] This embodiment provides a topology diagram G for inter-chip interconnection networks. I The optimization method aims to achieve optimal interconnect performance of multi-core integrated systems by adjusting network topology and data transmission parameters.

[0199] In the subproblem P2 of interconnection network optimization, the optimization objective is defined as:

[0200] min{σ(G I )=σ(t i ,f i )}

[0201] Among them, t iG represents I Network topology types, including Mesh, Ring, Torus, Crossbar, Clos, and Fattree, are defined as follows:

[0202] t i = <t1,t2,…,t j >

[0203] At the same time, f i The size of the flit representing the data transfer is defined as:

[0204] f i = <f1,f2,…,f j >

[0205] Specifically, network topology parameter t i The value range is [0,1,2,3,4,5], which corresponds to six topological structures: Mesh, Ring, Torus, Crossbar, Clos, and Fat tree.

[0206] Data transmission parameter f i The value range is [32, 64, ..., 1024], which represents the size of flit in bytes.

[0207] The following objectives and constraints need to be considered during the optimization of interconnection networks:

[0208] Optimization objective: By adjusting t i and f i Minimize the system's average parallel memory access time σ(G) I ).

[0209] Constraints: The power budget P must be satisfied. T and cost budget C T Limitations are imposed to ensure the feasibility of the optimization results.

[0210] For optimizing network topology type and flit size, this embodiment uses an exhaustive search method to find the optimal solution within the parameter range. The optimization process is as follows:

[0211] A: Initialization: Set t i and f i The value range is initialized to all possible values ​​of the corresponding parameter [0,1,2,3,4,5] and [32,64,…,1024].

[0212] B: Search process: Traverse all possible combinations of network topology and flit size, and calculate the average parallel memory access time σ(G) for each parameter combination. I ).

[0213] C: Optimal solution selection: based on average parallel memory access time σ(G) I Based on the calculation results, the optimal combination that satisfies power consumption and cost constraints is selected.

[0214] D: Termination condition: The search process terminates after all parameter combinations have been traversed, and the optimal network topology is output. and flit size f i * .

[0215] Through the exhaustive search and performance calculation model described above, this embodiment can effectively optimize the interconnection network of a multi-core system and determine the optimal network topology type. and flit size f i * The optimized network topology not only minimizes cache access time but also meets power consumption and cost budget constraints, thereby improving the overall system performance.

[0216] The present invention also provides a design space exploration device for cache hierarchy in a multi-core system, comprising:

[0217] The architecture diagram acquisition module is used to acquire the storage hierarchy diagram and network topology diagram of the multi-core integrated system. The storage hierarchy diagram is the connection diagram of the core, cache, and DRAM nodes of the multi-core integrated system, and the network topology diagram is the connection diagram of the nodes within / between the cores of the multi-core integrated system.

[0218] The model building module is used to build a system performance model based on the storage hierarchy graph and network topology graph; the system performance model includes a cache miss rate model, a latency model, and a memory parallelization model; and calculates the optimal performance parameters and optimal cache parameters of the storage hierarchy graph respectively;

[0219] The model optimization module is used to solve the system performance model and generate target cache parameters with optimal performance.

[0220] The model generation module is used to generate a target cache architecture diagram based on the target cache parameters, and use the optimal cache parameters corresponding to the target architecture diagram as the cache parameters of the multi-core integrated system.

[0221] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for exploring the design space of cache hierarchy in a multi-core system, characterized in that, Includes the following steps: (1) Obtain the architecture topology diagram of the multi-core system; the architecture topology diagram includes a storage hierarchy diagram and a network topology diagram; wherein, the storage hierarchy diagram is a hierarchy connection diagram of the cache subsystem of the multi-core system, used to characterize the connection relationship between each level of cache; the network topology diagram is a core connection diagram of the multi-core system, used to describe the communication connection relationship between cores; (2) Based on the architecture topology diagram, construct a system performance model; the system performance model includes a cache miss rate model, a latency model, and a memory parallelization model; (3) Optimize the system performance model to obtain target cache parameters; generate a target cache architecture diagram based on the target cache parameters and the storage hierarchy diagram; including: (3.1) Definition of the optimization objective function: Based on the performance requirements of the multi-core system, cost and power consumption constraints are set. Combined with the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application. Through recursion, referring to the formula for average parallel memory access time, the cache miss rate model, latency model and memory parallelization model are integrated into the performance model. (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the optimization subproblem of the cache subsystem and the optimization subproblem of the interconnection network topology. The two-layer optimization algorithm is used to solve the problem alternately until the performance converges or the preset iteration limit is reached. The target cache parameters that meet the constraints are obtained, and the optimized cache architecture diagram is generated.

2. The method for exploring the design space of cache hierarchy in a multi-core system according to claim 1, characterized in that, Step (1) includes the following sub-steps: (1.1) Modeling of cache hierarchy and access latency: Based on the storage hierarchy of the multi-core system, a cache hierarchy diagram is defined to represent the hierarchical relationship of each cache level and its access latency characteristics; (1.2) Bandwidth and delay modeling of the inter-core interconnection network: Construct an inter-core interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation and communication delay characteristics between cores.

3. The method for exploring the design space of cache hierarchy in a multi-core system according to claim 2, characterized in that, Step (2) includes the following sub-steps: (2.1) Cache miss rate modeling: Based on the storage hierarchy diagram and network topology diagram, combined with the hit probability of each cache level and data flow characteristics, a miss rate calculation formula is defined, and a cache miss rate model is constructed to estimate the cache miss rate of different cache levels; (2.2) Cache access latency modeling: Based on the connection path between nodes, establish a cache access latency model, calculate the point-to-point communication latency between two nodes based on the latency model, including zero-load latency and queuing latency; quantify the average access latency of each cache level; (2.3) Memory parallelization modeling: Combining cache miss rate and inter-slice network latency, calculate the average hit concurrency and the proportion of pure miss cycles, and construct a memory parallelization model to quantify the memory access parallelism of the cache level.

4. The method for exploring the design space of cache hierarchy in a multi-core system according to claim 1, characterized in that, Define storage hierarchy diagram Each node Represents a core, cache unit, or storage unit, and the node The weights are: when hour, With the core as the weight, ,in For ideal instruction cycles (CPI). Delay at the core; when hour, For cache units, the weight is ,in For cache size, For cache line size, For cache relevance, For cache replacement strategy, For cache access count; cache replacement strategy The replacement algorithms used include FIFO, LRU, LFU, and random strategies; when hour, For DRAM cells, the weight is ,in For DRAM access latency, For DRAM bandwidth; Storage hierarchy diagram The edge in Represents a node and The connection relationship between nodes, when nodes and When connected, Otherwise, it is 0; Define the topology of the chip-to-chip interconnect network. Each node Represents a core particle The nodes on the network, which are cores, L1 / L2 caches, network interfaces, or routers, or are simply D2D interfaces; the weights of the edges. Represents the transmission delay between two nodes; edge Indicates adjacent nodes and The connectivity between them.

5. The method for exploring the design space of cache hierarchy in a multi-core system according to claim 3, characterized in that, Step (2.2) includes: Zero-load latency is defined as being related to the distance between the source and target nodes, and its calculation method includes based on the chip interconnect network topology. Perform modeling, define For the node To the node The transmission delay consists of two parts: header delay and packet delay. and serialization delay ; Header packet delay By node and Shortest path between And the delay along the path, serialization delay Related to data packet length and transmission rate; the formula is as follows: ； ； in, For transmission rate, For data packet length, This refers to the size of the data slice; Indicates the number of stages in the routing pipeline. Indicates the total delay along the path; Queuing delay is defined and modeled based on the G / G / 1 model, and the computation node... Queuing delay The delay is a node Multiple input channels With single output channel Queue delays between; Calculate total queuing delay The delay is through the path On each router The sum of queuing delays; Ultimately, the transmission delay is obtained. Zero load latency and queue delay The sum of.

6. A design space exploration apparatus for implementing a cache hierarchy in a multi-core system as described in claim 1, characterized in that, include: The architecture diagram acquisition module is used to acquire the architecture topology diagram of the multi-core particle system; The architecture topology diagram includes a storage hierarchy diagram and a network topology diagram; wherein, the storage hierarchy diagram is a hierarchical connection diagram of the cache subsystem of the multi-core system, used to represent the connection relationship between caches at each level; the network topology diagram is a core connection diagram of the multi-core system, used to describe the communication connection relationship between cores; The model building module is used to build a system performance model based on the architecture topology diagram; the system performance model includes a cache miss rate model, a latency model, and a memory parallelization model; The model optimization module is used to optimize the system performance model and obtain target cache parameters; The model generation module is used to generate a target cache architecture diagram based on the target cache parameters and the storage hierarchy diagram.

Images