Design space exploration method for cache hierarchical structure in multi-core particle system

Abstract

The invention discloses a design space exploration method for a cache hierarchical structure in a multi-core particle system. The method aims at optimizing the cache subsystem in the multi-core particle system, and the system performance is improved by reasonably configuring the cache hierarchical structure and the interconnection network topology between the core particles. The method comprises the following specific steps: 1) modeling a cache miss rate and network delay: modeling the cache miss rate and the network delay as a function of a cache hierarchical structure and interchip interconnection network parameters; 2) optimization problem definition: defining an optimization objective and minimizing concurrency average storage access time (C-AMAT) under the constraint of cost and power consumption; and 3) solving by using a double-layer optimization algorithm: respectively optimizing the cache subsystem and the interconnection network between the chip grains through the double-layer optimization algorithm. The method provides an effective solution for cache optimization of the multi-core particle system, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuits, and in particular to a design space exploration method for a cache hierarchy structure in a multi-chip system. Background Art

[0002] With the increasing complexity of computing systems and the increasing prevalence of multi-core systems, memory performance optimization has become a core issue in computing architecture research. Existing research has proposed a variety of memory performance modeling and optimization methods, but they still have many shortcomings. The concurrent average memory access time (C-AMAT) model extends the functionality of the traditional average memory access time (AMAT) model by incorporating memory concurrency into performance calculations. However, its optimization focuses primarily on local memory characteristics and lacks global performance optimization capabilities. The hierarchical performance matching (LPM) method decomposes the memory system performance optimization problem into local subproblems. While it improves overall performance, its applicability to multi-core sharing and consistency issues is limited. The data sharing-aware analysis model effectively estimates the shared cache miss rate and addresses some data consistency issues, but its scalability is insufficient in heterogeneous environments and complex multi-threaded workloads. The shared memory analysis model (SASMM) and optimal cache partitioning techniques perform well in terms of shared cache hit rate and dynamic cache partitioning, but they still lack flexibility when dealing with dynamic workloads and heterogeneous memory requirements. Design tools and optimization models for multi-chip systems (such as IntLP, NN-Baton, and Monad) have made significant progress in functional partitioning, resource optimization, and design space exploration. However, there is still room for improvement in memory hierarchy optimization and inter-chip communication efficiency. Furthermore, while multi-chip integration methods based on the 64-core RISC-V processor and cost sensitivity analysis of multi-core systems on a chip address cost and scalability issues in the early stages of system design, they lack attention to runtime dynamic performance optimization. In heterogeneous systems, optimization methods for latency constraints and dynamic scheduling are often limited to single-architecture scenarios and struggle to adapt to the complex requirements of multi-chip systems. Overall, existing technologies have achieved significant progress in memory performance measurement, cache management, and design tools. However, in multi-chip systems, challenges remain, such as insufficient global performance optimization, limited dynamic adaptability, insufficiently thorough cost-performance trade-offs, and a vast design space. Further research and improvement are needed to meet the demands of future high-performance computing systems. Summary of the Invention

[0003] The purpose of the present invention is to provide a design space exploration method for the cache hierarchy structure in a multi-chip system to address the problems of high design complexity, low performance optimization efficiency and insufficient adaptability of the existing multi-chip system cache hierarchy structure, which can achieve the optimal configuration of cache access performance while meeting the cost and power consumption constraints.

[0004] To achieve the above objectives, the technical solutions provided by this application are as follows:

[0005] The present invention provides a method for exploring the design space of a cache hierarchy in a multi-chip system, including the following steps:

[0006] (1) Obtaining an architectural topology diagram of a multi-chip integrated system; the architectural topology diagram includes a storage hierarchy diagram and a network topology diagram; wherein the storage hierarchy diagram is a hierarchical connection diagram of a cache subsystem of the multi-chip integrated system, used to characterize the connection relationship between caches at each level; and the network topology diagram is a chip connection diagram of the multi-chip integrated system, used to describe the communication connection relationship between chiplets;

[0007] (2) constructing a system performance model based on the architecture topology diagram; the system performance model includes a cache miss rate model, a delay model, and a memory parallelization model;

[0008] (3) Optimizing the system performance model to obtain target cache parameters; generating a target cache architecture diagram based on the target cache parameters and the storage hierarchy diagram.

[0009] Furthermore, the step (1) includes the following sub-steps:

[0010] (1.1) Cache hierarchy and access latency modeling: Based on the storage hierarchy of a multi-chip system, a cache hierarchy diagram is defined to represent the hierarchical relationship of caches at all levels and their access latency characteristics.

[0011] (1.2) Bandwidth and delay modeling of the inter-chip interconnection network: Construct an inter-chip interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation, and communication delay characteristics between chiplets.

[0012] Furthermore, the step (2) includes the following sub-steps:

[0013] (2.1) Cache miss rate modeling: Based on the storage hierarchy diagram and network topology diagram, combined with the cache hit probability and data flow characteristics of each level, a miss rate calculation formula is defined and a cache miss rate model is constructed to estimate the cache miss rate at different cache levels.

[0014] (2.2) Cache access latency modeling: Based on the connection paths between nodes, a cache access latency model is established. Based on the latency model, the point-to-point communication latency between two nodes is calculated, including zero-load latency and queuing latency. The average access latency of each cache level is quantified.

[0015] (2.3) Memory parallelization modeling: Combining cache miss rate and inter-chip network latency, the memory parallelization parameters related to concurrent memory access time are calculated to evaluate the overall performance of the multi-chip system.

[0016] Furthermore, the step (3) includes the following sub-steps:

[0017] (3.1) Definition of the optimization objective function: Based on the performance requirements of the multi-chip system, setting cost and power constraints, and combining the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application. Through recursive relationships and referring to the average parallel memory access time formula, the cache miss rate model, latency model, and parallel memory access time model are integrated into the performance model.

[0018] (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the cache subsystem optimization subproblem and the inter-chip interconnection network topology optimization subproblem. The two-layer optimization algorithm is used to alternately solve the problem until the performance converges or reaches the preset iteration limit. The target cache parameters that meet the constraints are obtained, and the optimized cache architecture diagram is generated.

[0019] Furthermore, we define the storage hierarchy graph G M (V,E), where each node v ijk ∈V represents a core, cache unit or storage unit, and node v ijk The weight is:

[0020] When j = 0, v ijk is the core, and the weight is w(v ijk )= <cpi(v ijk ),ls(v ijk )>, where cpi(v ijk ) is the ideal cycles per instruction (CPI), ls(v ijk ) is the core delay;

[0021] When 0 <j<h k When v ijk is a cache unit with a weight of w(v ijk )= <cs(v ijk ),cl(v ijk ),as(v ijk ),

[0022] rs(v ijk ),ac(v ijk )>, where cs(v ijk ) is the cache size, cl(v ijk ) is the cache line size, as(v ijk ) is the cache associativity, rs(v ijk ) is the cache replacement strategy, ac(v ijk ) is the number of cache accesses; the cache replacement strategy rs(v ijk ) The replacement algorithms used include: FIFO, LRU, LFU, and random strategies;

[0023] When j = h k When v ijk is a DRAM unit with a weight of w(v ijk )= <dt(v ijk ),bw(v ijk )>, where dt(v ijk ) is the DRAM access delay, bw(v ijk ) is the DRAM bandwidth;

[0024] Storage hierarchy diagram G M Edge e in (V,E) ijk,mno ∈E represents node v ijk and v mno The connection relationship between them, when the node v ijk With v mno When connected, e ijk,mno =1, otherwise 0;

[0025] Define the inter-chip interconnection network topology G I (N,C), where each node n p,q ∈N represents a node on a core q, which is a core, L1 / L2 cache, network interface or router, or just a D2D interface; node n p,q The weight ω(c(n p,q ,n r,s )) represents the transmission delay between nodes; network connection c(n p,q ,n r,s )∈C connects adjacent nodes; the core particle interconnection network topology G I The number of core particles in (N,C) is n α , and each core particle where |R i | represents the number of cores in the i-th core particle;

[0026] The cache miss rate model and network delay model of the multi-chip system are constructed through the storage hierarchy diagram and network topology diagram to optimize the storage access performance under power and cost constraints.

[0027] Furthermore, the cache miss rate model M(v ijk ) is calculated by node v ijk The reachable path Π(v ijk ) to determine the cache size, cache line size, cache associativity, and replacement policy polynomial fitting, where Π(v ijk ) represents the distance from any core to node v ijk The set of all nodes on the reachable path; cache miss rate model M(v ijk) based on different cache replacement strategies, including FIFO, LRU, LFU and random strategies.

[0028] Furthermore, the step (2.2) includes:

[0029] Define zero load delay, which is related to the distance between the source node and the target node. The calculation method includes the following steps: I Modeling, defining L(n p,q ,n r,s ) is the slave node n p,q To node n r,s The transmission delay consists of two parts: the header packet delay L h (n p,q ,n r,s ) and serialization delay L s (n p,q ,n r,s );

[0030] According to the transmission rate μ, data packet size s i , flit size f and number of router pipeline stages η to calculate the serialization delay L s (n p,q ,n r,s ) and header packet delay L h (n p,q ,n r,s );

[0031] Define queuing delay, which is modeled based on the G / G / 1 model and computes nodes. The queue delay This delay is the node Multiple input channels With single output channel The queuing delay between

[0032] Calculate the total queuing delay W(n p,q ,n r,s ), the delay is through the path Γ(n p,q ,n r,s ) on each router The sum of the queuing delays;

[0033] Finally, the transmission delay D(n p,q ,n r,s ) is the zero load delay L(n p,q ,n r,s ) and queuing delay W(n p,q ,n r,s ) and.

[0034] Furthermore, the step (3) includes:

[0035] Problem decomposition: The optimization problem is decomposed into two sub-problems through a two-level optimization algorithm, namely, the storage hierarchy graph G M The optimization subproblem P1 and the network topology graph G I The optimization sub-problem P2;

[0036] Iterative optimization: The optimization process adopts an iterative approach. In each iteration, only one sub-problem is optimized, and the decision variables of the other sub-problem are fixed as constraints.

[0037] Termination condition: When the target performance index σ meets the convergence condition or reaches the maximum number of iterations, the optimization process terminates.

[0038] Furthermore, the step (3.2) includes:

[0039] A two-level optimization framework is used, where subproblem P1 is a storage hierarchy optimization problem, and its goal is to optimize G M To minimize the average concurrent memory access time σ(G M );

[0040] Subproblem P2 is the interconnection network topology optimization problem, whose goal is to optimize G I To minimize the average concurrent memory access time σ(G I ).

[0041] The present application also provides a design space exploration device for a cache hierarchy in a multi-chip system, comprising:

[0042] The architecture diagram acquisition module is used to obtain the storage hierarchy diagram and network topology diagram of the multi-chip integrated system. The storage hierarchy diagram is a connection diagram of the core, cache, and DRAM nodes of the multi-chip integrated system. The network topology diagram is a connection diagram of the nodes within and between the chips of the multi-chip integrated system.

[0043] A model building module is used to build a system performance model based on the storage hierarchy diagram and the network topology diagram; the system performance model includes a cache miss rate model, a delay model and a memory parallelization model; and calculate the optimal performance parameters and optimal cache parameters of the storage hierarchy diagram respectively;

[0044] A model optimization module, configured to solve the system performance model and generate target cache parameters for optimal performance;

[0045] A model generation module is used to generate a target cache architecture diagram based on the target cache parameters, and use the optimal cache parameters corresponding to the target architecture diagram as cache parameters of the multi-chip integrated system.

[0046] The present invention has the following beneficial effects:

[0047] Through cache performance modeling and design space optimization, cache access time is significantly reduced, improving the overall performance of multi-chip systems;

[0048] The hierarchical optimization algorithm effectively reduces the complexity of the optimization problem and improves design efficiency;

[0049] Supports flexible optimization for various application scenarios and can meet specific performance requirements;

[0050] While optimizing performance, it also takes into account cost and power consumption constraints to improve the economy and feasibility of system design.

[0051] The present invention provides a systematic solution for optimizing cache performance of a multi-chip system, and can realize efficient cache hierarchy design in a high-performance computing environment.

[0052] Compared with the existing technology, the present invention can comprehensively optimize the configuration of the cache subsystem by integrating multiple models, reduce the performance loss caused by cross-chip communication delay and cache misses, meet the stringent requirements of high-performance computing systems on cache architecture, and thus improve the overall computing efficiency of multi-chip systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 Schematic diagram of the process of the present invention;

[0055] Figure 2 is the architecture diagram of the present invention; wherein, Figure 2 (a) is the multi-core system architecture diagram. Figure 2 (b) in the figure is the cache hierarchy diagram. Figure 2 (c) in the figure is the topology diagram of the interconnection network between core particles. DETAILED DESCRIPTION

[0056] In order to more clearly illustrate the technical solution of the present invention and the beneficial effects thereof, the technical solution of the present application is now described in detail in conjunction with the accompanying drawings and specific embodiments. It should be particularly pointed out that the embodiments described below are only for the purpose of facilitating the understanding of the technical content of the present application and should not be regarded as limiting the scope of protection of the present application. Under the premise that ordinary technicians in this field do not need creative work, any other implementation methods made based on the disclosure of the present application should be deemed to fall within the scope of protection of the present application.

[0057] In this application, the term "embodiment" is used to refer to a possible implementation method or technical solution. The repeated mention of the term "embodiment" in the specification does not mean that all references belong to the same embodiment, nor does it mean that different embodiments are mutually exclusive. In fact, based on actual needs and technical background, the features of different embodiments can be appropriately combined and adjusted to achieve more optimal technical effects.

[0058] In the context of this application, "include" or "comprising" is non-exclusive, meaning that in addition to the listed contents, other relevant contents may also be included. For example, a method may include steps that are not explicitly listed, and an apparatus may include other modules or units that are not mentioned.

[0059] The terms "connect", "connected" and the like in this application are not limited to physical connections, but may also include electrical connections, whether direct or indirect.

[0060] When implementing this application, technicians can reasonably adjust the order or structure of the various modules and steps in the technical solution according to specific circumstances. This flexibility applies not only to the design of methods and devices, but also to the manufacture and deployment of products or equipment. In addition, for ordinary technicians in this field, further designing and optimizing the technical solution of the present invention based on the disclosure of this application is a routine technical activity and should also be regarded as an expanded application of the present invention, and does not constitute a transcendence of the scope of the technical disclosure.

[0061] Through the above description, those skilled in the art can fully understand the technical content and implementation of the present invention. The technical solution of the present invention can be adjusted and optimized according to specific needs. In particular, for complex multi-core systems, the design of their storage architecture and interconnection network can achieve significant performance improvements and design flexibility through the optimization methods provided in this application.

[0062] The core technology of this invention is to model and optimize the cache hierarchy and interconnection network of the multi-chip system, and propose a two-level optimization method that combines performance modeling and design space exploration to achieve the optimal configuration of the cache access performance of the multi-chip system while meeting the cost and power consumption constraints of the system.

[0063] The present invention proposes a design space exploration method for a cache hierarchy in a multi-chip system, comprising the following steps:

[0064] (1) Construction of cache and interconnection model: Based on the cache hierarchy and inter-chiplet communication characteristics of the multi-chiplet system, a cache hierarchy diagram and an inter-chiplet interconnection network topology diagram are constructed. This includes the following sub-steps:

[0065] (1.1) Cache hierarchy relationship and access delay modeling: Based on the storage hierarchy of the multi-chip system, a cache hierarchy diagram is defined to represent the hierarchical relationship of caches at each level and their access delay characteristics.

[0066] (1.2) Bandwidth and delay modeling of the inter-chip interconnection network: Construct an inter-chip interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation, and communication delay characteristics between chiplets.

[0067] (2) Cache performance modeling: Establish a cache miss rate model, cache access latency model, and memory parallelization model based on a multi-core system to quantify performance indicators. This includes the following sub-steps:

[0068] (2.1) Cache miss rate modeling: Based on the hit probability and data flow characteristics of each level of cache, a miss rate calculation formula is defined.

[0069] (2.2) Cache access delay modeling: Based on the connection paths between nodes, a cache access delay model is established to quantify the average access delay of each level of cache.

[0070] (2.3) Memory parallelization modeling: Combining cache miss rate and inter-chip network latency, the memory parallelization parameters related to concurrent memory access time are calculated to evaluate the overall performance.

[0071] (3) Definition and solution of performance optimization problem: Under the condition of meeting the system cost and power consumption constraints, the optimization objective function is defined based on the performance model to minimize the average concurrent memory access time of the application. The specific sub-steps include:

[0072] (3.1) Definition of optimization objective function: Combining the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application.

[0073] (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the cache subsystem optimization subproblem and the inter-chip interconnection network topology optimization subproblem, and the two-layer optimization algorithm is used to alternately solve them until the performance converges or reaches the preset iteration limit.

[0074] This invention introduces a two-layer optimization approach, breaking down the complex cache performance optimization problem into subproblems that are solved incrementally, effectively reducing the complexity of the problem. Simultaneously, combined with the layered performance matching method (LPM), it achieves collaborative optimization of the cache subsystem and the interconnection network, significantly improving system performance while balancing cost and power consumption constraints. This invention is applicable to a variety of application scenarios and provides strong technical support for the design of high-performance multi-chip systems.

[0075] See also Figure 1By building a storage hierarchy diagram and a network topology diagram to model the multi-core system, and combining it with the performance model to optimize and solve, we can finally achieve the optimal design of the cache subsystem. Figure 2 The embodiments of the present invention will be described in detail.

[0076] like Figure 2 As shown in (a) in FIG. 5 , in this embodiment, the multi-chip integrated system is modeled by two diagrams, which respectively represent its cache subsystem and interconnection subsystem.

[0077] Storage hierarchy diagram G M (V,E) is an undirected graph used to describe the storage hierarchy of a multi-core integrated system. ijk ∈V represents a core, cache unit or memory unit. Where i is the node number, j is the cache level, and k is the core number. When j = 0, node v ijk Indicates the core; when 0 <j<h k When node v ijk Represents a cache unit; when j=h k When node v ijk Represents a dynamic random access memory (DRAM) cell. ijk,mno ∈E represents node v ijk and v mno The connectivity between, for example, the connection between two cache units or the reachability between the core and the cache. ijk and v mno If there is a direct connection between ijk,mno =1, otherwise 0.

[0078] Specifically, if Figure 2 As shown in (b) of Figure 2, the storage hierarchy diagram includes multiple cores and their corresponding L1 cache (L1Cache) and L2 cache (L2 Cache), and two cores share the L3 cache (L3 Cache). For example, node v 131 and v 121 There is an edge between them, indicating that v 131 Contains v 121 On the other hand, v 131 and v 321 There is no edge between them, indicating that there is no containment relationship between them.

[0079] The weight w(v ijk ) is defined as follows:

[0080]

[0081] Among them, for the core node j = 0, the weight is <cpi(v ijk)>, indicating the ideal instruction cycle number of the core. <j<h k The cache node, weight is the cache parameter set <cs(v ijk ),cl(v ijk ),as(v ijk ),rs(v ijk ),ac(v ijk )>, respectively represents the cache size, cache line size, associativity, replacement strategy and cache access times. For j=h k The DRAM node has a weight of <dt(v ijk ),bw(v ijk )>, represents the access latency and bandwidth of DRAM. The replacement strategy rs(v ijk ) The replacement algorithms used include: FIFO (first in, first out), LRU (least recently used), LFU (least frequently used), and random strategies.

[0082] Interconnection network topology diagram G I (N,C) is another undirected graph that describes the node connection structure between and within core particles. Each node n p,q ∈N represents a node in a core q, which can be a core, L1 / L2 cache, network interface (NI), router, or D2D interface between cores. p,q ,n r,s )∈C represents the adjacent node n p,q and n r,s If q = s, then node n p,q and n r,s are in the same core particle; otherwise, they are in different core particles. The weight of the edge ω(c(n p,q ,n r,s )) represents the transmission delay between two nodes. I The number of core particles in (N,C) is n α , and each core particle where |R i | represents the number of cores in the i-th core particle.

[0083] The cache miss rate model and network delay model of the multi-chip system are constructed through the storage hierarchy diagram and network topology diagram to optimize the storage access performance under power and cost constraints.

[0084] like Figure 2 As shown in (c) of Figure 1, the interconnection network topology diagram shows a two-core system, which includes the connection relationship between the core particles and the core particles. The core particles communicate with each other through the D2D interface, and the core particles are connected internally through routers.

[0085] In step (2), by combining the cache hierarchy diagram and the core-to-core interconnection network topology diagram, the system performance is modeled based on the cache miss rate model, the latency model, and the memory parallelism model. The specific implementation is as follows:

[0086] In order to quantify the performance of the multi-chip integrated system, this embodiment uses the cache hierarchy graph G M Chiplet interconnection topology diagram G I Build a performance model. The modeling process is described in detail below.

[0087] ① Mapping relationship between cache nodes and network nodes

[0088] Defining a function Return to G M Zhong and G I Network node n p,q The corresponding cache node v ijk ,Right now

[0089] ② Modeling the average latency between cache levels

[0090] To estimate the average delay between adjacent cache levels, this embodiment calculates the point-to-point delay based on the delay model and traffic theory. Defined as:

[0091]

[0092] in, Represents slave node n p,q To node n r,s flow rate, c=1,…,g.

[0093] ③Average parallel memory access time modeling

[0094] Define the average parallel memory access time for cache level j-1 as Indicates the average time taken for each memory access in one CPU cycle. It is related to the next cache level. The average parallel memory access time has a recursive relationship:

[0095]

[0096] Among them, H(v i(j-1)k ) is the hit time of cache level j-1, C H (v i(j-1)k ) is the average hit concurrency of cache level j-1, M(v i(j-1)k ) is the miss rate of cache level j-1, κ(v i(j-1)k ) is the pure miss cycle ratio of cache level j-1, is the average latency between cache levels j-1 and j.

[0097] ④Performance modeling

[0098] The access time of the cache subsystem σ(G M ,G I ) represents the total time consumed by cache access operations in a multi-core system, i.e., the performance model, which is expressed as:

[0099]

[0100] Among them, H(v ijk ) is the hit time of cache level j in core k, C H (v ijk ) is the average hit concurrency of cache level j in core k, M(v itk ) is the cache miss rate of cache level j in core k, κ(v itk ) is the pure miss cycle ratio of cache level t in core k, is the average latency between cache levels j-1 and j, n α is the total number of core particles, h k is the total number of cache levels of corek, |R k | is the core number of core particle k, and t is the recursive series from the product to j.

[0101] Through the above performance modeling method, this embodiment can accurately evaluate the cache access performance in the multi-chip integrated system, providing a basis for subsequent optimization.

[0102] In step (2.1), the cache miss rate is modeled to optimize the cache subsystem of the multi-chip integrated system. The specific implementation method of cache miss rate modeling is described in detail below in conjunction with the storage hierarchy diagram.

[0103] In the embodiment of the present application, the cache miss rate is calculated by storing the hierarchical graph G M (V,E) modeling, where G M Represents the storage hierarchy diagram of the multi-core integrated system. Each node v in the diagram ijk ∈V represents a node in the cache hierarchy, which includes a core, cache unit, or memory unit.

[0104] In cache miss rate modeling, for each cache node v ijk , define the set of all upper nodes on its reachable path as Π(v ijk )={π1(v ijk ),…,π n (v ijk )}, where π x (v ijk) represents the xth node on the path, and n is the total number of nodes on the path. This set contains v ijk All upper-level nodes in the cache hierarchy that have an associated relationship (e.g., inclusiveness) with it.

[0105] On this basis, node v ijk The cache miss rate M(v ijk ) is modeled by the following formula:

[0106]

[0107]

[0108] Among them, Λ(v ijk ) is a function related to cache, which is composed of cache size (Cache Size,cs(π x (v ijk Cache Line Size, cl(π x (v ijk ))) and association (Associativity, as (π x (v ijk ))) composition; a x , b x , c x and d x is a coefficient related to the cache miss rate; α x , β x , γ x and δ x is the corresponding polynomial order; rs(π x (v ijk )) is the cache replacement strategy, including FIFO (first in first out), LRU (least recently used), LFU (least frequently used) and Random (random replacement); is a function that maps replacement strategies to numerical values.

[0109] For example, for node v ijk , its path set Π(v ijk ) contains all nodes in the upper layer. Calculate Λv by combining the weights of cache size, cache line size and associativity ijk ), and calculate M(v according to the mapping value of the replacement strategy ijk ), thus obtaining the final cache miss rate of the node.

[0110] This modeling approach accurately assesses the miss rates of caches at each level in the cache subsystem, providing data support for cache optimization. Combined with other models, the cache architecture of multi-chip integrated systems can be further optimized to achieve more efficient system performance.

[0111] In step (2.2), the transmission delay between nodes is analyzed and modeled to achieve the optimization prediction of system performance. I (N, C) Describe in detail the specific implementation of delay modeling.

[0112] In an embodiment of the present application, the delay model is used to estimate the transmission delay between two core nodes, and the transmission delay consists of two parts: zero-load delay and queue delay.

[0113] ①Zero load delay

[0114] Zero load delay refers to the time required for a data packet to be transmitted from the source node to the destination node in the absence of contention. p,q ,n r,s ) is delayed by the header packet L h (n p,q ,n r,s ) and serialization delay L s (n p,q ,n r,s ) consists of two parts, and its expression is:

[0115] L(n p,q ,n r,s )=L h (n p,q ,n r,s )+L s (n p,q ,n r,s )

[0116] Header packet delay L h (n p,q ,n r,s ) by node n p,q and n r,s The shortest path between l(n p,q ,n r,s ) and the delay on the path, the serialization delay l s (n p,q ,m r,s ) is related to the packet length and transmission rate. The formula is as follows:

[0117] l h (n p,q ,n r,s )=ψ(η·l(n p,q ,n r,s ))

[0118]

[0119] Where μ is the transmission rate, s iis the packet length, f is the data slice size; η represents the number of routing pipeline stages, ψ(η·l(n p,q ,n r,s )) represents the total delay on the path.

[0120] The total length of the shortest path is calculated recursively using the Bellman equation. The path starts from node n p,q and n r,s The formula is:

[0121]

[0122] in, Indicates the distance between adjacent nodes in the path.

[0123] ②Queue delay

[0124] Queue delay refers to the waiting time for data transmission due to data contention. Queue delay W(n p,q ,n r,s ) is the sum of the queue delays of each routing node on the path, and the path is Γ(n p,q ,n r,s ). In the G / G / 1 model, each routing node on the path The queue delay The expression is:

[0125]

[0126] in, Indicates the queue occupancy ratio, is the packet transmission rate, is the service rate, and are the correlation coefficients of data arrival rate and service rate respectively.

[0127] The total queue delay is calculated as:

[0128]

[0129] ③Total transmission delay

[0130] Finally, the total transmission delay D(n p,q ,n r,s ) is obtained by adding the zero-load delay and the queue delay:

[0131] D(n p,q ,n r,s )=L(n p,q ,n r,s )+W(n p,q ,n r,s )

[0132] By modeling the transmission delay between nodes using the above method, the performance impact of network communication in multi-chip integrated systems can be quantified, providing a theoretical basis for optimized design.

[0133] In step (2.3), the memory access characteristics of the cache subsystem are modeled by defining the average hit concurrency and the pure miss cycle ratio. The specific implementation is as follows:

[0134] Memory parallelism modeling aims to quantify the hit concurrency and miss overhead of memory accesses in multi-chip systems, thereby providing guidance for system performance optimization. The following details the specific modeling process.

[0135] Average hit concurrency C H (v ijk ) is used to indicate the concurrency of cache hit events during memory access, and its calculation formula is:

[0136]

[0137] Among them, α(v ijk ) represents the total number of memory accesses of the core, defined as:

[0138] α(v ijk )=IC(v ijk )·f mem (v ijk )

[0139] IC(v ijk ) is the number of core instructions, f mem (v ijk ) is the average number of memory accesses per instruction, H(v ijk ) represents the core hit time (calculated in CPU cycles), h(v ijk ) is the number of pure hit cycles of the core, x(v ijk ) is the mixed hit / miss cycles of the core, is the hit concurrency of the core in the i-th CPU cycle, and n is the total number of CPU cycles.

[0140] Pure miss cycle ratio κ(v ijk ) is used to describe the ratio of core miss cycles to total miss cycles, and its calculation formula is:

[0141]

[0142] Among them, m(v ijk ) is the number of pure miss cycles of the core.

[0143] For C H (v ijk ) and κ(vijk ), which is modeled via regression models to capture its relationship with cache parameters.

[0144] Average hit concurrency C H (v ijk ) is:

[0145]

[0146] Among them, f x , l x and g x is the coefficient, θ x , ι x and ξ x is the polynomial order, π x (v ijk ) is from the core to v ijk The xth node on the path, mpki(π x (v ijk )) is the number of memory accesses per thousand instructions of the path node, ac(π x (v ijk )) is the number of cache accesses to the path node, is the DRAM bandwidth.

[0147] Pure miss cycle ratio k(v ijk ) is:

[0148]

[0149] Among them, m x and o x is the regression coefficient, and is the polynomial order, is the miss rate of LLC (last level cache).

[0150] By modeling the above average hit concurrency and pure miss cycle ratio, the memory access parallelism characteristics of the cache hierarchy can be effectively quantified, providing a theoretical basis for optimizing cache configuration.

[0151] In step (3), an optimization problem is defined in combination with the performance model, aiming to achieve the optimal performance of the multi-chip system under the constraints of cost and power consumption. The specific implementation is as follows:

[0152] ① Cost modeling

[0153] The architectural cost of a multi-chip system is determined by the cumulative cost of each component (including chip and DRAM) and the chip assembly / integration cost c int constitute.

[0154] The cost of a single chip Cdie (v ijk ) can be expressed as:

[0155]

[0156] in, represents the chip area, c1 is the cost per unit silicon area, and Y die Is related to the chip area Defect density The limit yield related to the clustering factor a is calculated as follows:

[0157]

[0158] For DRAM cost C dram (v ijk ) can be expressed as:

[0159]

[0160] Among them, bw(v ijk ) is the DRAM bandwidth, c2 is the cost per unit bandwidth, U bw The capacity provided per unit of DRAM bandwidth.

[0161] The cost modeling formula for the entire multi-chip system is:

[0162]

[0163] ②Power consumption modeling

[0164] The power consumption of a multi-chip system is the cumulative power consumption of each component. The system power consumption modeling formula is as follows:

[0165]

[0166] Among them, P core (v ijk ) is the core power consumption, P dram (v IJK ) is the DRAM power consumption, P ROutER (v Ijk ) is the router power consumption, r k is the number of routers in each core particle.

[0167] In step (3.1), the problem definition is carried out, which is specifically implemented as follows:

[0168] Based on the above performance, cost and power consumption model, the performance of the multi-chip system is optimized to minimize the average memory access time σ(G M ,G I). Among them, the cost and power consumption of each chip shall not exceed the set threshold C T and P T The optimization problem is defined as follows:

[0169] min{σ(G M ,G I )}

[0170] dtG(G M ,G I )≤C T

[0171] P(G M ,G I )≤P T

[0172] The optimization goal is to minimize the cache access time of the application under the power consumption and cost budget. The search space of the above problem is very large. For example, consider a system containing n γ cores, n α A system of core particles, core particle i contains |R i | cores, the total number of cache levels is h i , the number of cross-chip network interconnection topologies is t, and the optional size of the j-th layer cache is The available DRAM sizes are

[0173] In step (3.2), the goal is to achieve optimal performance of the multi-chip system by efficiently exploring cache configurations and interconnect networks in a large-scale design space through a two-layer optimization algorithm that combines Layered Performance Matching (LPM) and Branch-and-Bound (BNB).

[0174] In a multi-chip system, due to the complexity of cache configuration and interconnection network design, the design space needs to be decomposed into two sub-problems: (a) Memory hierarchy graph G M Optimization subproblem P1, (b) Inter-core and intra-core interconnection network topology G I Optimize subproblem P2. Its optimization objectives are:

[0175] min{σ(G M )}

[0176] min{σ(G I )}

[0177] Bi-level optimization iterates by alternately optimizing P1 and P2. In each iteration, only one sub-problem is solved, while the variables of the other sub-problem are fixed as constraints. For example, in the first iteration, P1 is solved and the decision variables of P2 are fixed; in the second iteration, P2 is solved. When the performance indicator σ converges or reaches a predefined upper limit of iterations n, the performance indicator σ is optimized. max When , the algorithm terminates.

[0178] The iterative convergence condition is:

[0179] σ i -σ i-1 ≤δ or i≤n MAx

[0180] Among them, σ i is the cache access time of the i-th iteration, δ is the convergence condition, n max is the upper limit of the number of iterations.

[0181] ① Storage hierarchy diagram G M optimization

[0182] Search Space Definition The goal of the memory hierarchy graph optimization subproblem P1 is to optimize the cache hierarchy h i , the number of cores in each core particle|R i |、Cache parameter w(v ijk ). The optimization objective can be expressed as:

[0183] min{σ(G M )=σ9h i ,|R i |,w(v IJk ))}

[0184] Among them, h i = j > represents the maximum number of cache layers for each core particle; |R i |=<|R1|,|R2|,…,|R j |> indicates the number of cores per core particle;

[0185] Including cache size, cache line size, associativity, and replacement policy.

[0186] This subproblem is solved using the branch and bound algorithm. The decision variables are represented as tree nodes It is defined as the a-th node in the b-th layer. Its form is:

[0187]

[0188] Each tree node Associate a target value​ and cost and power consumption

[0189] The branching rule is that the search tree is expanded step by step in levels.

[0190] Hierarchical Performance Matching (LPM) pruning. Hierarchical Performance Matching (LPM) is a method for evaluating the performance of hierarchical memory systems based on data flow analysis. Using the aforementioned performance model, the LPM model is used to verify the utilization of each cache tier.

[0191] LPM matching rate LPMR (v ijk ) is defined as the ratio of the request rate to the supply rate of cache level j, as follows:

[0192]

[0193] Among them, λ(v ijk ) represents the request rate; v(v ijk ) represents the supply rate; μ(v ijk Indicates the miss ratio of learn cycles.

[0194] According to the above definition, the smaller LPMR (v iJk ) indicates higher memory performance of cache level j.

[0195] The pruning rules defined in this embodiment are implemented as follows: Combined with LPM pruning, when the node LPMR (v ijk ) exceeds the set performance threshold, pruning is performed.

[0196] Through the above method, this embodiment achieves efficient exploration of the design space of the multi-chip integrated system, optimizes the cache configuration and interconnection network topology, and improves the overall system performance.

[0197] ② Inter-chip interconnection network topology diagram G I Optimization

[0198] This embodiment provides a topology diagram G for the interconnection network between core particles. I The optimization method aims to achieve the optimal interconnection performance of multi-chip integrated systems by adjusting the network topology and data transmission parameters.

[0199] In the interconnection network optimization subproblem P2, the optimization objective is defined as:

[0200] min{σ(G I )=σ(t i ,f i )}

[0201] Among them, t iIndicates G I The network topology types include Mesh, Ring, Torus, Crossbar, Clos, and Fattree, which are defined as:

[0202] t i = <t1,t2,…,t j >

[0203] At the same time, f i Indicates the flit size of data transmission, defined as:

[0204] f i = <f1,f2,…,f j >

[0205] Specifically, the network topology parameter t i The value range is [0,1,2,3,4,5], corresponding to the six topological structures of Mesh, Ring, Torus, Crossbar, Clos and Fat tree respectively.

[0206] Data transmission parameter f i The value range is [32,64,…,1024], which indicates the size of the flit in bytes.

[0207] During the interconnection network optimization process, the following objectives and constraints need to be considered:

[0208] Optimization goal: By adjusting t i and f i , minimize the average parallel memory access time σ(G I ).

[0209] Constraints: Power consumption budget P must be met T and cost budget C T constraints to ensure that the optimization results are feasible.

[0210] For the optimization of network topology type and flit size, this embodiment uses an exhaustive search method to find the optimal solution within the parameter range. The optimization process is as follows:

[0211] A: Initialization: Set t i and f i The value range of is initialized to all possible values ​​of the corresponding parameters [0,1,2,3,4,5] and [32,64,…,1024].

[0212] B: Search process: traverse all possible network topologies and flit size combinations, and calculate the average parallel memory access time σ(G I ).

[0213] C: Optimal solution screening: According to the average parallel memory access time σ(G I ) calculation results, and select the optimal combination that meets the power consumption and cost constraints.

[0214] D: Termination condition: When all parameter combinations are traversed, the search process terminates and the optimal network topology is output. and flit size f i * .

[0215] Through the above exhaustive search and performance calculation model, this embodiment can effectively optimize the interconnection network of the multi-core system and determine the optimal network topology type. and flit size f i * The optimized network topology not only minimizes cache access time but also meets power consumption and cost budget constraints, thereby improving the overall performance of the system.

[0216] The present invention also provides a design space exploration device for a cache hierarchy in a multi-chip system, comprising:

[0217] The architecture diagram acquisition module is used to obtain the storage hierarchy diagram and network topology diagram of the multi-chip integrated system. The storage hierarchy diagram is a connection diagram of the core, cache, and DRAM nodes of the multi-chip integrated system. The network topology diagram is a connection diagram of the nodes within and between the chips of the multi-chip integrated system.

[0218] A model building module is used to build a system performance model based on the storage hierarchy diagram and the network topology diagram; the system performance model includes a cache miss rate model, a delay model and a memory parallelization model; and calculate the optimal performance parameters and optimal cache parameters of the storage hierarchy diagram respectively;

[0219] A model optimization module, configured to solve the system performance model and generate target cache parameters for optimal performance;

[0220] A model generation module is used to generate a target cache architecture diagram based on the target cache parameters, and use the optimal cache parameters corresponding to the target architecture diagram as cache parameters of the multi-chip integrated system.

[0221] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A design space exploration method for cache hierarchy in a multi-chip system, characterized by: The steps include: (1) Obtaining an architectural topology diagram of a multi-chip integrated system; the architectural topology diagram includes a storage hierarchy diagram and a network topology diagram; wherein the storage hierarchy diagram is a hierarchical connection diagram of a cache subsystem of the multi-chip integrated system, used to characterize the connection relationship between caches at each level; and the network topology diagram is a chip connection diagram of the multi-chip integrated system, used to describe the communication connection relationship between chiplets; (2) constructing a system performance model based on the architecture topology diagram; the system performance model includes a cache miss rate model, a delay model, and a memory parallelization model; (3) Optimizing the system performance model to obtain target cache parameters; generating a target cache architecture diagram based on the target cache parameters and the storage hierarchy diagram.

2. The design space exploration method for cache hierarchy in a multi-chip system according to claim 1, characterized in that: The step (1) includes the following sub-steps: (1.1) Cache hierarchy and access latency modeling: Based on the storage hierarchy of a multi-chip system, a cache hierarchy diagram is defined to represent the hierarchical relationship of caches at all levels and their access latency characteristics. (1.2) Bandwidth and delay modeling of the inter-chip interconnection network: Construct an inter-chip interconnection network topology diagram to describe the communication connection relationship, bandwidth allocation, and communication delay characteristics between chiplets.

3. The design space exploration method for cache hierarchy in a multi-chip system according to claim 2, characterized in that: The step (2) includes the following sub-steps: (2.1) Cache miss rate modeling: Based on the storage hierarchy diagram and network topology diagram, combined with the cache hit probability and data flow characteristics of each level, a miss rate calculation formula is defined and a cache miss rate model is constructed to estimate the cache miss rate at different cache levels. (2.2) Cache access latency modeling: Based on the connection paths between nodes, a cache access latency model is established. Based on the latency model, the point-to-point communication latency between two nodes is calculated, including zero-load latency and queuing latency. The average access latency of each cache level is quantified. (2.3) Memory parallelization modeling: Combining cache miss rate and inter-chip network latency, the memory parallelization parameters related to concurrent memory access time are calculated to evaluate the overall performance of the multi-chip system.

4. The design space exploration method for cache hierarchy in a multi-chip system according to claim 3, characterized in that: The step (3) includes the following sub-steps: (3.1) Definition of the optimization objective function: Based on the performance requirements of the multi-chip system, setting cost and power constraints, and combining the cache hierarchy and interconnection network parameters, the optimization objective is defined as minimizing the average concurrent memory access time of the application; Through recursive relations and referring to the average parallel memory access time formula, the cache miss rate model, latency model, and parallel memory access time model are integrated into the performance model. (3.2) Application of the two-layer optimization algorithm: The optimization objective function is decomposed into the cache subsystem optimization subproblem and the inter-chip interconnection network topology optimization subproblem. The two-layer optimization algorithm is used to alternately solve the problem until the performance converges or reaches the preset iteration limit. The target cache parameters that meet the constraints are obtained, and the optimized cache architecture diagram is generated.

5. The design space exploration method for cache hierarchy in a multi-chip system according to claim 1, characterized in that: Define the storage hierarchy graph G M (V, E), where each node v ijk ∈V represents a core, cache unit or storage unit, and node v ijk The weight is: When j = 0, v ijk is the core, and the weight is w(v ijk )= <cpi(v ijk ), ls(v ijk )>, where cpi(v ijk ) is the ideal cycles per instruction (CPI), ls(v ijk ) is the core delay; When 0 <j<h k When v ijk is a cache unit with a weight of w(v ijk )=<cs(v ijk ), cl(v ijk ), as(v ijk ), rs(v ijk ), ac(v ijk )>, where cs(v ijk ) is the cache size, cl(v ijk ) is the cache line size, as(v ijk ) is the cache associativity, rs(v ijk ) is the cache replacement strategy, ac(v ijk ) is the number of cache accesses; Cache replacement strategy rs(v ijk ) The replacement algorithms used include: FIFO, LRU, LFU, and random strategies; When j = h k When v ijk is a DRAM unit with a weight of w(v ijk )=<dt(v ijk ), bw(v ijk )>, where dt(v ijk ) is the DRAM access delay, bw(v ijk ) is the DRAM bandwidth; Storage hierarchy diagram G M Edge e in (V, E) ijk,mno ∈E represents node v ijk and v mno The connection relationship between them, when the node v ijk With v mno When connected, e ijk,mno =1, otherwise 0; Define the inter-chip interconnection network topology G I (N, C), where each node n p,q ∈N represents a node on a core q, which is a core, L1 / L2 cache, network interface or router, or just a D2D interface; node n p，q The weight ω(c(n p,q , n r，s )) represents the transmission delay between nodes; network connection c(n p,q , n r，s )∈C connects adjacent nodes; the core particle interconnection network topology G I The number of core particles in (N, C) is n α , and each core particle where |R i | represents the number of cores in the i-th core particle; The cache miss rate model and network delay model of the multi-chip system are constructed through the storage hierarchy diagram and network topology diagram to optimize the storage access performance under power and cost constraints.

6. The design space exploration method for cache hierarchy in a multi-chip system according to claim 5, characterized in that: Cache miss rate model M(v ijk ) is calculated by node v iik The reachable path Π(v ijk ) to determine the cache size, cache line size, cache associativity, and replacement policy polynomial fitting, where Π(v ijk ) represents the distance from any core to node v ijk The set of all nodes on the reachable path; Cache miss rate model M(v ijk ) based on different cache replacement strategies, including FIFO, LRU, LFU and random strategies.

7. The design space exploration method for cache hierarchy in a multi-chip system according to claim 3, characterized in that: The step (2.2) comprises: Define zero load delay, which is related to the distance between the source node and the target node. The calculation method includes the following steps: I Modeling, defining L(n p,q , n r，s ) is the slave node n p，q To node n r，s The transmission delay consists of two parts: the header packet delay L h (n p,q , n r，s ) and serialization delay L s (n p,q , n r，s ); According to the transmission rate μ, data packet size s i , flit size f and number of router pipeline stages η to calculate the serialization delay L s (n p,q , n r，s ) and header packet delay L h (n p,q , n r，s ); Define queuing delay, which is modeled based on the G / G / 1 model and computes nodes. The queue delay This delay is the node Multiple input channels With single output channel The queuing delay between Calculate the total queuing delay W(n p,q , n r，s ), the delay is through the path Γ(n p,q , n r，s ) on each router The sum of the queuing delays; Finally, the transmission delay D(n p,q , n r，s ) is the zero load delay L(n p,q , n r，s ) and queuing delay W(n p,q , n r，s ) and.

8. The design space exploration method for cache hierarchy in a multi-chip system according to claim 4, characterized in that: The step (3) comprises: Problem decomposition: The optimization problem is decomposed into two sub-problems through a two-level optimization algorithm, namely, the storage hierarchy graph G M The optimization subproblem P1 and the network topology graph G I The optimization sub-problem P2; Iterative optimization: The optimization process adopts an iterative approach. In each iteration, only one sub-problem is optimized, and the decision variables of the other sub-problem are fixed as constraints. Termination condition: When the target performance index σ meets the convergence condition or reaches the maximum number of iterations, the optimization process terminates.

9. The design space exploration method for cache hierarchy in a multi-chip system according to claim 8, characterized in that: The step (3.2) comprises: A two-level optimization framework is used, where subproblem P1 is a storage hierarchy optimization problem, and its goal is to optimize G M To minimize the average concurrent memory access time σ(G M ); Subproblem P2 is the interconnection network topology optimization problem, whose goal is to optimize G I To minimize the average concurrent memory access time σ(G I ).

10. A design space exploration device for cache hierarchy in a multi-chip system, characterized in that: include: The architecture diagram acquisition module is used to obtain the storage hierarchy diagram and network topology diagram of the multi-chip integrated system. The storage hierarchy diagram is a connection diagram of the core, cache, and DRAM nodes of the multi-chip integrated system. The network topology diagram is a connection diagram of the nodes within and between the chips of the multi-chip integrated system. A model building module is used to build a system performance model based on the storage hierarchy diagram and the network topology diagram; the system performance model includes a cache miss rate model, a delay model and a memory parallelization model; and calculate the optimal performance parameters and optimal cache parameters of the storage hierarchy diagram respectively; A model optimization module, configured to solve the system performance model and generate target cache parameters for optimal performance; A model generation module is used to generate a target cache architecture diagram based on the target cache parameters, and use the optimal cache parameters corresponding to the target architecture diagram as cache parameters of the multi-chip integrated system.

Images