Multi-Core Group Cache Architecture with Inter-Core-Group Buses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increase in the number of CPU cores and cache capacity leads to prolonged latency due to increased distance between CPU cores and caches, resulting in a cache or memory latency bottleneck that hinders CPU performance improvement.
Innovation Solution
The implementation of multiple core groups with intra-core-group buses and inter-core-group buses between caches in different core groups allows for efficient data access and storage across memory spaces, reducing latency by enabling parallel processing and interleaved data storage in caches and memories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of CPU cores or cache capacity is increased beyond the miniaturization rate of semiconductors, then the CPU performance is improved, but the distance between CPU core and caches relatively increases, prolonging and deteriorating the latency between CPU core and cache
Solution Approach 1:
The system divides the cache hierarchy into multiple levels (L1, L2, L3 caches) and organizes CPU cores into groups, where each core group has dedicated caches. This segmentation allows shorter access paths for frequently accessed data while providing capacity for larger datasets, resolving the contradiction between increased cache capacity and access latency.
Solution Approach 2:
The patent introduces a new dimension to the cache hierarchy by adding L3 caches that are shared among core groups but physically closer than traditional L2 caches. This creates a multi-dimensional cache structure where data can be accessed through multiple paths (L1→L2→L3 or L1→L2→memory), reducing the effective distance and latency for cache misses.
2Reliability
If additional provision of one layer of caches to the CPU cores between the L1 caches and the L2 caches is made, then the cache miss rate in the CPU cores is reduced, but the number of cache pipelines increases, deteriorating the latency between the main memory and the CPU cores
Solution Approach 1:
The patent merges multiple L2 caches into shared L3 caches that serve multiple core groups. Instead of having separate L2 caches for each core group (which would increase pipeline complexity and latency), the L3 caches provide a unified caching layer that reduces the number of independent cache pipelines while maintaining low miss rates through larger capacity and shared access.
Solution Approach 2:
The L3 caches serve multiple functions: they act as an extended cache for individual core groups, provide a shared resource for inter-core-group communication, and function as a buffer between the cache hierarchy and main memory. This multi-functionality reduces the need for separate specialized structures, simplifying the overall system while improving reliability.
Data Source
AI summary
An arithmetic processing apparatus includes: first and second core groups each including cores, a first to an Nth (N is plural) caches that process access requests from the cores, and an intra-core-group bus through which the access requests from the cores are provided to the first to Nth caches; and a first to an Nth inter-core-group buses each provided between the first to Nth caches in the first and second core groups respectively. The first to Nth caches in the first core group individually store data from a first to an Nth memory spaces in a memory, respectively. The first to Nth caches in the second core group individually store data from an N+1th to a 2Nth memory spaces, respectively. The first to Nth caches in the first core group access the data in the N+1th to 2Nth memory spaces, respectively, via the first to Nth inter-core-group buses.