A hierarchical memory access system and method based on monolithic three-dimensional integration

Abstract

The application belongs to a kind of hierarchical storage access system and method based on monolithic three-dimensional integration, system includes: computing layer, for integrating multiple processor cores, each processor core is physically independent from each other, and is equipped with private cache respectively;Storage management layer, for coupling last level cache with corresponding memory controller respectively, forming multiple last level cache-memory controller coupling unit;Storage layer, for integrating multiple main storage units, part of the data of the main storage unit is cached in the last level cache-memory controller coupling unit;Wherein, the computing layer, storage management layer and storage layer are directly electrically connected through the interlayer interconnection hole of distributed setting. The application improves the effective bandwidth and access efficiency of the storage system from the architecture level.

Description

Technical Field

[0001] This invention relates to the field of computer architecture and monolithic 3D integration technology, and in particular to a hierarchical storage access system and method based on monolithic 3D integration. Background Technology

[0002] With the continuous expansion of multi-core processors and artificial intelligence accelerators, the number of processor cores in computing systems continues to increase, and memory access concurrency is significantly improved. As a result, the overall system performance is gradually being limited by memory access bandwidth and latency. How to efficiently organize the connections between computing units, cache levels, and storage systems at the chip scale has become a key technical problem in high-performance processor design.

[0003] To overcome the physical limitations of traditional two-dimensional architectures in terms of interconnect bandwidth and latency, monolithic 3D integration (M3D) technology has been proposed in recent years. However, existing monolithic 3D integration architectures still employ the hierarchical storage architecture of centralized shared last-level cache found in traditional two-dimensional systems at the system organization level. For example... Figure 1 As shown, in this type of architecture, multiple processor cores access a unified shared last-level cache through their respective private L1 and L2 caches. The shared last-level cache serves as a centralized storage node on the computing side, connected to multiple memory controllers via an on-chip interconnect network or bus, and further communicates with off-chip dynamic random access memory.

[0004] In this architecture, the shared last-level cache handles a large number of memory access requests from multiple processor cores. It not only needs to perform data caching functions but also serves as a central node for the aggregation and forwarding of memory access requests. As the number of processor cores and the concurrency of memory access continue to increase, the access bandwidth, arbitration logic, and interconnection capabilities between the shared last-level cache and the memory controller gradually become significant bottlenecks restricting system performance.

[0005] Specifically, such as Figure 1 As shown, memory access requests from each processor core must first be centrally processed into a shared last-level cache, and then sent to multiple memory controllers via a unified on-chip interconnect or bus. In high-concurrency access scenarios, this centralized access path is prone to causing request congestion at the shared last-level cache, making it difficult to fully utilize the physical bandwidth provided by the memory controller, thereby limiting the improvement of the overall throughput of the storage system.

[0006] Although monolithic 3D integration technology significantly shortens the communication distance between the compute layer and the storage layer at the physical level through vertical stacking and high-density interlayer interconnection, existing solutions only perform a simple 3D mapping of the hierarchical storage architecture of the shared last-level cache in terms of spatial layout. They do not systematically reconstruct the functional division and structural relationship between the last-level cache and the memory controller for the characteristics of 3D integration. As a result, the shared last-level cache is still located on the compute side and centrally handles a large number of memory access requests, making it continue to be a key factor limiting the performance of the memory system. This weakens the overall advantage of monolithic 3D integration in improving memory access bandwidth.

[0007] In summary, existing technologies still suffer from problems such as unreasonable cache hierarchy and memory controller organization, limited bandwidth of shared last-level cache, and difficulty in fully releasing the potential bandwidth of the memory system under monolithic 3D integration conditions. Summary of the Invention

[0008] The purpose of this invention is to provide a hierarchical storage access system and method based on monolithic three-dimensional integration. While making full use of the advantages of vertical interconnection, the structural relationship between the last-level cache and the memory controller is reconstructed to reduce the constraint of shared cache on memory access bandwidth and improve the overall access efficiency of the storage system.

[0009] To achieve the above objectives, the present invention provides the following solution: A hierarchical memory access system based on monolithic 3D integration includes: The compute layer is used to integrate multiple processor cores, each of which is physically independent and equipped with its own private cache. The storage management layer is used to couple the last-level cache to the corresponding memory controller, forming multiple last-level cache-memory controller coupling units; A storage layer is used to integrate multiple main storage units, wherein a portion of the data in each main storage unit is cached in the corresponding last-level cache-memory controller coupling unit; The computing layer, storage management layer, and storage layer are directly electrically connected through distributed interlayer interconnect vias.

[0010] Optionally, the last-level cache-memory controller coupling unit includes a last-level cache submodule, a cache control submodule, and a memory control submodule; The last-level cache submodule is used to store data copies within the address space of the corresponding main storage unit; The cache control submodule is used to calculate the cache potential score of the corresponding data block based on the access characteristics of the memory access request, historical hit status and performance counter indicators, and generate cache replacement and data prefetching decisions based on the cache potential score. The memory control submodule is used to manage the upper-level main storage unit coupled with the last-level cache.

[0011] Optionally, the final-level cache submodule includes a cache data array and a cache tag array, wherein the cache data array is used to store data blocks read from the upper-level main storage unit, and the cache tag array is used to record the address information and status information of the corresponding cache data blocks.

[0012] The present invention also provides a storage access method implemented according to a hierarchical storage access system based on monolithic three-dimensional integration, comprising: Receive memory access requests from the processor core and perform address resolution on the memory access requests; Check the last-level cache flag based on the access address of the memory access request to determine whether a cache hit has occurred; When a cache hit occurs, data is read from the last-level cache and a memory access response is returned; When a cache miss occurs, the memory access request is sent to the memory controller coupled to the last-level cache, and the main memory unit is accessed. The data buffer receives the data returned by the main storage unit and generates a memory access response.

[0013] Optionally, reading data from the last-level cache and returning a memory access response includes: writing the read data into the read data buffer via the internal data path and returning the memory access response upwards to complete the access process.

[0014] Optionally, sending the memory access request to a memory controller coupled to the last-level cache and accessing the main memory unit includes: The memory access request is sent to the memory access queue, and the memory access request is queued and scheduled according to the preset storage access scheduling strategy to control the access operation of the lower-level main storage unit. When the main storage unit completes the access and returns the target data, the target data is transmitted back to the cache control submodule.

[0015] Optionally, after the main storage unit completes the access and returns the target data, the following steps are included: calculating the cache potential score of the corresponding data block based on the access characteristics of the memory access request, historical hits, and performance counter metrics, and comparing the cache potential score with a preset threshold to determine whether to perform a last-level cache update operation.

[0016] Optionally, updating the last-level cache may include: When the cache potential score is greater than a preset threshold, the cache control submodule performs a last-level cache update operation, writes the returned data into the last-level cache data array, and synchronously updates the corresponding cache tag. When the cache potential score is not greater than a preset threshold, the cache control submodule does not update the last-level cache content, and the returned data is only used to generate the current memory access response.

[0017] The beneficial effects of this invention are as follows: (i) Eliminating the bandwidth bottleneck of centralized shared last-level cache: In the hierarchical storage architecture of traditional shared last-level cache, memory access requests from multiple processor cores need to be uniformly converged to a single last-level cache for arbitration and forwarding. As the number of processor cores and the concurrency of memory access increase, the port bandwidth and internal arbitration logic of the last-level cache easily become system performance bottlenecks. This invention, by splitting the centralized shared last-level cache into multiple last-level cache-memory controller coupling units, distributes memory access requests to multiple independent cache and control paths at the architecture level, thereby avoiding the centralized convergence of requests to a single point and significantly improving the overall parallelism and scalability of the last-level cache layer.

[0018] (II) Fully Unleashing the Potential Bandwidth of the Memory Controller and Storage System: In traditional architectures, even if the underlying storage system possesses high parallel access capabilities, it is still limited by the forwarding capabilities of the shared last-level cache, making it difficult to fully utilize the available bandwidth of the memory controller. In the architecture proposed in this invention, each last-level cache forms an integrated coupled unit with its corresponding memory controller and establishes a direct access path with its corresponding main storage unit. Memory access requests can directly reach the target memory controller without going through the unified scheduling of the centralized last-level cache, thereby effectively improving the parallel working efficiency of the memory controller and allowing the bandwidth capabilities of the underlying storage system to be more fully released.

[0019] (III) Reducing the Complexity and System Overhead of Cache Coherence Maintenance: In traditional shared last-level cache architectures, the last-level cache often serves as a shared resource across multiple processor cores, requiring complex cache coherence protocols to maintain data consistency in multi-core systems, increasing hardware implementation complexity and access latency. In this invention, each last-level cache-memory controller coupling unit only caches data within its corresponding main memory address space. There is no data sharing relationship between different last-level cache blocks, thus eliminating the need to introduce cross-cache coherence protocols at the last-level cache layer. This design significantly reduces the scale of control logic and communication overhead required for cache coherence maintenance while ensuring functional correctness.

[0020] (iv) Shorten storage access paths and reduce access latency: By physically and logically integrating the final-level cache with the memory controller, and combining this with the high-density inter-layer interconnects provided by monolithic 3D integration technology, this embodiment effectively shortens the signal transmission path between the processor core and the target main memory unit. Compared with traditional two-dimensional architectures or simple stacked 3D structures, memory access requests in this embodiment can be completed via a direct path of "local interconnect—coupled unit—main memory unit," reducing the additional latency caused by cross-layer and cross-module forwarding, and improving the determinism and response speed of memory access.

[0021] (v) Excellent architectural scalability and implementation flexibility: The distributed last-level cache-memory controller coupled hierarchical storage architecture proposed in this invention has good scalability. As the number of processor cores or main storage units increases, the number of last-level cache-memory controller coupled units can be increased accordingly to achieve coordinated expansion of storage bandwidth and computing scale.

[0022] Furthermore, the implementation of each coupling unit can be flexibly configured according to specific application requirements. For example, the capacity of the final-level cache, the type of memory controller, and the type of corresponding storage medium can all be set independently, enabling the present invention to adapt to high-performance computing systems of different scales and application scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the hierarchical storage architecture of the shared last-level cache according to an embodiment of the present invention; Figure 2 This is a diagram illustrating a hierarchical storage access system architecture based on monolithic 3D integration, according to an embodiment of the present invention. Figure 3 This invention presents a distributed last-level cache-memory controller coupled hierarchical storage architecture. Figure 4 This is the final-level cache-memory controller coupling unit structure according to an embodiment of the present invention; Figure 5 This is a flowchart of a hierarchical storage access method based on monolithic 3D integration, according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] To address the limitations of shared last-level cache bandwidth and the difficulty in fully releasing the potential bandwidth of the memory controller in existing monolithic 3D integrated architectures, this embodiment proposes a hierarchical memory access system based on monolithic 3D integration, comprising: The compute layer is used to integrate multiple processor cores, each of which is physically independent and equipped with its own private cache. The storage management layer is used to couple the last-level cache to the corresponding memory controller, forming multiple last-level cache-memory controller coupling units; A storage layer is used to integrate multiple main storage units, wherein a portion of the data in each main storage unit is cached in the corresponding last-level cache-memory controller coupling unit; The computing layer, storage management layer, and storage layer are directly electrically connected through distributed interlayer interconnect vias.

[0028] Specifically, such as Figure 2 The system in this embodiment is constructed by vertically stacking multiple layers of circuits on the same silicon wafer, forming three functional layers from bottom to top: a computing layer, a storage management layer, and a storage layer. The functional layers are directly electrically connected through a large number of distributed inter-layer vias (ILVs), thus forming a high-bandwidth, low-latency vertical interconnect channel.

[0029] like Figure 2 As shown at the bottom center, the compute layer is primarily used to integrate multiple processor core modules. Each processor core is physically independent and equipped with its own private cache to form a core-oriented, low-latency local storage layer.

[0030] The processor core module in the computing layer establishes a direct connection with the storage management layer above it through vertical interlayer interconnect vias, so that storage access requests from the processor core can be transmitted to the upper storage structure without having to go through long distances of on-chip lateral interconnects.

[0031] The storage management layer, located between the compute layer and the storage layer, is used to integrate the last-level cache module and the memory controller module tightly coupled to it.

[0032] like Figure 2 As shown, the storage management layer is provided with multiple final-level cache units and multiple memory controller units. The cache units and memory controller units are arranged adjacent to each other in physical location and are connected by short-distance metal interconnects.

[0033] By integrating the last-level cache and the memory controller into the same storage management layer, the access congestion problem caused by the centralized deployment of the last-level cache on the compute side in traditional architectures is avoided, enabling the last-level cache to more directly serve the data buffering and request scheduling process in the storage access path.

[0034] like Figure 2 As shown at the top, the storage layer is used to integrate the main storage unit, which can be a dynamic random access memory (DRAM) or other types of volatile or non-volatile memory, such as resistive random access memory (RRAM), embedded dynamic random access memory (eDRAM), etc.

[0035] The main storage cells in the storage layer are directly connected to the storage management layer below through a large number of interlayer interconnect vias, enabling the memory controller to access the corresponding storage cells with a shorter physical path, thereby significantly reducing access latency and increasing storage bandwidth density.

[0036] In the aforementioned three-layer structure, interlayer interconnect vias are distributed in an array between the functional layers to carry the vertical transmission of data, address, and control signals. These interlayer interconnect vias can be formed using a high-density vertical interconnect structure based on monolithic three-dimensional integration technology, with interconnect dimensions and spacing significantly smaller than those of traditional three-dimensional interconnect structures achieved through through-silicon vias (TSVs).

[0037] Compared to TSV-based 3D integration, the interlayer interconnect vias do not need to penetrate the entire silicon wafer thickness, and have the characteristics of short interconnect length, low capacitance and parasitic effects. They can support higher interconnect density and lower transmission latency, making them more suitable for carrying the data and control signal transmission requirements in high-concurrency memory access scenarios.

[0038] Compared to the traditional two-dimensional architecture that relies on long-distance on-chip interconnects and centralized bus arbitration, this embodiment achieves high spatial proximity between the compute layer, storage management layer, and storage layer through three-dimensional vertical stacking and high-density inter-layer interconnects. This effectively shortens the storage access path, reduces memory access latency, and further unleashes the performance potential of the distributed last-level cache and memory controller coupling structure in terms of bandwidth.

[0039] like Figure 3 As shown, this embodiment proposes a distributed, last-level cache-memory controller coupled hierarchical storage architecture for monolithic 3D integrated computing systems. While maintaining the organization of the processor core's internal private cache hierarchy (including L1 and L2 caches), this embodiment restructures the organization of the last-level cache and memory controller, as well as their connection to the processor core, at the architecture level.

[0040] Specifically, this embodiment no longer adopts the traditional centralized shared last-level cache structure. Instead, it splits the originally uniformly configured shared last-level cache into multiple independent last-level cache blocks, each tightly coupled with its corresponding memory controller, forming multiple last-level cache-memory controller coupling units. Each coupling unit establishes a direct access path with the underlying main storage unit, thereby constituting a distributed, parallelized storage access system.

[0041] Figure 3 A schematic diagram of the distributed last-level cache-memory controller coupled hierarchical storage architecture proposed in this embodiment is shown. Figure 3 As shown in the middle section, the L2 cache of each processor core is connected to multiple final cache-memory controller coupling units via an on-chip interconnect network or bus structure. The on-chip interconnect network can be implemented in the form of bus structure, crossbar switch structure, ring interconnect, etc.

[0042] Unlike the traditional centralized shared last-level cache structure where all memory access requests are uniformly converged to a single last-level cache node, in this embodiment, memory access requests from the processor core are distributed to the target last-level cache-memory controller coupling unit according to the storage address mapping relationship or memory channel partitioning rules, thereby forming multiple parallel distributed access paths.

[0043] Figure 3 The local interconnect path in the code represents the direct access channel between the processor core and the target coupled unit. Through this local interconnect path, memory access requests do not need to go through the unified arbitration and forwarding of the centralized last-level cache, which effectively reduces the probability of cross-unit access and the bandwidth contention caused by centralized arbitration, and alleviates the bottleneck problem in high-concurrency memory access scenarios from the architectural level.

[0044] like Figure 3As shown in the lower part, the system is equipped with multiple final-level cache-memory controller coupling units (e.g., coupling unit 1 to coupling unit 4). Each coupling unit includes at least one final-level cache module and a memory controller module closely connected to it, and the two are regarded as a whole functional unit in terms of physical structure and logical control.

[0045] In this coupled unit, the last-level cache is used to determine cache hits and buffer data, while the memory controller is used to schedule memory access requests, control memory access timing, and manage the interface with the underlying main storage unit. Through this integrated design, the data interaction path between the last-level cache and the memory controller is significantly shortened, reducing cross-module communication latency.

[0046] Unlike traditional architectures where the last-level cache is primarily located on the compute side and serves as a shared resource across multiple processor cores, this embodiment logically migrates the last-level cache to the storage side and integrates it with the memory controller. Simultaneously, the original shared last-level cache is split into multiple independent last-level cache blocks. Each last-level cache block is used only to cache data within the address space corresponding to its coupled main storage unit, and does not cache data from other main storage units.

[0047] Based on the above structural division, there is no data sharing relationship between different last-level cache blocks. The data in the last-level cache exists only as a copy of the data in the corresponding main storage unit. Therefore, there is no need to maintain a cache consistency protocol between different last-level caches, which significantly reduces the complexity and hardware overhead of system consistency maintenance.

[0048] like Figure 4 As shown, the final-level cache-memory controller coupling unit in this embodiment is an integrated functional module used to handle memory access requests from the processor core, manage the cache, and control access to the upper-level main memory unit. Logically, the coupling unit includes a final-level cache submodule, a cache control submodule, and a memory control submodule.

[0049] The final-level cache submodule stores data copies within the address space of the corresponding main memory unit. It internally includes a cache data array and a cache tag array. The cache data array stores data blocks read from the upper-level main memory unit; the cache tag array records the address and status information of the corresponding cache data blocks. This final-level cache submodule only caches data within the memory address space managed by its coupled memory controller and does not involve caching data from other main memory units.

[0050] When a memory access request arrives at the coupling unit, the cache control submodule matches the cache tag array based on the request address to determine whether the cache has hit or missed.

[0051] The cache control submodule is used to coordinate the access process between the last-level cache submodule and the memory control submodule. Its main functions include cache replacement decision, cache data update, and memory access request scheduling.

[0052] like Figure 4 As shown, the cache control submodule may include cache access control logic, cache replacement and prediction logic, and related state management logic. Specifically, the cache replacement and prediction logic can calculate the cache potential score of the corresponding data block based on the access characteristics of the memory access request, historical hit rates, and performance metrics, and generate cache replacement and data prefetching decisions based on the cache potential score.

[0053] In the event of a cache hit, the cache control submodule directly reads the target data from the last-level cache submodule and returns the memory access response downwards; in the event of a cache miss, the cache control submodule initiates a memory access request to the memory control submodule and updates the cached data and cache tags after the data is returned.

[0054] The memory control submodule is used to manage the upper-level main storage unit coupled with the last-level cache. Its functions include memory access request queue management, storage access scheduling, read and write control, and interface control with the upper-level storage media.

[0055] like Figure 4 As shown, the memory control submodule may include a memory access scheduling module, a memory access request queue, and an interface module connected to dynamic random access memory or other storage media. When a cache miss occurs, the memory access request from the cache control submodule is sent to the memory access request queue of the memory control submodule, and the memory access operation is executed according to a predetermined scheduling strategy.

[0056] After the storage access is completed, the memory control submodule returns the read data to the cache control submodule to complete cache filling or directly return the access result to the processor core.

[0057] like Figure 5 As shown, a storage access method implemented based on a hierarchical storage access system with monolithic 3D integration includes: Receive memory access requests from the processor core and perform address resolution on the memory access requests; Check the last-level cache flag based on the access address of the memory access request to determine whether a cache hit has occurred; When a cache hit occurs, data is read from the last-level cache and a memory access response is returned; When a cache miss occurs, the memory access request is sent to the memory controller coupled to the last-level cache, and the main memory unit is accessed. The data buffer receives the data returned by the main storage unit and generates a memory access response.

[0058] Specifically, in the last-level cache-memory controller coupling unit, memory access requests are processed in the cache submodule, cache control submodule, and memory control submodule according to a predetermined collaborative workflow, thereby achieving the integration of cache access path and memory access path.

[0059] When a memory access request from the processor core enters the coupling unit via the upper-level cache, the cache control submodule first parses the memory access request and performs a matching check on the last-level cache tag based on the access address to determine whether the memory access request hits the last-level cache.

[0060] Furthermore, reading data from the last-level cache and returning a memory access response includes: writing the read data into the read data buffer via the internal data path and returning the memory access response upwards to complete the access process.

[0061] Specifically, when a cache tag matches successfully, the cache control submodule determines that the memory access request is a cache hit request and directly controls the last-level cache data array to read the corresponding data block. The read data is written to the read data buffer via the internal data path and a memory access response is returned upwards, thus completing the data access process in the case of a cache hit.

[0062] During this process, memory access requests do not need to enter the memory control submodule and the lower-level main storage unit, avoiding unnecessary storage access operations and reducing access latency.

[0063] Furthermore, sending the memory access request to the memory controller coupled to the last-level cache and accessing the main memory unit includes: The memory access request is sent to the memory access queue, and the memory access request is queued and scheduled according to the preset storage access scheduling strategy to control the access operation of the lower-level main storage unit. When the main storage unit completes the access and returns the target data, the target data is transmitted back to the cache control submodule.

[0064] Specifically, when a cache tag fails to match, the cache control submodule determines that the memory access request is a cache miss request and forwards the request to the memory control submodule for processing.

[0065] After receiving a cache miss request, the memory control submodule sends the request to the memory access queue and queues and schedules the request according to a predetermined storage access scheduling policy, thereby controlling the access operation to the lower-level main storage unit. When the main storage unit completes the access and returns the target data, the data is transmitted back to the cache control submodule.

[0066] Furthermore, after the main storage unit completes the access and returns the target data, the following steps are included: After the main memory returns the data, the cache control submodule does not perform cache filling operations for all cache miss requests. Instead, it calculates the cache potential score of the corresponding data block based on the access characteristics of the memory access request, historical hit status, and performance counter indicators. Based on the comparison result of the cache potential score and the preset threshold, it determines whether to update the last-level cache.

[0067] Furthermore, updating the last-level cache includes: When the cache potential score is greater than a preset threshold, the cache control submodule performs a last-level cache update operation, writes the returned data into the last-level cache data array, and synchronously updates the corresponding cache tag. When the cache potential score is not greater than a preset threshold, the cache control submodule does not update the last-level cache content, and the returned data is only used to generate the current memory access response.

[0068] like Figure 5 As shown, the cache control submodule calculates the cache potential score of the corresponding data block based on the access characteristics of the memory access request, historical hit data, and performance counter feedback information, and compares the cache potential score with a preset threshold. When the cache potential score is greater than the preset threshold, the cache control submodule performs the last-level cache update operation, writes the returned data into the last-level cache data array, and updates the corresponding cache tag synchronously. When the cache potential score is not greater than the preset threshold, the cache control submodule does not update the last-level cache content, and the returned data is only used to generate the current memory access response.

[0069] The selective cache filling mechanism described above can effectively prevent low-value data from frequently occupying the last-level cache space, thereby improving the utilization efficiency of cache resources.

[0070] In this embodiment, the cache potential score is used to characterize the overall benefit and risk level of the target data block between the cache layer and the main memory layer. The cache control submodule constructs a unified scoring function by fusing row access statistics from the memory controller side and system operating status information. The cache potential score Score(i) can be expressed as: ; in: RowHit_counter(i) represents the cumulative number of row buffer hits triggered by the target data block within a preset statistical window. This parameter is maintained by the row buffer status detection logic on the memory controller side and is used to characterize the contribution of the data block to the continuation of the currently active DRAM row access segment, reflecting the row access locality characteristics of the memory access request. Conflict_counter(i) represents the number of times the target data block triggers a row switching operation within the statistics window. It is accumulated when the access causes pre-charging and reactivation operations, and is used to characterize the degree to which the data block disrupts the continuity of row access. Queue_Depth represents the number of pending requests in the current memory request queue. This parameter is provided by the memory controller's scheduling queue length register and is used to reflect the current main memory access load level of the system. RowMissBurst represents the intensity of row misses within a preset time interval, used to characterize the fluctuation of row switching pressure in stages. The "≪1" is a shift operation that performs weight scaling on this statistic to reduce hardware implementation complexity. This embodiment also provides a memory access response generation and return mechanism: Regardless of whether the memory access request is a cache hit or a cache miss, the final retrieved data is written to the read data buffer, and the cache control submodule returns a memory access response to the upper-level cache or processor core, completing a full memory access operation.

[0071] Each last-level cache-memory controller coupling unit is connected to at least one dynamic random access memory, such as Figure 3 The dynamic random access memory 1 to dynamic random access memory 4 are shown.

[0072] When a processor core initiates a storage access request, the request is first processed through the private cache of the compute layer; in the event of a cache miss, the request is sent directly to the corresponding last-level cache block in the storage management layer through the inter-layer interconnect via.

[0073] If the last-level cache is hit, the data is returned to the compute layer via the inter-layer interconnect vias; if the last-level cache is not hit, the request does not need to go through centralized cache arbitration, but is directly forwarded by the memory controller coupled to the last-level cache block to the corresponding main storage unit in the storage layer.

[0074] Due to the tight coupling between the last-level cache and the memory controller in terms of spatial location and logical control, the transmission path of storage access requests between the cache layer and the memory controller is significantly shortened, reducing unnecessary cache contention and bandwidth competition, thereby improving the effective utilization of memory bandwidth.

[0075] In this embodiment, the last-level cache is no longer a globally shared centralized cache resource. Instead, it is divided according to storage address space or memory channel, with each last-level cache block caching only a portion of the data in its corresponding main storage unit. Since there is no data sharing relationship between different last-level caches, there is no need to introduce a cache consistency protocol at the last-level cache layer, thereby simplifying the control logic and reducing system overhead.

[0076] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A hierarchical storage access system based on monolithic three-dimensional integration, characterized in that, include: The compute layer is used to integrate multiple processor cores, each of which is physically independent and equipped with its own private cache. The storage management layer is used to couple the last-level cache to the corresponding memory controller, forming multiple last-level cache-memory controller coupling units; A storage layer is used to integrate multiple main storage units, wherein a portion of the data in each main storage unit is cached in the corresponding last-level cache-memory controller coupling unit; The computing layer, storage management layer, and storage layer are directly electrically connected through distributed interlayer interconnect vias. The final-level cache-memory controller coupling unit includes a final-level cache submodule, a cache control submodule, and a memory control submodule; The last-level cache submodule is used to store data copies within the address space of the corresponding main storage unit; The cache control submodule is used to calculate the cache potential score of the corresponding data block based on the access characteristics, historical hit status and performance indicators of the memory access request, and to generate cache replacement and data prefetching decisions based on the cache potential score. The memory control submodule is used to manage the upper-level main storage unit coupled with the last-level cache.

2. The hierarchical storage access system based on monolithic three-dimensional integration according to claim 1, characterized in that, The final-level cache submodule includes a cache data array and a cache tag array. The cache data array is used to store data blocks read from the upper-level main storage unit, and the cache tag array is used to record the address information and status information of the corresponding cache data blocks.

3. A storage access method implemented according to the hierarchical storage access system based on monolithic three-dimensional integration as described in any one of claims 1-2, characterized in that, include: Receive memory access requests from the processor core and perform address resolution on the memory access requests; Check the last-level cache flag based on the access address of the memory access request to determine whether a cache hit has occurred; When a cache hit occurs, data is read from the last-level cache and a memory access response is returned; When a cache miss occurs, the memory access request is sent to the memory controller coupled to the last-level cache, and the main memory unit is accessed. The data buffer receives the data returned by the main storage unit and generates a memory access response.

4. The storage access method according to claim 3, characterized in that, Reading data from the last-level cache and returning a memory access response includes: writing the read data into the read data buffer via the internal data path, and returning the memory access response upwards to complete the access process.

5. The storage access method according to claim 3, characterized in that, Sending the memory access request to the memory controller coupled to the last-level cache and accessing the main memory unit includes: The memory access request is sent to the memory access queue, and the memory access request is queued and scheduled according to the preset storage access scheduling strategy to control the access operation of the lower-level main storage unit. When the main storage unit completes the access and returns the target data, the target data is transmitted back to the cache control submodule.

6. The storage access method according to claim 5, characterized in that, After the main storage unit completes the access and returns the target data, the following steps are taken: based on the access characteristics of the memory access request, historical hits, and performance counter metrics, the cache potential score of the corresponding data block is calculated, and the cache potential score is compared with a preset threshold to determine whether to perform a last-level cache update operation.

7. The storage access method according to claim 6, characterized in that, Updating the last-level cache includes: When the cache potential score is greater than a preset threshold, the cache control submodule performs a last-level cache update operation, writes the returned data into the last-level cache data array, and synchronously updates the corresponding cache tag. When the cache potential score is not greater than a preset threshold, the cache control submodule does not update the last-level cache content, and the returned data is only used to generate the current memory access response.

Images